A couple of versions ago there was introduced a bug that the game ignores controller inputs while you are panning around with right-click.
Now a new bug has also emerged where if you use middle mouse button to pan, the view keeps jumping back, so you can't pan this way together with zoom anymore.
It sort of makes flying a PITA because looking and zooming the view around is something I've grown to do habitually in FS. Though I don't know what the "way of working" is. Maybe those bugs take so long to be fixed because no one else really has the same problem I do.
Actually my most preferred way to look around is FSX/Xplane where you can use the hat switch, but in FS2020 the hat switch makes the camera jump to specific preset locations instead of letting you freely look.