The game occasionally bugs out on me with stuck movement keys and I have spam my right mouse button to bring up the sights. This happened to me deep in Greased Palms and Neon Tomb.
Is your screen mode fullscreen? If not, try switching over to it. Really odd UE4 issue where inputs can hang on screen modes other than fullscreen