Just one thought: you are not thinking about the Earth rotation
Just one thought: you are not thinking about the Earth rotation as camera shaking, are you?
The staight line is due to Earth rotation, and the deviation from a straight line is due to camera shaking.