The reference point can be whatever you want it to be.
It's very convenient to make yourself the reference point. When you do that, then all the directions, displacements, speeds, and momentums ... relative to the reference point ... are exactly what they look like to you !
