Numeric Entries "recoil" when pulling away the Apple Pencil

I am not sure how to search for the issue within this forum in case it was asked before - so please forgive me if this is a duplicate query.

When I am trying to adjust the X or Y position of say a reference image, I would use the Apple Pencil within the input box and drag the entry until it reaches the position I want. However, when I release my Apple Pencil, somehow there will be a recoil (probably due to my unsteady hand I suppose while gripping my Apple Pencil?) and my reference image will shift slightly off from my original position. This happens to other entry when I will be dragging. So what I will normally do is to drag the reference to the position I need, read the numeric value and after the recoil, manually key in the exact number that I need. This is quite inconvenient.

What I would like to know is if there’s a better technique of dragging or lifting off or entering those numeric entries without having this recoil? Thank you.