Drilling a hole in the right spot just takes a little planning.
-Spot the hole location with a center punch. This must be done unless you are using a milling machine and center bit.
-measure, double-triple check it is in the right place.
-Drill the hole, but way undersize.
-Check the location again, it is is a little off, center it up with a Dremil tool. Drill bits follow existing holes.
-Drill it a little larger, check it again. IF it is now perfect..
-Drill to the tap size.
*Never drill a hole for a tap in one shot. Drills love to wander. They always drill a little oversized. IF you make a pilot hole they will drill the correct size.
That seem Like a lot of messing around. It is. Filling holes is worse. When it matters, I always do the above. Having a lock bolt off center in the tang drives me nuts.
I do have a milling machine and would only consider it to be a stout drill press, I do not try to drill holes by the numbers on the cross feeds. I do what I described above.