Accurately drilling a hole can be a challenge even in a milling machine with all axes locked, apart from the quill movement.
I start with a centre drill. The centre drill is short and so it doesn’t wander around the mark.
Drill gently to ensure that the centre is perfectly aligned and drill into the material forming a small chamfered edge. This chamfered edge will help to align the next drill bit.
Drill the hole through the workpiece using a drill that is slightly under size. I normally use a drill that is 0.5mm smaller diameter than the final hole size required. This will then mean that the final drilling operation is more likely to be accurate in size.
Don’t use too much pressure as you don’t want the drill bit wandering.
Use cutting fluid if appropriate as this will reduce the temperature of the cut and avoid any local work hardening.
Finally drill the part to the actual size required. Again, use cutting fluid to reduce the heat and take a reasonably gentle cut.
This process is equally applicable in the pillar drill, lathe or even with a hand held drill.
The other key part of accurately drilling a hole is to use sharp drill bits.