It requires a bit of investment, but a few taper pin reamers of various sizes come in handy for a variety of hole size adjustment issues. The ebony loading blocks below are for .32 and .50 cal round balls. I drill the holes undersize and then use a taper pin reamer, from both sides, to precisely adjust the hole size to grip the ball (any size ball and any patch thickness) the way I want.
I also use these reamers to open up the bores of cast ram rod pipes, etc. The reamer in the photos below is a Number 8. I use #s 7,8, & 9 the most and those three have been worth more than what they cost over the years in making hole adjustments quick, easy and precise.
