Drill a hole in a piece of hard wood (maple, cherry, etc) as close as you can to the size you want, with whatever is a nominal drill bit size. Then, with a reamer or a rat-tailed file, make it the size you want. Sneak up on it, and try patched balls in it to get the fit you want. A loading block with only two or three holes is all that you'll need. Otherwise, the lube will dissipate and leave it too dry. Seal the holes with tung oil, bees wax or the like, and you'll be good to go. I use one when I hunt, and have made use of it several times for a quick second shot, even if I don't need to take it.
You'll likely find that the hole you need is larger than what you thought you'd need.