Matt - if you're going to buy balls (expensive), then use the .628's. If your bore is .660" and the balls are .628", they will be .032" smaller than the bore and you should easily be able to load a nice denim or mattress ticking patch.
We used to adhere to the old rule (maybe it was only our rule) of a ball being .030" smaller than the bore of a smoothbore. This easily allowed the use of a .020" or even thicker patch. Most of us today, use a ball that is .020" to .025" smaller and the same .020" to .0225" patch. The thicker patches hold lots of lube whether an oil/grease or water based (oil or grease for hunting) lube and will also help compress the ball without damage to it.
In the loads we use, there is compression of the ball itself - is it not just a sliding fit in a tube. If a sliding fit, it will not effectively push down ALL of the fouling from the previous shot and it will build up, eventually preventing you from loading- a ball and patch stuck 1/2 way down is not acceptable to us, so we use combinations that prevent exactly that. There is never more than 1 shot's fouling in the bore and that is pushed down when loading the next ball. As far as accuracy is concerned, perhaps your gun wants a loose fit and wants you to wipe it often - it's up to your gun. I refuse to do that with my smoothbore and perhaps that's why it isn't very good past 30 or 35yards. To that range, it will hold 1 1/2", but goes to pot by 50 yards. For me, it's merley a shot gun that I sometimes will shoot balls in. I refuse to have to wipe the bore while shooting.
If you do the math on bore and patch thickness:
.660" bore - .628" ball = .032" difference. .032" divided by 2 = .016" per side. It would appear that a .016" to .018" patch would perfectly fit with that sized ball, however there would be no or very little compression/pressure of the patch into the ball to assist in cleanign while loading. The pressure against the side of the bore would be minimal. Yes - it would load with almost no effort sliding down a 'freshly wiped and clean bore by the weight of the rod - something some guys seem to like, but then, they also seem to like to wipe the bore every shot or so.
It all depends on what YOU want to so. Spend the day wiping the bore, or simply load and shoot, load and shoot. the first is less critical of patch thickness - in fact you don't even had to use a patch, just a thin wad over the ball to hold it, will do. Even that will shoot into 1 1/2" to 2" at 25 yards.
That same .628" ball, if of pure lead, will easily load with a .022" denim patch, and if the crown is nicely rounded as it should be, will start easily without cutting the patch - but- you wil probably need a short starter. You will not be able to thumb start it - if you want to use nothing more than your thumb, you'll need to use thinner material and may find you have to carry lots of cloth for wiping the bore. For those who have to wipe and dry their bores while shooting a 20 to 50 shot trail, with rifle or smoothbore, what do you do with all the dirty patches?
This is my 20 bore's muzzle.
This is how I made it look that way with 320 grit (fine) emery cloth. Wrist rotation pluss periodically rotating the gun to keep it even. A cleaning patch shoved 3" down the tube will collect debris and it easily removed with long tweezers or long needle nose pliers. If not available, a worm on a rod or hooked piece of wire will suffice.
