Have you considered roughing your balls? Smooth bore shooters at N-SSA and the USMLIT both rough their balls for greater accuracy. There are several ways to rough them including the use of ferrier files, or, rolling them between a hard rubber sheet and a piece of 60 grit sand paper on a hard rubber sanding block. I use the latter method for roughing my balls for smooth bore pistol. The theory is that the rough texture mimics the dimples of a golf ball, and aids in the ball flying true. You should note that roughing the surface of the balls increases their diameter slightly. So, you should test the balls with different size patches.
I have not done a side to side comparison of roughed and smooth balls, but use this more on faith at this poiint. What does definitely contribute to greater accuracy in a smooth bore pistol is an increase in velocity. It has come to the point that many British shooters will use as much as 35 to 40 grains of 4F Swiss in .45 caliber pistol. They claim superior results. I am now using 25 grains of 4F in a Pedersoli Tatham & Egg repro with good results.
Best regards, John C.