Jerry - I shoot 85gr. 2F in my smoothbore with patched round ball, for targets out to about 60yards. I know this gives about 1,200fps to 1,300fps velocity. For shooting the longer 100and 110yard shots on our trail, I use 100gr. 2F. This works - sometimes. That is, sometimes I hit but more often, I miss the long shots.
I do not know the answer to your question, about stability. A ball that is launched from a smoothbore, is not stable, as there is no rifling to impart a stabilizing rotation to the ball.
What happens, is sooner or later, the ball will take on a spin and veer off it's intended path.
I have seen this, in binoculars when one of the guys was shooting. The ball arcing towards the gong at 100or so yards, then just before getting there, seemingly in the last 20 (looks closer) or so yards, veering hard off to the left, right or down & missing by 3 or 4 feet. I've only observed this a few times, but when I did, the ball never climbed, only veered left, right or a combination with down. This veering happened with one particular fellow who shot 65gr. 2F in his 20 bore guns.
I fully suspect 120gr. 2F would likely work better, but have not tested that myself.
The higher velocity shooting guys, like Taylor (85gr. 2F I think in his 42" bl.) and Norm who uses 85gr. 3F in his 42" bl., the ball does not veer, but arcs to the target. I have not witnessed as many of the higher speed shots.
I think, now this is just an impression, that increasing my charge to 100gr. 2F from 85gr. 2F, gives me more hits on the longer range gongs.
