When we were discussing this yesterday, the whole "condor-zone" issue, let alone ball-diameter, never even occurred to me... and it should have.
That, in itself, explains a lot about the wild results you were describing.
Yep: your "fix" is going to be more in patch (thickness and material) and lube; THEN start your search for your "pet" powder charge.
Couldn't figger out quite what was going on for you; I'd found that barrel to be quite surprising, for an import/production barrel. That's why you've got it. But that sub-size ball is your problem you were reporting.
I'll see what I've got for thicker, natural-fiber patching. We've still got a whole raft of worn, well-washed cow-country clothes to choose from.
Meanwhile, there's this:
http://americanlongrifles.org/forum/index.php?topic=4330.0and, do you still have a grain-weight reloading scale? As mentioned, a little weighing and measuring (mic-ing potential patch materials, etc) might reveal some issues and potential fixes.
Another option: shelve the entire project, 'til, as you say, my grandkids are a little more grown, and you feel better about casting your own ball, using a custom, full-size mold.
Toward that direction, has anyone here done any accuracy testing with the lead-free balls in a larger-bore smoothbore? Next ball-size up from the (nominal) .54RB is a .601": much closer to full diameter 20ga, than their .54 cal offering. Steve's living in brush country, with heavy forest just up the slope: long, desert-country shots aren't an issue... but the groups he was reporting (and the lad's a shooter) weren't at all acceptable for hunting.