The crown, as Dan brought up is usually the culprit in this situation - or the bore has been shot with pyrodex and allowed to rot a bit. The pits will create fouling traps and a rough surface for loading. The pits caused by perchlorate are sharp edged and will grab a patch, but usually tear it as well. Since they come out in good shape, perhaps the crown just needs a little smoothing.
I like a 'softer' more rounded corner crown, with the edges of the crown a little more rounded than they appear in Dan's picture. Since the barrel will come off easily, chucking it in a lathe, spinning it and just using a bit of wet-or-dry or emery 320grit pressed against the muzzle's crown with your finger will smooth it and make loading tight combinations quite easy. If a lathe is not handy, just clamp it in a vice and use the end of your thumb to press the paper or emery cloth into the muzzle and rotate your hand, every 10 seconds or so, rotate the barrel 90 degrees. This will keep the crown even and centred. Depending on the crown that is already cut, as in Dan's picture, softening, or radiusing that crown may take only 15 seconds or a couple minutes depending on it's condition.
Here, we all use a .005" under ball and a .018" to .0225" patch loading without difficulty. Not wanting to use a .010" or other ridiculous thickness patch is good and seaking advice on how to go about it is also good. All of us use a substancial short starter as well and we all use the rifle's 3/8" or whatever diameter hickory rod.
Note, the horn in this picture holds over 1 1/2 pounds of powder. It is a large horn. The reason we use moose antler (bases) is it is heavy and hard. Note the short stud on the starter handles. We used to just smack the ball into the bore, but I found doing that, reduces accuracy minimally, due to the off centre force moving lead sideways, slightly. Using the stud (hitting the top of the handle with the palm) makes the entry straighter almost like using a bullet starter - but not quite that accurately.
How the ball is started makes a difference and can be noted by using a stip of patch material, pushing or smacking the ball into the bore, with withdrawing it to check how centered the ball was, by the rifling marks. One blow starts the ball - not tap, tap, tap with a hammer or bang bang, bang with a mallet as I've seen in some videos. Hitting the ball more than one blow deforms it's upper surface, albeit the nose is of less importance than it's bottom. Every starter I use, has a cupped end, that causes no damage to the ball.
Smoothing the crown on a muzzle.
.45 and .58 muzzles
All of these crowns allow pressing the patched ball down into the bore with little strain on the rifle's stock, just using a "choked" rod or the short starter stud with the large knob in your hand. I prefer to use the starter as it's faster, easier and pushes a perfectly centred ball down some 6" before putting the rod on it. This helps 'save' the rod from my hamfistedness. (is that a word?)