As was said above, I believe a large part of the problem is dry patching the bore.
I was taught that if you wipe between shots, you use a damp/wet patch followed by a dry one. This keeps the fouling soft. Don't use a super tight patch for wiping either. You don't want to drive a mass down to the bottom of the bore (I think you did this).
Around here, if the Southern weather is right, I'll get a hard grunge ring in the bore around where the ball seated. If I feel this, I go with a wetter patch to loosen it up. Using wet patch lube usually minimizes this in my experience.
Note that a too loose ball/patch combo will allow lots of blow-by and really foul a bore.
I was also taught that the vent pick is not necessarily to clear the vent hole, but is instead meant to create a flame channel into the middle of the powder charge. This speeds up the iginition. Logically, I pick each load, right before I prime the pan.
Yes, I do wipe the frizzen, pan and flint edge with a damp patch after each shot. Without wiping, the grunge in the pan can really suck in moisture, and on a bad day will get no flash at all. Wipe the pan, voila! Back working again. Alcohol is great for this wiping if you want to carry it. I have found a brush to be useless for this grunge cleaning.
This of course is for range shooting, and the plan differs for elsewhere.
I am no expert at anything, but am happy to share what I have learned. Especially when I know the "why" behind the method.
Best wishes and God Bless, Marc