How are you removing the fouling you push down the barrel in front of the patched jag? I submit that you are blocking the vent with this fouling.
I never clean until I get home from the range. And when I pour in my next powder charge and ram a patched ball, I can see new powder at the vent through the hole. Ignition is instantaneous and there is no build up of fouling at the breech or in the bore.
How do you know the thickness of your patches? Patches sold commercially that say they are .020" thick, are not, for example.