I use doubled flannelette on the jag when cleaning. I clean my ML rifles, all of them with the barrel removed & with the breech shoved into a container of cool tap to cold water. I have never had any buildup of fouling in the bore itself when using mink oil or neetsfoot oil. I use tight combinations, maybe that's the difference - Taylor too & LB, and Neil, Hatchet Jack, & the rest.
The cleaning patch, pumping up and down seems to wipe out all the oil (from the last shot) in the first pass or two - we use the same patch for all the cleaning strokes.
I repeat, I have NEVER had any buildup as Gander suggests. 2 years ago during the winter trail shooting, Taylor used Neetsfoot oil almost every Sunday due to it's being warmer on his old fingers than an alcohol/water based lube. He would fire typically up to about 55 shots int hat one day's shooting. He never had to wipe his bore during the day and he never had to resort to a solvent to cut oil fouling in his bore when he cleaned his rifle barrel after the shooting was finished. He never had to wipe the bore while shooting. There was no oil/fouling buildup. If you do have to cut the grease or oil in the bore, try something like WD40. Then clean with water as normal for BP. Since Taylor had a rusting episode with Ballistol while on a hunting trip, I've never tried it.
Unburnt powder does not, can not remain in the bore - too much flame work going on, but solid residue from the previous shot (56 or 57% of the charge results in solid waste. Some of that solid waste adheres to the inside of the bore after a shot. The lube on your patch remaining in the bore softens that fouling. That 'fouling' is pushed down with the loading of the next lubricated patched ball - or, if not - the fouling will build up as you are using a (too) loose combination imo. The only way anything builds up inside the bore where the ball occupies and flies, is if the patch/ball combination is too loose - again, in my opinion. It can not build up with a tight, properly lubricated combination.