That can only happen if you leave the barrel on the stock and only use patches for cleaning. If you remove the lock, then plug the vent, fill the barrel with water, let it sit a bit, then pull the plug letting the water drain out the vent, then use patches to wipe the bore clean, repeat a couple times, the "crud" will not be able to build up in the inner cone of the vent liner.
Some of us remove the barrel for cleaning whether it is attached with pins or wedges and stick the breech end in water, then pump water in and out with a tight patch on the jag. That way, "crud" can never build in the vent.