First, we all know that we have opinions on things and some are different...so this post is not challenging yours, just adding other viewpoints of consideration to loaded Flintlocks and pulling balls, etc.
In my opinion the loading process is far more prone to the extremely remote chance of an accidental discharge than while pulling a ball...ie: if we are to believe the sometimes mentioned concern that powder is being poured into a barrel that may have just been fired, the notion that an open vent could invite however remote, the possibilty for an extraneous ignition source to come into play, and the sometimes mentioned notion that compression is involved, etc. And during the loading process, and these conditions, the ramrod hand is over the muzzle.
However, speaking only for me, I load my Flintlock in the garage before leaving the house for an early morning hunt...when loaded, I tape over the vent, and put a finger cot over the muzzle. There is no internal ignition source inside the bore, and there is no access to the bore for an external ignition source.
After a hunt, I peel the tape off the left side of the stock where I had stored it, and retape the vent for the trip home, get there, lay the rifle sideways on a carpeted workbench and slide the ball out with a ball puller.
No internal bore ignition source, no external ignition source can come into play, not pouring powder into a just previously fired barrel, no compression involved, only takes 10 seconds to turn the screw 3 times and slide the ball out.
So IMO, I have far less risk of an AD when I pull a ball compared to reloading the Flintlock at the range.
Not saying I'm right...just offering my .02 cents