Here's something to think about. You do all of your target shooting and sighting-in activities from a big range box, and load with a stainless steel range rod. Then you shoot your trail walks from a hunting hag and horn. So when you go hunting, you take your minimal set (loading block and charging tubes). You have developed almost no muscle memory, and when you are faced with a "must load this next shot fast", you fumble and make mistakes that cost you your opportunity for the coup de grace, and your wounded buck gets up and runs away.
That's a stereotypical scenario for sure, and it wasn't meant as a criticism of your loading and shooting regime. What surely works best for me is to use all of the same equipment for all of one's shooting. That way, your hands will fly automatically to your accoutrements and you'll load your rifle instinctively. I load a bullet board with lubricated precut patches and balls when I hunt, but I practice with it all so when the time comes, as in the unfortunate scenario I described above, you can load your rifle fast without hardly taking your eyes off the buck.