I have to make myself reset the butt back into my shoulder almost every shot. I tend towards too light a hold in the shoulder with light recoiling rifles and varying this shot to shot will in itself, cause vertical changes.
Going to a .495" ball will help - don't change the patch material. use an oil-type lube. Hoppe's #9 plus is cheap for the shooting you get from one quart bottle. LHV,if it's available is probably the best patch lube for target I've found in 36years of shooting. Hoppe's Plus is probably second with spit third.
Weather! - this all changes when switching from winter to summer shooting - spit for lube and 2F works better for me in the summer - both .45 and .40 - go figure. My tight, cold weather grouping goes to pot when things warm up, so NP- I just switch to spit and 2F. My loads in 2F are roughly 10gr. more than my 3f charges in the .45, however in the .40, it's 10 gr. less- go figure. Every barrel is unto it's own as far as loads are concerned. We can ask what everyone is using and trends will generally show- especially with the better 'placers'. By placers, I'm talking about those who actually put in some time developing a load that will help them 'place' in the winner's circle. Others just pick a load and try to make the gun shoot that load. Sometimes that works - generally, it merely allows them to sit and watch the 'placers' get their well earned trophies.
It all depends on what you want from the gun - the best, or just something that rings some of the steel targets. To routinely outshoot "today's smoothbore shooters" some of whom we shoot with weekly, one has to make the rifle shoot well - their smoothbores certainly do. Seems less trouble to get one of those shooting well than a rifle.
There is more than one smoothbore shooter up here in Alberta and BC who think that on gongs, a rifle has NO advantage to 125 yards. There are those who shoot the smoothie so well, that that they usually beat all the rifles on steel. This has happened more than once at the Hefley Creek Rondy - with steel sils. to 100 yards. As a matter of fact, it happened last year for the second year in a row. The top smoothbore shooter beat all the rifles for top score. Good thing they have separate contests for smooth and rifles. I know this makes the rifle shooters sound weak - yes - I guess that's the case, but we're trying.