I'd be tempted to try experimenting with maybe a .570 ball (or maybe even .562) and a thicker patch (pillow or mattress ticking .018 to .025 thick). It may be that if you had more cloth and less lead, the progressive loading difficulties you describe might be reduced.
I have found some guns to be extremely sensitive to the patch lube being used, too. Some of my rifles are very happy with a spit patch. On another rifle, I used a variation of Dutch Schulz's recommended lube: I make up a mixture of olive oil and water (1:7 ratio) plus a drop of dishwashing detergent so the oil will separate. Shake it to mix. Put the patching material in to saturate it. Lay the material out flat and let it dry, and use it after it dries. (Dutch's recipe uses water soluble machine oil instead of olive oil/soap, but I haven't been able to find any locally.) In shooting off a rest, I saw a very noticeable shrinkage of group size when I went from 1:6 oil: water to 1:7, and got a better group than using spit lubed patches.
If you were to shoot 5 shot groups with .570, and .575 balls, with .011, .018, and .020 thick patching, lubed with spit, Hoppes, 1:6 Olive Oil, and 1:7 Olive oil, you would be shooting 2X3X4=24 five shot groups. That might take several trips to the range, but would give you a pretty complete record of what your rifle likes and doesn't. And, since your rifle is a double, you could probably limit your test to whichever barrel you feel most confident is the 'best'.
Good Luck, SCL