I've seen a reference to an Oct 1972 Muzzle Blasts article where Dr Gary White documented some very stout loads in a custom rifle (caliber unspecified): "Averaged 1850 fps with 140 gr, 1940 fps with 168gr, 2040 with 196 gr, 2222 fps with 224 gr, then dropped to 2196 with 252 grains...with a load of 280 gr he was still under the muzzle vel he measured at 224 gr" And, of course there was a "Kids don't try this at home" warning.
Indicates to me the the point where you get no increase at all in velocity, or a decrease in velocity, is probably way up there beyond what a sane person would normally use or need.