A .32 will shoot dime-sized groups at 25 yards - ALL DAY& fairly easily given a decent shot, good eyes & GOOD ball/patch combination.
A mere 20 to 25gr. of 3f is all that is needed at that range.
Shooting that well at 50yards, however, will usually take a lot more powder and may never happen, that is, resulting in dime sized groups with a .32.
A .45 will my much easier to get to shoot those sized groups at 50 yards, but at that, dime-sized is a good trick for open sights at 50 yards from a rest. Nickle sized groups are quite possible with .40's, .45's and even .50's - and maybe a .36 as well - but a .32 - possible, but not with the .32 I had. My .32 would put 5 consecutive shots that you could cover with a dime at 25 yards, using only 20gr. 3F. It would give the same accuracy (albeit different poi) with charges increasing by 5gr. weight, right up to 40gr. 3F at 25 yards. All charges were stricken using ctg. case powder measures with roughly a .30 cal bore. The smaller the hole diameter in the powder measure, the closer each thrown powder charge will be to the previous one. Using small diameter powder measures and good technique, your charges will not vary more than a couple tenths of a grain.
The larger the ball, the greater the POTENTIAL accuracy as the range increases.