On the note of shooting it, if you really know how to take care of a muzzleloader, and can clean it completely, I don't think shooting it will "hurt" it if you are careful.
That said, I am a little concerned by the question about how much 2f you can stuff down the barrel. The question that you SHOULD be asking is, "At what general load in a .50 have y'all gotten the best accuracy from your .50 gun?" I would start at 55 grains or so, and shoot a group at 50 yards. Put up another target at 50 yards and increase to 60 grains. Keep doing that until your group size shrinks, then begins opening up. The smallest group size will be your powder charge. I would guess somewhere around 70 to 75 grains with a .495 roundball and .015 to .018 heavy canvas all cotton patch. You should hear it start to "crack" around 65-70 grains. Remember, it is of no importance how much powder you can stuff down, or how fast the ball goes, if you cannot hit that at which you are aiming.
You have to ask yourself now, "Do I really want to shoot this expensive gun enough to determine the load it likes?" If not, you may want to market it and buy something else.
I hope this does not offend. It was not intended to do so, but rather to be helpful.
Best wishes, and God Bless, Marc