I tend to think of the .54 as a general-purpose caliber for North America (or similar sized game from Europe, etc). Other than moose and the big bears, it seems adequate in power while still being manageable for most shooters. A possible difference today is that people tend to not load up to a point where they get best accuracy; period records suggest that half ball weight (so, ~110-115 gr) was a common charge. Loads around 60-80 grains are common today. If I hunted moose more often, or in places where bears are an issue, I'd probably go with a 16-20 gauge rifle like Daryl and Dan, or my 10-11 gauge smoothbores.
A possible factor to consider is that, at least in what is now the US, many people choosing a rifle were also considering the defensive use of the rifle. The military went to .54 after dabbling with the 1792 Contract Rifles in .47-.49 caliber. I've not dug into it but have always been a bit curious for their reasoning in increasing the caliber for the .54 caliber 1803 and subsequent rifles until the adoption of the Minie ball.
For targets, I bounce back and forth between the .50 and .54. When chasing accuracy, recoil can become a factor (I suspect that is part of the reason for the popularity and success of the .40). I think the place for the .54 in target shooting is where winds are a big factor.
In handguns, particularly smooth bores, I like the .54. In my experience smooth bore pistols seem tend to get best accuracy with heavier charges, and the .54 is about all I want to stand for 40-80 shots in a day. In pistols rifled with a rifle twist, the same thing comes up. I don't have enough experience with them to say how well a fast-twist .54 pistol would do, but then again, I don't recall seeing an original rifled .54 caliber pistol with a barrel rifled faster than 1:36.
