Well, I ain't very smart so bear with me. So, if you put a .50 cal ball between your shoulder and your buttplate and shot your gun the ball would go through your shoulder just like the one you fired from the barrel?
I have spun deer clear around with a .62, but I didn't spin around when I shot the gun......
https://en.wikipedia.org/wiki/Newton%27s_laws_of_motion, specifically the 3rd law is the important one here. It says that for the action of the ball + patch + powder ejecta going out the barrel, there is an equal reaction going the opposite way. But just as the business-end action is dispersed over *everything* going forward, the equal reaction is dispersed over *everything* going backwards. Which is (a) the gun and (b) you. So while the energy going forward is mostly (but not entirely) concentrated in the ball, simply because that's the heaviest bit, whereas the energy going backwards is dumped into the rifle (then dumped into you). You're familiar with how a heavier bullet has a lower muzzle velocity than a ball for the same charge? Well imagine how slow a bullet would fly if it weighted as much as the gun! That's essentially why the gun doesn't fly through your shoulder, it's so heavy that the "butt velocity" is too low, and the impact area is so much larger than a bullet. This is why light rifles feel like they kick more (they don't kick *more* but they kick *faster*), and why narrow butt plates hurt more.
You can use Newton's third law to calculate the free recoil of a rifle/shotgun/pistol (
https://en.wikipedia.org/wiki/Free_recoil). Free recoil is the recoil of the gun if you're holding it loosely, where it's basically free to recoil without resistance. Usually you want to solve for the recoil speed, not the recoil energy or momentum, since the recoil speed is what causes the "pain". This is useful if you're trying to design a rifle for someone who is recoil sensitive (or a load for their existing rifle).
But the upshot is that yes, the energy dumped into that deer by the .62 ball is *less* than the energy dumped into you when you shot it - less, because some of the energy went into making the powder and patch fly downrange, and because the air slowed the ball down some before it hit. It's possible that the ball really did spin the deer around, if the deer were much lighter than yourself and the rifle was loaded pretty heavily - certainly a .62 is getting to the point where a heavy load will thump you pretty severely.. In my part of the country the deer are very small (frequently 70-90 lbs). But this sort of thing can also happen because the deer flinched or jumped crookedly because of a damaged leg or ribs, causing him to spin himself around.