It seemed like a lot to me, too, but the book's numbers are all consistent for different ball diameters, and muzzle velocities.
Keep in mind that those numbers are the wind drift for a 10 to 20 mph
crosswind. If the wind were blowing at less than a 90 degree right angle, some of that wind force is downrange, not crossrange. For example, a 20 mph wind blowing into your face but off to the side 45 degrees would resolve into about a 14 mph crosswind and a 14mph headwind.
If you don't have a wind gauge or wind measuring instrument, wind estimation is more an art than a science. It may be the wind wasn't blowing exactly from the side, or maybe the wind velocity was lower than it felt. Here's a link to a Weather Service site for visual cues on estimating wind:
http://www.wrh.noaa.gov/pqr/info/wind.phpGoing back to your original numbers, a left to right wind, and hitting 1" right of point of aim at 50 yards, either your rifle would shoot a couple of inches to the left on a still day, or your estimation of crosswind speed was too high. You were right, I think, not to drift your front sight to compensate for one-time wind conditions.
SCL