I think most of the weight you are seeing is probably because you haven't yet trimmed down the "rails" to either side of the rod groove. How much of the ramrod should show? The number 50% gets thrown out there, but it actually varies. Having more than 50% of the rod showing is common. On the half stock Daryl just showed, you can see that only about 25% of the ramrod is covered.
I'm assuming you saw this Jaeger on Clay's website intro?
https://www.youtube.com/@grumpygunsmithofwilliamsburg/videos
Notice how much of the ramrod is showing. It is very common for people to cut the ramrod groove deeper than it needs to be, and that may be the case with your stock as well.
If you do need to cut down the rails on the ramrod channel, then do it very slowly and carefully. It's very easy to go too fast here with a rasp and accidentally remove more than you meant to. And if just using a rasp, it's also hard to be consistent. So if you have to take off a lot, you can use a rasp to start with, but stop well before you get near your final surface and switch to a sandpaper jig, something like this image below, that will help you get to final depth consistently. This is with a piece of 220 sandpaper held to a 4"x8" wood block using glue or double sided tape, and then a split piece of ramrod screwed or tacked down over the sandpaper, with countersunk holes.
Don't use coarse heavy sandpaper on this, as you need to be able to control this well and you don't want it to leave deep scratches in the wood. If the sides of the forearm have already been shaped a bit (as yours have), then the rails on the rod groove aren't very thick and you have to cut them very carefully.
If you make the split piece of ramrod just a tiny bit thicker/deeper than you need your final groove, then when this jig stops cutting, you will have just enough wood left for final sanding or scraping.
