Go for it Nick!
When I was younger, in 1981, I made a rifle, not all by hand as you want to do, but with using only hand tools. I also made the patchbox, all the inlays, sights, muzzle cap and RR pipes, engraved everything and relief carved it. I still have the gun and still remember each and every slip of the knife and engraving tool. And not a single one of them bother me a bit!
Hang around, and you'll find most guys here are really helpful,,, and a small handful, not so much...
John