I did the bulk of the work on my first rifle with a set of "12 for $10" razor-blade-stuck-in-a-dowel carving tools, a pair of cheap buck brothers full-sized gouges, couple of full-sized Stanley chisels, and a 4-1 rasp from the home improvement store. Kept them sharp on a $5 pocket oilstone from Walmart. Ideal? Hardly, but they did work. Admittedly, the Stanley chisels were and remain pretty good tools.
I've done an awful lot of work over the years, not just on guns, on a little Workmate folding bench, because it was all I had. Aggravating to use, but better than nothing. Living in an apartment, a full-sized bench is out of the question...
These days the tools options have changed - the hardware store variety have gotten a lot worse, but there are options like flexcut carving tools that weren't available 20 years ago. I still think that, if you can find someone who can hep you with a certain specialized tasks, like drilling the ramrod hole and installing the vent liner, then you can do the rest with very minimal equipment.
Honestly, if you want to build a gun, get a lock, barrel, and a decent piece of wood, and set to work. Buy what you need when you need it, have patience, and you will find it a lot more manageable (if a much longer endeavor!) than you think.