I have a small (90A?) Harbor Freight flux-core wire welder that I bought to play around with small stuff; it cost less than $100 and works on 16g up to 3/16" pretty well if the settings are right. Works amazingly well once you get the hang of it but you have to learn how to use it and protect against spatter, or else learn to love cleanup. At the economical end of the spectrum, a small welder will pay for itself in just a few uses. Think being able to weld back up a tang or lockplate versus buying a new one and starting over. The flux core wire is indistinguishable (to me at least) from mild steel as far as finishing; I would like gas shielding option, though. The same benefits could be had with a small stick welder, but it takes longer to learn. I know cheap is never a popular option, but for the hobbyist who only uses the welder on small stuff a few times a year, there really is no need to go into the multi-thousand dollar rigs. Dan's Hobart handler is a good example of more than adequate for hobby use and a handy size for many tasks.
That said, I am moving toward forge welding ultimately, for the authenticity, just need to finish the forge. Also, a perfectly good alternative is riveting and brazing, which your torch or a forge will do. So, I doubt you'll regret a welder if you don't get insane about what you spend, but be realistic about what you want to do with it and make sure there aren't better options first.