I'm very much a hobby woodworker. I'm retired recently, so I have more time and hope to get better at it and to make more projects in the coming months and years. There are species that I simply don't on the have on the property which I enjoy working with (Butternut, Black walnut, etc.), but for the most part, I agree with you. For the record, I own over 100 acres, but only about 70 of that is timbered and it's not all hardwood, although it's mostly hardwood northern forest.
I just discussed the possibility of having a guy with a smaller sawmill bring it to the property to mill several logs for me soon. I'll probably do a Cherry (still have some Cherry from a tree I cut a few years ago), a Hickory, an Ash (it's all going to die anyway) at least one Maple (I have hard and soft maple and would love to find one with a lot of character) and possibly something else, I'll have to think about it. I have a large American elm on the property and I presume it's going to die on me at some point, but I understand that it can twist and be pretty hard to work when it dries. I'll also need to decide whether to let all of the lumber air dry or take it to someone with a kiln (the sawmill guy doesn't have one).
If I had a 100 acres of hardwood forest, I would never buy wood for any project from any one.