In my experience FWIW, some Buff horn will start to sprout hair after being worked and some will not. I use lard to heat and shape my horns and I used to think this helped. It might, but I have had this problem with a few that were done this way. Crack cream for hands seems to help keep horns "alive" longer. Zim's is one brand but there are many available at the drug stores. Between bugs, ultraviolet light, and just drying out, horns are always going to be a little unstable.
Steve