I inlet when I have the form final rasped to shape. This is with a fine rasp, not a coarse one. All the chew marks from the coarse rasp are gone. I inlet the box, shaping to the form as best I can while inletting. I bend a little more here or there as I inlet it deeper. When close, I file everything smooth as possible.
If it's a thin sheet box, I will then sand the box in place. Use an eraser to clean up the wood.
If it's a cast box, I usually remove it from the stock for sanding and polishing. While it's out., I like to scrape the stock at this point. If it leaves the box a little proud of the stock, that's ok.
Sometimes the box gets scratched during finishing the wood, so spot sand or burnish the scratches out. Steel wool does a nice burnish on brass.