Right or wrong, I always do the final scraping, whiskering, staining, and finishing with the mounts/inlays in place. I take iron mounts off after staining for their finishing. As brass and silver usually don't get any sort of color treatment, I would just clean them in place, if necessary, after the final wood finishing is done. Ballistol will take the worst of tarnish off brass or silver almost instantly without harming the wood.
If finishing involves wetting the stock with water, all inlays and mounts must be in place for that to avoid having the inlets close up so that you can't get the inlay or mount back in without more cutting. A word of warning, make sure that all inlays/mounts, or the bevels thereof, are flush with the wood and NOT even the slightest bit below the surface. Scrape along the edges to make sure all is flush and level. Wood above an inlay can swell over the inlay causing a chip out when the inlay, such as a patchbox finial, is removed. Been there, done that, and it is a real bummer when it happens.
Along those same lines, mounts should just drop in a stock without forcing them. This is critical for the barrel/tang, set triggers and lock. If these things are tight, it will shorten the life of the stock. All stocks shrink over time. If they haven't air dried for a number of years before you use it, it will shrink even more. When the stock shrinks around the barrel, the forearm splits, and you can even get splits down the center of the barrel channel right to the breech. I have seen antiques with stocks that look like they exploded where the barrel was inlet particularly tight. You can never put them back together right. You have to glue in a linen strip to hold everything together. Just some advice, if you want your gun to still be around in 200 years.