Some inlays are intended to stand proud of the wood, but for those that are desired to be flush, I file and sand them down once they are installed. If a bevel is filed under the edges and the mortise is cut properly, the process of filing the inlay down flush with the surrounding wood will result in the appearance that the wood grew around the inlay- a perfect fit. Once this is done, I would never want to remove the inlay and disturb that fit. Also, many inlays are attached with nails and tacks that would not be practical to remove. I've never had an issue cleaning the stain or finish from an inlay, just wipe it off while it is wet.
*Since the inlay is filed down and "blended" with the wood, this is best done before staining. I wouldn't want to be filing and sanding the inlay after the wood is finished.