All the work of inletting, cutting in sights, etc, will certainly affect the old brown. Leave it alone, and see what you have in the end. Then you can decide to clean it up, or brown over top of the original finish.
I do not believe the nice brown will survive the build process unscathed.