Since I card all my browning/rust bluing with a wire wheel I remove all steel/iron for carding. No matter how careful you are applying the solution I think you are going to get a small amount that will flow under the metal and continue to rust if not neutralized. With that said I would "yank it off".
DMR