It's your gun right? Do what you want! I put a low rear sight on mine for hunting deer, since it always shot low without one. Big difference! Gave me more confidence in my shot.
I don't shoot many smoothbore matches anymore, but if I do I just push it out of it's dovetail and go at it. It has a witness mark so I can put it back in the same position.
For good practice you can put the sight on and shoot it in the rifle matches! Or is that illegal somewhere?