Don't use a stone unless it is very fine grit, and you can keep it square when you use it. You would do better with a good ball file running slow in a hand drill. The best way to go would be to have it re-crowned by a competent gunsmith on a lathe. Stones wear away as you cut, and they don't always wear evenly. A proper sized ball file will cut very evenly, and very accurately, but it has to go slow and not chatter. A ball file most often will do as well as the lathe for a problem such as yours, but the lathe is the ultimate best answer.
