I started using poster putty to keep gravers clean, and have recently used it when filing with small files that had excess magnetisms.
I keep a hunk handy for stabbing/wiping gravers on, or roll a ball of it across the file. Gets filled-knead it around a little for a fresh surface. Some tools are worse than others for sure.
Demagnetizing is a best option, just haven't made/acquired a gizmo for such yet.