The first clue is weight, if all are dried to the same moisture content. Silver maple is light, red maple is medium weight, and sugar maple is very heavy. Specific gravity of silver maple is .44, of red maple, .55, and of sugar maple, .68. These are averages of course. Weight and hardness go hand in hand. There will be overlap of the softest sugar maple and the hardest red maple. But the hardest sugar maple will be close to hickory or the oaks in density.
Some sugar maple has a pinkish cast to the wood.