Sonny, it takes practice - once you get the hang of it, it will work fine. Also, the leather tab (or as you call it, the "tongue") needs to be softened some and the hole "worn" larger or stretched from use. Try wetting the leather tab and forcing the leather button through about 15 or 20 times. Of course, you can change it out for a brass stud but the brass stud was a very rare item at the time and considered too expensive to be wasted on something like a soldier's cartridge box. Keep working on it, you'll get used to it. Also, remember, you don't button the cover every time you pull a cartridge out. When firing in ranks, the soldier would simply drop the cover in place. He would button it only when on the move.
And yes, the boxes were large.
BrownBear, where do you get your Sam Brown studs? Are you using the US or English Style? I have been looking for a source for the British pattern stud. The US Sam Brown types seen around today are strictly 20th Century, the English style date back to the late 19th Century.