I rub mine pretty hard (just because I do everything hard
) and it is holding up very well (I've had it about 2 years now). When I first got it, it did shed a lot, but as it was used and cleaned that problem dwindled.
Maybe you need to clean it. I run mine under water every week; then once a month I use a dab of soap with my water.
I have gotten an extra set of pads so that I can keep stamping while one set of dirty pads are drying.