It's still cute, but it's not as nice as it could be.
So how can it be fixed?
An eraser. It doesn't have to be new or pretty, although if you want to get one exclusively for this purpose, that would be a great idea.
Feeling lazy, I just walked over to my kitchen junk drawer and found an old, sad looking eraser that somehow survived the school year. Seriously, it's bad:
Then I used the eraser to lightly buff the surface of my stamp.
That's it. I flipped my circle piece over and re-stamped the image on the back. Here's the result:
Big improvement, right?