No, I never keep my pins either--David has one on his lapel, I think.
We also crashed our first SSS party. We got the letter warning us about the party (our second cruise), but never got the invite. Fortunately we had met a couple who were frequent cruisers--they told us about it, and dragged us along. No pins, however. Left a bit of a bad taste, I guess. Nice party, free champagne and caviar, otherwise, no biggie.