There are LOTS of other islands in Thailand that have been far less spoilt than Ko Samet!!!!... (thank god).
I would agree that it's probably the best suited option somewhere that close to BKK but I would totally disagree that White Sands Beach (Hat Sai Keaw) is what you sound like you want. It's the busiest and most crowded beach on the island...think jet skis, banana boats and wall to wall resorts.
If you are willing to pay REALLY top $$$, then Ao Kiu Na Nok would be perfect for you. It used to be the last truly unspoilt and undeveloped beach on the island but now a 6 star resort has taken over it. The resort has done its best to discourage riff raff like me from using its beach by blocking public footpath access to it.
If 80,000B a night sounds a bit steep then Tubtim beach might be a reasonable quietish alternative but don't expect to have the place to yourself.
More info here
Ko Samet travel guide for backpackers. Includes hotel and hostel reviews