Unless you are really into temples then 4 days is about right to cover the major ones and perhaps a visit to a floating village out of town. Forget the one most tourists go to (Chong Khneas). Then you could take a bus or hire a taxi to Phnom Penh and spend 2 days there visiting the sights. Fly from here to either Hanoi or Saigon if you can change our already booked flights.