No, don't just turn up looking for a room.... at most (if not all hotels) you'll pay rack rates which are more expensive than internet bookings.
Good 4 star hotels in good locations... Pathumwan Princess (opens onto MBK) Belaire Princess and Amari Boulevard in Sukhumvit Soi 5, Sofitel Silom (Silom Rd.) and Imperial Queens Park in Sukhumvit Soi 22.

Have a look on any of the internet sites like sawadee, wired-destinations dot com and asiatraveldotcom and you'll find these and a lot more.