Do I have to book online?

No, you can stop by and book on site without a reservation. However, if you book online, then you have fixed your place and get the cheaper prices.