Yeah I was at the Southern 100 last year, great racing and the sidecars were crazy. As you say there are 4 main corners but there are also some very fast bends which have good vantage points.
Olivers Mount is good and its easy to get to.
Also if you want to go to watch racing on the Isle of Man Mountain Course the Festival of Motorcycling (Classic TT/Manx GP) is excellent especially if you like bikes from the 1980's back to some of the very first bikes, unfortunately like the TT you have to book your accommodation and ferry about a year in advance to get what you want.