Temperamental Problem with Heater switch

Hi folks.

Could someone please perhaps give me an insight into what might be wrong? I have had a browse at some forums but ideally I'd need help on my specific problem.
it started last winter. I tried to switch my heater on and got nothing. After I started the engine, I gave it a few minutes of twiddling and the heater came on. I though maybe because it was really cold that might have been the problem. Anyhow it all seemed fine but then setting number 4 stopped working. A few days later it was ok again. No problems for months. The during the summer it starts again.. This time trying to put aircon on. Nothing... And then repeat. a Few twiddles and it works. However now, Setting number 3 does not work but setting 4 does!
Can someone tell me what might need fixing? Please don't tell me I need to take the steering wheel off!