phone-in polls - a statistical plague on both our houses

According to Reinventing TVNZ GayNZ and others are incredibly concerned about the use of phone-in polls in news reporting. Fair enough - these have got to be amongst the most dubious news-creating devices out.

I just cannot believe that any journalist would base a news story on the results of a phone-in poll. They are statistically criminal for so many reasons it's hard to know where to start, but I'll try:

1. the sample is completely self-selected, ie there is no randomness that would mean you could meaningfully extrapolate - the poll ONLY represents the views of the exact people who rang in, NO ONE else.

2. multiple call-ins are not taken into account. Those who hit redial effectively get twice the say of those who only call once. What might be useful would be some analysis of how many voters on each side called multiple times - it might reveal that there was a minority who felt strongly enough to call many times, which would be interesting.

3. of course it is exclusive to people who own phones, without a toll bar, and can afford to call. And were watching Holmes, oops Close Up At 7, (or whatever) and thus knew about the poll.

Ok enough of this - check out the excellent cartoon from Ross at Dorking Labs for a good sum-up of it all.