Reader Poll Roundup: Series Nine Edition

When I drew my comparisons last year between Series Seven and Series Eight, I had to do a little handwaving because of the difference in the lengths of those series (thirteen and twelve episodes, respectively). For similar reasons, the switch between Doctors was difficult to quantify. This time I can draw more direct parallels as both the number of episodes and the current Doctor are the same between S8 and S9; S7 will also get a look-in.

Beginning with the average (mean) ratings of episodes, we see the usual ups and downs over the course of the series. To get the average rating for any given episode, each star rating (e.g., 5 stars) was multiplied by the number of votes it got, the results added, and the sum divided by the total number of votes. For Series Nine, we get the following:

Although there are some peaks and valleys (more on that later), we see that the ratings were fairly consistent throughout the series. Fully half of them fall within less than a third of a star of each other (from 3.45 to 3.75 stars).

If we rearrange the data to show how the episodes scored from highest to lowest, we can see the trend more clearly (note that the episodes are different colors between the above and below charts; apologies for the vagaries of my visualization software):

Last year I noted the large spread between the highest- and lowest-rated episodes, which amounted to a difference of more than a star-and-a-third (1.36 stars). The jump in Series Nine is even more pronounced, with top-rated Heaven Sent scoring 4.37 stars to bottom-ranked Sleep No More's 1.95 stars (more than two standard deviations below the average), which makes nearly a two-and-a-half-stars gap (2.42).

Averaging ratings across the entire series, we get a series-average of 3.58 stars, with a standard deviation (σ) of 0.65 stars (indicative of the huge spread). For comparison, Series Eight's score was 3.53 ± 0.46 and Series Seven's was 3.76 ± 0.34. For kicks and grins, I tried throwing out the high and low scores and recalculating. Without Sleep No More dragging things down (Heaven Sent isn't really that far off trend), the average S9 score was 3.66 ± 0.38, which is a much more reasonable-looking σ.

Series Nine was overall fairly comparable with both its predecessors. It had three 4.00-star or above episodes (Heaven Sent, Under the Lake, and The Witch's Familiar), just like S8 did, though all three of them had higher ratings (4.37, 4.27, and 4.27 respectively) than the single highest-rated episode over the previous two years (Hide, which came in at 4.23 stars). Similarly, three of its episodes were rated less than three-and-a-third stars (The Girl Who Died: 3.23; Face the Raven: 3.15; and Sleep No More: 1.95), while S8 had four episodes at those levels and S7 had two.

Each episode had a wide vote distribution, show here by percent of overall respondents:

Zero-star votes were out in force again this series. Although Face the Raven garnered the largest overall percentage of goose eggs (22% of its voters), Sleep No More (SNM) really took the prize. SNM exhibited not just one but two unprecedented characteristics in my reader polls. The first is that—for the first time since I started these ratings polls—SNM garnered more zero-star votes than five-star votes. Even if we throw those out, the overall trend in votes can be approximated with a line of positive slope (rather than any sort of "bell curve"), wherein an increase in votes corresponds with a lower star rating.

Especially with peculiar distributions like that in play, I again had to see how much of a difference all those zeroes made (24 votes out of 291 across the entire series, or 8%—the most ever). It turns out, the difference is quite large: an increase of 0.32 stars to a new series average rating of 3.90 ± 0.61. (Extending my experiment above, if I throw out the high and low scores from my no-zero-star-ratings calculation, the overall series rating improves even further to 3.99 ± 0.36.) Sans zeroes, then, S9 handily beats both S7 (3.79 or 3.80 without high/low scores) and S8 (3.70 or 3.72) by the same metrics.

As last year, two episodes improved their overall scores by more than one full σ after eliminating zero-star votes: Face the Raven (Δ = 0.90) and Hell Bent (Δ = 0.62). Those particular episodes were character- and story-arc heavy, which suggests that fans reacted strongly, whether in the positive or negative, to those installments. Nor were they entirely alone. Using the no-zeroes ratings, six of the episodes—fully half the series—rated better than 4 stars each (top-ranked Heaven Sent improved to 4.52 stars), compared to three before. Even poor, unloved SNM hit nearly two-and-a-third stars (2.31) using this method. Clearly leaving in the "No stars" option is having a significant effect on the overall ratings.

All of this is, of course, a statistics of small numbers. Reader participation continues to decrease during the regular series (polls are open for two weeks, starting on the broadcast date), though opening up the voting for all episodes again after the finale added a few votes for each episode. I intend to do that again next series.

On the up side, by the end of voting, the final episode had raked in thirty-five votes, the most for the series. Half the episodes had twenty-six or more votes, while the other half had twenty or less. Just as in previous years, there was a rough inverse bell curve to the participation rate with a mid-series peak for a single episode (last year it was Kill the Moon, which seemed to bring in an outrage vote; this year's The Girl Who Died was clearly a curiosity because of guest star Maisie Williams). More accurately, two "halves" of the series each had an inverse bell curve, with participation peaking at the beginnings and ends.

I could probably keep finding new permutations to calculate and graph, but I'll stop here. What did you think of the poll results? Did anything in particular surprise or dismay you? If there is any further data manipulation you'd like to see, just let me know in the comments, and I'll be glad to oblige.



Sleep No More was unquestionably the weakest episode of the season. But I would have ranked it the same or better than about a third of the episodes of last season. It wasn't THAT bad and while the story and characters were undistinguished for Doctor Who the presentation of looking out of the eyes of various characters gave it points for being new and different. I like experimental episodes. When you try new things you might crash and burn but also, it might be fantastic!

There was far less rediculous anti science this season. What a relief!

Personally I voted all the episodes this season a 3 or 4 except Sleep No More which I think I gave a 2. Nothing was so excellent that it deserved a 5. Nothing was so terrible that it recieved a 1 or less.

I notice you didn't include the Christmas episodes as part of the seasons. I found the Christmas episodes from this and last year quite good, far better than some of the previous Christmas episodes. Perhaps you should compare all the Christmas episodes you have numbers for against eachother.

By Kara S (not verified)
mrfranklin's picture

Good idea. Gimme a tick...

By mrfranklin
mrfranklin's picture

It turns out I have polls for four Christmas specials: The Snowmen, The Time of the Doctor, Last Christmas, and The Husbands of River Song. The number of votes ranged from sixteen (Last and THORS (to date)) to thirty-five (TTotD). The group average rating is 3.79 ± 0.50, or 3.92 ± 0.38 without no-star votes.

Individual ratings span from 3.11 stars (TTotD) up to the current high score of 4.31 stars (THORS). TTotD also had the most zero-star votes (three); if those are removed from the calculation, it improves to 3.41 stars.

Not sure if that tells you everything you wanted to know, but that's what I have off the cuff. :)

By mrfranklin

I really didn't care for The Time of the Doctor and The Snowmen was mawkish. I did like Last Christmas and THORS very much. I also really liked the Christmas Carol episode. The Widdow and the Wardrobe was dismal.

By Kaara S (not verified)
mrfranklin's picture

I found that the ratings I gave some of the Christmas specials at the time are not necessarily how I'd have rated them now. :\ I'm pretty much in agreement on all your assessments here, though. :)

By mrfranklin
Real Time Analytics