In May of this year, I spent a long, slow day at work penning an article comparing the ratings a network television series receives on IMDb with its fate in the renewals and cancellations. The conclusion was reasonably straightforward, but the numbers were fascinating to analyse, and given the level of discussion generated, I felt compelled to take the concept further.
This time, I sought to leave Nielsen out of the picture entirely, and instead ask the question of whether a television series gets better during a season, and during its lifetime. I wanted to know whether those episodes in a series that blew my mind were received in a similar fashion by most other fans, and vice versa for those episodes which fell a bit flat.
IMDb visitors, in addition to giving a series an overall star rating out of 10, can also rate individual episodes out of 10. It's worth noting that the difference in the quantity of votes between the two can be rather large, and from my analysis, season and series premieres, tend to have higher vote counts than other episodes. Also, bear in mind that individual episode ratings have no bearing on a series' overall rating.
A term I've seen referenced on the internet on this subject is survivor bias, not to be mistaken for survivorship bias, which has its roots in economics. In television terms, it proposes that the critics view and rank the first few episodes, but if they call it quits on a series, only those that like the series are left to rank its episodes, which can in theory skew the results. From what I've seen, survivor bias presents itself the strongest right at the beginning of a series but doesn't flow on to future seasons. Bringing dreaded Nielsen back into the fold for a moment, this makes sense because series premieres often rake in big audiences, but said audiences drop and flatten out fairly quickly.
To analyse the data, I'm using a site called graphtv.kevinformatics.com, a beautifully simple site that allows you to simply type in the name of a series and see its data in graph format. If I don’t mention your favorite series in this article, you can go to the site and check your own shows out yourself. The graphs on the site are also a lot larger, and are interactive.
To kill two birds with one stone, let's look at some long running medical dramas for starters. FOX's House MD was on our screens for eight terrific seasons, and sits among the best rated television series on IMDb - an 8.9 rating from 229,445 ratings to be precise. Its lowest rated episode (7.7) came in its final season, and its two highest rated episodes (9.8) came in its fourth season. Two of its 8 seasons trended down, while 6 trended up, with the steepest slope seen in season 5. Surprisingly, House MD trended down over its 8 seasons, as can be seen by the white line in the graph below:
Entering double figures this year is ABC's longest running current scripted series, Grey's Anatomy. It's received 120,109 overall ratings, which averages 7.7. Two episodes in season 6 touched 9.3, with the lowest rated episode airing in season 3, garnering a 7.3 rating. Seasons 6 through 8 have trended flat, but its ratings surged last season. Like House MD, it trends down marginally across its lifetime, as seen in the white trendline in the graph below:
Off the top of my head, ER and Scrubs are two other long running medical dramas. ER trended up overall, and surged in its last season, while Scrubs went the other way.
Crime dramas are another popular genre, and one that has a good selection of shows this upcoming season. There's also a few stalwarts around. CBS's NCIS enters its twelfth season this month, and its data makes for interesting reading too. The series has an 8.1 overall rating from 63,279 viewers - a surprisingly small sample size given its longevity. Like House MD, NCIS underwent a cast shakeup in its third season which must have had an impact on its popularity as the season trended down, but the absence of Cote de Pablo last season saw season 11 trend slightly upward, albeit to a lower extent than previous seasons. Overall, the series trends up. Its graph is below:
Fellow CBS series Criminal Minds chalks up a double figure season count this month also. Its ninth season saw a much steeper incline in individual episode ratings than any other time in its history. Its one of the only series I've looked at in which the first season trends down, albeit marginally. Criminal Minds has two dozen episodes with an individual ranking higher than 8.5, but none beyond 8.9. Its lowest rated episode came in at 7.4, and aired in its freshman season. However, despite the ninth season rally, the series trends marginally down, as can be seen below:
ABC's Castle is also worth a look. Compared to most other series, Castle's trendlines are very consistent, though the episode ratings vary wildly. Three of its season finales sit at a 9.0 rating, and all but 2 of its premiere and finale episodes rank above the series trendline. I can't help but take a stab at the most recent season finale, which sits more than half a point below the trendline (I wonder why). An 8.3 overall rating from 88,826 votes reflects its popularity, but interestingly its first season, which some argue is its best, is its lowest rated. Its graph is below:
FOX's Bones is a mixed bag if there ever was such a thing. It's another series entering double figures this month. A total of 83,541 votes have been cast, giving Bones an 8.1 overall rating. Only two episodes from its more than 200 have received a 9.0 rating (there are none rated higher) one being the most recent season finale. The series' lowest rated episode came in season 8 and garnered a 6.3 rating. I'm sure a Bones fan will let me know why that episode wasn't well received in the comments. Aside from that, the overall series trendline heads downward, while each season bar a couple tends to head upward. Its graph is below:
Enough of the veterans, let's have a look at some newer series, yet well established series from the last few years. Person of Interest is a great place to start, especially given that it's my favourite television series. It's received an 8.4 overall rating from 78,894 votes, and you know a television series is a goodie when you take into account that not one episode has an individual rating less than 8.1, with the three highest episodes, all from season 3 and including the season 3 finale, have a 9.8 average rating from well over a thousand votes each, one of the best sample sizes per episode you'll see. It's not surprising that each season and the series overall all trend upwards, with nearly two thirds of episodes in season 3 receiving ratings higher than 9.0, compared to 5 in its freshman run.
ABC's Once Upon A Time is consistently one of SpoilerTV's most popular series. IMDb users also rate it, with 117,907 ratings equating to an 8.1 average. Once Upon A Time has been on the air for three seasons, and its last season saw a strong increase in its episode ratings, with the season finale garnering the highest episode rating yet, a 9.5 averaged from 855 votes. All of its premieres and finales sit higher than their respective season trendlines too. Its graph is below:
Another ABC series scoring well on IMDb is Scandal. After three seasons of varying lengths, Scandal enjoys a 8.0 rating, though the sample size is rather small at 30,669, which shows in this case that social media popularity doesn't necessarily mean IMDb popularity. Despite its current fame, the series premier is the lowest rated individual episode by a large margin, and only one episode has garnered a rating higher than 9.0, though the third season finale equals that rating. Despite the low rated series premiere, the trendline for season 1 is steep, and the two most recent seasons share a similar upward gradient. See the graph below:
May wasn’t particularly good to me in terms of freshman series being picked up for a second season, but let's look at a few of the bigger names anyway.
James Spader's The Blacklist led the way by more than a few country miles. The IMDb data corroborates this, with a steady first season trendline beginning at 8.0 and ending past 8.5. The series has an overall 8.2 rating courtesy of 43,601 votes, and each episode receives a good sample size too. While no episode has fallen below a 7.8 rating, only a third of its episodes actually sit above the trendline, as seen in the graph below:
The 100 was CW's big hit with its 13 episode first run beginning in late March. Like Scandal, its series premiere was the least popular among IMDb voters, garnering a 7.5 rating, but its freshman season finale sat at the other end, with a 9.0 rating. The survivor bias theory may hold some weight here, with the premiere receiving around 50% more votes than most of the other episodes. The season went on to earn an overall average of 7.5 based on 18,849 ratings. Its graph is below:
FOX's Sleepy Hollow made for interesting reading. It garnered a 7.8 overall rating courtesy of 29,331 votes, but only one episode was given a rating equalequalling that, with none rated lower. The 100 also shares this trait. The highest rated episode in its 13 episode run was the season finale which received a 9.1 rating, and was the only episode to break the 9.0. As you'd expect, its overall trend was upwards.
The last freshman series I'll take a look at is ABC's Agents of SHIELD. Much like Sleepy Hollow it started rather slow with no episodes in the first half of the season threatening the 8.5 benchmark. Again like Sleepy Hollow, only one episode from the season garnered a rating lower than its overall 7.4 rating from 62,319 votes. The top half of the season saw no episodes rated lower than 8.1, with the most well received episode garnering a 9.4 rating, compared to the season finale's 9.2 rating.
It's also worth taking a look at some cable series. The most popular cable series of the last few years is without a doubt HBO's Game of Thrones. Game of Thrones sits among the highest rated television series on IMDb, with a 9.5 rating sourced from 636,266 ratings. Each episode receives thousands of ratings, making for a great sample size. All four seasons feature a steep upward trendline, and it should come as no surprise that the overall series trendline heads in the same direction.
Showtime's Homeland was snubbed at this year's Emmy Awards, and its trendline for season 3 probably explains why. Its two lowest rated episodes aired last season, though it also equalled its highest rated episode last season, a 9.2. The series has an overall 8.5 rating garnered from 152,985 ratings, and the series trendline runs as near as makes no difference to horizontal, as you can see below:
As online companies take up the challenge of producing their own series, there aren't many ways to determine the success of a series among its fans because television ratings become irrelevant. Netflix is one such example, with flagship series House of Cards and Orange Is The New Black both having screened their sophomore seasons. The former has a 9.1 overall rating from 142,859 votes, with the sublime season 2 premiere and finale both taking the title as the series' top rated episodes to date, sharing a 9.6 rating. House of Cards has been trending upward steadily, with the trendline for the second season beginning at a higher point than the first.
Orange Is The New Black is enjoying plenty of acclaim in the wake of its second season. The series premiere has the series' lowest episode rating at 7.9, while the season 2 finale garnered a 9.6 rating, surprisingly high considering no other episode has a rating beyond the 9.0 benchmark. Overall the series has an 8.5 rating from 91,197 votes.
Finally, does Breaking Bad really deserve its accolades? IMDb users think so, with an enormous number of votes for each episode. It ended its 5 seasons with a 9.6 overall rating from 545,616 votes, and after a flat first season, it there was only one direction the trendlines could head.
If you've made it to this point in the article, you deserve a pat on the back because you’ve read just over 2100 words, so now's a good time to bring things to a close. IMDb isn't a perfect source for determining the true quality of a series, but it's probably the best you'll find, it's easy to analyze, and it has a solid sample size. Of course, a series' overall rating and its individual episode ratings will always be subject to die-hard fans rating each episode a full 10, but the same can be said for trolls who give everything the lowest possible rating.
From the series I've examined and featured in this article, the majority of seasons trend upwards from their season premieres, but it's not uncommon to find a series that trends down overall. Survivor bias may or may not be present in a lot of the data, and each series will be subject to countless other factors which determine the ratings it receives.
It goes without saying that there's countless other series that I haven’t mentioned in this article, so please feel free to look your favorites up on graphtv.kevinformatics.com and post your findings in the comments below.
Thank you very much for reading, and please share this article to your fellow fans using the social media buttons on the left hand side of your screen.
Jimmy.
That was a brilliant article Jimmy, thank you for putting it together so clearly.
ReplyDeleteLove that POI, one of my favourite shows, gets recognised over at IMDB with a high score , and also thanks for the graphtv site, I think I might be spending some time there.
Awesome job.
Cheers Sharon, really glad you enjoyed the article.
ReplyDeleteAnd yes, that website is something where its easy to lose track of time while you're on it!
Thanks @Jimmy good stuff. And yes, that site is very cool.
ReplyDeleteCheers @DarkUFO, I'm guessing you'll be spending some time there too ;-)
ReplyDeleteIt's obvious that episode rating will go up, because people who are not satisfied with show will gradully give up and stop voting.
ReplyDeleteAddressing Scandal? I think the high ratings are coming from the younger generation who loves the stupidity and anything goes of the show. The show stopped being a political drama in Season two. So, the newer viewers love the fact you don't have to think to watch. The best years of Scandal was the lower rated years IMO. The show really took off when Fitz was shot. Season 3 was terrible writing, the the ratings were good. So, now it is just a guility pleasure show that will never win an Emmy, but probably keep high ratings because it is mindless tv.
ReplyDeleteIt might seem obvious but the graphs don't show that at all. If the only people that were left watching and voting were always 100% satisfied then all the later episodes would be getting a perfect 10 score. You also wouldn't have a season trending upward, then the following season beginning lower than where the previous season ended up.
ReplyDeleteThis isn't scientific or anything, it's just a different way of assessing how fans perceive a series.
Jimmy - simply fascinating! I especially love the in-season trend lines. This a very well executed idea and analysis - great job!
ReplyDeleteThanks a lot @dave, really glad you enjoyed the article!
ReplyDeleteWell despite the target audience the series garners, most episodes have barely 150 individual ratings, so its not like viewers are flocking to IMDb to rate the episode they've just watched - they probably head to Twitter instead.
ReplyDeleteWhoa, This must have been a heck of work to compile all thses numbers. Thanks a lot for it Jimmy, it's very intereseting, mainly the season trends.
ReplyDeleteI think what Bazooka means is that if people who didn't like the show give up and stop voting, then only people who enjoy the show will vote and will obviously give it a high rating, so later episodes end up higher. So with less votes from haters later on, a series trend will continue to rise as long as the loyal fans continue to vote. But I'm not 100% sure that's the case because as you can see, some of the show's lowest ratings come in its later seasons like House.
ReplyDeleteFascinating article though, I really enjoyed it.
Interesting article. My highly unscientific observation is that serialized shows almost always seem to peak some time in the season 2-4 range. Quality procedurals though can sometimes produce their best episodes in later years.
ReplyDeleteOn the discussion about unsatisfied people dropping off, it would seem that for a show declining in quality, some unhappy people would drop off after a period of time, and become less active in activities like voting on the show, but at the same time there will be more people crossing the line from happy fan to newly unhappy fan, but still engaged engaged enough to vote for a little while.
Excellent work.
ReplyDeleteI think that firstly people who don't like the show leave and it goes as you say, but as the series continues, people tend to stay even if they're dissapoited on storylines - the longer you were invested in the show, the harder it is to quit but also possible dissapointment grows... e.x. it'd me interesting to see how the Mentalist looks like in this kind of analysis
ReplyDeleteThis is a really good article!
ReplyDeleteYou nailed it on the head.
ReplyDeleteI think the number of episodes that a show does in a 8 or 9 month season is the most significant. Comparing a show that does 22-23 shows with one that does 6 makes no sense because good writing takes time. Let's see the House of Cards team do 23 shows over 8 months and see what that looks like. Could you get the same group of people to work like that over half a dozen seasons? Kevin Spacey has said he wouldn't work like that in the past when he moved to England.
ReplyDeleteNice stuff, thanks.
ReplyDeleteWooooow THose are some intense graphs! NIce work.
ReplyDeleteI totally agree. For me Scandal is mindless tv. I think S2 was great but I quit after the last episodes because the show had lost everything that made it special in the first place. The same thing happened with Grey's in S3/S4.
ReplyDeleteit's without a doubt a very good job what you did here, but the truth is that these polls are not really useful to rate the quality of a show, because sometimes people who love the actors get obsessed and start voting as crazy rabbits for the show where they are working in, and in the CW there are 2 very good examples of this, Supernatural and B&TB, no matter how bad or repetitive the plots are (talking as a person who used to watch these 2 shows) people will do whatever it takes to make them look successful and good. BUT!!, the good thing that gives your job more credibility is that you didn't put any of those shows in your list, so maybe these poll were voted for real people who actually enjoy the show for its quality and not for its cast.
ReplyDeleteThe Ghost in the Machine
ReplyDeleteFor me, personally, a show tends to get better as it goes, through Season 1 and 2, then later in age starts to not get as good for multiple reasons (repeat story lines, introduction of forced characters, more screen time for disliked characters or dragged out characters/story, etc.) but that's a matter of opinion. Sometimes I just get nit-picky, other times I'm just missing the feeling past seasons gave me because that's what I've come to expect from the show.
ReplyDeleteMost of my shows have weak spots, but most have never truly disappointed me. There have only been a few where I really disliked their late/final seasons because of what they did to some characters and/or mythology re-writes.
Person of Interest has been the best show to me quality wise. Besides some nit-picks I've really enjoyed all the episodes and it's consistent and continues to have complex connections and doesn't do anything I'd consider stupid. It's smart and takes itself seriously.
It really depends on the series. Some series have a very slow start. NCIS is one of them.
ReplyDeleteYeah I didn't discuss this at all in the article. From the shows I know of, you can see where the filler episodes were, but you can also see where the big episodes were, so I guess it evens out
ReplyDeleteThanks very much!
ReplyDeleteThe website I used makes it a lot easier! Thanks for the comment
ReplyDeleteHere's The Mentalist here
ReplyDeleteHere's The Mentalist here. Interestingly, the Red John episode is the lowest rated, but also has the most votes for it
ReplyDeleteYeah I agree this isn't very accurate results on if a show gets better or worse as it goes on. tracking the Nielsen Ratings would have given better and more accurate results, not IMDB user ratings.
ReplyDeleteNielsen does not track what people thought of an episode.
ReplyDeleteThank you:) and about the Red John episode - I suspected as much - it was a mystery 6 years in the making so a lot of people wanted to see it but also a lot got dissapointed
ReplyDeleteAs @DarkUFO said, Nielsen doesn't collect this sort of data, it only collects audience metrics, not opinion based data like IMDb.
ReplyDeleteWell the title only says "Do TV series get better or worse over time," that can mean many things. I guess a more accurate title for the purposes of this would be "Do peoples opinions of TV shows get better or worse over time,"
ReplyDeleteWe assumed people would actually read the article before commenting.
ReplyDeleteGreat article!! As a math teacher I love numbers so :) It was really interesting even if I don't really "trust" the IMDB data for some shows (for various reasons that some people already explained here).
ReplyDeleteThe Once Upon A Time numbers are a great proof of that, I still love the show very much but I think objectively the first season was the best one. Now, I think the S3 results are more about the shipping than the writing :)
I would love the same article with the Spoiler TV Reader Popularity Episode Tables data :) I trust them more because I "know" the people who are voting here (I think there's more diversity and plenty different people who focus on more aspects of the shows they're voting for)
Anyway, this article still is amazing because I love analysing those numbers so thank you !
It's partly true, partly not. True, because the classic network shows' schedule is practically a factory work, you need to have an episode roll down from the line at every eight-nine days. And isn't true (or probably right), because when somebody sits in front of the TV and decide which show they will watch in the next 60 minutes from the daily roster, they don't care about the problems of the writing, they expect to be entertained th best way and each show has equal chances.
ReplyDeleteIf good writing takes more time, or requires less episodes, it's the problem of the channel, not mine. Do more, shorter shows if that's the solution, but I won't give a free pass to crappy stories just because twenty years ago it wasn't a problem.
My little accountant heart is so happy right now! Stats, trends, graphs - I can't, it's just too perfect!
ReplyDeleteI haven't really given much thought to the IMDB ratings, mostly I go there to win arguments about which actors were in movies. However, this article is absolutely fascinating and did get me thinking about my viewing habits. I usually give a show 3/4 episodes to win me over as it usually takes the cast a few episodes to get their rhythm. If I like the show enough to watch it week to week, then I usually stick with it to the end unless it gets repetitive or WAY off track. If I like it but not enough to keep up with it week to week, then I usually binge watch it on Netflix - which is generally the only place I rate shows. As for rating individual episodes, I'm not very consistent but when I do, it is only on the STV "What did you think..." polls.
Great read! Thanks for putting the time and energy into this!
Still, it's practically the same not because an episode got low numbers means that it was a total trash and vice versa, and again, I'm not trying to underestimate Jimmy's work, it was a great job!!, all I'm saying is that there are some fandoms that distorts the results, because I'm pretty sure that even if they hated the damn episode, they gave it 10 points just to keep supporting the freaking show and actors. Now in my opinion a series starts going down after the 4th. season and therefore these shouldn't last more than that.
ReplyDeleteLook at that glorious 10 near the end of Breaking Bad. Ozymandias, the king of all TV episodes. I'm surprised you didn't mention it!
ReplyDeleteAs others have already said: Great work, Jimmy! (I like your name by the way)
Haha glad to be able to please a math teacher! If you haven't read my previous article that I linked to in the first paragraph then you'll be in math heaven!
ReplyDeleteTotally unscientific, but usually a TV show first gets better, reaches its peak around the second or third season and then the quality is falling. It doesn't reflect in the ratings immediately, and there are a few exceptions, but no matter what the votings say, I don't think that the shows which have an up curve necessarily got better, but that the people who got tired of the show just don't watch anymore at this point, leaving the die-hard fans.
ReplyDeleteBeing based on ratings tells me that it's not necessarily the quality of the episode, but the quality of the marketing of the episode in conjunction with the show's popularity. To put it another way "ZOMG Caskett!!!" Is going to get more eyeballs, but you can only go to that well so many times.
ReplyDeleteHi Jimmy, always enjoy your analysis! Great job!
ReplyDeleteOn your impression, have you seen any connection between ratings and the numbers (Nielsen)?
it depends on the series
ReplyDeleteGreys seems steady which is good
ReplyDeleteThe Blacklist is good
Once upon a time, is doing well - and i loved the finale (that was long time coming for Emma Swan episodes)
I wanna know what Revenge got?
What about Beauty and the Beast?
ReplyDeleteThe walking Dead started off small and guanine a whole lot as seasons and episodes went on
ReplyDeleteThanks @Mercy, glad you liked it. I made a point of not comparing Nielsen data with IMDb data as I wrote the piece because they are mutually exclusive. The ratings data tends to trend down over time as viewers adopt online and DVR viewing. An episode that was seen by more people (generally premieres) often has more votes, which can mean more negative votes which can skew particular episodes. Case in point was The Mentalist with its 'Red John' episode.
ReplyDeleteYou don't see many Jimmy's on the internet!
ReplyDeletehttp://graphtv.kevinformatics.com/tt2193041. You're welcome to look at its data here. Its been heavily skewed by fans giving high ratings to all episodes when the overall series rating is at 7.2 probably because non-fans have given it low ratings.
ReplyDeletei was also dissapointed with the Red John episode but still i think they should have given a 22 episode season rather than ending in the mid season . Moreover the show was telecast in a pretty bad time slot so thats also one of the reason for low rating (18-49).
ReplyDeleteThat's not true in the case of fans growing dissatisfied with the direction of a show. Perhaps that's why a lot of series trendlines actually slope downward. But it might be true that the survivor effect results in episodes being rated higher than they would have otherwise.
ReplyDelete