How To Rate On Netflix

The Netflix Project: Part 1 (Retention)

This personal project has been a long time coming. If I have to choose three things that I am passionate about, they would definitely be: food, statistics, and TV shows — in no particular order. To maximize my utility, I've been putting food and the TV together for the longest time. If you picked a moment at random in my life, there is a high probability I would have been eating and watching the telly.

I had an idea to put another two of my favorite things together — statistics and TV shows, but there was too much good content on the combined prowess of Netflix, Hulu, and Prime for me to step away. With the quarantine, all plans for the summer was scrapped and I finally got around to doing it.

How I Procured My Data

Netflix has a functionality that isn't commonly heard of but extremely useful. It allows you to download all of your viewing history as a "csv" file. Overall, my Netflix persistence paid off with roughly 1000 data points over the past 3 years. That's almost a show a day. And we haven't even included my Hulu, Prime, and CBS accounts yet.

One limitation of the data is that Netflix overwrites your viewing history with the latest date of viewing. For example, Since I watched Knock Down The House (Shoutout to AOC there) twice, it would only report the second time I watched it and not the first. However, since I have hopped from one friend's account to the next over the past three years (my cheap self tried never to mooch off someone for too long), I was able to capture some of the re-watching that provides more color to my analyses. This quirk would result in some bias against shows that are rewatched immediately vis-a-vis shows that are rewatched after some time. However, since the former is a much rarer behavioral pattern for me than the latter, this bias isn't a game-changer.

After procuring my viewing activity, I wanted to get a rich set of variables such as episode length, number of seasons and episodes, genres, tags, and actors. I was able to scrape the IMDb website for this information, as you can see from the red boxes below.

I definitely got more information than I needed for this first part of the analyses, but all that information in escrow is going to come in handy someday.

Measuring "Retention Rate"

The first metric I wanted to investigate for each TV show was how willing I was to finish the show after starting on it. To determine this, I created a simple metric called "Retention Rate" which is the total time spent on a TV show (Number of episodes watched * Total episode length as a proxy for total time spent on an episode) divided by the total show length (Number of seasons * Number of episodes * Total episode length).

Why is this even important? First off, it is if you are like me — looking for something long term. There is nothing more antagonizing (there actually probably is) than browsing Netflix during dinner time and going through all of the "recommendations" before realizing that you've inadvertently finished your dinner and you've got to get back to work.

One caveat I must add is that if I had my way, I would not proxy total time spent on a TV show using the total episode length as I doubt I finished every episode I started. Unfortunately, Netflix doesn't tell you how long you spend on any given episode. This is unlikely to be an issue for the top few shows I've watched as I probably finished every episode I started. Still, this creates an upward bias on retention rate, but more so on the shows where I couldn't even get pass an episode (yes I'm talking about you Glee…)

Another issue of pairing my Netflix viewing activity in the numerator with data about TV shows from IMDb on the denominator is that shows with a staggered release from cable to Netflix will be disadvantaged here. For example, I follow Schitt's Creek rather faithfully but the results won't reflect that. While all of the 6 seasons reported on IMDb, not all 6 are available for viewing on Netflix (come on get it together Netflix I need to watch what happens with David and Patrick!!). Evidently, there will be a substantial bias towards original Netflix content.

My Netflix Recommendations If You're In For the Long Haul…

I first start off with my recommendations of things to watch in general. To filter to TV shows, I looked at any show with more than 1 episode recorded on IMDb. In the following chart, I rank all Netflix TV shows by retention rate, and then by total time spent.

Generally, true crime shows (When They See Us, Narcos, Jeffrey Epstein), feel-good shows (Jane the Virgin, Queer Eye, Cheer), and sitcoms (Schitt's Creek, On My Block) do pretty well at least in my book. Not surprisingly, there is an overwhelming number of limited series content since these are shorter.

Any good and budding data analyst will know that there is substantial variation amongst the Top 20. For example, while I watched all 6 hours of Cheer, can that really be compared to me watching more than 100 hours of Jane the Virgin? Although, I must say that if I could watch 100 hours of Jerry doing mat talk, you know I would. As such, I segment my data into three pieces to make better comparisons.

First, I look at long-running TV soaps.

Surprisingly, there is no substantial Netflix bias here. The only bias is perhaps towards shows which have finished airing such as Jane The Virgin and Scandal. Overall, I must give a shout-out to Narcos. Season 3 was amazing — don't understand how Matias Varela was overlooked for an Emmy there. And yes, I did watch Season 3 more than once.

Overall trends seems to be in favor of shows about family (Jane the Virgin, Shameless, The Fosters), crime (Narcos, American Crime Story, HTGAWM), and dystopia (Black Mirror, TWD).

Next, I investigate sitcoms and reality TV.

Unfortunately, I don't watch that many sitcoms and thus could only display 10 of my favorites. And I would say that this is an unpopular opinion but I didn't even really like The Good Place after the first season. Yet again, shows about family (Schitt's Creek, Kim's Convenience), social issues (Unbreakable Kimmy Schmidt, Dear White People), and coming-of-age shows (On My Block, The Inbetweeners, Derry Girls) do well here.

Next, I rank my favorite limited series.

As expected, there is a higher retention rate on average for all limited series, probably just because they are short, and usually suspenseful so I watch till the end for the big reveal. One exception to the rule — Crash Landing On You — on the charts solely because my wife wanted to watch all of it (thankfully there's only one season, although I appreciate all the Korean Fried Chicken that was featured on the show).

What Not To Watch

Finally, you might be interested to know what NOT to watch. To find a set of shows not to watch, I took a slightly different approach than simply looking at the shows with the lowest retention rates. I'll explain why using the converse of what we've previously done does not work as well. If we look at shows with the lowest retention rates, they will be invariably be long-running TV shows where I just watched one episode. However, would they necessarily have a lower retention rate than a limited series that couldn't even catch my attention for a hot minute?

Thus, I instead decided to filter to shows where I only watched just 1 episode, and then ranked them by episode length. This is again a proxy for the total time spent on each of these shows. Generally, I'd expect that I could not even get through the first episode of these shows. Therefore, the shorter the episode length, the more un-watchable it probably was (harsh words… but that is what the world is like my friend).

Disclaimer: This is a personal opinion. No offense primarily to the huge Community community (see what I just did there?) out there, as well as the huge Aziz Ansari following — I still love him. Maybe one day I'll get through all of the Master of None.

Well, thanks for indulging me in something already so indulgent, and I'll report back next time on another metric I want to investigate— Bingeability.

*Update: See here for Part II!