The Problem With Populism, Part Two

In a previous post, I discussed the IMDB Top 250 List, and why it's essentially meaningless and not a useful tool for comparing movies to each other. In short, the problem with it is populism, everyone gets a say, no matter how wrong they are about it. Additionally, films that predate the internet face a massive handicap in terms of exposure and the number of rankings they achieve. Populism is a complex, multi-faceted problem, but one way to simplify things is to simply adopt clearly defined terms for the adjudication of films, terms that put everyone on the same page. This would eliminate 100 people with 100 different favorite films from shouting at each other.

Now, perhaps I've been unfair and the IMDB list deserves to be defended. It is based on a weighted ranking system that accounts for varying numbers of votes,after all. As IMDB discloses...

The formula for calculating the Top Rated 250 Titles gives a true Bayesian estimate:

weighted rating (WR) = (v ÷ (v+m)) × R + (m ÷ (v+m)) × C
Where:
R = average for the movie (mean) = (Rating)
v = number of votes for the movie = (votes)
m = minimum votes required to be listed in the Top 250 (currently 25000)
C = the mean vote across the whole report (currently 7.0)
For the Top 250, only votes from regular voters are considered.

So let's compare the math between The Shawshank Redemption (#1, 9.3/10) and Citizen Kane (#63, 8.5/10).

So, this isn't a math blog, but I did do all the calculations, per IMDB's disclosure, and my result was not 9.3, but rather 9.06 (-0.24).

Likewise, for Citizen Kane (the most frequently cited "best film ever made"), the result, factoring for 233,777 votes - less than 1/5 of the votes of Shawshank - was not 8.5 but rather 8.08 (-0.42). So both movies dropped, but the movie with significantly fewer votes dropped more. I'm intrigued. Let's look at another set of movies with a similar disparity.

The Dark Knight is #4 on the list, with a 9.0 rating and the second highest number of votes. Do the math and the 9.0 rating falls to 8.86 (-0.14). That's not much of a drop.

The Third Man though, #98 with an 8.4 rating has only managed to get 88,061 votes. The math shows a weighted average of 8.01, a drop of 0.39 points.

I think I'm seeing a trend here. But why? Is it a conspiracy? Not really. The idea is just that the fewer votes a film has, the closer it gets pushed toward the report mean, in this case, 7.0. The more votes a film gets, the closer the weighted average gets pushed toward the actual mean rating for that particular movie. For The Dark Knight the mean rating is 8.9.

What does it all mean? Well, it comes back around to the fact that populism is inherently unfair and is no way to decide the merits of art or commercial products, let alone things that are in fact a combination of both of those things. Just for fun, I scaled The Third Man to see what would happen if it had the same number of votes as Shawshank (while maintaining the same proportional distribution) and the equation works out to 8.23 instead of 8.01. That's a hefty difference when the only thing that changed is the volume of votes.

This is all rather pointless though. If you've been paying attention you'll have noted that none of the results I've gotten have matched the stated average ratings. There's a problem somewhere, and IMDB is aware of it. They put it there.

IMDb publishes weighted vote averages rather than raw data averages. Various filters are applied to the raw data in order to eliminate and reduce attempts at 'vote stuffing' by individuals more interested in changing the current rating of a movie than giving their true opinion of it. The exact methods we use will not be disclosed. This should ensure that the policy remains effective. The result is a more accurate vote average.

So they basically just hide the way they actually figure the ratings out. Lest you think I'm just making this up, CLICK HERE.

This is all so very Don Quixote of me, I'm well aware. My quixotic screaming into the internet demanding justice for old movies is just a modern way of tipping at windmills. I won't get IMDB to change anything. But, maybe, just maybe, I can get more people to watch and then rate these old movies and restore their glory that way. Or, on a much less ambitious note, perhaps I can just get a few kids to realize the difference between popularity and merit, and understand that the IMDB Top 250 List is deeply flawed and shouldn't be taken too seriously. I'm willing to settle for that.

The Problem With Populism, Part One

I love rankings and lists... Usually. The one list that I really can't stand is the IMDB Top 250, because it's such a ridiculous case of populism trumping merit. For those that aren't aware, just because a movie set a box office record doesn't mean that it was the best movie ever made. It just means that it had the most appeal. Think of the movies as potential dates. There are some that are extremely attractive, just sexy beyond belief. Those are the summer tent-poles. Now, while it's not impossible for a summer blockbuster to be great (Jaws, Star Wars, Inception, etc.), the odds are pretty good that the sexy package is going to be lacking something, and more often than not, they lack brains or some other merit. Don't get me wrong, there's nothing wrong with dumb popcorn flicks. But they shouldn't be ranked among the the best films ever made. That's what makes the IMDB Top 250 so infuriating.

The medium of film has been around for nearly 120 years. 120 years, let that sink in. Before there was a Ford Model T, there was film. Before the world population reached 2 Billion, there was film. Before 240-some-odd wars could be fought, there was film. So why is it that on the IMDB Top 250 list, 126 (of the 250) films were released in the past 25 years? Were the first 95 years of filmmaking just a warm up for this explosion of groundbreaking work that many of us have seen within our own lifetimes? NO. It wasn't. The problem with the IMDB Top 250 list is that it depends on the people, the uneducated, ill informed masses to rate movies with no pre-qualifications. Anyone can rate The Dark Knight Rises (#50, 8.5/10) an 8 or 9, without ever having seen Sunset Blvd. (#41, 8.5/10) which is, when judged on its artistic (not commercial) merits, a vastly superior film.

I hate saying this, but TDKR was terrible. I paid my $20 to own it, I've watched it dozens of times, I love the trilogy, I love Christopher Nolan... But TDKR was a pile of garbage. Was it slick? Sure. Did it have star power? Yep. Was it riding a wave of anticipation? Of course. Was it a good movie? NO! It's just the truth. TDKR shouldn't be rated any higher than 7.5 for no other reason than the massive plot holes. That would remove it from the Top 250 list entirely, which would be great. It just doesn't deserve to be there. But because it has the 11th most ratings (because practically everyone has seen it) and most people aren't qualified to judge the merits of films, there it is, at #50 on the list. (Speaking of number of ratings, there are only 3 films in the top 50 most rated that were released before Star Wars: Episode IV - A New Hope in 1977. Those three films are the Godfather, The Godfather Part II, and One Flew Over the Cuckoo's Nest. All released in the 70's. That means that not a single film from the first HALF of the film age ranks in the Top 50 when it comes to number of ratings. But I digress... ) TDKR is ranked higher than 77 of the films on the AFI's 100 Years, 100 Movies Tenth Anniversary List, which only includes American productions. Do we really want to say that TDKR is the 24th best American film ever made? Is that okay with you? Because it makes me nauseous.

So, I've identified a problem, but what's the solution? Well, for starters, we need to agree on the different ways of comparing films (and other commercial/artistic endeavors). The IMDB list, in its populist glory, should bear very little actual consideration. Box Office returns are the industry standard for measuring the popularity of films. How many butts ended up in the seats? That's what matters to the studios and the bean counters therein. Given my own predilection for the old and dusty things in life, I'm especially fond of accounting for the Box Office Grosses when adjusted for inflation. Once inflation is taken into account, a little old 3 hour 40 minute long Civil War epic from 1939 sits at #1 all time, with more than a half a billion dollar cushion between itself and Avatar at #2. That's right, commercially, Gone With the Wind and Avatar put the most butts in seats and earned the biggest (relative) box office grosses. That's a fact, there's no debating it. On the flip side though, what is there to say about the artistry of these films?

Gone With the Wind is seemingly a lock in the top ten of any list to be made (except the IMDB Top 250 where it's #155!) because of its iconic status, enormous legacy, and let's not forget the 13 Oscar nominations and 8 wins.

Avatar on the other hand, is not as beloved. It won't have a profound legacy in 70 years. It's a rehashed story full of tropes and verges on the unoriginal at times. It was nominated for 9 Oscars, but only won 3. It was a tremendously important film from a technical standpoint though, as James Cameron pushed the technological envelope to the max in making it, and those breakthroughs will pervade Hollywood forever.

Two epic movies. Two massively popular movies. One reliant on grandeur and old Hollywood glitz and glamour, the other created virtually around actors in an empty soundstage. The art of filmmaking and storytelling and the art of technological achievement, side by side. This is an apples to oranges comparison.

A lack of any defined parameters for the measuring of a thing's goodness is the single biggest problem with populism.