Regardless of the introduction of sycophantic reviewers, the 3/5 = fresh thing has always been a pretty half-ass threshold imo, and that a fact that a film can be "100% fresh" on RT on the basis of every single reviewer saying "yeah it's nothing special but it's fine, 3 stars" is fairly easy to misinterpret.
I will say that when I used to go to the theaters, which was before the pandemic and I started a family I used metacritic.
I found that any time I went to something that was red, I absolutley regretted it and it was terrible. Yellow was more hit or miss and top green scores were pretty good.
Exceptions were comedy where a lower score could still mean a good film, and politics oriented films, where a bad film with a media approved message could get a really good score even if it sucked.
It’s sad to not get a reliable indicator of that and someone should just resurrect the old score and call it Bad Apples. Since the actual score seems transparent, why not develop a competitor.
Its absolutely hopeless. The rankings are just totally gamed because Rotten Tomato still carries some semblance of credibility (although in this post Trump winning era, sources of credibility can get eroded more quickly)
It does not get clearer than when a political movie comes out. 2018 was an interesting year, two movies came out that really allowed me to get a clearer picture of what was going on: "Knock Down The House" and "Death of a Nation".
When "Knock Down The House" (documentary featuring the leftwing US politician AOC) came out, I got interested in scraping the data off of Rotten Tomaotes and studying it.
Before making any moves, I first watched the movie for myself in a theater (and also got to see a live Q&A with the director to understand her thought process)
The movie had at the time a 100% rating from critics and ~80% from viewers. After watching it, I would concur with the viewer ranking but felt that the critic ranking was unusually high. Seriously? 100%? (It has now gone down to 99% but still). In regards to the viewer ranking I conceded that I was probably biased which is why I also ran this experiment on Death of a Nation (also saw in theaters but to a room with only one other moviegoer).
Knock Down the House eventually got featured on Tucker Carlson like a year after release(I think it coincided with Netflix making it free on youtube). I watched in realtime how the movie critic score kept going down and down and down to where it is now (11%). Dumping the scraped data, I ran a simple analysis and discovered a large portion of the people ranked it with no comments, or simplistic things like how stupid AOC is and many had had no other ratings other than this movie or the only other film is the one featuring Illhan Omar (another politician hated by the right).
For Death of a nation, the scores were flipped. A whopping 0% in the critic score(12 reviews) while the user score stood at a respectable 87% (again at the time when I did my scraping yet again we saw tons of 1 movie reviewers). Yeah the more royally sucked and was painful to watch but 0%? That was a bit fishy. This essentially killed any credibility that I had in Rotten Tomatoes.
I started to trust places like /r/movies and /r/AMCsAList only to get burned by that as well when promoted movies would end up being terrible and then when I went back to criticize the films, I would get criticized and downvoted to non visible status. It was not a definite signal but gave me the feeling that there is a lot of astroturfing going on there as well.
Furthermore, these movies promoted on Reddit would typically be in 3/5 range on Rotten tomatoes which further made me think there is no real way to get a real signal if a film is likely to be good or not.
What I started to do was not a great metric but has helped cut down on the cruft: Follow specific actors/directors I really liked and ones that I felt were in it to make good films. As an example, actresses like Mary Elizabeth Winstead have turned down big roles in favor of indie films or other interesting scripts to the detriment of her career but the films have been more enjoyable and interesting. In each film I also find other actors to follow and if I start to see more studio promotion of a specific person (for example Anya Taylor Joy after The Queens Gambit) I start to caution away and sometimes just drop that actor from the list. In her case I stuck to films she worked with other people that I determined I liked(like Edgar Wright) before feeling like there is too much promotion going on and just dropping her from the list. Other than this, I fill out my list with fnchises I like or subjects that I always give a chance to (science/space, etc.).
I know I am leaving out a lot of potential good films but the noise has become unbearable to the point where I don't want to waste my time anymore.
A few years later Rotten Tomatoes introduced "Verified Reviews". I thought this will be amazing as now it will only include people with skin in the game (ie. verified to have paid for a ticket)...except now this has been completely hijacked as well.
Going back to the example of political films now what the right wing does is they have a billionaire finance a film through some intermediary group then free tickets to the movie are given out at conservative events. People books seats to the movie, promote it on social media, post a "Verified Review" and then often don't show up to the screening. I have discovered sold out screenings to some of these movies but when I went to the theater to see some other film, i'd often peek into the screening of these films only to find they are almost empty. The movie plays regardless of if someone shows up or not. Furthermore some of these films actually have a code that they show at the end of the film to gift a free ticket to a friend so the box office numbers of these films are inflated and its all a bunch of hogwash.
Like I said its hopeless. In a way we saw the rise of this new fake world play out in Hollywood before it took over social media and the rest of the internet now. How will it end? People trusting only what they already know they like or from trusted friends and everything else is ignored.
This article has got me thinking of an interesting idea though: What if we go and determine which of the critics are known to provide reviews that reflect our tastes (maybe by reading reviews of movies that we enjoyed), then pull only those review data and compute a new Tomato score based only on those critics? We could toss the Rotten Tomatoes tomato meter in the trash and get back to a legitimate review that you could use as a positive signal again.
It’s interesting that people pay close attention to one-size-fits all number (regardless of the pros and cons of the methodology used to derive said number). I find RT really useful for collating the reviews from “top” reviewers in one place: over the years, I know how my interests align with the tastes of particular reviewers, and I don’t have to look in multiple places to get a snapshot view of their opinions.
Would there be some selection bias as well? As info about movies becomes readily available, generally the people who go see movies would have decided that they would probably enjoy said movie, and write favourable reviews
If they're actual movie reviewers then their job is to go see films regardless of whether they think they'd personally enjoy them. Some of the best reviews come from reviewers who have to go and see something they absolutely hate and would never go see for their own entertainment.
Regardless of the introduction of sycophantic reviewers, the 3/5 = fresh thing has always been a pretty half-ass threshold imo, and that a fact that a film can be "100% fresh" on RT on the basis of every single reviewer saying "yeah it's nothing special but it's fine, 3 stars" is fairly easy to misinterpret.
a mean rating of 3 can only be 100% fresh if the variance is 0
Wouldn't we expect the most truly mediocre movies to have the lowest variance in opinions?
That is what "every single reviewer saying '[...] 3 stars'" means, yeah.
Maybe the rating system should be changed from fresh and rotten into clonally propagated and heirloom to reflect this nuance
Yeah. Pixar movies are often close to 100%. IMDb ratings are usually far more reasonable.
I will say that when I used to go to the theaters, which was before the pandemic and I started a family I used metacritic.
I found that any time I went to something that was red, I absolutley regretted it and it was terrible. Yellow was more hit or miss and top green scores were pretty good.
Exceptions were comedy where a lower score could still mean a good film, and politics oriented films, where a bad film with a media approved message could get a really good score even if it sucked.
It’s sad to not get a reliable indicator of that and someone should just resurrect the old score and call it Bad Apples. Since the actual score seems transparent, why not develop a competitor.
Its absolutely hopeless. The rankings are just totally gamed because Rotten Tomato still carries some semblance of credibility (although in this post Trump winning era, sources of credibility can get eroded more quickly)
It does not get clearer than when a political movie comes out. 2018 was an interesting year, two movies came out that really allowed me to get a clearer picture of what was going on: "Knock Down The House" and "Death of a Nation".
When "Knock Down The House" (documentary featuring the leftwing US politician AOC) came out, I got interested in scraping the data off of Rotten Tomaotes and studying it.
Before making any moves, I first watched the movie for myself in a theater (and also got to see a live Q&A with the director to understand her thought process)
The movie had at the time a 100% rating from critics and ~80% from viewers. After watching it, I would concur with the viewer ranking but felt that the critic ranking was unusually high. Seriously? 100%? (It has now gone down to 99% but still). In regards to the viewer ranking I conceded that I was probably biased which is why I also ran this experiment on Death of a Nation (also saw in theaters but to a room with only one other moviegoer).
Knock Down the House eventually got featured on Tucker Carlson like a year after release(I think it coincided with Netflix making it free on youtube). I watched in realtime how the movie critic score kept going down and down and down to where it is now (11%). Dumping the scraped data, I ran a simple analysis and discovered a large portion of the people ranked it with no comments, or simplistic things like how stupid AOC is and many had had no other ratings other than this movie or the only other film is the one featuring Illhan Omar (another politician hated by the right).
For Death of a nation, the scores were flipped. A whopping 0% in the critic score(12 reviews) while the user score stood at a respectable 87% (again at the time when I did my scraping yet again we saw tons of 1 movie reviewers). Yeah the more royally sucked and was painful to watch but 0%? That was a bit fishy. This essentially killed any credibility that I had in Rotten Tomatoes.
I started to trust places like /r/movies and /r/AMCsAList only to get burned by that as well when promoted movies would end up being terrible and then when I went back to criticize the films, I would get criticized and downvoted to non visible status. It was not a definite signal but gave me the feeling that there is a lot of astroturfing going on there as well.
Furthermore, these movies promoted on Reddit would typically be in 3/5 range on Rotten tomatoes which further made me think there is no real way to get a real signal if a film is likely to be good or not.
What I started to do was not a great metric but has helped cut down on the cruft: Follow specific actors/directors I really liked and ones that I felt were in it to make good films. As an example, actresses like Mary Elizabeth Winstead have turned down big roles in favor of indie films or other interesting scripts to the detriment of her career but the films have been more enjoyable and interesting. In each film I also find other actors to follow and if I start to see more studio promotion of a specific person (for example Anya Taylor Joy after The Queens Gambit) I start to caution away and sometimes just drop that actor from the list. In her case I stuck to films she worked with other people that I determined I liked(like Edgar Wright) before feeling like there is too much promotion going on and just dropping her from the list. Other than this, I fill out my list with fnchises I like or subjects that I always give a chance to (science/space, etc.).
I know I am leaving out a lot of potential good films but the noise has become unbearable to the point where I don't want to waste my time anymore.
A few years later Rotten Tomatoes introduced "Verified Reviews". I thought this will be amazing as now it will only include people with skin in the game (ie. verified to have paid for a ticket)...except now this has been completely hijacked as well.
Going back to the example of political films now what the right wing does is they have a billionaire finance a film through some intermediary group then free tickets to the movie are given out at conservative events. People books seats to the movie, promote it on social media, post a "Verified Review" and then often don't show up to the screening. I have discovered sold out screenings to some of these movies but when I went to the theater to see some other film, i'd often peek into the screening of these films only to find they are almost empty. The movie plays regardless of if someone shows up or not. Furthermore some of these films actually have a code that they show at the end of the film to gift a free ticket to a friend so the box office numbers of these films are inflated and its all a bunch of hogwash.
Like I said its hopeless. In a way we saw the rise of this new fake world play out in Hollywood before it took over social media and the rest of the internet now. How will it end? People trusting only what they already know they like or from trusted friends and everything else is ignored.
This article has got me thinking of an interesting idea though: What if we go and determine which of the critics are known to provide reviews that reflect our tastes (maybe by reading reviews of movies that we enjoyed), then pull only those review data and compute a new Tomato score based only on those critics? We could toss the Rotten Tomatoes tomato meter in the trash and get back to a legitimate review that you could use as a positive signal again.
It’s interesting that people pay close attention to one-size-fits all number (regardless of the pros and cons of the methodology used to derive said number). I find RT really useful for collating the reviews from “top” reviewers in one place: over the years, I know how my interests align with the tastes of particular reviewers, and I don’t have to look in multiple places to get a snapshot view of their opinions.
Would there be some selection bias as well? As info about movies becomes readily available, generally the people who go see movies would have decided that they would probably enjoy said movie, and write favourable reviews
If they're actual movie reviewers then their job is to go see films regardless of whether they think they'd personally enjoy them. Some of the best reviews come from reviewers who have to go and see something they absolutely hate and would never go see for their own entertainment.
Fun fact. You can click into Popcornmeter and see verified audience reviews versus all reviews.
E.g. Audience who went to see 2025's Snow White loved it. Those who haven't seen it hated it. Who is more biased?
Obvious to anyone using the site or aware of is ratings, including the author, but it is good to see some analysis as evidence.
RottenTomatoes has been rotten for over a decade. IMDB user ratings are much more useful, but still far from perfect.
Stopped reading when I saw a bar chart being used for correlating critic and audience scores.
It's not the best way to visualise a correlation, I agree, but the article is interesting and full of valuable information, so why stop reading?
You don't need to comment if you didn't read.