Page Index Toggle Pages: 1 Print
Hot Topic (More than 10 Replies) Weighted ratings... (Read 6,472 times)
josephk
Platinum Member
*****
Offline



Posts: 3,981
Location: Canada
Joined: 01. Apr 2002
Weighted ratings...
09. Mar 2009 at 17:46
Print Post  
(reposted from the thread on Whole New Thing)

I would suggest removing the "weighted rating" function on the database.

I was the one who initially suggested this feature and the formula that's used to calculate the ratings is based on the formula that IMDb uses. The reasoning behind it was that the top list should reflect popularity as well as ratings. What I thought it would achieve was that if two movies had an equal score (say 8.5), the one with 15 votes would appear first on the list, and the one with only 5 votes would appear second. I thought it would adjust the ratings only slightly. Like, the one with 15 votes would get a "weighted rating" of 8.51 and the one with only 5 votes would get a "weighted rating" of 8.49, or whatever.

The way it actually works is bizarre and incredibly annoying. This movie has 5 identical votes of 9 out of 10. And yet the "weighted rating" is 8.05. That makes absolutely no sense.

All the weighted ratings seem to do is consistently lower the ratings of well-liked movies. Maybe it only makes sense in a context where films get hundreds or thousands of votes like on IMDb.

It was an experiment, and I think it failed. So I would suggest removing it, if it's not too complicated.

PS: I think the "all by rating" listing function actually uses the raw ratings, not the weighted ratings. Which makes this function ever more unnecessary.
  
Back to top
 
IP Logged
 
Quantum
BA Administrator
*****
Offline


BA Member

Posts: 788
Location: UK
Joined: 18. Jan 2005
Gender: Male
Re: Weighted ratings...
Reply #1 - 09. Mar 2009 at 19:25
Print Post  
The original weighting function basically drags the rating of a movie towards the average rating of all the movies in the database - with the amount the rating is dragged from its original rating determined by the number of votes. So a movie with more votes is dragged by a lesser degree than movies with fewer votes. Or another way of looking at it is that all movies start with an average rating and then get pulled towards a higher or lower rating depending on the number of votes they've received.
It makes some sort of sense, although I'm open to alternative suggestions - whether that's removing the weighted rating completely, or an alternative weighting function.
It shouldn't be particularly difficult to remove the weighted rating - it's mainly just a question of making sure I don't miss any references to it in the code...

As for the list all by rating function, that can be sorted by either the mean ratings (which is the default) or by the weighted ratings.
  
Back to top
 
IP Logged
 
josephk
Platinum Member
*****
Offline



Posts: 3,981
Location: Canada
Joined: 01. Apr 2002
Re: Weighted ratings...
Reply #2 - 09. Mar 2009 at 20:01
Print Post  
Yeah, I kind of understand the formula. Like I said, I was the one who originally researched this and suggested it. I just didn't realize to what extent it would make a difference. I think probably that formula is meant to be used with a very large sample base, like on IMDb. With our fairly dozen or two voters, it all seems a bit arbitrary.

However, I had forgotten that it was possible to switch from weighted to mean rating on the list, which is actually kind of a neat feature if you want to compare popularity vs. pure rating.

If I was better at math, I might try to adjust the formula so that the "weight" applied on the ratings isn't quite so heavy. But I really have no idea how to do that.

Either way... maybe it's worth keeping after all. I don't feel very strongly about it. I just remember Cal having some objections to the system when we were first trying it out, and since I was the one mostly pushing for it at the time, I thought I would let you guys know that I don't find it as useful as I thought it would be. But if no one really feels that strongly about it, we might as well keep it, I guess.
  
Back to top
 
IP Logged
 
TheSwede
Platinum Member
*****
Offline


Tack ska du ha.

Posts: 1,069
Location: USA
Joined: 18. Sep 2006
Gender: Male
Re: Weighted ratings...
Reply #3 - 10. Mar 2009 at 03:38
Print Post  
That's interesting. I never did much research into what the "weighted" rating meant.  I figured it weighted the most recent ratings more than the older ones... which didn't make much sense to me.  At least this sort of makes more sense.  But like you say, J, it's pretty much nonsense with so few ratings.

Maybe I'll take a look at it sometime.  I love statistics and math...but I don't know this particular equation yet.
  
Back to top
 
IP Logged
 
josephk
Platinum Member
*****
Offline



Posts: 3,981
Location: Canada
Joined: 01. Apr 2002
Re: Weighted ratings...
Reply #4 - 10. Mar 2009 at 04:02
Print Post  
Here's the formula, by the way. It's the same one that IMDb uses for the top 250:

weighted rating (WR) = (v ÷ (v+m)) × R + (m ÷ (v+m)) × C

where:
  R = average for the movie (mean) = (Rating)
  v = number of votes for the movie = (votes)
  m = minimum votes required to be on the list (at IMDb it's currently 1300 / here, it's 5)
  C = the mean vote across the whole report (currently 6.7 on IMDb / don't know what it is on BA)
  
Back to top
 
IP Logged
 
josephk
Platinum Member
*****
Offline



Posts: 3,981
Location: Canada
Joined: 01. Apr 2002
Re: Weighted ratings...
Reply #5 - 10. Mar 2009 at 04:07
Print Post  
You know, I still find it kind of useful in our case for the purpose of compiling the list. In other words, I like having the option of viewing the top films by either pure rating or weighted rating.

Maybe the solution would be to NOT display the weighted rating on the film profiles, but only in the weighted rating list. Because I find it misleading when you see a movie for which everyone voted 9 or more but on the profile it says: "Weighted rating: 8.1" or whatever. Only in the context of all the other weighted ratings does this make sense. Because, basically, it's impossible to have a perfect score on the weighted scale.

I guess that would be my preferred solution: not display the weighted ratings on the profiles, but still compile it with the option to view the list using the weighted scale.
  
Back to top
 
IP Logged
 
TheSwede
Platinum Member
*****
Offline


Tack ska du ha.

Posts: 1,069
Location: USA
Joined: 18. Sep 2006
Gender: Male
Re: Weighted ratings...
Reply #6 - 10. Mar 2009 at 16:55
Print Post  
Yeah, that does seem like the best solution and application, JK.
  
Back to top
 
IP Logged
 
Quantum
BA Administrator
*****
Offline


BA Member

Posts: 788
Location: UK
Joined: 18. Jan 2005
Gender: Male
Re: Weighted ratings...
Reply #7 - 14. Mar 2009 at 14:07
Print Post  
Weighted ratings are now no longer displayed on the individual movie & TV show pages. There are still available in the Rating lists.
  
Back to top
 
IP Logged
 
josephk
Platinum Member
*****
Offline



Posts: 3,981
Location: Canada
Joined: 01. Apr 2002
Re: Weighted ratings...
Reply #8 - 15. Mar 2009 at 03:51
Print Post  
Cool. Thanks. I hope no one objects. But I guess if they had an opinion, they would have posted it here.
  
Back to top
 
IP Logged
 
cal-Q-L8
Platinum Member
*****
Offline


Admin

Posts: 8,002
Location: Australia
Joined: 30. Oct 2001
Gender: Male
Re: Weighted ratings...
Reply #9 - 30. Mar 2009 at 05:31
Print Post  
I don't think my opinion really matters anymore, so I'm keeping out of these discussions for my own sanity,   Smiley
  
Back to top
 
IP Logged
 
josephk
Platinum Member
*****
Offline



Posts: 3,981
Location: Canada
Joined: 01. Apr 2002
Re: Weighted ratings...
Reply #10 - 15. Aug 2009 at 16:42
Print Post  
concerning the minimum number of votes required for the average rating / top list inclusion...

It's currently set at 5. But considering how few people actually vote on films, I wonder if we shouldn't bring it down to 3.

What do you think?
  
Back to top
 
IP Logged
 
Page Index Toggle Pages: 1
Print