Think Draw Forums
Forums - Think Draw Feedback - A response to the Top 5 comments

AuthorComment
21. 5 Feb 2009 20:13

anotherronism

That presents the same problem as just counting votes. The people at the end of the month don't stand a chance.

22. 5 Feb 2009 20:21

anotherronism

And finally: The rolling 30-day total vote count would really, really, really, really stick it to the blitzers. They would actually be helping their intended victim make Top 5. Take that dirty blitzers! Ha!

23. 5 Feb 2009 20:37

MugDots2

You know, I'm not half as smart as thinkle peep I am.

24. 5 Feb 2009 22:36

yahoo

Ron - the big question in your maths is what you were using as C. Assuming the average rating on the site is 4 I get a very different picture.

Still using your example of:

A: 4 5's
B: 6 5's
C: 7 5's & 1 4
D: 10 5's and 1 1

Instead of this:

Assuming a minimum vote count of 2 the results work out to this:
A: 4.942528736
B: 4.956896552
C: 4.865517241
D: 4.665782493

Still assuming minimum vote count of 2, but average of C=4 I get:

A: 4.33
B: 4.75
C: 4.7
D: 4.54

Which is looking pretty good already (although obviously needs testing with higher vote count).

So - what C were you using? or have i got the formula wrong?

25. 6 Feb 2009 03:17

marg

my 2 cents -

my picture in the January Top 5 was done in about 20 minutes, quite late on the morning of New Year's Day (i.e. after the end of the month), and so it wasn't blitzed (like nancylee's kite, which was done even later)

.. if you didn't have 'highest rated', I don't think anyone would ever ask for it

Why not just remove 'highest rated' altogether and sort everything on 'most rated' [the way you are doing, which I'm guessing is accumulative ?] for Top 5 and Showcase, plus having 'Most recent' ?

{but being able to have the most rated by month would be spectacular !]

26. 6 Feb 2009 03:19

marg

oh, so sorry.. December and hawk !

27. 6 Feb 2009 03:30

Login

MugDots2, Thank you from the bottom of my heart for introducing some humour to all this. It was the first good laugh I've had on TD for ages.

28. 6 Feb 2009 04:17

candr

Matthew, when I mentioned TD being abe to KNOW who's doing the unethical voting, I didn't mean that it would be public for everyone to see -- but that they would be able to monitor something like that and remove voting capabilities from someone who does it. The main problem is lack of accountability. Shouldn't we be accountable to someone, even if it isn't public?

It wouldn't even be necessary to monitor everyone, but if there is a complaint, they could look into it and monitor the offender to see if there is any patterns in this behavior. Just a thought...

29. 6 Feb 2009 04:34

anotherronism

Yahoo: I ran this through Excel and didn't save the sheet.

I actually averaged all the votes in the example given:

A: 4 5's
B: 6 5's
C: 7 5's & 1 4
D: 10 5's and 1 1

Total Score/Total Vote Count: 20+30+39+51/4+6+8+11
140/29 = 4.8276

30. 6 Feb 2009 04:38

anotherronism

But I see your point. Over a greater range the average WOULD go down and would help weight the whole thing more properly. Hence it's being used by IMDB and Amazon.

This is my problem with 'simulated' data. I never think of everything.

I would LOVE to know the Mean for all images on ThinkDraw.

I am NOT a mathematician, I just play one on TD!

31. 6 Feb 2009 04:47

anotherronism

I'm gonna run some sample data which might more closely approximate a real-world month-end on TD. In the words of Da Gubner: I'll Be Back!

32. 6 Feb 2009 04:52

marg

LOL, Mogin and LugDots2.. you made my day - many thanks !

33. 6 Feb 2009 05:40

anotherronism

And the results are in:

Here's what I've done:

I created 3,000 Fake Ratings
I gave them a random number of votes from 0 to 50
I gave 5 stars to 85% to 100% of those Votes
I split the raminder between One and Four Stars
I then got a Mean for the entire report (worked out to be 4.930692)
I then ran the Formula across all the data with a minimum vote count of 10

And the results:
Ratings of 50 votes with all 50 getting fives obviously won.
But here's the rub.

I then added a single record with 50 Fives and a Single Four
This record jumped to 1,046 in the rankings.

So I started playing.

Turns out that single Four needs an additional 87 fives to climb back into 1st Place.

What I've learned (I think) is that the "I only give five stars" voters are doing just as much damage as the blitzers. You could argue they are actually doing far MORE damage. By only voting on things they think deserve a Five they raise the overall average SO HIGH that a single Four falls way, way short.

The formula does work and Yahoo is correct.

BUT - it only works in a valid ratings system where there is an even distribution of votes across the range of options.

I'm making major assumptions here. But so many people say they only give Five Stars or No Rating. I do this myself. So I assumed 85% of all votes fall into this category. Even if it's only 50% the weight of a single Four or (God Forbid) a single One becomes so significant as to be unrecoverable.

Note: 3,000 is probably a low number but I'm assuming 100 images per day over a 30 day range. If you consider this low then remember the rankings are done by-theme and I think 3,000 is a good number in that regard.

Comments welcome. I can share my data set with anyone who would like to play with it. But not in a forum post. It would be a long, long message

Ron

34. 6 Feb 2009 06:31

Soda

Right we would need to start voting on every piece and not just on the ones we think should go into the top 5....(or find the ones we vote on only to be those with a 5)

The way a lot of people do it now is like the no vote, vote system.... there seems to be very little gray....

35. 6 Feb 2009 08:49

matthew

...Think Draw like the "highest rated" because the think "most rated" is not fair to the pics done at the end of the month... The problem is "highest rated" is not fair to the pics done at the begining of the month.

...I bet if I did identical pics... ONE on the first of the month & ONE at the end of the month, the latter would make the top 5 while the 1st never stood a chance...

...As soon as a pic gets a 4 in the system we use now it is dead... Pics done early have all month to get their 4 vote...

I still say that a default vote of 1 on every pic will give the results we want.

... one 5 plus the default (D) = 3 average (5 average in current system)

...(10) 5's +(D) = 4.64 average (5 average in current system)

... (19) 5's + (D) = 4.8 average (5 average in current system)

...(27) 5's + (1) 4's + (D) = 4.83 average (4.96 average in current system)

...(27) 5's + (1) 1 +(D) = 4.72 (4.85 average in current system)

...Now one voter can not make or break a pic... if a few agree it is a 4 or a 3 then that would effect it... Also we would not have to surf through pages and pages of perfect 5 pics that only have the 1 vote of the creator in order to see the ACTUAL high rated pics...

36. 6 Feb 2009 09:22

anotherronism

Matthew... I'm gonna plug this into the data I created earlier and check out your theory.

Honestly I had dismissed it earlier because it did the same thing to every pic and, in my mind, that would simple yield the same results.

But I was wrong (maybe)... It actually forces everyone into the middle with a struggle upwards instead of starting at the top with nowhere to go but down.


Hmmm....

As I've said before: Who'd've thunk it would be so complicated

37. 6 Feb 2009 11:34

Ernie

My head hurts! I just wanna draw purdy pictures! I didn't know i would need a maths degree to use the forums!

38. 6 Feb 2009 11:57

puzzler

I'm with you Ernie. I'm beginning to loose the will to live!

39. 6 Feb 2009 11:58

puzzler

-o

40. 6 Feb 2009 12:28

Soda

Matthew that does seem a lot more fair. Since if you think of it then we would have to climb instead of only being able to fall. Opposite of some grading systems in schools where the goal is to let you pass/get a good grade. Here the goal is to climb... I agree... that should work.

I'll let Ron do the math though :p