Latest revision as of 20:54, December 21, 2008

Weighted Voting

What exactly is a weighted voting system trying to accomplish?

1) Give more voting sway or power to people who have been at the site for a while and know there are more ways to vote something besides a 1 and a five. Hopefully resulting in better content highlights.

2) Dissuade people from making alt accounts with the purpose of giving their site a better score or competing sites a lower score. As well as providing a long term goal for new users to achieve in order to attract them to become part of the community for the long term.

Some things to keep in mind:

1) The less we give to an automatable action, the better. We want to avoid giving the opportunity to create hundreds of drones which automate some votes and then sit dormant until six months later their voting power is great enough to drastically change site scores.

2) How does voting power change over time? ie: If I vote 5 on a site in Jan when my vote weight is 2.0 effectively giving it 10 points, what happens when my voting score is increased to 4? If I go back later vote it a 4.0, what happens then?

3) How should voting score be applied to votes? If a user has 2.0 voting power should that affect the score by applying a vote of 10 to the site if the user votes a 5 or is the preferred method to try to keep voting weights as an integer (as opposed to a float) and then make that many votes (ie a 2.0 voting power would net a the voting power of 2 users)?

That all being said, I think I want to focus less on passive rewards and more on active rewards. Possibly create an activity score, commenting score (again some things that wouldn't be hard to automate), site moderation scores (ie. if a user marks a site as nws when the majority say its ws, lower their score or vice versa) which would be popped into the algorithm.

This is a giant step for YTMND and could possibly be a great deal of work depending on which path we end up taking, but I appreciate the time you're spending on it. Max 07:05, August 22, 2006 (CDT)

My god I forgot how hard the NARVs bite when they see something they don't like. Looks like if Weighted Voting ever comes, it will come with most people unaware of what the hell it means. Aw well.

Thanks very much for the imput Max. That first one was kinda written half blind as to what you and everyone wants, so I now find it pretty silly in retrospect. I'm gonna write up another one over the night that is more direct with what you said.

Peace, Mewchu11 22:07, August 23, 2006 (CDT)

The problem with normal distribution

Most of the algorithims seem to suggest that votes should be normally distributed with a median of 3, but such a distribution also assumes that there are an equal number of sites worthy of 5 stars and 1 stars. Is that really the case? At all?

Even max, whose voting is mostly restricted to random looks at the front page, has an average under 3 and like 60% more 1s than 5s. To end up with an average of 3 or as many 1s as 5s, one really has to go out of their way to avoid sites they suspect might be bad, or just refrain from voting on the majority of them.

If our voting system really assumes that there are an equal number of awful sites as there are good ones (rather than what should be apparent - that the chaff outweights the cream by orders of magnitude,) the system will be just one more factor contributing to the already over-inflated scores sites are getting and further discourage low votes, as users afraid of screwing up their weighting will bump votes up to 2s and 3s.

To recap, a 3 median punishes users who visit more than their preferred pockets of the site and vote accordingly, and also artificially inflates the scores of the mass of sites which deserve 1 and 2 stars. Each instance of pointing out a bad site will now trigger not only the unavoidable revenge downvoting, but a loss in that user's vote weight.

Perhaps nudging the median to 2 would be called for?--Inkdrinker 22:25, September 20, 2006 (CDT)

--

This is a valid point, as well as the fact that people are more likely to vote on a site if they really hate or love it.

+-----------+
| AVG(vote) | (this is from 12,377,951 votes.)
+-----------+ 
|    3.5529 | 
+-----------+

The average site score rounded to the nearest value:

| score | number  |
+-------+---------+
|     1 |  44247 | 
|     2 | 100608 | 
|     3 | 133445 | 
|     4 |  71395 | 
|     5 |   2130 |

Vote spread:

| score | number  |
+-------+---------+
|     1 | 2742685 | 
|     2 |  781141 | 
|     3 | 1313178 | 
|     4 | 1972549 | 
|     5 | 5569626 | 

Max 20:41, September 21, 2006 (CDT)

--

Good ideas, the only downside is that a non-linear rating system (the actual value being the number of stars squared, for example) is not very intuitive, at least not to me... And if the median is not going to be the middle of the scale, non-linearity exists in some form or another. I'd also like to point out that the number of 4 votes vastly outweight the number of 2 votes, same for 5s vs. 1s; this implies that we don't need to worry that much about people wasting a lot of time downvoting chaff. The system self-regulates by promoting better sites to be viewed anyway, with top rated/most viewed/up and coming/worthwhile, and as a result people use their votes on a scale of those better YTMNDs. If someone decides to venture off into the wilderness and downvote a bunch of chaff, that only emphasizes how great the better stuff is when they come back and vote on decent things. Wallet 18:28, September 22, 2006 (CDT)

Typical weights are 1/variance. I like the idea of enforcing a simple Gaussian. SqueeAndSpleen 23:25, April 15, 2007 (CDT)

Original TODO Note on Weighted Voting/Site Deletion

!- THIS WILL REQUIRE A LOT OF DISCUSSION!
!-- I don't feel this is a feature I should add without a LOT of user and moderator feedback.
!-- This will GREATLY sway the score of sites (and possibly in the future users etc).
!--
!-- I really don't want people to resort to automating the use of the site to get higher score.
!-- and I know they will so I'm not sure if it's even worth it to add this (I could randomly
!-- show captchas but that would be more annoying than worth it.


@- Create a score for each user based on a plethora of information:
@-- Signup date. (Older is better.)
@-- Number of votes (obviously people with > x thousand votes would get nulled.
@-- Site Score divided by Number of Votes ( this would be recursive )
@--- Try not to give users like Syncan super powers, and try not to make new users completely useless.
@-- Your vote compare to average vote (Do we have to assume the average vote on each site is correct?)
@--- Or if not at what number is a site's rating considered correct? (10 votes? 50? 100?)
@-- Activity on the site (a simple calculation of how active a user is)
@-- User votes (a rep system like vbulletin but much less swayable)
@-- Warning or reward points given out by moderators and administrators.


@-  Votes would be a multiplier on User score*User vote.
@-- There would have to be a maximum and minimum multiplier.

@-  Treasure/Trash System
!-- There is too much garbage on the site, and it shouldn't be up to mods to clean it up, it should
!-- be up to users.

@-  Based on your score you get a certain number of "Treasure" or "Trash" votes per day (or week/month etc)
@-  Newly created sites will have "Treasure" and "Trash" buttons available to all logged in users.
@-  During this "trial period", site profiles will have a separate tab with trash/treasure votes
@-  and comments from each user.


@-- Sites with a certain number or percentage of "Trash" get added to the deletion queue:
@--- The sites will be listed on the deletion queue page, and when viewed will have a notice near the
@--- top warning of deletion. The site will be listed for ~ 2 weeks and if it isn't saved by "Treasure"
@--- votes, it is deleted. Possibly force a change to "PRIVATE" instead.

@-- Sites with a higher percentage of "Treasure" votes will be highlighted on the front page or other
@-- pages.

Max 21:05, October 21, 2006 (CDT)

Know it's been a little while, and I'm still mulling over a proper answer, but a thought:

To me it seems like our biggest aid against automation of dupe accounts is a variable based on number of GOOD sites. Perhaps a boolean modifier that states that if the user hasn't made a certain contribution in a certain time (lets say a site with above a 3 rating after 10 votes in a month) that all other postive factors cease to be added for the period it rings true.

Mewchu11 22:25, October 21, 2006 (CDT)

What about people who jsut want to visit YTMND without making sites? They shouldn't be punished for having a life (unlike us).

Wallet 19:49, November 15, 2006 (CST)

Experience Points

Newgrounds has a very interesting system for handling how much "voting power" each user has, and it's basically an experience point system.

For every site you vote on, you get 2 points, after voting on 5 sites, you can deposit your 10 points. You can only deposit points earned through voting once per day, and voting on more than 5 sites does not count. As you gain EXP, you level up. As you level up, your votes will have more power. But, gaining EXP to level up gets harder as you go on. Levels 1-9 only require 50 more experience points to get up to, but getting to Level 9 requires 1175 more points instead of 50! BUT...we could also make it so that 1'ing a site doesn't give you any experience points! This will prevent downvoters from getting too powerful, and allow longtime users to REALLY support good sites, by giving them the power of many votes in one vote!

You could also earn experience points in other ways, like by having sites reach certain goals, sponsoring sites, and other ways! This system will give veteran users alot of power, and stupid downvoters and newbies, not much...

KKyuubi 07:36, October 22, 2006 (CDT)

First off, a weighted voting system has GREAT potential if it is well thought out. I am pleased to see that there is progress being made in this direction.

I would add that from my perspective, it should not be assumed that there is roughly an equal number of quality sites on ytmnd as lame sites. For the sake of argument it could be said that the ratio is along the lines of 5 to 1 in terms of the amount of work, originality, humour and wit incorporated into the sites.

I agree that there should be more weight behind the votes of a user who has created many sites. That weight should also be determined by the quality of the user’s sites but not so much that in the end the user breaks even in weight with a user who has made no sites (assuming all his sites are pitiful) although he may come very close.

Weight from sign up date should have some weight but nowhere near as much as the quality of one’s site or the number of their votes cast. I feel it is important that the number of votes a user’s sites receive should add to that user’s vote weight significantly more than their number of views (if this is even to be taken into consideration).

Perhaps number of votes a user has cast and his/her site’s ratings (with 100+ votes) should be of the most weight. I haven’t worked with any numbers here but this has all just been some food for thought.

I would also note that the Treasure/Trash system would be relatively ineffective as trash sites would not receive a significant amount of negative “Trash” ratings to be deemed deletable. As things stand, if a site is lame (or just downvoted) it fades into obscurity, often with less than 100 views and 10 votes. -NiteSky Pacific Time 9:26pm April 15th 07

Could the powers that be possibly install LaTeX support for the Wiki?

It would sure make any sort of discussion involving math much easier to read. Gimme' a shell and I'll do it. SqueeAndSpleen 23:29, April 15, 2007 (CDT)

The function may also be written as or .

markup available at http://memory-alpha.org/en/wiki/Help:Math_markup

sort of buggy but i think it works Max 05:56, April 16, 2007 (CDT)

Dedicated Creators vs Dedicated Viewers

In my experience, there is no causal relationship between the users who have high site ratings and thier participation in the voting system. Users with high rated sites usually adopt a bias towards avoiding downvoting whenever possible in order to protect thier site ratings. All of the most famous users are guilty of this (BTape comes to mind as the most obvious example). There are also cases like Roy4l, who submit multiple creations, but have a dismal participation record in the voting system. By contrast, many users who offer criticism of others' work on a regular basis usually suffer the consequences in the form of revenge votes against thier own site ratings. A system that gives extra weight to users with high site ratings will empower a group of YTMNDers who will not actually make use of that benefit, and disempower the group that participates the most. I suggest that site ratings, and site creation history should have no effect on an indivual's weighted vote. --Baron 23:49, April 15, 2007 (CDT)

I am going to assume this in response to the algorithm I added tonight. I use both average site score and average vote deviation is exactly for this reason. Someone can be a "shitty" voter and make "great" sites and have the same vote power as someone who makes "shitty" sites and "great" votes. The fact is, people who time and time again make high rated sites obviously have some grasp on what is "good". Either way, with my algorithm, more power is given to those who vote accurately than those who make good sites. On top of that, you can be penalized (with a negative weight) for voting badly, but not for making bad sites. Max 05:20, April 16, 2007 (CDT)

I think it should also be noted that highly ranked sites aren't even necessarily the most enjoyable sites because ytmder's are so political, they'll vote high if a site conforms to their fad ideology, even if it isn't that great. Giving weight to people with higher ranked sites will empower people who make "popular" and not necessarily good sites. With a strong enough fanbase, a faction like nintendo fanbois could upvote their sites into a higher status of voting power. Now I suppose in theory if most of the people on the site actually enjoyed all nintendo sites then such a model could create a lot of enjoyment, if not originallity, but when sites are being upvoted simply out of a sense of obligation or nastaliga, it doesn't help anyone, or at least doesn't let the site reach it's creative potential. Also on a side note I'd like to thank everyone involved for setting up this system, i think it will be a big improvement. --momthinksimcool 16:19, May 12, 2007 (CDT)

I have one question though for Max or anyone who knows the answer. When you say "vote accurately" do you mean simply have a good vote spread or voting similarly to the way everone else does? Because if you mean the latter that could encourage people just jump on the band wagon of higher rated sites, or avoid voting on more controversial sites. In theory we should want people to vote how they actually perceive the site as opposed to how everyone else does, and also by getting outside of the dominant frameworks "what constitutes a good site" people are more likely to think and produce creatively. --momthinksimcool 16:28, May 12, 2007 (CDT)

Small request: page numbers for full votelist

Since there's all this talk about votes, perhaps it wouldn't be too difficult to modify the user page tab to allow the viewing of ALL your votes?

It would be nice to be able to see all your votes from before the Favorites feature was implemented. Hard as i tried to save every site of interest since 2004, am still sure to have missed quite a few..

esp. since if weighted voting is implemented existing users with many votes would need a way to fix their average.. for us upvoters to perhaps downgrade some 5s since the initial excitement has died down..

Skullers 03:25, April 17, 2007 (CDT)

thumbs up/thumbs down voting

if i recall correctly, there used to be an entry in the TODO about implementing a thumbs up/thumbs down voting system. was this abandoned? and if so, why? --X1011 19:30, May 3, 2007 (CDT)

A case against the "clean start" idea

I'm coming into this discussion pretty late so for all I know everything has already been decided, but I'd just like to say that a system that contains compromises will best serve the best amount of people. The idea of a clean start has some good points because many people who might be quite capable of judging a site might have voted poorly in the past for self interest reasons, and creating steep hierarchy will discourage new users. But I also think there's the other side of it that, people who have actually been thinking about the sites they've voted on are more likely to have developed a critical sense then someone who just votes one way. Obviously there are exceptions in both cases. Also people who have been here longer or more actively will have a better critical sense on average. I think rather than starting from scratch or starting with an intimidating hierarchy, a hybrid start with small weighting on more experienced and better voting past members would be a good compromise. After the new voting system comes out people will be more inclined to vote with all 5 digits, but that doesn't necessarily mean they'll know how to and may just throw votes around arbitrarily to keep their numbers even (I suspect this will be a problem anyway). My point is though that if there are reasons to start weighted and reasons not to then a small relatively tame compromise might be the best solution (although on a practical level this might not be possible at this point). --momthinksimcool 16:58, May 12, 2007 (CDT)

For example a relatively tame and simple starting point could be to give small weighting difference between the people who have been here for a month and not, because the people who haven't been here for a month probably wouldnt know that they were missing anything anyway (and wouldn't actually be missing much).

Or maybe additionally you could give a small weighting difference between people who've voted extremely and those who at least made a small effort to use some of the other numbers. If the difference is small enough and easy enough to overcome then it shouldn't anger split voters, down voters, and upvoters too much because they were aware of that they were voting improperly anyway, and it would be a small temporary award to those who made the effort previously.

this is why we can't have nice things

i see a big problem with weighted voting. 

if you want to downvote, all one needs to do is run script that will search for shit sites, up-vote several of them (won't make a difference because they are so low already), and take your new "heavy" vote and downvote away. upvoting is just as simple. i don't see a way around this. weighted voting seems like it can just as easily be used to the rogue voter's advantage.

limiting rogue voters to 3 star spread

Is it possible to only enable 1* and 5* votes for voters leaving comments on the site they want to five or one? This could limit rogue voters to only 2,3 or 4 stars and promote feedback on sites blah blah blah.