Revision as of 20:55, April 15, 2007

Come up with a reasonable weighting algorithm that wont enrage half the site.

-Max

This is the place where we attempt to do just that.

Mewchu11's second algorithm

This one takes max's thoughts on the subject more to heart.

Theory in writing, watch this space

Pilcrow's algorithm

Site creation and site appreciation have only a tentative relationship to one another, and seniority and past success do not necessarily correlate to the ability to rate a site intelligently. An algorithm based on these will not necessarily improve the ratings, and it has a secondary effect of creating an elite.

If voting must be made more intelligent, make it intelligent by attacking the extremist voters. The strongest way to effect a site's rating is to 1 or 5 star it, but statistically, someone who is really participating in the community will rate sites throughout the spectrum. In fact, the MAJORITY of their votes should be 2-4 stars with only the very worst and very best sites getting 1 or 5 star ratings. So, track the voting history of people supplying the votes, the more their votes resemble a bell curve from 1 to 5, the more weight their votes get. Downvoters and upvoters will least resemble the bell curve and will therefore have their votes discounted the most.

For a second factor, give people higher ratings if in addition to rating a YTMND, they also included comments, as this gives the author further feedback on how to please their audience. That factor should carry significantly less weight than the bell curve factor however.

One could also add factor for other forms of community participation - such as forum posting, wiki posting, and site posting. These should be very small elements though, since one should not want to encourage people to pad any of those counts.

A forumula similar to 0.8 * Bell_Curve_Metric + 0.1*Comment_Frequency + 0.05*Forum_&_Wiki_Posts + 0.05*Sites_Submitted would stike me as about right.

Mewchu11's First algorithm

Personally I would love to see weighted voting take back YTMND from the NARV's but at the same time keep them happy. Therefore I am proud to unveil my version of the weighted algorithm. As is anything on a wiki, this page is open to editing. For the sake of logic, suggestions to edit this algorithm or any other should first be taken to the edit page. Also, if you were going to ask, yes I am going to make a simple YTMND on this sometime tomorrow for the sake of people adding their opinion.

Variables

A: Number of sites with a rating higher than 3 after at least 10 votes (if this is 0, 1 or 2 we treat the variable as 3)

B: Average Rating of all sites (if this is Undefined, we treat the variable as 3)

C: Average Rating of other sites

D: Donations

E: Days since join

The Algorithm

Voting Power = [(A / 3) * (B / 3)] * [Ca * Da] + Ea

Finding Ca from C

Ca is determined not as a raw value, but as a deviation from 3.

If 1.0<C<1.5 or 4.5<C<5 then Ca = .5

If 1.5<C<2.0 or 4.0<C<4.5 then Ca = .75

If 2.0<C<2.5 or 3.5<C<4.0 then Ca = 1

If 2.5<C<3.5 then Ca =1.25

Special case in Ca

If user has less than 10 votes overall, Ca = 1 regardless of the above

This promotes fair and average voting, and moreover cripples rampant upvoters and downvoters.

Finding Da from D

D is an outside modifier that scales the whole equation based on how long the user has been around. For the sake of simplicity when I say “Month” we can approximate it to 30 days.

< 1 month = 1

1 to 2 = .9

2 to 4 = .8

4 to 6 = .75

6 to 8 = .8

8 to 9 = .9

9 months to 1 year = 1

1 year to 1.5 = 1.25

1.5 to 2.0 = 1.5

2.0 to 2.5 = 1.75

> 2.5 years= 2

This works to both give the newbie a chance but punishes slackers who don’t improve their scores in other ways (by making good sites or buying their way in). The four to six month marks the worst of it, after that the punishment ceases and they are rewarded with a gradual increase in voting power. Once senior ranks (a year onward) are achieved voting power will increase rather dramatically with time, by this point the user should have a good eye of what’s good after all.

Finding Ea from E

D is a vote increase by a fixed donation rate

1 D (a buck after paypal) = .2 DA

aka 5 bucks weighs your vote up a single vote. I would suggest a cap somewhere as so people can’t become insanely weighted (I’d say the limit would be 25 bucks/+5vote strength) This function gives people a reason to donate other than out of love for YTMND.

Note of Incompleteness

I am aware that this might seem off to some people, and to them I say TELL ME. I don't think it's perfect, just usable. I want to see what other people are thinking, otherwise I wouldn't have put it on the wiki! One big thought I'm having is if there should be a maximum vote weight, and if so what if...

Wallet's algorithm

(OK fine I'm a lazy bastard, come up with a good w yourself using standard deviants or something)Wallet 19:15, October 24, 2006 (CDT)

This method puts a smaller emphasis on vote strength and a greater one on vote skew. The algorithm holds the following true:

• The average vote for the site as a whole should be 3; if the average is, say, 4, then the ratings 1-4 are precise, but the ratings 4-5 are vague.
• Similarly, if someone is a downvoter, their high votes should have more weight than their low votes, and an upvoter's low votes more weight than their high votes.
• The tendency for many members to vote "hot-or-not" makes a mockery of the famous 5-star system, and shouldn't be rewarded.
• No matter what measures are taken on a person's vote, a vote lower than the site's average must never raise the average, and similarly a vote higher than the site's average must never lower the average.

The following proposal differs from Mewchu11's First in that it puts little weight on a member's supposed contribution. It differs from Pilcrow's in that skew is not punished across the board, but rather compensated for. Like all algorithms on this page, it should be evaluated on its logical merits, not on individual number values which can be changed without altering the spirit of the algorithm.

Assumptions

This algorithm assumes it would take way too much computing power to retroactively change the weights of votes already made without the user manually re-voting, or look up all the data on all users that have ever voted on a page. The only things assumed to be known by the algorithm before internal math are:

• The number of each vote 1, 2, 3, 4, or 5
• The number of votes on the site being appraised
• The average rating on the site being appraised

With this info we can derive a lot of information, such as how much a vote in an unweighted system should change the average: simply take the number of votes N and the rating R and the result based on a vote V would be

(NR + V)/(N+1)

This is the assumed system, since it can't be a significant distance from the true current method, and so an algorithm based on it requires less work on Max's part to translate into a final implementation.

The process of weighting votes in this algorithm is to change the value V such that the above fraction outputs a different average vote. If we want to give extreme weight to votes, giving them a V of 0 or 6 is not unbecoming, as long as we limit the bounds of the final rating: a vote weighted to 0 might bring a one-vote-of-5 page down to 2.5, but can't bring the score below 1.00.

Exploit Prevention

General weighting regardless of skew is really a method of exploit prevention, either preventing a new member who has never seen other sites from voting unfairly, or a sock puppet account from upvoting friends and downvoting enemies. I believe no purely mathematical method can prevent such exploits and it requires diligence on the part of the administration or automated methods outside the scope of vote weighting; however, a good catch-all method is to give an automatic -50% weight vs. the average to all votes made by accounts that have not both existed for 1 month and logged 100 votes, or some other measure of activity:

(2NR + V)/(2N+1)

The following is also vulnerable to moderate exploitation with very careful and diligent vote-changing; though this is a small flaw, it should be compensated for. The easiest method is to limit changing votes. Vote-changing could easily be limited to 25 per day, but I know that I for one wouldn't have much trouble using this up, as I often change votes once or twice even on my first visit to a page as I reconsider my rating. Logging vote times henceforth might be useful; then vote-changing could be unlimited for sites a user has never rated before today, but limited to 25 per day on sites the user first rated more than a day ago.

An Average of 3

The entire pholisophy of this algorithm is that the natural state for any given page is a rating of 3.00; that is, all votes on YTMND whatsoever, after weighting, should together average to 3.00. It's important to recognize that five votes of five stars doesn't make 25 stars; we're dealing with an average, so we must calibrate our scale so that the median vote is our "zero".

Part 1: Tending to 3

As such, a user's votes should average 3.00 after weighting. As an example, assume a user's current average vote after weighting is less than 3. In this case, less weight is given to votes below the user's average, and more weight to those above.

Any method you imagine will work fine, but the following is a simple one derived from the requirements: Let s equal the site's current rating, and v equal a user's vote. Regardless of how we want to weigh it,

If v<s, s must not increase

If v=s, s must not change

If v>s, s must not decrease

We can safely ensure this by only weighting the difference, d, where d=v-s, by multiplication of a weight, w, which is non-negative.

I'm still working on the exact values, but the gist is that your final weighted vote, V, looks a bit like:

V = v + dw "The final counted vote is the original vote, plus the difference (d) times w"

I don't yet have a nice equation to determine w, but you can think of it in general terms based on wether a vote is inside or outside a user's average vote a. A vote v is here considered "outside" a if it is both further from 3.00, and on the same side of 3.00 as a, while an "inside" vote is either closer to 3.00 or on the opposite end of 3.00 relative to a.

E.g. if a user's average vote is 3.5, a 4 or 5 is "outside" and a 1, 2, or 3 "inside". Feel free to edit this if you think of clearer terminology.

So, consider the weight w as such:

If v is outside a, w<1

If v is inside a, w>1

If v=a, w=1

Note this means that if a person's average vote is 2, and the person votes nothing but 2, the weighted vote will be 2. The algorithm presumes that virtually no one only votes one rating 2-4 absolutely all the time, and so given enough time, the weighting will tend towards correct. A more robust system would, naturally, be more complicated. As for people voting only 1 or 5 all the time, this is covered by the following section:

Part 2: Punishing Binary Votes

Most users are divided into two camps: those that reserve 1s and 5s for more extremely bad/good pages thus keeping their votes in a logical perspective of preference, and those who only vote 1s and 5s, thus giving their vote more power against the average. The logic behind only voting 1 or 5 is that one can better force the average to approach the rating the user would actually give the site; thus users attempt to exert as much dictatorship as possible on the democratic vote.

To discourage such behavior, the best way is to make it a moot point. First let's forget the "tending to 3" section and assume that 2, 3, and 4 have a constant application on the average (as 2, 3, and 4). Let 1' be the number of 1-star votes, 2' be the number of 2-star votes, and so on. For a given user, the actual recorded numbers would be:

For a vote of one,

2 - 2^-(1'/2' - 1) "Two, minus two to the power (1'/2' minus one)"

For a vote of five,

4 + 2^-(5'/4' - 1) "Three, plus two to the power (5'/4' minus one)"

(Note that these require a non-zero 2' and 4' - I suggest minimum values of 1)

The end result of this is, if you have ten times as many 1s as 2s, your 1s act as 2s, and if you have ten times as many 2s as 1s, your 1s act as 0s. (There's a little rounding involved, don't any math nuts get on my case.)

This satisfies three conditions:

• If you almost never vote extreme, they will be worth even more than their unweighted numerical amount.
• If you give an even number to all votes 1-5, they will all act accurding to their unweighted numerical amount.
• If you only vote 1 or 5, after you vote 10-20 times, all your subsequent votes will act as though you're only ever voting 2 or 4.

The result of this is that the necessary evil of only voting 1 or 5 for maximum weight has been eliminated, since maximum weight is achieved only with moderation in the use of extreme votes.

Final Result

Though w has as yet been poorly-defined, we can now give a final weight by slapping the two methods together crudely:

V = v + dw

If v = 1 or 5, apply the following:

For a vote of one,

V' = V + 1 - 2^-(1'/2' - 1)

For a vote of five,

V' = V - 1 + 2^-(5'/4' - 1)

V (or V' as necessary) can now be applied to (NR + V)/(N+1) as-is to change the page's average, or additionally weighted with other considerations toward longevity, community contribution, number/ratings of the user's own pages, donation, etc.

Veteran System

Intro

This idea is designed to be as basic and easy as possible, so that everyone will understand how it works, and how this will help YTMND. If you have any comments or suggestions, please include them in the final section of this page, and optionally you can add your username too.

Purpose

- Encourage more user participation

- Allow YTMND veterans slightly more control over the overall ratings of ytmnd sites. This control is a privilege that will be given in exchange for their long-term dedication to YTMND. Note that users are not initially considered veterans when the system is put into place, it must be earned through balanced voting.

- YTMND veterans are expected to set a good example by voting as fairly as possible. By exercising the new control granted by this system, they will be providing newcomers with helpful feedback which in turn will establish a better framework for the judging of site context by the community as a whole.

- New users will be motivated to comment and vote more fairly across the 1-5 star spectrum in order to gain access to these privileges. During this process, they will hopefully learn to be more objective critics, and to be more outspoken in general.

Star Levels

There are 3 star ranks in total. As users progress, they will slowly gain more ability to influence the overall rating of YTMNDs. All YTMND users (even current veterans) will initially start at the Blue Star level, as this level system does not start counting votes until the date this system is put into effect. All votes made before this system takes effect will not be counted. This is to ensure that all users begin equally under the new system, and to encourage more voting by all users in general. Also, it would be too difficult to apply this new system to all sites that users previously voted on. This is probably the easiest and fairest way to implement the changes.

Blue Star: This is the standard vote, all registered users begin here when the system is implemented. Blue Star votes are counted as one vote towards the total votes, and change the site rating by one vote. (equivalent to our current voting system)

Green Star: This vote is still counted as one in regards to the vote total of a site, but it is worth two votes in regards to its effect on a site's overall rating. Prerequisites: To become a Green Star Member, you must make 10,000+ "comment votes" on the new voting system, and within this total there must be at least 1,000 votes that fall within each category of 1,2,3,4,5 stars respectively. Note: Only votes with comments attached will be counted towards this total.

Gold Star: This is the highest ranking level. This vote is still counted as one in regards to the vote total of a site, but it is worth three votes in regards to its effect on a site's overall rating. Prerequisites: To become a Gold Star Member, you must create 100 sites within the new system, and receive 100,000 votes (non-comment votes included) within the new system. In addition, you must submit 100,000+ "comment votes" to other sites within the new system, and within this total there must be at least 10,000 votes that fall within each category of 1,2,3,4,5 stars respectively. Note: Only votes with comments attached will be counted towards this total.

Conclusion

Note that the Star Levels are difficult to achieve, and this is done for a reason. Only the most dedicated members should have access to such privileges, because the consensus is that objective voting takes time to learn. The only way to become more objective is through direct experience by viewing a great multitude of different sites over time. Also, remember that all users' stars will still appear red in the comments section. This is to ensure that prejudice does not arise, such as favoring green members over blue, etc. The color change is ONLY visible to the user making the vote, no one else will know you're green or gold unless you tell them personally.

Old Votes: If it is possible for old votes to be counted towards this sytem, I would be in favor - I only suggested this system because it seems easier than to redefine the old votes in this way. Assuming that old votes can't be counted, users would simply need to vote on old sites again as they gain higher rank, so that the green stars and gold stars would then take effect.

Feedback

IMPRESSIVE

Darth Vader is that you? Thanks.

I think that encouraging comment spam is a definite no-no. I for one find it extremely useless to see a 5-star comment that says simply "WIN", since this doesn't at all help me to learn why it is the site has the rating it does, and so neither contributes to my future sites for tells the other people viewing the profile anything useful. Wallet 19:19, October 24, 2006 (CDT)

^ I can agree on that point Wallet. It's not an option that is of great importance here, considering the problems it could bring. So after removing the comment requirement, all votes would count.. but there is another weak point to address. Users might employ bots to vote for them, in order to gain access to the privileges mentioned. One possible solution to this could be to allow a set maximum (100-200 for example) of votes per day from each user. It wouldn't stop the bots, but would slow their progress by a lot. Then again, I think max mentioned that he's working on a solution for bots.. so maybe that won't be much of a problem. roq

AaronD12's First algorithm

I wrote this algorithm without knowledge of the other algorithms above. My primary purpose was to restrict downvoters without any significant disruption to regular users.

My thoughts on this process is that everyone wants their wonderful YTMND site to be voted a "5". Yes, there are some spoof sites that are made (e.g., the most annoying sound sites) that are destined for "1"s, but I think a majority of people want their sites to receive a "5". When I created my first YTMND, I was disappointed in the score of around 3. Turned out downvoters took a chunk of the score from my site, even though my site really wasn't that good.

Down with downvoters

To keep downvoters in check, a simple averaging scheme would take their average vote, subtract it from 5 (the highest vote possible), and that would be their minimum vote. This would possibly change the voting scheme somewhat, causing people to vote "5" for sites that are great more often.

With other algorithms stating that there should be a bell curve, peaking at "3", is like saying that a "C" in school should be the target for all students. People don't try to score "C"s in school; they go for "A"s. Think of the voting in terms of letters (1=F, 2=D, 3=C, 4=B, and 5=A), and you understand what I mean.

Examples

If a user has an average score of 2.6, round that number down to 2 and subtract it from 5. That would make the lowest score that user could vote on any site to be a "3".

Average of 4.2? Lowest vote possible would be a "1". Average of 3.6? Lowest vote possible would be a "2".

Code Snippet

Here's a code snippet. I don't know which language YTMND is programmed in, but here is my idea in PHP/MySQL:

<?php
$myresult=mysql_query("select avg(score) from votes where userid=$_COOKIE['userid']",$database); // retrieve average score
$myrow=mysql_fetch_row($myresult);
// calculate average score, leaving only the whole number, rounded down, subtracting from 5 (the maximum possible vote)
$minimumvote=5-intval($myrow[0]);
?>

Suggestions are welcomed! Thanks, AaronD12

Feedback

This is certainly an interesting system. I completely agree that people claiming the "3 peak bell curve" as a good thing is ridiculous and backwards. - BTape

This is a decent system, but I can see 1 major problem. This will not help stop splitvoters. For example, if a person has 1000 5-votes and 200 1-votes (avg vote would be a 4.33) that person would still have the power to vote 1s. When his avg vote drops below 4.00, he only has to upvote more sites (probably his friends) to continue downtoving. "but I think a majority of people want their sites to receive a "5"" True, but most sites don't deserve a 5. - LUE1337 (yea, it's really me)

Max's attempt at an algorithm

After recently writing a rule-based system for the moderation queue priority system (which prioritizes sites that need to be moderated based a set of predefined rules), I've decided to finally give writing a weighted vote system a shot.

One major flaw in this algorithm is that it incorrectly assumes that votes users make are based entirely on the content of the site where in reality, the user who created the site can also be a large part of the final score of their site.

VARIABLES

DYNAMIC VARIABLES BASED ON EACH USER'S HISTORY (with examples)

AVERAGE_DEVIATION = 0.875

This is the average vote deviation from the final score of each site the user has voted on.
This means a user who votes 5 on 10 sites that average a 1.5 score, the user would have a 3.5 average deviation.

AVERAGE_SCORE_GIVEN = 3.25

 The average vote the user gives across all sites.

AVERAGE_SITE_SCORE = 3.8

 The average score of this users sites.

LENGTH_OF_MEMBERSHIP = 16 months

 The length of the user's active membership on the site in months.

RULE BASED VARIABLES

BASE_SCORE = 2.71

 This is the average score of all sites, which would obviously change over time. 
 This defines a "middle of the road" score.

BASE_MULTIPLIER = 1.0

 The base multiplier of a regular user's vote.

MODERATOR_MULTIPLIER = 1.5

 The base multiplier of a moderator's vote.

SUPERMOD_MULTIPLIER = 2.0

 The base multiplier of a supermod's vote.

ADMIN_MULTIPLIER = 3.0

 The base multiplier of a admin's vote.

BASE_TIME_SEGMENT = 3 months

 The time segment at which a user's membership should start affecting their vote weight.

TIME_SEGMENT_MULTIPLIER = 0.1

 How much each TIME_SEGMENT should increase the vote weight.

SITE_SCORE_DIVIDEND = 0.33

 The segments of the average site score above BASE_SCORE that should add to a user's weight.

SITE_SCORE_MULTIPLIER = 0.25

 The multiplier of the above segments by which to increase the weight.

USER'S SITE SCORE ALGORITHM

FLOOR((AVERAGE_SITE_SCORE - SITE_SCORE_BASE)/(SITE_SCORE_DIVIDEND)) * SITE_SCORE_MULTIPLIER = 0.75

     (     3.8           -    2.5         )/(        0.33       )  * 0.25
     (                 1.3                )/(        0.33       )  * 0.25
     (                         3.9                              )  * 0.25       

USER'S AVERAGE VOTE DEVIATION ALGORITHM

This needs tweaking!

BASE_SCORE - ($AVERAGE_DEVIATION/$AVERAGE_SCORE_GIVEN)) = -0.285

USER'S MEMBERSHIP LENGTH ALGORITHM

(LENGTH_OF_MEMBERSHIP/BASE_TIME_SEGMENT) * TIME_SEGMENT_MULTIPLIER = 0.5

FINAL VOTE WEIGHT

Add USER'S SITE SCORE ALGORITHM + USER'S AVERAGE VOTE DEVIATION ALGORITHM + USER'S MEMBERSHIP LENGTH SCORE ALGORITHM + BASE_MULTIPLIER.

IN ACTION

[ max ] final weight: 7.8377

 - base: 3 + site_score: 1.0584 + vote_score: 2.5793 + membership: 1.2

[ inkdrinker ] final weight: 4.0397

 - base: 2 + site_score: -0.3727 + vote_score: 1.2791 + membership: 1.1333

[ dcmm ] final weight: 4.3299

 - base: 1 + site_score: 0.2613 + vote_score: 2.5019 + membership: 0.5667

[ Teknorat ] final weight: 4.4273

 - base: 1 + site_score: 0.2042 + vote_score: 2.6564 + membership: 0.5667

[ Punisher ] final weight: 2.3846

 - base: 1 + site_score: 0.4194 + vote_score: 0.4319 + membership: 0.5333

[ Decisi8ns ] final weight: 3.8204

 - base: 1 + site_score: 0.6592 + vote_score: 1.6612 + membership: 0.5

[ Pineapplesarebetter ] final weight: 3.528

 - base: 1 + site_score: -0.7827 + vote_score: 3.1107 + membership: 0.2

This is a pretty rough draft, but it seems to be somewhat accurate and doesn't reward any group of users too thoroughly. Let me know if you have some users you'd like me to calculate.

Max 21:55, April 15, 2007 (CDT)