My Proposal To Revamp The XP / Level System
Note: I am mostly putting this here for posterity. I think it is a good idea, and I don't want to forget it.
The sheer number of discussions, debates, back and forth, and rabblerousing that the leveling / XP / merit / Honor Roll / "site is dying" has caused could (and often does) fill a book or two. I think this topic is important enough that it should be considered more deeply, more often, by more users.
The goal of this writeup is to identify what I think are the most important components of a good XP / level system, and then analyze how the pre-Honor Roll system failed to meet those components, how the current system fails to meet those components, and how my system would meet those components.
The Components
The chief components that are vital to a good leveling system are:
- Attainability - A new level should be attainable within a reasonable frame of time.
- Clarity - The requirements for attaining the next level should be clear.
- Positive Reinforcement - All contributions to the site should be a step towards attaining the next level.
- Proportionality - All contributions should be rewarded in proportion to their quality (as ascertained by the community).
Old System
The pre-Honor Roll system succeeded very well on the Clarity component. If you wrote X number of writeups and gained Y number of XP, you leveled up.
The Positive Reinforcement aspect was fairly well defined, although nuked writeups used to contain a more injurious 5XP penalty. In general, though, writeups were created so fast (and votes flowed so freely) that on the whole, more people were moving up than down.
By keeping standards and expectations rather low, its Attainability component was not that much of a factor, but over time, as standards rose and noding material lightened up, the failure of this system became more and more imminent. Additionally, the exponential rise in the level requirements towards the end of the spectrum (Level 10 and up) was untenable from the "reasonable frame of time" aspect - even with almost no standards, writing 2,500 writeups would be a feat bordering on codependency.
The system's biggest failure, of course, was the Proportionality reward. A writeup was a writeup was a writeup. A person who submitted 10 +4/-1 writeups was doing just as well as someone who posted 2 writeups that each received a C! and +12/-0 writeups - even better, when you considered the weight that # of writeups had in determining levels.
Current System
With the introduction of the Honor Roll, we fixed the Proportionality component in a direct fashion. If you write better than the typical noder, you received more credit in your # of writeups component of leveling up. However, this was done at the sacrifice of all of the other components of a good system:
- Clarity suffered as people could no longer simply look at their XP and number of writeups to determine where they stood. The Honor Roll is rather basic statistical math, but its application immediately added a layer of obfuscation and obscurity to what had been an extremely clear system.
- At first glance, it appears that the Honor Roll also solved the Attainability issue, but in fact, its impact on Positive Reinforcement was so severe that the percentage of new users from any given year who have achieved Level 2 and have then gone on to achieve any other Level has decreased every year since 2004.
- Ah, Positive Reinforcement. The horror stories of spiregrain deleting nodes to raise his merit, of mauler, passport, and golFur afraid to post something nominal yet unglamorous lest they disturb their precious Merit (a trait I haven't been guilty of, but have cursed myself frequently for lacking it) - the Honor Roll system is ripe for gaming and strategy, and worst of all, it raises the possibility that one can move backwards in levels and attainment - even by posting new material. That is simply not a good system.
Proposed System
The Honor Roll's sole purpose was to address the issue of "a writeup is a writeup is a writeup." If someone wrote an excellent writeup, we felt they should receive some sort of "writeup bonus", thereby reducing the number of writeups they needed to level up. Unfortunately, the system we created is too dependent on other noders, wasn't very clear, wasn't that attainable, and didn't involve an absolute positive reinforcement - no backsliding allowed.
My proposed system would essentially reimplement the old system, with one additional change.
Each user would have a "writeup bonus" variable stored in the database. This number would start at 0 for all users, and can only go up. This number would simply be added to your actual number of writeups, which would then determine what level you were (along with the usual XP requirements.)
Since I'm doing the writing here, I'll use me as an example.
I've written 576 writeups (not including this one.)
By our current level system, I should be level 7, and leaning close to level 8.
However, if my writeup bonus was 304, then my total writeup # for leveling purpsoes would be 880, and I would be level 9 (just like I am today.)
"Well, alright, Kyle, that sounds pretty straightfoward, but how did you come up with 304 for your writeup bonus? How does that number change?"
Well, this is my proposed formula, but I am not adverse to another formula:
- You get one extra writeupbonus for every writeup with over 30 reputation. (For me, that's 150.)
- You get another extra writeupbonus for every writeup over 60 reputation. (19.)
- And you get a third writeupbonus for writeups over 90 reputation. (3.)
- You get an extra writeupbonus for every C! on a writeup after the 3rd one, but with a limit of 3 writeupbonuses per writeup (so only C!s 4 through 6 count for your total). (132 for me.)
I don't necessarily think this is how the final system ought to be, by the way. I think it rewards outstanding writeups at the expense of perhaps just a really solid writeup, and I'm not quite sure how to reconcile that yet. But the ultimate point here is:
- The system is very clear. If you get a lot of C!s or a good reputation (or both!), you'll move up faster.
- The system is very attainable. I got a 60% bonus using my formula - that jumped me two levels. I'm going to build a page to test some other people later and see how they do, and probably tweak the numbers to come up with a better mousetrap.
- The system is very proportional. Write good, get more rewards. It's that simple.
- The system is very positively reinforced. There's no bonus for removing writeups, no punishment for only posting average stuff, and no disincentive not to throw up a bunch of stuff and see what sticks.
Anyway, just a thought. More later.