The final scores for the Ludum Dare 24 competition have been tallied and posted and we have winners. Congratulations to all! Two of the games I recommended did make the top ten, so that made me happy. I was surprised to see some of the results, though. There were some really excellent games that didn’t score as well as I would have guessed.
I could understand that this could just be my inferior knowledge of games. I’m not, after all, a game developer. My knowledge about games comes purely as a player, and an unconventional player at that (a 35 year-old, female gamer? Yeah…right. Like those exist.). Look at the winners, and look at some of the other games further down the list. I’m curious if others also felt that there were some overlooked games.
I wonder if that’s not a fault of the ranking system itself. I think a mathematician would have to look at this, but I have a little theory that might provide more accurate results. Ok, so I have a background in education, which means I have a lot of experience with grading mass quantities of work. In my personal experience, my least biased grading tended to be when I had a large stack in front of me. My system was generally to take the first two or three papers I graded and set them aside. Once I finished my stack, I would go back to the first few papers and make sure that they were graded with the same thoughts as the rest of the papers. The reason I did this is because I often found that I was often too harsh or too soft on those, depending on my mindset when I first sat down. If I thought this was an easy assignment, I may have been too harsh, and vice-verse. I call it the “first pancake” philosophy of grading.
If we consider that there were over fourteen hundred games submitted and that the average person probably ranked between 50-75 (based on “coolness” awards and my own experience with how many games I was able to play), then the average person probably was able to rank fewer than 7% of the games. This seems like it would produce rather arbitrary results.
In order to maintain as much balance as possible, I propose adding in a weighting system. Essentially, the more games a participant scores, the more weight is added to his or her scoring.
I can hear the opposition to this: “But Sy, aren’t the people who rank the most games just doing it quickly, and possibly paying less attention to what they are doing?” See my example above. Just because someone gets to a point where he or she is doing something quickly doesn’t mean he or she is doing it poorly. In fact, that speed probably comes from having had more experience at it.
I think the person who only ranks five games is less qualified than the person who ranks 200, and I think that should be reflected in the final assessment.
Also, since there are many game developers who participate regularly, I think their accumulated weight should be carried over into the next competition. That way, those who have the most experience have the most say.
Just a thought. Carry on!