Update
After a discussion with Chris from wolftickets.ai, I did some more feature engineering and feature selection. The most important ones were an impact stat, redoing the ratios, and lowering the amount of features to about 30, down from 80-100.
Divide by zero
Ratios (fighter1_stat / fighter2_stat) have always been important to the algorithm, they’re a standard marker of how much more effective one fighter was in one area compared to the other fighter. The major problem is when the denominator (the bottom number) is 0 then it comes out as NaN which the algorithm can’t interpret. Some statisticians say to use a very tiny number in place of the 0 like, say, 0.000001. Others say just set the ratio value to a standard output, maybe just mark it 100 in all cases. Previously I was just setting the denominator to 1 so the ratio would just be the numerator. 10/0 becomes 10/1 = 10. None of these make me happy especially because when I average the ratios over a fighter’s careers wonky stuff happens. So I’m doing something new and it seems to be performing well: (2 * numerator) + 1. This gives the fighter who shut down their opponent in this area a healthy boost but it’s also fairly realistic. If you got 1 takedown and your opponent got 0, it’s not unreasonable to think you would’ve got 3 takedowns before your opponent got 1. Second, it works for both high numbers and low numbers. You landed 40 leg kicks to your opponents 1? Fairly reasonable to assume you’d get about 81 leg kicks before your opponent got 1. It’s far from perfect but it doesn’t mess up career averages nearly as much and feels fairly realistic.
The advantage of this is now I get to add ratios to many more stats than before. Before I was just ratio’ing stats I knew would be highly unlikely to be 0, such as strikes thrown, but now I can ratio basically everything. Nice.
Impact
Impact is a simple stat that measures how much you affect your opponent’s style: (opponent_stat_avg / opponent_stat_against_you). This is a super interesting one that I kick myself for not thinking of myself. It helps measure how good you counter wrestle wrestlers, how much you shut down a striker’s strikes, just generally how much you change your opponent’s normal game plan.
As it stands now it’s a very course measurement. In certain stats like total number of strikes it won’t take into account Imagine you’re fighting a striker and knock him out in 30 seconds. You impact his total strikes thrown dramatically but it’s not a wonder measure of how many strikes he would’ve thrown if it went 5 rounds. That’s OK because I also added lots more per_sec stats and the majority of the final features are things like strikes_per_sec_impact which is a much more holistic measurement of how you impacted their style.
Results
The results of these changes made the average accuracy and logloss improve by about 1 to 1.5%. That’s pretty great. We’re sitting at about 64% accuracy, about .645 logloss now without any odds included. That means this algorithm is just shy of being as accurate at measuring odds as the real bookie odds. Additionally, since we improved the feature quality, we can now lower the amount of features needed. This is fantastic because it means there’s less room for the algorithm to overfit the training data and give me inaccurate metrics for how it will perform over unseen data.
I haven’t done a ton of testing yet, just a few extra unit tests, so might still be some bugs here but we’ll let it play out for a few events and then go back and triage where we think something is going wrong.