By Alex White, Head of ALM Research at Redington
This presents an immediate question: is the goal to minimise the average number of guesses or to make it as likely as possible that you get the answer within six guesses? Even in a simple, closed universe like Wordle, you still have to define your goals. The goals highlighted above are difficult to calculate, so I proxied them with the number of possible answers ruled out on average and in a worst-case scenario. The highest-scoring word turns out to be the same for either (ruling out 2,100 and 1,529 words) and is footnoted .
Now, there’s far more complexity to the problem – for instance, a guess may be superior even if it ruled out fewer words if it gave more information about those it didn’t rule out, or if it allowed a superior second guess. For example, ‘arise’ scored higher than ‘arose’, but ‘arose’ allows ‘until’ as a second guess, covering all the vowels and most common consonants. So it may lead to better results after the second guess. Given that the best and 50th best words on the first metric scored similarly (2,100 and 2,038 words removed), it seems very likely that the highest-scoring first word is not actually the best one overall.
This is a perfect problem for an AI to solve, and I’ve no doubt that an AI could find the optimum way to play Wordle, if you defined optimum for it.
Nonetheless, even in my simplified proxy to a fairly simple problem, these choices have an impact – the correlation between the minimum and average number of solutions ruled out is 69% (the rank correlation is 68%). The worst guess, ‘immix’, works predictably poorly on both metrics (ruling out 960 solutions on average and 228 in a worst case). But a c.70% correlation is not 100%, and still leaves a lot of room for differences. It highlights how in a real world case like investments, subtle differences in goals can lead to bigger differences in portfolios and, therefore, in outcomes.
This becomes much more critical if AI gets involved. Suppose you wanted an AI to set an investment strategy. Ex-post, the optimum portfolio for any investor is likely to be a reckless gamble that just happened to work – such as a maximally leveraged combination of long and short options that would never have been viable ex-ante. So, clearly, you need constraints and risk tolerances.
To set those, you’d need a model. AIs are fantastic at finding efficient, creative solutions to well-defined problems. Unfortunately, that means they’re great at finding shortcuts; an AI trained to survive at Tetris for as long as possible worked out how to pause the game. If you use VaR95 as a risk metric for investments, an optimiser might go short an option with a 4.9% chance of being struck.
Any model will have holes, and finding holes in a model is probably an easier problem for an AI than finding rules for a perfect strategy. You could iterate and use AI to design the model; but again, data-mining is a lot easier than finding a grand unified theory of finance. As an example, buying companies beginning with A, covering Apple, Amazon and Alphabet, would have probably outperformed everything else over the last two decades.
Predicting that technology can’t do something is almost guaranteed to be wrong. But even in a simple guessing game with a defined solution, defining the goal precisely can matter. As AI becomes more integrated into our processes, we may initially find it most useful in helping to clarify what the goals really are. But it may be a little while yet before Skynet takes over.
|
