GPT-4 will change the world.

Last December, I played a few chess games against ChatGPT. These always ended the same way: ChatGPT would play an accurate opening, until it forgot where its pieces were and started playing illegal moves, with full confidence of course. The truth is, GPT-3 does not know how to play chess. Playing a game against it reveals its true nature as a stochastic parrot that merely produces a believable-sounding answer from its training set. I wrote in December:

ChatGPT cannot play chess at a human level (yet). It is clearly aware of the game and able to accurately play mainline openings. But the moment the game moves out of theory, ChatGPT can no longer keep up. This shows that the language model doesn’t (yet) have any understanding of chess fundamentals, but merely repeats moves and phrases that commonly occur in a documented chess game.

ChatGPT’s confidence, combined with its “bending” of the rules of Chess, became something of a meme on the Chess side of the internet, with Reddit posts that hit the front page and YouTube videos receiving millions of views. We laughed at it. Mocked it. Used its deficiencies to justify the superiority of humans over machines.

Then, GPT-4 arrived.

I have gained rating points since December. My current Chess.com ELO rating sits at 1435, which indicates an intermediate player. While GPT-4 is marketed as a significant step up over GPT-3, I did not expect it play particularly well. So, I started a game. Here’s what happened.

I lost.

And not only did I lose, I got blown off the board, checkmated in 20 moves!

I tried to use the unusual Polish Opening (1. b4) as an anti-GPT strategy, as there are significantly fewer games played in these positions than in the popular openings. It didn’t seem to matter: GPT-4 handled the position well and took advantage of my mis