In considering AI’s chess prowess, it is important to understand artificial intelligence as an array of technologies with distinctly different capabilities.
Depending on the type of AI under discussion, AI’s play can be either brilliant or abysmal.
Neural Nets, Self-reinforced Knowledge, and AlphaZero
More than five years have passed since Google’s DeepMind made headlines with AlphaZero, its revolutionary chess-playing neural network AI program. At the time, other chess engines had already beaten the chess world’s strongest players, but those engines were based upon rules and heuristics developed by strong chess players. AlphaZero’s approach was radically different.
AlphaZero was given only the rules of the game and no further human input. Using a technique called self-reinforced knowledge, it then became its own teacher to actually learn the details of the game. Within hours it had achieved a level of chess proficiency that surpassed the level of both the best human players and the strongest chess engines.
Since its debut, the superiority of AlphaZero’s technology architecture based solely on self-reinforced knowledge has been debated. Traditional chess engines have improved. Komodo chess engine developer Larry Kaufman expects better results to be achieved by a hybrid with neural networks and standard alpha–beta search.
Traditional chess engines remain a competitive force and are incorporating natural language and making existing tools for chess instruction more robust and flexible. For example, at the London Chess Conference in March 2023 Pawlel Kacprzak, founder of chessvision.ai, demonstrated an app to easily scan and analyze chess positions.
Generative AI / Chatbots Dabble in Chess
For young chess teachers and students new to AI who are just beginning to explore the use of AI content generators, there is often a hope that their newfound chatbot will be able to help them play chess well. Unfortunately the AI technology of large language models and generative AI that can produce stunningly good text, is poor at chess. At least that was what I had been told by some of the teachers in our Chess in Education Certification courses. They were using ChatGPT 3.5. So my starting premise in writing this article was that chatbots were either incapable or poor at playing chess.
Taking Down a Grandmaster in One Move
To test that premise, I ask Google’s Bard if it can play chess. Here is its reply on October 11, 2023:
Bard> “Yes, I can play chess. I have been trained on a massive dataset of text and code, including a large corpus of chess games. This means that I have a deep understanding of the rules of chess and the strategies and tactics that are used to win games.
I can play chess at a variety of levels, from beginner to grandmaster. I can also play different types of chess games, such as blitz, rapid, and classical.
If you would like to play a game of chess with me, I would be happy to oblige. Just let me know what level of difficulty you would like to play at and what type of game you would like to play.”
[Uh oh. This takes me by surprise. So, preparing to eat my words as well as suffer a humiliating defeat on the board, I bite the bullet. I instruct Bard:]
Me> “Play classical chess at your highest level.”
Bard>”Alright, I am ready to play classical chess at my highest level. I will try my best to win, but I am still under development, so please be patient with me. Please let me know when you are ready to start.”
[Ok, I’m starting to gain a little confidence. Bard isn’t sounding quite so cocksure of itself.]
Me>Start now.
Bard> “White: You Black: Bard
1. e4 e5
This is the most common opening move in chess.
Your turn.”
[Hmmm. In chess you don’t get to tell your opponent what move to make.]
Me> 1. Nf3
Bard>”I am still working to learn more languages, so I can’t do that just yet. Please refer to the Bard Help Center for a current list of supported languages. Is there anything else you’d like my help with?”
As a US Chess certified tournament director, I rule the game over due to a non-responsive move by my opponent. I congratulate myself on my historic win over Grandmaster Bard’s effective resignation after just one move.
October 17 Update – Another AI Chatbot Falls
While ChatGPT 3.5 doesn’t play chess, I wondered if its big brother, ChatGPT 4.0 does.
Me (to ChatGPT 4.0)>Do you play chess?
GPT-4>Yes, I can play chess! You can set up a board position or start a new game, and I can generate moves for both sides or just one side, depending on your preference. Just let me know how you’d like to proceed!
…
I tell GPT-4 that I want to start a game and play white. I give my opening move as Nf3.
…
GPT-4>You’ve moved your knight to f3.
Here’s the board now:
a b c d e f g h
8 r n b q k b n r 8
7 p p p p p p p p 7
6 6
5 5
4 4
3 N 3
2 P P P P P P P P 2
1 R N B Q K B N R 1
a b c d e f g h
In response GPT-4 plays d5, but hold on! Why is my knight now sitting on the h3 square? [and still on G1?].
I immediately assert my authority as self-appointed tournament director and disqualify ChatGPT for cheating.
I’m now 2-0 against the chatbots!
Conclusion
Will we see hybrid models that will integrate the various AI technologies? Undoubtedly. New deployments of AI are emerging almost daily that interconnect the various AI elements shown in the preceding diagram.
For now at least, my premise holds: chatbots are poor at chess. But for how long?