Meet ChessAgine

Intro

Hey all, I'm back with something I have been working on secretly and silently, and I've finally reached the beta stage where I can open-source it and make it public! It is ChessAgine, a chess GUI powered by an Agent Agine that converts AI models like OpenAI, Claude, and Gemini into chess-aware chess buddies who have access to engine lines, chess databases and mainly a working in-memory chess context. Think of ChessAgine as Agent + Engine combination, a powerful tool that allows you to talk to the current position you are analyzing, do interactive chess puzzles with Agine from the Lichess puzzle database, and do AI-powered game review with Agine, where Agine can point out mistakes and blunders, not even that Agine uses capabilities to its best strength with Agine-powered annotations that you can generate as boilerplate for the position you are analyzing and add on top of it, which no current Chess GUIs provide. Agine takes chess training to another level. Now with Agine, you can ask for possible explanations on why the engine prefers certain variations over others when the eval is 0.0. You can listen to Agine and even ask Agine to output in your native language. One last thing, Agine is a fully open source, free Chess GUI where you bring your provider's (OpenAI, Claude, Gemini) API key, you pay for what you use directly to the provider, external features like game reviews, chess DB access, puzzles are free, and you can use the ChessAgine GUI without AI, also! I have also created a completely new protocol called Chess Context Protocol (CCP) to allow your favourite chess GUIs to support Agine in their application, giving Chess app devs the ability to convert the GUI into AI-powered Agine!

There is a lot to unpack here, so tie your seat belts and let's go for a ride :)

(Agine analyzing the position)

How it all started

Before I go into showing around chessAgine and talking about CCP, I want to talk about how it all started, the eureka moment or the big bang, or really the struggle of analyzing chess all by myself in my room. It was 1 year ago that I was analyzing some games I played online, and was trying to analyze a position where the engine was not helping me at all, all the lines were dead equal, and I was struggling to calculate which variation I should play to get some advantage, while I was playing around with some lines I tried to see if there were any master games or someone I can take inspiration from, and again nothing I was about to call my friend who is 2500 rated, but it was too late at night and it seemed I have hit the dead wall. Then I thought of something: what if I could "talk" to the chessboard? The idea sounded wild, but I knew it wasn't that "hard" after all, I'm a coder, and just like all programmers, I pulled off a "programmer move" and started playing around with ChatGPT and I give it the position FEN, and yea as we all know it hallucinated so bad it thought it was a endgame, didn't know legal moves and yea it was really a stupid thing to do. Though I was defeated, I didn't give up I felt there was definitely some way to make LLMs aware of chess, then I tried the naive approach by giving ChatGPT some engine lines and asking for the best move, and obviously, it gave me the answer. I asked why, and then it struggled and yeah, I was hit by a dead wall again. I did code up a "proof-of-concept" back then, I showed it to some friends, and they said I was "crazy" and doing all this was "useless", and back then, I sort of agreed and shut down my chessxplain project, which is the ancestor of ChessAgine. I was kind of sad I wasn't able to solve that problem, so I let it go, forgot about it, only when I faced a similar problem somewhere else...

https://youtu.be/I0_WQ5Qd9Jw?si=Rst89lPh7T04P0R4
(The original chessxplain demo, it's poc and very wrong, yet still a first step in LLMs explaining chess moves)

DojoAI

While I was a developer at ChessDojo, I was helping the ChessDojo team with some of their development problems, and one most common business problems faced, answering the same question again and again and again. At ChessDojo, students would often encounter the same problems with the training problem, and volunteers and helpers would answer them again and again. It felt like it was time to automate this, so I asked the team if I could create an AI chatbot that could answer questions on the fly and help Dojo with such common FAQ and save everyone's time. While working on this project, I mainly used Vector Database and a RAG (retrieval augmented generation) for the dataset of questions and answers it was a quite common design pattern, and at the end of the project, DojoAi was indeed smart enough to answer the question according to the context, and even the Sensei's (ChessDojo trainers) were impressed by such technology, the LLM with enough context was able to answer questions about dojo like it knew everything about the program, though it did hallucinate like all models I started thinking about that night when I wanted the chessboard to "talk", I remember I give chessxplain the engine lines, but what about full chess board context? The problem I was given up a year ago didn't sound too hard of a problem now. If an LLM with enough context from sample Q/As can generate a valid response, there is definitely a way that an AI agent can answer chess queries with enough chess context, though, in theory, it seemed obvious and true; the problem came in how to create that context.

(ChessDojo AI, answers questions about setting up Discord with the right context)

Chess Context Protocol (CCP)

After that, I started experimenting and playing around with this idea. It seemed I was getting somewhere because now there were many sophisticated AI tools like agents, tool calling. I thought if I give enough "tools" to the chess engine, then it should be able to call a Chess engine, and come up with the best moves. The only real problem was that even with the tool calling "understanding" the idea seemed impossible, because chess is a dynamic game, unlike the FAQ system used at Dojo, where there were N Q/A the AI can fetch the context from, for a given chess game or position there will be at most N legal moves and there are no context databases the AI can fetch, ok there is engines and some chessDB to give "best move" info, but there the models even though can use reasoning were not probably using their best capabilities. Days went by, and I was about to give up again, but this time I thought to think small. Rather than worrying about everything I thought I can compute small functions that determine context at a time, this included material balance, king safety, space this were easy to calculate so I thought calculating enough context at the runtime, and making the Agent follow a certain step by step approach with enough engine and external data then Agine should be able to "think" and properly communicate. The idea is quite similar to how chess engines give scores to position elements, but with CCP, the scoring is mapped to a prompt context, which is sent to the agent for a proper response. I consider the small functions as calculating "themes" and these themes are evaluated at position runtime (no depth analysis like engines, as this will make context window overload and the agent mess up). CCP is divided into two protocols: client and server. As most modern chess GUIs follow the client-server architecture, the client side is responsible for handling external resources like engines, databases, and properly scoring them and writing a valid prompt which is sent to the server, which does what I explained above, calculate N themes and also the board state which is finally injected to agent at the runtime, the prompting uses CoT(Chain of thought) to get better response back to end user. ChessAgine is an example of CCPC (Chess Context Protocol Client) as well as CCPS (Chess Context Protocol Server) via this Agine can answer much better than your GPTs, Claudes, CCP and Agine sit on top of this models, they take advantage of pure context and prompt engineering to make the model more aware of valid data which than it can resort to its training data for valid answers, these models are defiantly trained on a lot of chess material, its just we are not tapping into it, with Agine and CCP you now can.

(A very high level of CCP)

ChessAgine Demo

Now that you have some high-level understanding of what's happening behind the scenes now it's time to see Agine in action! I have recorded a YouTube video demo that goes over the ChessAgine platform and how to set up Agine for your personal use. One thing to note, as I said earlier, you need to have your own API key for base models like OpenAI, Claude or Gemini. Since this is Agine in beta, you also might encounter some hallucinations, but you would see the great difference in chess awareness from Agine vs the model itself.

https://youtu.be/1bbPl8_ZVyA?si=vMB05o-uALOqL-TH

Conclusion

Agine might raise some people's eyebrows, and people might even say it's no way compared to a chess coach and AI vs human intelligence debate, and I completely agree, ChessAgine isn't meant to be the next AI chess coach, or a tool that will replace your GM coach who has won many chess competitions. Agine is just a chess tool to better understand chess positions via understanding themes, and having an LLM to guide you to understanding the position by asking some questions and giving you an analysis in words. Of course, it will not match a GM's strength, but it will help you do puzzles in a fun way and also give you some ideas on picking a line where engines say everything is equal. Just like all AIs, Agine will make mistakes and its ok! We humans also make mistakes and learn from them. What is great about it is giving you a quick analysis so you can build upon, start an annotation on a position for you because you never were able to come up with an idea, it's there to support you and your coach, not to replace them. Technology is a tool at the end of the day, anyhow. I hope you had a great read. For all the ChessLise fans out there, it might again be integrated into the Discord app if there is enough usage!

Before I sign off, I want to thank many people who indirectly helped chessAgine development. This includes the devs of the Stockfish engine, who provided the beast to the world, ChessKit devs for their open source game review features I modified and took inspiration from, Lichess devs for their amazing API, and many more! I also want to give a big thanks to @Mattchess and @Frahde, who constantly checked my Agine project, performing beta testing and giving feedback and motivating me to push new features.

Thanks for reading!
Noob

ChessAgine CCPC (Website)
ChessAgine Discord
Source Code