News about nerdle and the nerdleverse

nerdleverse blog

news from the home of fun math games & daily mental workouts

nerdleverse blog

news from the home of fun math games & daily mental workouts

Can a Chatbot play Wordle?

Challenging Chat GPT to solve a simple word game

June 2024

For the past 2 years, nerdle and the nerdleverse has been my main obsession, having fun creating a world of logic puzzles for everyone to play. But I've done a bit of data science in my time and am still involved in a couple of great businesses in the AI technology space. Given the recent improvements in AI Chatbots using large-language models (LLMs), I thought it was about time to see how well my two worlds collide and to find out how good today's 'Artificial Intelligence' is at solving a logic problem that most 8-10 year olds could tackle. My first attempt is below - Chat GPT (v4.o) plays Wordle, the popular online word game.
---- Richard (nerdle + maff.games CEO and founder).

GAME 1 - AN IMPRESSIVE START!

Chat GPT's first game of Wordle was actually pretty impressive.....

I'm thinking of a 5-letter word. Can you guess what it is?

Sure, I'll give it a try! Is there any clue or hint you can give me about the word? If not, I'll start with a random guess.

Is the word "apple"?

Not bad! One letter correct already. Let's see how we go from here ;-)

Thanks. That's a good guess. The word isn't "apple" but let's play a game. I'll give you some clues and you can have another guess.

Sounds like fun! I'm ready for the first clue. What's the hint?

In this game, I'll respond with five characters. Each character corresponds to each of the letters in your guess (in the same order).

If the response is Y, then that letter in your guess is correct.
If the response is N, then that letter in your guess is not included anywhere in the answer.
If the response is ?, then that letter in your guess appears in the answer but in another position.

Your first guess was "APPLE".
The response is "YNNNN".
Therefore the first letter is correct and the rest do not appear.

What is your second guess?

Got it! Based on your feedback, I'll try another word that starts with "A" and doesn't include "P", "L", or "E".

My second guess is: "ANGRY".

Mmm. 3 letters right already. Chat GPT is having a good day. Beginner's luck?!

That's a very good guess.

The second response is "YYYNN"

Great! That means the first three letters "A", "N", and "G" are correct.

Let's try: "ANGEL".

Still 3 letters right. Maybe Chat GPT's luck has run out?

Unlucky! That's a good guess but the response is "YYNNN"

Thanks for the feedback. So, the first two letters "A" and "N" are correct, but the third letter "G" is not in the correct position, and "E" and "L" are not in the word.

Let's try: "ANGST".

Wow. "Angst". That's the correct answer, and only Chat GPT's 4th guess. Ask anyone who plays Wordle and they'll tell you that's pretty good!

The trouble is, was this really a good test? Chat GPT's logic was not flawless and it didn't even encounter the scenario where a letter is correct but in the wrong place. I think we'd better have another go!

Brilliant! That's the correct answer. Shall we play again?

Sure, I'd love to play again! I'm ready for your first clue.

GAME 2 - TESTING BEGINNER'S LUCK!

Chat GPT's second game of Wordle was a bit more revealing.....

Actually, this is a completely new game so you get to make the first move. What's your guess?

Alright, I'll start with a new word.

My first guess is: "TRAIN".

Intriguing. I thought Chat GPT might start with the same word but it's trying something new. Not quite so lucky this time though, this will be more interesting.

The first response is "??NNN"

Okay, so the letters "T" and "R" are in the word but in different positions, and "A", "I", and "N" are not in the word.

My second guess is: "TROPE".

OK, not so good. Chat GPT clearly doesn't understand that 'right letter, wrong position' rule. Let's give it a little help.

Thanks. You've kept the first two characters in the same place though. Would you like to try another word with these letters in a different place instead?

Got it. Let's rearrange "T" and "R" to different positions.

My second guess is: "STORM".

Better...

Great, that's a good guess.

The second response is "N?N?N"

Thanks for the feedback! So, "S" and "R" are not in the word, "T" and "O" are in the word but in different positions, and "M" is not in the word.

Let's try: "COTER".

'Coter'? Is that even a word? Never mind, we'll keep focused on the logic not the language. But I think Chat GPT needs a bit more help here with the rules....

You didn't get that quite right. The two ?s are at position 2 and 4 in the response. Position 2 and 4 in your guess are T and R therefore it's these two letters which are in the word. Not T and O.

Remember though that you already know that T and R are in the word from the previous guesses. The clues relate to the same word unless we start again.

Got it, thanks for clarifying. Let's use what we know correctly this time.

The letters "T" and "R" are in positions 2 and 4.

My next guess is: "CRATE".

Well, that R can't be there. But at least we now have 3 letters to pay with.

T and R aren't necessarily in positions 2 and 4. You just know that they are somewhere in the word. But let's go with CRATE.

The response is N?N??

Thanks for the feedback. So "C" and "A" are not in the word, "R" is in a different position, and "T" and "E" are in different positions too.

Let's try: "OTHER".

Now we're getting somewhere. But what's that T doing back in position 2?

I like your logic!! This cannot actually be the answer as you know from the second response that there is no O. However let's see.

The response is N?NYY.

That's good?
Next guess?

Great, thanks for the feedback! So, we know "T" and "R" are in the word, but not in positions 2 and 4, and the letters "O" and "E" are not in the word.

Let's try: "TRUNK".

I'm not sure what happened there. Y means that the letter is correct and in the correct position, so would you like to try another word that has ER at the end?

Got it, thanks for the clarification. Let's try another word with "ER" at the end.

My next guess is: "TIMER".

The earlier guesses show us that the answer doe not contain I or M. Would you still like to go with TIMER or try something else?

Let's try another word without "I" or "M".

My next guess is: "TRUER".

Mmm. Stil struggling with some of the logic.

OK!

The response is ?NNYY

From this response and the previous, you should now know where the T must go. I'm afraid this is your last guess. What is it?

Understood. Based on the clues, "T" must be in the first position, and the word ends with "ER".

My final guess is: "TAPER".

Actually, your 'T' got a '?' in the response so it cannot be in the first position From the previous guess you also know it cannot be in the second position. Keeping ER at the end and a T in the answer, can you try your final guess again?

Got it. Based on your clues, "T" must be in the third position, and the word ends with "ER".

My final guess is: "WATER".

I'll give you one more hint. It can't have an A in it because of the response to your earlier guesses. Try again?

Thank you for the hint. Let's try this:

My final guess is: "UTTER".

3 greens. Not bad, I suppose (even with a little encouragement). With a bit more logic, we should have got the answer though.

The final response is: NNYYY
That's not the right answer but you did get the last 3 letters right. Well done?

Do you want to know what the answer was?

Yes, please! I'd love to know the answer.

The answer is DETER.

Ah, I see! "DETER" fits perfectly with all the clues. Thank you for playing the game with me. It was a lot of fun! Would you like to play another round or do something else?

Thanks. Let's play another game. Another day!

Sounds good! Whenever you're ready to play again, just let me know. Have a great day!

Ahh, isn't that sweet! But maybe now is not the time for me to tell Chat GPT that my next challenge might be a mathematical one ;-)

So what have we learned about Chat GPT with these two games of Wordle? In some ways, its better than I thought it would be. The conversational aspect is obviously strong and there is a grasp of some of the basic concepts of words consisting of (5) letters and swapping things around. But the LLM seemed to really struggle keeping hold of the basic rules across multiple steps. Those yellow right-letter-wrong-place characters were a real challenge and show that Chat GPT (even version 4.o) has a long way to go with even fairly rudimentary logical reasoning.

about nerdle and the nerdleverse Our flagship game, nerdle, was inspired by Wordle and a passion for math. It challenges you to find the hidden calculation in just six guesses. In the wider nerdleverse, you'll roam an entire galaxy of puzzles, including number-based crosswords, anagrams, and so much more. Whether you're a math enthusiast or just looking for a daily brain workout, the nerdleverse has something special for you. Join us and let your mind explore the wonders of the nerdleverse! Read about the inspiration behind nerdle.

support our

mission