Yesterday I saw this tweet about using GPT chat.

OMG WTF pic.twitter.com/I2hE0e5ppq

— cts (@gf_256) December 1, 2022

I had never tried GPT chat and this got me very keen in doing so!
I quickly tried it on some of the code examples we use in our courses as examples of hard-to-find bugs or
code in need of refactoring and I was more than amazed that one code challenge that
1/100 engineers find was found ‘easily’ by GPT chat and it also provided the suggested refactor to remove it.

Today Advent Of Code started so I’m naturally curious how far I can come with minimal guidance of these code-writing Ais.

If you want to do the challenges yourself or with your Ai friend visit Advent Of Code.

Bots at my disposal

• CoPilot
• GPTchat

Let’s Go!

Day 1

Times:
Challenge_1: 15 seconds
Challenge_2: 2 minutes

Question, Input, Ai used:GPTchat

Day 1 | Challenge 1

I wrote a little instruction on where to find the question that needed to be answered, that I wanted the answer printed, and please don’t use any libraries.

Then I just pasted the whole question in full to the GPT chat.

And lo and behold, GPT produced the following program

with open("input.txt") as f:
  lines = f.read().strip().split("n")


max_calories = 0
current_calories = 0


for line in lines:
  
  if not line:
    max_calories = max(max_calories, current_calories)
    current_calories = 0
  else:
    
    current_calories += int(line)


max_calories = max(max_calories, current_calories)


print(max_calories)

I copied the code and ran it on my machine, it produced a number!…
I pasted the number into Advent Of Code as the answer and it was the correct answer!
Wow…

Day 1 | Challenge 2

On the second part of the challenge I yet again just pasted the whole page of information,
But this time I was way less precise in my instructions.

One of the features of GPT chat is that it remembers! So I wanted to use that
and act as if I was basically talking to a human.

I pasted this as my prompt




… Drumroll!

The initial answer was this.

I yet again took the code and ran it on my machine.
…
It gave me an answer and I was hopeful!
I pasted it into the website and… Wrong answer!
No…
What now ? do I just run it again and hope for a better result ?

That is not how I would have done it with a human programmer, let’s try the same here.
Advent Of Code gave me a hint that the answer was too low so I just told GPT chat the same thing.

This is pretty mindblowing, now only does the new solution work, but it also tells me
what was wrong with the other one!

I have not even understood the challenge myself, since I have not even read it.
But GPT3 chat was able to use my prompt to ‘reason about’ the previous answer and how to fix it.

Amazing.

Day one complete! See you tomorrow!

Day 2

Results:

• Challenge_1: 20 minutes
• Challenge_2: hours
• Tries: 30+

Question, Input, Ai used:GPTchat

Today it was certainly not faster to use the AI than to code oneself.
It was hard to use the challenge text as is, I tried a few times.

I did have more success when telling the AI it was a story about a programmer
that got challenged by some elves.
The main issue seems to have been mixing up what is actually supposed to be calculated.

At first, I didn’t want to understand the question myself, but it was hard to tell it what
to do without actually reading the text and understanding the challenge myself.

The mistakes the AI did, was to calculate the score for both players, and also
reversing how the scores were calculated.

When you have an ongoing conversation with ChatGPT, it seems it loses its memory after a few prompts, so I ended up restarting and reading the question myself. Then I only wrote down the relevant details of the challenge and skipped all the elf stuff.

I still was a bit too loose and ended up having to do one last correction as you can see in the images or GIF below.

Day 2 | Challenge 1

Day 2 | Challenge 2

I started with wishful thinking just adding the new requirements as is.
that did