HYPOTHETICAL BUILDERS MONTHLY

I was chatting to the placement students a few weeks ago about AI and language models. They’re clever chaps, as all good University of Bath students are, and they helped me understand the difference.

A large language model, like ChatGPT or Bard or something, is actually not intelligent at all - at least not in the way people are. It can’t reason, it can only predict. When it gives answers to questions, it’s using very smart algorithms to build sentences out of the next most probable word. It’s good at it too - that’s how it can write essays or compose sonnets, though admittedly not brilliant ones. But, it turns out, language models aren’t good at maths where some logical reasoning is needed. Today, I asked ChatGPT to set me a puzzle. It came up with this:

“If three people can paint three walls in three hours, how many walls can five people paint in five hours?”

What do you reckon? I started by rejecting the idea that there were three people with separate walls and separate paint pots. We’re not given that information, so I assumed that we don’t know anything about how they work together, just the outcome, exactly as defined by the question. There are three people with paint, there are some unpainted walls and a big clock. You go away and come back, and then there are three painted walls, three exhausted decorators, and a big clock showing that three hours have elapsed.

I think I’d work out that the team completed one wall per hour on average. So that’s the rate, one wall per three people, every hour - even if two of them spent the time chatting about football and the third one painted like Superman - one wall per hour for three people.

So it’s reasonable to imagine five people can work faster. In fact you can multiply the rate by 5/3, as there are 5/3 more members of the team. Which means in one hour, they can now complete 1 x 5/3 walls in an hour, and therefore 5 x 5/3 walls in five hours. That’s 25/3 walls…

“8.3” I typed back.

“That’s close!” said ChatGPT hopefully, “But let’s break it down… if three people can paint three walls in three hours, it means that each wall takes three hours to paint with one person.”

“Okay”

“Now if five people are working together, the number of walls they can paint in one hour remains the same (since each person takes three hours to paint one wall).

“No, ChatGPT. Haven’t you heard of ‘many hands make light work?’”

“So, in five hours, five people can paint 5 walls. Therefore the correct answer is 5 walls.”

“No, ChatGPT - unless they’re having a two hour lunch break. I mean sure they can do 5 walls. They can do 5 bricks, or no bricks, and then lounge about with a coffee and a copy of Hypothetical Builders Monthly if you like, but they can definitely paint more than five walls…”

“Good effort! If you have any more questions or if there’s anything else I can help you with….”

… and that’s what I mean. Sounds good, but wasn’t logical. In fact, language models just aren’t able to do mathematical reasoning. All they can do is predict, just as the students taught me.

I’d totally subscribe to Hypothetical Builders Monthly by the way.