In the last year AI has become the dominant topic of conversation in technology. There has been a lot of attention to how it changes the world of work, but the effects don’t stop there. I thought I would try a fun experiment disguised as a quiz to explore this.

Learning by trying to fool people

Inspired by a recent session at M 365 North, I thought I would take the ideas a little bit further. I volunteer for a local scout group, and it offers a chance to see the world from a very different perspective. Our young people aren’t just digital natives, they can barely believe there was a world before internet ate everything. Cub scouts are between 8 and 10 years old. Minecraft is older than them.

To test if this digital savviness translated into the ability to tell the difference between reality and a digitally enhanced version of reality, I created a quiz in PowerPoint. Each slide had but one simple question. Real or fake? We’d go through the slidedeck together one evening session, and get them to record if they thought the slide depicted something which had been digitally manipulated with generative AI. At the end I would total up the score and see how they did.

Scouting has badges which encourage participants to try new activities. Each badge has requirements which are quite broadly defined. We can sometimes be more than schools, which have a curriculum which can’t always keep pace with the feverish pace of technology. So an exercise where they used their skill and judgement to tell what is fake in a rapidly changing online world is arguably a valuable life skill.

My starting point was to make an obvious fake video to set false expectations of how convincing a fake would be. Using D Id I created very unconvincing avatar reading a script I had created by asking Bing Chat to create some text about what AI can do.

I decided that as I was a know and trusted figure to them I would fake myself to lend credibility to my message. That seems unfair, but people who wish to decieve and fool you for sinister reasons will use social engineering based around well-known respected figures. Look at how many dubious ads feature Martin Lewis, or Elon Musk.

Using D ID again, I was able to animate a still image of myself reading an script. The audio file and still image were “real” about the video was something synthesised from these. Fakery can be subtle.

Next for a bit of fun I added an obviously fake animation of a sarcophagus talking in a pleasant female Irish accent. Obviously the picture was real, but I used https://fakeyou.com/ to create a suitable voice, and animated a picture I had taken eariler in the year with D ID.

Taking the voice generator idea further, I then created an image of a polar bear hunting penguins with Bing Image Create. I then created an voiceover of an uncannily good David Attenborough explaining how the polar bear’s favourite prey was a penguin. Once I stuck a BBC logo in the corner it looked pretty convincing to somebody who didn’t pay attention in geography.

The fake voiceover was a lot of fun to play with. The quality of the fake voices is variable, so some experimentation was required to find a passably good impression of a recognisable voice.

It turned out that 8 to 10 year olds were actually pretty good at spotting the fakes. In fact, anecdotally as good as adults. So, the next step was to try some rather more abstract challenges. I picked a series of photos of real places that looked implausible, like an ultra-modern city skyline, or a fantastic castle, and generated DALL-E 3 images with a prompt based on describing the scene depicted in the photo. With a few attempts it was very easy to produce a similar and exotic looking photo. Next, I found some real pictures of pets dressed in ridiculous costumes, and generated some similar DALL-E 3 images. AI can generate really endearingly cute images of cute pets.

I then mixed the fake and real up. Once again, the wisdom of children shone through, and they were usually able spot the fake. Rather tellingly, not all of them could, and my hit rate at passing off fakes markedly improved.

I also included a few additional pictures in the style of famous artists, alongside the real versions. Needless to say, few could tell the difference (myself included) aside from the one or two astute enough to notice the “real” picture included an artist signature. My conclusion, which is hardly ground breaking, is that most people can’t tell AI generated art from human art. Given that AI generated art is essentially a resampling of human art.

I tried something similar with the written word. Using ChatGPT I created the opening scene of a story starting with “It was a dark and stormy night” . Predictably, few could tell which was the real story that opened with those famous words, and the AI generated one. The widely mocked, overly complex and flowery Victorian prose was no match for the tersely written and exciting opening sentence from ChatGPT. The conclusion from this is that most people, not just cub scouts can’t tell the difference between writing from a large language model and a human.

My final trick took some real effort to create. I got a trial account from OverDub and uploaded several hours of video of myself to allow the service to buidl a model of how I sounded. Alas, the free account allowed only a very limited vocabularly, which to a person as fond of words as myself seemed a terrible injustice, and forced me to create a rather uncharacteristically simple script. Them, using D ID, I was able to create a video of me talking starting once again with a still image. I was pretty pleased with the result even if it took a lot of effort to produce. Alas, even when I slipped this into the other samples of me talking, it was easily spotted as a fake by many.

Try for yourself

Want to see the slide deck? There’s a lot more sections than I described, and of course by now I have spoiled the surprises, but if you are still curious try it for yourself