The ABC of AI use in content production from Down Under

A clearly defined experiment in AI use for content production has yielded some interesting results

by Rob Corbidge

Published: 19:11, 17 October 2024

Rapidly obvious in the world of content and the utilisation of AI systems is that focus is everything. AI is a tool, and like all tools it has applications that is useful for, and like all tools there are applications it has no use for. Fortunately, it does look like we're leaving the "opening a box of chocolates with a hammer" stage of understanding.

We're suckers for a good headline at Content Aware, and coming from the analogue/digital crossover era myself, the British newspaper tradition of word play and puns in headlines, were appropriate, is deeply ingrained in my own journalistic DNA.

Yet, we're also realists, and the pressure of the production workflow in many media operations is one we know. Indeed, Glide itself is designed and built with ease of production steps as a prime consideration.

One of the clearest descriptions of the use of AI in media production, and its successes and tribulations, landed on our virtual desk this week in the shape of Australia's ABC detailing their own experiments with using Microsoft's Copilot AI to assist in the writing of podcast titles and descriptions.

The ABC team approached the suggested use case in a highly methodical manner, and such an approach means they were able to produce the kind of clarity about the results achieved that are extremely useful for others thinking about going down the same technical path. We recommend the read.

It's a rule-of-thumb in editing that if the headline for a piece of content is difficult to write, then it's probably the content that is at fault. That is to say, if the content itself lacks focus and direction, then the apposite headline, or title, will be that much harder to pin on it.

We all know, both as creators and consumers of content just how important the "sell" on a piece of content is. As an example, it's not for nothing that a whole of industry of YouTube thumbnail creators exists. A poor image and title on a video can leave even the best actual content with a dusty play button.

What is of particular interest in the ABC experiment is exactly the amount of effort put into building a prompt that would yield the desired results. As they put it: "Producers were asked to copy and paste the prompt into their Copilot and then add in their podcast episode notes and any extra instructions for the LLM to work with. They would review the LLM’s output and, if time permitted, optionally adjust their prompt and run again."

Notably, all producers involved in the project were specialists in the field of their respective podcasts. Any errors produced by the AI would therefore be picked up by a human. Also importantly, "there was no public output" for the AI task. That is to say, "the tool would still only be generating suggestions for the producers as an assistant". 

Personally, I subscribe to the "breadmaker" school of utility. Lots of people buy breadmakers, but many go unused after the first few months' flush of doughy excitement. Any tool must work to earn its place in the kitchen, and the same rule applies to content production. If an AI system can save time and therefore money, and/or increase the creative potential of the team in which it is employed, it has earned its place.

So what of ABC's experience? Well, they discovered "colon bias", and it is not a euphemism. . The vast amount of podcast material the LLM had ingested caused it to reproduce the "Title: here are the words" style of title that is highly prevalent in the contemporary podcast universe. As the ABC team put it "despite our best efforts to stamp them out via the prompt instructions, we couldn’t stop the number of suggested titles with colons in them". 

So far, so LLM. It's all about the training data. Seemingly, you can train too hard.

Yet, and this is the point that can't be made enough, ABC found that teams using the system "clearly saw it as a useful starting point for ideas, suggestions and possible angles for episode titles and descriptions".

Augmentation of human effort. That's the name of the AI game at this present point in the development of the technology. While we rarely talk about ourselves in these comment pieces, it is also wonderfully aligned with our own approach at Glide, and therefore very pleasing to hear, that these tools are here to augment human creativity, and not to replace it.