Red pill or blue pill? Truth cannot be discovered by AI alone

Technology cannot solve all the issues of bias, prejudice, and ignorance presented by humans. And nor should it.

by Rob Corbidge

Published: 16:53, 15 September 2022

Last updated: 08:39, 16 September 2022

Efforts to use AI for such purposes as to weed out bias in news stories, to moderate immoderate social media platforms, and to fact check articles and identify "fake news" to prevent its dissemination, have increased in recent years in step with the technology to do so.

Such efforts are the other side of the same technological coin which has allowed in the first place the colossal surge of unfiltered information this age is experiencing. For it to be used to try to put the genie back in the bottle - or at least the most offensive parts of the genie - is worth a rueful smile.

The problem is with people, not with tech. The people thought what they've always thought, and behaved as they've always behaved, but now the distribution threshold has grown from just friends, family, and work colleagues, to potentially most of the planet. 

In this informational blizzard, attempts to separate the ice from the wind are of interest, and it is with that in mind that we've been trialling The Factual for a week, our interest spurred by the news that Yahoo had purchased it this month. 

The Factual sits around an algorithm that analyses more than 10,000 news articles daily to check how they measure within certain parameters - those parameters forming a definition of "informative". Importantly, according to co-founder and CEO Arjun Moorthy, the algorithm was not trained from existing data, or directed to look at any particular sites. Meaning it is working as cleanly as possible.

The four main pillars of this analysis are: 

Site quality/reputation

Author authority

Diversity of sources - are there quotes etc

Content tone/opinion

Speaking from the point of view of an experienced editor, it's a fairly good spread of criteria - although the application of such rules can have unintended consequences. For example, the fact of an author writing a great deal about a story could just mean they're obsessed beyond its wider interest. You laugh? We've seen it.

On the evidence of a week of use, The Factual provides a reasonable service. There's nothing shocking there - UK sources such as The Guardian and The London Evening Standard were represented alongside American sources in a set of articles chosen about protesting the monarchy in Britain, for example.

This brings us to a caveat though. The Factual is heavily dominated by US domestic concerns, and that does make it harder for a UK reader to discern bias, but not impossible of course. 

The stories it picked up in topics in which we have a good grounding certainly featured a reasonable spread of opinions around an identifiable centrist stance. It would have been interesting to have used it during the most tempestuous parts of the Covid situation. 

When an article was featured by The Factual, but with a lower criteria grade, then often it was apparent why - a simple unsupported assertion, using an arguably emotive term, in an article about a recent European election was an obvious issue.

Another article suffered by being an agency report published on an established news site without a byline. The fact that the article came from the Associated Press, who apply rigorous editorial standards, meant little. No author - no authority.

Penalising news articles for containing few or no quotes is an excellent criteria. With personality journalism, we are in danger of the person telling the story becoming the primary evidence of its truth.

Having a rigorous news routine, a relic of my former editorial career, there's not a great utility in a service such as The Factual. Like many, I use a broad set of sources, both domestic and foreign. If I want to know what is happening in a particular country from the perspective of the people who live there, then domestic sources, even if biased, are often more useful. That represents a truth an algorithm will struggle to identify.

It's quite possible that Yahoo, as a news aggregator, is thinking in terms of advertising. We can all appreciate how the advertising industry is keen to avoid controversy, difficult associations, or unfortunate placement. It's not a new conundrum that advertising has to sit alongside editorial, and there's no perfect solution to it. A service such as The Factual will filter out such risks.

Yet there can be no doubt that some ideas once held as indisputable are now entirely incorrect, and new ideas must be allowed to surface. Fact is more mutable than we'd care to admit, and often a lot less bland than we'd like.