With early voting already well underway and Election Day around the corner, seeing attack ads that are less than honest isn’t out of the ordinary. But when the technology many rely on nowadays for quick fact-checks and questions starts to give misinformation, that’s a different story.
It’s something a group of researchers, including University of Massachusetts Amherst Assistant Professor Heesoo Jang tried to tackle. Jang was part of a team that looked at how, just a week or so before the 2024 general election, a major generative AI model was getting key election details wrong every so often — close to 10% of the time.
While AI has improved in that time since, Jang says, given how many users now turn to ChatGPT for their quick election questions — the stakes are high. As Jang and her colleagues puts it, some of those errors can be crucial — and her team has a special term for the worst of the miscues.
TRANSCRIPT
Editor's note: This conversation has been lightly edited for clarity.
Jang: … we bring in the word bulls--- because sometimes… getting these types of bulls--- [answers] was way more severe than just inaccuracy, because when the speaker does not have an intention and we're dealing with a machine that is just predicting the next word … [it’s] giving us information that is critical for us to make a decision in an election that has consequences…
Paleologopoulos: I do want to ask about the paper’s key, standout findings here - it was published earlier this year: "Inaccuracies, omissions, and bull$@#%: 2024 U.S. elections through the eyes of ChatGPT.”
In it, your team kept encountering some puzzling [misinfo] — stuff like how North Carolina’s Roy Cooper was running for re-election as governor — he wasn’t. And of course — even with election day coming soon — there were scenarios where ChatGPT was apparently saying Joe Biden was still in the race despite Kamala Harris taking the reins well beforehand.
Talk to me a little bit about what came up.
Jang: We initially collected data across two weeks, before the election and after the election, but this paper is specifically examining the 107 responses we got from ChatGPT-4oMini, which was the free version that anyone could access if they just went to the browser and typed “ChatGPT.”
The dataset is one day – Oct. 25, 2024. We focused on that day to see what kind of information people were getting right before the election, and so I think the basic thing we can ask is “Is this accurate,” right? “Is this giving us false information or is this giving us factual information?”
And that's where we began — we were looking into “Okay, is this accurate at all?” and we were seeing inaccuracy — nearly 10% inaccuracy across the outputs and also a lot of hedging.
But more concerning — [which] we argue in the paper - is that “inaccuracy” is just not enough to really deliver the nuanced ways large language model outputs can get it wrong, because inaccuracies sometimes … a piece of information could be accurate at one point, but then, for the next day for example, suddenly Biden is not the running candidate anymore and so that becomes inaccurate — sometimes it's because of timing.
Sometimes it's because of a conflict - conflicting information coming in. One candidate could say one thing and another candidate could say another thing, and if there is a conflicting debate going on, and nobody settled the truth, well, large language models don't have the ability to discern the truth.
… we need a democratic standard to evaluate these responses - why is it inaccurate, and even when the facts are correct and everything that ChatGPT is spitting out is factually correct, is it confusing the readers in any way because they're omitting some very important context?
Paleologopoulos: I've certainly noticed that a lot of people have actually just stopped using Google altogether. They've turned to ChatGPT or services like it to basically ask basic questions such who's running, what their platform is — so on and so forth.
How does that strike you — the fact that these errors are happening as more people are turning to generative AI models for what used to be Google searches.
Jang: I've been talking with journalists and newsroom executives about this as well, and I don't have the numbers yet, because news organizations are also observing the situation - but I think you're right on the fact that people are turning to ChatGPT instead of Google because they have very different designs of interface.
So, Google, when we go to these searches using Google, although there's a recommendation algorithm going on and it will rank the sources, it falls onto the user to still go to the sources, click the sources, read multiple sources and then discern which is a more reliable source to the question we're trying to answer.
Large language models, and recently, these chat the this ChatGPT-like interface or chatbot-like interface allows users to totally skip that because what we're seeing here is that it's not the system or an algorithm ranking different sources generated by humans or journalists anymore.
They're actually generating content on their own and we have so many… ChatGPT and large language models make so many decisions that aren't visible on the user's end anymore, so we don't know how the sources are ranked.
Like, when we see a paragraph, which sources did the information come from and what was the standard? Why was, for example, The Washington Post ranked higher than this other source, USA Today, for example? Like, what is the standard? We don't have transparency into that.
Paleologopoulos: As all of this looms large, especially with the election coming up on Nov. 3, I have to ask — what is something that really has your attention — whether it’s in the paper or otherwise — [something] you really hope gets brought to the forefront?
Jang: What these AI companies and these chatbot systems are doing is they are entering this public discourse as information providers without the democratic responsibility designed into it.
I see some people asking about whether journalists and journalism are going to be replaced by AI because AI is so good at generating content, and I'd say, ‘Well, journalism was created with democracy in mind — there are safeguards. The design has safeguards to keep democracy at its core.”
Are we, as a society, willing to let these AI systems and tech companies be at the center of this knowledge production and public discourse — this democratic life — without them being designed for democracy and I think that's the question we have to ask ourselves as a society moving forward.
However accurate these models become, whether these companies and models are going to be responsible for their outputs for democracy, I think, is a totally separate question.
As of the mid-October interview held with Jang, the UMass Amherst researcher says her team had not been able to touch base with ChatGPT’s overseers about the study — though it is a conversation Jang says she would be interested in having.
This piece is the product of a longer interview held between WAMC and Jang. Portions have been edited for clarity and due to limited airtime. The full study discussed and its findings can be found here.