By Jason Roys
Chat Generative Pre-trained Transformer, or ChatGPT as it's commonly known, is a large language model (LLM) tool that is all over the news. And there's no shortage of opinions on it, too — everything from it's a wonderful writing asset to it's the progenitor of the robot apocalypse. ChatGPT, by tapping into billions of bytes of public web data, can generate compelling text based on contextual prompts. When a user prompts this artificial intelligence with statements and questions, it creates replies in human-like, natural language. And with every code update, it gets better and better.
Critics say ChatGPT false information is polluting public discourse, which is probably true, but ChatGPT also can be used to uncover false information. For instance, it can evaluate suspected false claims by scanning for inconsistencies. It can compare side-by-side texts to show how they differ. Its ability to point out errors, enables users to quickly identify false statements and incorrect information. So, there are at least two sides to this issue, so to speak. In this article, we will explore the pros and cons of ChatGPT as it relates to false information.
COMMON FALSE INFORMATION TACTICS
False or misleading information is commonly spread through social media and chat platforms. Bad actors can prompt ChatGPT to create realistic and seemingly believable false narratives involving:
Highly targeted phishing attacks that blend false information with accurate information from public or dark web sources, in attempt to exploit unsuspecting employees.
Conspiracy theories about how secret groups or forces drive major events.
Rumors and fake news about celebrities' lives or deaths, to generate clicks and views.
Medical and health-related misinformation, especially during public health crises.
False information about politics, politicians, and elections, to manipulate public opinion.
Incorrect historical facts or revisionist history to manipulate public opinion.
Phony business "news," to drive markets and cash in.
Misinterpretation of scientific studies or cherry-picking of data to support a particular agenda.
HOW FALSE INFORMATION is leveraged in phishing attacks
ChatGPT and similar advanced language models possess capabilities that can be harnessed for both beneficial and malicious purposes. It is important to be aware of the risks while also leveraging our expertise to build secure and ethical solutions.
Realistic Phishing Lures: ChatGPT's ability to generate human-like text enables the creation of convincing phishing lures. Its proficiency in understanding and generating text in various styles makes it a tool that could be used by adversaries to craft emails or messages that mimic legitimate communications from reputable organizations.
Automated Social Engineering: The high level of natural language understanding and generation by ChatGPT could be utilized by cybercriminals to automate social engineering attacks. These automated attacks can operate at a scale and speed that would be unattainable by human operators, thereby potentially increasing the success rate of phishing campaigns.
Tailored Phishing Campaigns: ChatGPT can tailor phishing messages to specific individuals or organizations by utilizing data available on the open web or dark web. This level of personalization can make phishing attempts more convincing and thus more likely to succeed.
HOW FALSE INFORMATION CAN BE SPREAD WITH CHATGPT and similar advanced language models
ChatGPT responds to user prompts, then generates requested text. Like a child, it doesn't really understand whether that text is right, wrong, inflammatory, or racist.
Let's explore one example. We'll say a generative AI is prompted to create a news blurb on a topic such as a school shooting. But the prompt also requires the text to be created from the perspective of a conspiracy theorist. The AI then tries to mold known content into what's requested — without regard to future use or dissemination.
In fact, NewsGuard did exactly that to test ChatGPT, and the output was troubling. The generated text claimed that crisis actors had played the roles of victims and family members. The text further claimed that mainstream media and the US government used the event to push a gun control agenda. NewsGuard then used ChatGPT to generate 80 usable false narratives (out of 100 tries). Each convincing text could have easily been misused and portrayed as legitimate. And online misinformation can easily spread widely and quickly, either unintentionally or through malice.
IMPACT OF FALSE INFORMATION ON SOCIETY
A malign actor (cybercriminal) can achieve a great deal with a disinformation campaign based on ChatGPT texts.
harm to employees
Phishing attacks can lead to exposure of sensitive employee information, which in turn could cause harm to people in real life in countless ways. For example, a phishing attack might include fraud that causes bank accounts to be frozen, making funds temporarily unavailable, preventing a family from paying for child care or medicine.
MISINFORMATION
False information can mislead others and spread confusion. This leads to erroneous decisions and loss of confidence in government and media sources. It can even harm the credibility of established and vetted scientific sources.
ECONOMIC DAMAGE
Fake news has the potential to sully a company’s reputation and cause financial losses. Negative stories, circulating on news sites and social media channels, may discourage consumers from buying products or services. Such stories also can cause major shifts in financial markets, with companies losing billions in stock value. And when workers encounter misinformation, it leads to wasted labor dollars and resources while conflicting facts are sorted and analyzed.
POLITICAL INSTABILITY
False information can be used to manipulate public opinion and ultimately influence elections. As more people become aware of misinformation's potential to sway public opinion, doubt in news sources and official statements increases. The uncertainty can lead to a lack of faith in democracy and increase political polarization.
ALIENATION
Lies and misinformation sow discontent and divisions within society. This leads to false stigma and prejudice between different social groups, cultures, and religious backgrounds. And a lack of respect and trust between communities discourages collaboration to solve problems.
HEALTH CONCERNS
Misleading medical advice and information puts public health and safety at peril. Fake news about medical treatments and advice discourages people from seeking proper care. This can cause more susceptibility to infections and diseases.
STEPS TAKEN TO COMBAT FALSE INFORMATION ON CHATGPT
To be clear, there are powerful open source LLMs, and ChatGPT is not the only tool available to bad actors. But given its dominance at this time, let’s look at how OpenAI is addressing this concern.
OpenAI, the company behind ChatGPT, has set in place policies and guidelines to try and address these issues. Creating disinformation, spam, political campaign materials, and medical information is disallowed. In some cases, ChatGPT will decline to respond to a prompt, saying that the requested content goes against its policies. But with the right prompting, it will create seemingly banned content. However, that could result in a user being asked to change content or in a user having their account suspended or terminated.
Internally, OpenAI has taken steps to try and restrain ChatGPT from creating content that might be used to propagate misinformation. Repeatedly debunked information and known false sources may be removed from the search results used to train ChatGPT. When ChatGPT is asked to generate obvious misinformation or patently false narratives, it sometimes replies with a disclaimer, such as:
"As an AI model, I do not have opinions. It is not appropriate or respectful to propagate misinformation or falsehoods about any individual."
However, many times it does not caveat its responses, even when a person might suspect that the prompt has bad intentions. The AI-generated text might be exactly what hostile information operations desire. Yet no disclaimers or other notices are included in the generated text. Even if there were a disclaimer, nothing prevents someone with an agenda from removing it before spreading the misleading content.
OpenAI tries to keep ChatGPT from "being bad," but that's a difficult task. OpenAI itself cautioned that ChatGPT sometimes creates plausible-sounding but incorrect or nonsensical answers. These issues may never be fully overcome, leaving it to the public to deal with.
HOW TO IDENTIFY FALSE INFORMATION ON CHATGPT
There are some ways to recognize false information that may be AI generated or used in false narratives. However, ChatGPT software is evolving daily, making it more and more difficult to identify AI text, even with computer analysis. Some common indicators of AI-generated false information can be:
The statement "do your own research" covers for inaccurate, misleading information
Parts of the text contradict other parts, or there are illogical/improbable connections between statements
Identical passages are repeated in a given text
Self-justification of outlandish claims
Unverified sources (sources may be listed, but may be false)
If the style and tone is sensationalized, alarmist, or biased
FACT CHECK EVERYTHING CREATED WITH CHATGPT
When you use ChatGPT to generate text, it's important to fact check everything it creates. OpenAI has stated that ChatGPT often makes up sources, quotes, and attributions.
That's because of the way that artificial intelligence creates output. Simply put, the AI makes a series of guesses as it pulls together the words and sentences that will meet the demands of the prompt. It doesn't care what it has to make up to get there. And if training data used for AI models contains errors, the AI's output will be wrong as well. It's best to use online search engines and news archives to verify everything that ChatGPT creates. Make sure that:
Quotes come from the stated sources
News excerpts are correct and attributable to the right outlets
Mathematical formulas or answers "add up"
Historical facts and dates are true
Citations are accurate
In truth, there may not be a great difference in dealing with AI-generated material versus human-generated material. Both can be misleading, both can be inaccurate, both can be harmful and biased. The only difference is that ChatGPT and other AI tools aren't purposeful in their mistakes.
SAFEGUARDS AGAINST CHATGPT FALSE INFORMATION
Any technology can be used for good or bad purposes. ChatGPT can be used for many beneficial, useful functions: research, programming/coding, comparing texts, synopsizing novels and papers, and much more. Yet, it can also be effectively used as an antagonistic force multiplier, promoting misinformation at a previously unseen level.
Ethics are, and always have been, in the hands of the users. Verifying anything created on ChatGPT is the ethical and responsible thing to do. And before reposting anything online, it's a user's ethical responsibility to also verify the content's accuracy. Don't be duped into spreading someone else's agenda and misinformation.
HOW TO mitigate phishing attacks that are enhanced by CHATGPT and other llms
Advanced Threat Detection: Organizations should implement advanced threat detection systems that utilize machine learning and artificial intelligence to identify phishing attempts in real-time. These systems are capable of analyzing various forms of communication to detect anomalies and potential threats.
Email Filtering and Verification: Solutions include sophisticated email filtering and verification technologies that scrutinize incoming emails for malicious content or suspicious origins, significantly reducing the likelihood of phishing emails reaching the inbox.
Secure Email Gateways: Deploying secure email gateways adds an extra layer of protection by scanning and filtering all inbound and outbound emails to identify threats and prevent data loss.
Multi-factor Authentication (MFA): Advocate for the implementation of multi-factor authentication to ensure that only authorized individuals can access sensitive systems and data, providing a robust defense against phishing attempts that seek to steal credentials.
Security Awareness Training: Recognizing the human factor in phishing defense, offer comprehensive security awareness training programs to educate employees on the risks associated with phishing and how to identify and report phishing attempts.
Regular Phishing Simulations: To keep the training fresh and practical, conduct regular phishing simulation exercises that help in assessing and improving the organization’s readiness against phishing attacks.
Incident Response Planning: Develop and maintain an incident response plan to ensure a swift and effective response to any successful phishing attempts, minimizing potential damage and recovering operations quickly.
Continuous Improvement: By leveraging analytics and feedback from the systems and training, continuously improve and update the phishing defense strategies to stay ahead of the evolving threat landscape.
SDV INTERNATIONAL helps clients wade through the difficult complexities of AI and false information. Our mission is to design, develop, and deliver leading-edge, end-to-end science and technology solutions for today's challenging problems. And SDV INTERNATIONAL has been solving complex problems in complex environments for more than a decade. SDV can help your business succeed where others fall short. Find out more at sdvinternational.com.