Scams are evolving with technology. Generative AI is a game changer for scammers and has been blamed for the rise in increasingly sophisticated phishing campaigns. It is also enabling scammers to add credibility to their schemes by providing a quick, cheap, and easy way to create fake websites, videos, documents, and photographs. Tasks that once required technical expertise can now be accomplished in seconds using widely available generative AI tools. These fabricated materials are commonly used to support false narratives, particularly in romance and investment scams, where fake apps and websites are frequently used to reassure potential victims.
But generative AI is also quietly lowering the barrier for opportunistic fraud. One emerging example is the use of AI-generated or AI-manipulated photographs as ‘evidence’. Platforms that rely on user-submitted images to resolve disputes are particularly vulnerable. In a recent case, an Airbnb guest was falsely accused of causing significant damage after a host allegedly submitted AI-manipulated photographs as proof.
A similar incident occurred on the Lyft platform, where a customer was charged a cleaning fee after being accused of leaving a mess in the back seat of the vehicle. The driver submitted what appeared to be photographic evidence, but the image still contained a visible Google Gemini watermark, revealing that it had been generated or altered using AI. After the customer disputed the claim, Lyft refunded the fee and permanently removed the driver from the platform.
Although both cases were eventually overturned, they highlight a broader challenge: when convincing fake evidence is easy to create, trust in digital verification systems becomes much harder to maintain.
The insurance industry is also facing the same challenge. Claims have traditionally relied on photographs to document vehicle collisions, property damage, and supporting evidence. With generative AI, however, convincing images of damaged vehicles, cracked furniture, stained carpets, or altered receipts can now be created or manipulated in minutes. As a result, insurers are increasingly having to question whether the images submitted as evidence genuinely depict damage or have been digitally created or altered, highlighting the growing need for more sophisticated methods of verifying visual evidence.
Additionally, AI can also be used to remove details from photographs, such as removing objects, vehicle registration plates, surrounding vehicles, or other physical evidence, which could affect the claim. By eliminating these contextual clues, fraudsters can make an image appear more consistent with their version of events while reducing the amount of evidence available for investigators to verify what actually occurred.
Generative AI is not changing the fundamental psychology of fraud. People have always committed fraud when they are sufficiently motivated, perceive an opportunity, can justify their actions, and are capable of executing the fraud (see The Fraud Diamond). What generative AI changes is the capability required to carry it out. Rather than requiring advanced skills, anyone can now generate or manipulate convincing photographs through simple prompts or image editing tools. In effect, AI reduces the level of capability needed to create convincing fake evidence, lowering the barriers to committing opportunistic fraud by people who would not have considered it in the past.
In a world where seeing is no longer believing, organizations will need to place less trust in photographic evidence alone and place greater emphasis on independent verification, which will likely come at a cost.
In the meantime, here are some tips on what to look for if you find yourself in a similar situation.
How to spot fake photographs
- Look for inconsistencies (e.g., lighting, shadows, objects etc.)
- Compare submitted images with previous photographs, reports, or other evidence
- Request multiple photographs taken from different angles or ask for a short video instead of a single image
- Reverse image search photographs to determine whether they have appeared elsewhere online
Where appropriate, verify evidence using independent sources (e.g., witnesses, CCTV etc.)
Martina Dove is a researcher and a published author specializing in scam psychology. This incorporates persuasion and social engineering techniques, as well as errors in judgment and other individual factors that increase scam vulnerability. She combines psychological science, behavioral insight, and real-world scam examples to explore how scammers influence decision-making and how victims can better protect themselves. Beyond academic writing, she works in the tech industry as a strategic product researcher, with focus on cybersecurity, observability and AI, including the responsible use of AI.
The opinions expressed in this post belong to the individual contributors and do not necessarily reflect the views of Information Security Buzz.