Novel Harm – When the Image Isn’t Real, the Harm Still Is
Human Edge Insight Series.
Jasmine Luke | Human Edge Consulting
Why is it that when the image isn't real, the harm still is?
I explored this position not long before I finished my last few weeks in policing.
A discussion was taking place surrounding operational matters involving deepfakes. During examination of certain facts, I noticed discomfort rising within me as certain moving parts were being highlighted. I wasn’t experiencing this discomfort alone. A group of highly experienced peers, accustomed to absorbing complex and sensitive information, shared similar concerns.
Online trends move quickly, and in harmful spaces, what attracts attention can escalate when yesterday’s novelty becomes familiar. The same thought can be seen with synthetic popularity when it emerged through circulated images of tidal waves hitting a popular landmark, images of a person represented with physical anomalies and this created a hype about whether a trending piece was fabrication or realism. And nobody seemed particularly uneasy.
Was this the makings of digital desensitisation, I wonder?
What happens then, when synthetic material challenges socially moral boundaries? When targets become celebrities that have passed on, or distortions of motion-imagery depicting unspeakable violence and terror-acts. Or, even more alarming, when novel harm preys on children and creativity in AI territories introduce a sinister capability?
Deepfakes and synthetic media raise safeguarding questions that organisations cannot leave to enforcement agencies to address alone.
How do systems respond?
I have been thinking about that question again over recent weeks, as professionals from law enforcement, child protection, advocacy, prosecution and technology gathered on the Gold Coast for the Youth, Technology & Virtual Communities Conference.
Different disciplines bring different pieces of the same problem, making the conference focus on collaboration particularly timely. Some emerging risks facing children no longer sit neatly within the boundaries our safeguarding systems were originally designed to recognise.
Deepfakes and synthetic media are simply one example.
Perhaps the question for organisations is no longer whether they understand the technology. It may simply be whether safeguarding systems are ready for the harm it can create.
When certainty cannot keep pace with harm, safeguarding cannot wait for certainty.
For me, that is where the safeguarding questions begin.
The technology changed. The human impact didn't.
A fabricated image may not depict an event that occurred.
The humiliation can still occur.
The fear can still occur.
The loss of control can still occur.
The reputational consequences can still occur.
For a child who sees their own recognisable face placed into sexualised or degrading content, explaining that the image isn't real does very little to make the experience not real.
That distinction matters.
From a frontline perspective, I think there is a risk that, in focusing too heavily on determining the authenticity of content, we overlook the authenticity of the harm. Determining what is real, altered or fabricated matters, but it is only one part of the response. Where law enforcement or specialist agencies can support questions of authenticity and evidence, organisations still have an immediate role in addressing the actual or potential harm arising from the incident.
And available data increasingly confirms that the issue is no longer theoretical.
Recent guidance from Australia's eSafety Commissioner reports increasing school-related image misuse, including cases where publicly available student and staff imagery has been harvested from school websites and social media and manipulated into fabricated content. Sharing school imagery in the age of AI: risks, deepfakes, and how to protect students and staff.
Their current deepfake position statement identifies risks including sexual exploitation, extortion, intimidation, harassment, reputational damage and psychological harm. Deepfake trends and challenges – position statement.
Law enforcement interventions and action is only one conversation. There are other important conversations alongside this, and perhaps there should continue to be more.
When an incident occurs, different systems respond appropriately by asking different questions.
Law enforcement may ask:
Has an offence occurred? What evidence exists? Who is responsible?
A regulator may ask:
What obligations apply? Were appropriate protections in place? What protective measures have been embedded to prevent further harm?
An organisation may immediately need to ask:
Who has been affected? What do we communicate? What action do we take? How do we support our people and community as an organisation?
There is another question worth asking alongside all of them: What has this incident revealed about our safeguarding system?
I see this being the question that moves an organisation from incident management into organisational learning. And synthetic media makes it particularly important.
Because an organisation may have excellent policies for photography, consent, online safety, child protection, staff conduct and incident reporting — while never having tested what happens when those systems collide in one completely new form of harm. Novel harm.
I don't think organisations necessarily need another policy every time technology produces another risk.
Instead, I would encourage asking this question: Would our existing safeguarding system know what to do?
If a synthetic sexualised image of a child surfaced tomorrow:
Who receives the disclosure?
Who determines immediate safety?
Who preserves information without inadvertently redistributing harmful material?
Who communicates with the child and family?
Who considers reporting and escalation obligations?
Who manages other children potentially involved?
Who communicates internally?
Who determines whether existing policies actually operated as intended?
And after the immediate response: Who asks what the incident taught the organisation?
That last question is where prevention begins.
Collaboration cannot exist only between agencies after serious harm has occurred. It also exists inside organisations before it does.
Between safeguarding and governance.
Between leadership and frontline staff.
Between policy and practice.
Between children, families and the adults responsible for hearing them.
Between technological awareness and human judgement.
Perhaps one of the challenges emerging technology presents isn't simply learning how to identify what is fake. It is ensuring that our systems remain capable of recognising what is real:
the harm,
the vulnerability,
the organisational responsibility,
and the opportunity to prevent the next incident.
We know technology will continue to change, and safeguarding frameworks will continue adapting around it. But organisations do not need to predict every emerging risk to become better prepared for one.
Safeguarding needs systems capable of communication, reflection and coordinated action when something unfamiliar arrives. Because when the image isn't real, the harm still can be.
My cover image isn't real. The room is.
It began with something real: my workspace — before it became something synthetic.
That distinction is visible here because I've told you.
Increasingly, the more important question may be what happens when the distinction is harder to see — and whether our systems remain capable of responding to the very real human consequences that can exist on either side of it.
Because when the image isn't real, the safeguarding responsibility still is.
————————
Digital Safeguarding & Emerging Risks
Human Edge supports organisations to strengthen awareness, prevention and response capability around deepfakes, synthetic media and emerging forms of digital harm.
