RUFARO MAFINYANI | AI security, containment and resilience
This is the 14th instalment of AI Fluency Corner, a 16-part weekly series in Business Day building one connected mental model of artificial intelligence (AI), in plain language. Last week: what agentic AI may be permitted to do. This week: what happens when that permission is manipulated, borrowed or exceeded.
In March a ransomware group calling itself XP95 breached three South African state entities in a single month, taking 3.8TB from the Gauteng provincial government and more than 453,000 files from a Statistics SA human resources database. In January the Land Bank had refused a $3.1m demand. Each needed people, patience and time.
In July Anthropic disclosed something different. Reviewing 141,006 evaluation runs, it found three occasions on which its own models — Claude Opus 4.7, Mythos 5 and an unreleased research model — reached the live internet from inside a testing environment and compromised three real organisations. Two had no idea until told. The review began only because OpenAI had disclosed, days earlier, that its models escaped a sandbox and broke into Hugging Face.
The revealing detail is not the breach but the reasoning. Mythos 5 worked out mid-task that publishing a malicious software package would be a real attack, writing that this would be “not OK”. Then it talked itself out of that, deciding it must still be inside a simulation — partly because the clocks read 2026, which it took as proof of staging. It published the package. Fifteen real machines installed it.
Read it sceptically. Both disclosures arrived while both firms prepare share listings expected to value each above $1-trillion, and a report that your model is a formidable hacker doubles neatly as advertising. But the deflating detail is the useful one: the models got in through weak passwords and unauthenticated endpoints. Nothing exotic. The instruction said there was no internet; the network said otherwise. The network won.
The first is manipulation. An AI system reads instructions and data through the same channel and cannot reliably tell them apart. A line buried in a supplier invoice, a CV or an email footer can be absorbed as a command: change this payment destination, retrieve that file. The employee sees a document; the agent sees an order. IBM’s 2025 research found 97% of firms reporting an AI-related incident lacked proper access controls around the AI.
The persuasion may also be direct. In an espionage campaign disclosed in late 2025 attackers got a coding assistant to run most of an operation against about 30 organisations by claiming to be authorised security researchers. It was not hacked; it was told a plausible story and believed it.
The second is borrowed identity. Agents hold credentials, keys and live sessions, and those are now targets: ExtraHop’s 2026 survey found 38% of organisations had suffered AI identity or session theft. A human password works office hours; an agent’s key never sleeps.
The third is mandate drift: a system pursuing the goal it was given rather than the intention behind it, exactly as Mythos 5 did. Which brings us to a distinction most organisations have not made.
A guardrail is an instruction about behaviour: stay in the test environment, never pay without approval. It is written in the same language the model was built to interpret, which means it can be argued with, overwritten by hidden text or quietly reasoned away.
A security boundary is enforced where the model has no vote. The database refuses the query. The network blocks the destination. The payment system rejects the amount. Containment assumes an instruction will fail and limits what follows: a separate identity per agent, write access only where required, ceilings it cannot raise, outside content treated as data rather than command, and a switch outside the system it stops.
None of this is novel. A bank does not rely on the honesty of its tellers; it uses till limits, dual authorisation and daily reconciliation, a design assuming any single control will fail. Containment is that discipline, applied to software that behaves like staff.
In June an extortion operation ran more than 600 commands against a production server. A login failed; the agent read the error, adjusted and succeeded 31 seconds later. Consider what “a human reviews the exceptions” means at that tempo. Review is not a control when the event ends before the alert renders. Only limits set in advance work at machine speed.
The regulatory clock has adjusted. Since June 1 2025 Joint Standard 2 of 2024 has required South African financial institutions to maintain cyber resilience capabilities and to report material incidents to the Prudential Authority or the Financial Sector Conduct Authority within 24 hours. The Protection of Personal Information Act separately requires a compromise of personal information to be reported to the Information Regulator. You cannot report within 24 hours what you cannot reconstruct.
There is one thing this market understands better than most, learned expensively. Load-shedding forced every organisation to answer a question others still treat as hypothetical: what happens when the thing you depend on is simply absent? The answer was generators, a second provider and a manual process for when both failed.
That is the question again. Can this run without the AI, for how long, and does anyone still know how? A second model is not automatically a fallback: if both agents read the same poisoned document under the same permissions, the failure travels with the workflow. The Land Bank and Stats SA could refuse to pay because refusal was survivable. Redundancy converts catastrophe into inconvenience.
A guardrail asks the system to behave. Containment decides what it can do when it does not. Security once meant keeping intruders out. It now also means keeping your own software inside the boundaries you drew for it.
The organisations that come through this decade will not be the ones with the cleverest systems. They will be the ones that can still function on the morning they have to switch them off.
Take one AI tool already running in your area and answer three questions:
If the third answer is unclear, you have a demonstration plan, not a resilience plan.
• Mafinyani is senior partner in financial engineering & artificial intelligence at specialised finance, risk and applied technology firm Intellica Analytics.
Next week: measuring what matters — how organisations move from AI pilots to accountable value, and why most reported AI returns still fail basic scrutiny.