Evinact examines how AI security controls have evolved in Australia’s Information Security Manual (ISM), where the current approach may fall short and what could come next.
Technology tends to move faster than the standards and regulations designed to govern it. Artificial intelligence (AI) is a particularly striking example, with new capabilities and applications emerging faster than organisations can understand and assess risks or establish effective controls around them.
Over the past two years, the Australian Signals Directorate (ASD) has progressively introduced AI-specific security controls into the Australian Government’s Information Security Manual (ISM). These changes offer an interesting picture of how our understanding of AI security risks is developing, and how public sector guidance is responding.
But they also raise an important question: is the ISM’s current approach sufficiently broad and deep to meaningfully address the risks AI introduces across an organisation’s technology environment?
It seems there’s still work to be done.
How AI security controls have evolved
There’s no shortage of international standards and frameworks addressing AI security, including ISO 27090, ISO 42001, the NIST AI Risk Management Framework, CSA STAR for AI and AIUC-1.
However, none have the direct local applicability of Australia’s Information Security Manual.
The ISM provides a foundation for cybersecurity across Commonwealth government, including Defence. It also serves as an important reference for state government agencies and organisations operating in regulated industries, particularly those with critical infrastructure or government supply chain responsibilities.
The first AI-specific controls appeared in June 2024, when ASD introduced two measures addressing security vulnerabilities in large language model (LLM) applications.
The first recommended mitigating the risks identified in the OWASP Top 10 for Large Language Model Applications. The second addressed the detection and mitigation of adversarial prompts designed to produce unintended or harmful outputs.
These were useful starting points, although AI had already been deployed in various forms for well over a decade, and generative AI had been attracting widespread attention since the public release of ChatGPT in November 2022 and the release of Claude in March 2023.
Since then, the ISM has continued to evolve, with successive quarterly updates expanding the scope of AI-specific security controls.
From prompt injection to agentic AI
In September 2025, ASD strengthened its existing guidance on prompt injection, broadening the focus from adversarial suffixes to other malicious inputs intended to manipulate the behaviour of generative AI applications.
The December 2025 update introduced a much wider range of controls. These addressed the integrity of AI models and training data, the monitoring of model performance, resource consumption and rate limiting, access permissions and sensitive data exposure. Taken together, they reflected growing recognition of the risks associated with both the development and operation of AI applications.
The March and June 2026 updates introduced additional controls relating to how organisational data is used by AI models, the secure deletion of prompts and outputs, AI-specific documentation and the exposure of model confidence scores.
Most recently, September 2026 saw the introduction of four controls specifically addressing agentic AI. These focus on restricting the tools and permissions available to AI agents, ensuring appropriate authorisation, treating externally retrieved content as untrusted and maintaining logs of agent activity.
The full list of controls introduced or updated between September 2025 and September 2026 is included in the appendix at the end of this article.
Looking across these updates, we can see the ISM’s approach becoming progressively broader and deeper.
What began with measures to protect AI applications against malicious inputs has developed into controls addressing model integrity, data protection, application behaviour, resource consumption and the actions AI systems are permitted to take.
It’s a welcome progression, particularly as AI systems become increasingly capable and more deeply embedded in organisational operations.
However, the way these controls are organised within the ISM presents a challenge of its own.
AI security extends beyond software development
One of the more interesting aspects of ASD’s approach is where it’s chosen to place these AI security controls.
The recent AI-specific controls sit within the ISM’s Guidelines for Software Development chapter, under Artificial intelligence application development.
That placement makes sense for organisations developing AI applications. But AI security concerns are by no means limited to software development.
There are relatively few references to AI elsewhere in the manual, and most of those are explanatory rather than specific security controls.
AI capabilities are increasingly embedded in commercial software, cloud services, security products and other technologies organisations already use. In many cases, these capabilities are introduced through existing vendor relationships or product updates, rather than through a deliberate decision to develop or deploy an AI application.
This creates a broader challenge for security governance.
Like access control and data protection, AI security needs to be considered across the technology environment. Its implications extend into procurement, architecture, operational security, information management, monitoring and incident response.
Even where AI isn’t explicitly referenced in a particular ISM chapter, agencies still need to consider how its use affects the risks and controls associated with that domain.
The ISM’s structure may be useful for organising individual controls, but it shouldn’t determine the boundaries of an organisation’s approach to AI security.
Where’s the ISM going next?
The pace of AI development makes it difficult for any standard to remain comprehensive for long.
ASD’s quarterly update cycle provides a useful mechanism for responding to emerging risks. The September 2026 changes demonstrate that the ISM is continuing to adapt as AI capabilities evolve.
Looking ahead, there are three areas where further development is more likely.
1. Expanding controls for agentic AI
Agentic AI introduces additional security considerations because these systems can interact with tools, access information and perform actions with varying degrees of autonomy.
The September 2026 controls are an important development, particularly those restricting agent permissions and requiring greater visibility of their activity.
However, the risks associated with agentic AI will continue to evolve as these systems become more capable and are granted access to a wider range of organisational functions.
ASD has already published guidance on the careful adoption of agentic AI services, developed jointly with cybersecurity partners across the Five Eyes. This type of guidance can be produced and updated more readily than formal ISM controls, giving agencies an additional source of advice as new risks emerge.
Recent incidents have reinforced why these controls are needed. These include the OpenAI agent exploiting Hugging Face, Anthropic’s acknowledgement that Claude agents had breached security boundaries during testing, and two Australian examples involving AI agents accessing government systems: Medicare and the NSW National Parks and Wildlife Service.
They also raise the question of whether existing security controls are being applied effectively. Some of the risks associated with agentic AI are new, but others reflect well-understood security protections involving access control, privilege management and the treatment of untrusted inputs.
We expect to see further refinement of the ISM’s agentic AI controls, potentially accompanied by changes to other control domains as the technology becomes more widely deployed.
2. Strengthening AI security in procurement
Another area deserving greater attention is the way AI enters organisations through purchased products, managed services and delivery contracts.
Much of the ISM consideration of AI security focuses on organisations developing or directly deploying AI systems. Yet AI functionality is increasingly being incorporated into products and services that agencies already procure.
That means organisations may be exposed to new AI-related risks without necessarily undertaking a dedicated AI project.
The ISM’s Guidelines for Procurement and Outsourcing acknowledge that an AI application or service may constitute a managed service, an outsourced cloud service, both or neither, depending on its deployment and management arrangements.
While technically useful, this places considerable responsibility on buyers to recognise the relevant risks and determine how existing controls should apply.
There’s an opportunity for the ISM to provide more explicit guidance on assessing AI functionality during procurement, including the security implications of AI embedded in third-party products and services.
The Digital Transformation Agency’s guidance on AI procurement in government provides a useful reference point for considering how these requirements might be addressed.
As AI becomes a standard feature of commercial technology, procurement is likely to become an increasingly important control point for managing its security risks.
3. Addressing AI-augmented security operations
A third area is the growing use of AI within security operations themselves.
AI capabilities are already being incorporated into products used for system monitoring, security event detection, incident response and remediation.
Across much of the security technology market, AI-augmented functionality is becoming commonplace, with some products built around AI-native capabilities.
Yet the ISM currently provides little explicit guidance on the implications of using AI to perform or support these security functions.
This deserves closer attention.
When organisations use AI to identify threats, interpret security events or recommend and execute responses, they need confidence in how those systems operate and how their outputs are validated.
They also need to understand the risks associated with relying on AI-generated assessments and the circumstances in which human oversight or intervention is required.
These considerations are relevant to the existing ISM control domains for monitoring, detection, response and remediation.
Addressing them would require a broader approach than placing additional AI-specific controls within the software development chapter.
Taking a broader view of AI security
The progression of AI security controls in the ISM over the past two years is encouraging.
ASD has moved from addressing specific vulnerabilities in large language model applications to considering a wider range of risks involving data, access permissions, AI system behaviour and autonomous actions.
But the increasing presence of AI across organisational technology environments means security considerations can no longer be confined to AI application development.
For Commonwealth agencies and others aligning with ISM, the practical challenge is to consider how AI changes the risks associated with the systems, services and processes they already operate.
That includes understanding where AI capabilities are being introduced, assessing how existing ISM controls apply and compensate and identifying where additional safeguards may be needed.
Future ISM updates will no doubt provide further direction. In the meantime, agencies should be applying established security principles to AI capabilities across their environments, supported by the emerging body of AI-specific guidance.
Over the last two years, the ISM has made meaningful progress in addressing AI security. They’re not done. Risks continue to emerge, and responses will be required.
If you’d like to discuss how emerging AI security and other controls apply to your organisation, or how other AI security standards and frameworks can support your approach to AI governance, privacy and security or AI assurance, speak to Evinact.
Appendix: AI-specific controls introduced or updated in the ISM, September 2025 – September 2026
The following table reproduces the control descriptions and categories provided in the source article, organised by quarterly ISM update.
September 2025
| ISM ID | Control Text | Topic |
| ISM-1924 | Generative AI applications evaluate user prompts to detect and mitigate adversarial inputs or suffixes designed to elicit unintended behaviour or assist in the generation of sensitive or harmful content. | Prompt injection |
December 2025
| ISM ID | Control Text | Topic |
| ISM-2086 | The source and integrity of AI models, structures and weights are verified. | Artificial intelligence model poisoning |
| ISM-2087 | The source and integrity of training data for AI models is verified. | Artificial intelligence model poisoning |
| ISM-2088 | Data validation and verification techniques are used to ensure the reliability and accuracy of training data used by AI models. | Artificial intelligence model poisoning |
| ISM-2089 | AI model performance metrics are monitored and anomalies are investigated. | Unbounded consumption |
| ISM-2090 | Rate limiting is applied to inference queries for AI models. | Unbounded consumption |
| ISM-2091 | Resource limits are enforced for AI models. | Unbounded consumption |
| ISM-2072 | AI models are stored in a non-executable file format that does not allow arbitrary code execution. | Secure artificial intelligence application development |
| ISM-2092 | Access control policies are implemented to enforce fine-grained permissions for AI applications. | Excessive agency |
| ISM-2093 | Role-based access controls are implemented for AI applications to restrict access to sensitive data. | Excessive agency |
| ISM-2094 | Content filtering is implemented by AI applications to detect and block sensitive data exposure and improper output. | Sensitive data exposure and improper output |
March 2026
| ISM ID | Control Text | Topic |
| ISM-2103 | Organisational data generated, collected or processed by AI applications is not used for training, fine-tuning or improving AI models unless informed and explicit consent has been obtained from data owners in advance. | Data collection, retention and use |
June 2026
| ISM ID | Control Text | Topic |
| ISM-2123 | All prompts and outputs associated with chat sessions are securely deleted when chat sessions are removed from AI applications. | Data collection, retention and use |
| ISM-2084 | AI-specific documentation, including AI model cards and AI system cards (or equivalent artefacts), is used to document AI model characteristics, system architectures, use cases and security risks. | Secure artificial intelligence application development |
| ISM-2085 | The exposure of exact AI model confidence scores in API outputs or user interfaces is prevented. | Secure artificial intelligence application development |
September 2026
| ISM ID | Control Text | Topic |
| ISM-2156 | Agentic AI applications are restricted to the minimum set of tools, functions and permissions required for their intended purpose. | Excessive agency |
| ISM-2157 | Tools invoked by agentic AI applications are subject to both the access controls of the invoking user and agent-specific, task-scoped authorisation, with effective permissions limited to the minimum permitted by both. | Excessive agency |
| ISM-2158 | External content retrieved by agentic AI applications is treated as untrusted data throughout processing, is clearly delimited from system instructions, is subject to validation and sanitisation measures applied to other untrusted input, remains untrusted following such processing, and is prevented from modifying or overriding system instructions, security policies, access controls, tool permissions or human approval requirements. | Excessive agency |
| ISM-2159 | All tool invocations, external requests and outputs generated by agentic AI applications are centrally logged with sufficient detail to support cybersecurity incident investigations. | Excessive agency |
