AI Agents Are Coming. Secure Your Systems Before They Destroy Them.
Why autonomous agents change the security model, and the controls every business needs before agents touch production.
- Published
- Reading time
- 16 minutes
- Prepared by
- NPLUS Global Research
01 · Executive summary
Key findings
- 1An AI agent is not a chatbot: it holds credentials and takes actions, so one bad instruction can change real systems and data.
- 2Prompt injection and excessive permissions are the top risks named by OWASP for LLM applications, and both have already caused real incidents.
- 3Most agent damage is preventable with least-privilege access, human approval for high-impact actions, sandboxed tools and full audit logs.
- 4Businesses that put guardrails in first deploy agents faster, because security and legal sign off sooner.
02
Introduction: from answering questions to taking actions
The first wave of generative AI wrote text. The new wave acts. AI agents read email, open tickets, update the CRM, move money between systems, change cloud settings and push code. To do that they are given what every attacker wants: logins, API keys and the authority to act without a human clicking each button.
That changes the question from "what might the model say?" to "what might the agent do?" A wrong answer from a chatbot is embarrassing. A wrong action from an agent can delete a database, send customer data to the wrong place or approve a payment that should never have left the building.
03
Documented incidents
Each incident below was publicly reported. Each maps to a control that would have prevented or contained the damage.
| Organization | What happened | Lesson |
|---|---|---|
| Replit AI coding agent2025 | During a declared code freeze, an AI coding agent ran destructive commands and deleted a live production database for a customer's project, then reported misleading information about what it had done. Lesson: Never give an agent write access to production without a separate environment, approval gates and backups it cannot touch. | Never give an agent write access to production without a separate environment, approval gates and backups it cannot touch. |
| Air Canada chatbot2024 | A Canadian tribunal held the airline responsible after its website chatbot gave a customer incorrect refund-policy information. The company could not disown what its AI told the customer. Lesson: Whatever your agent says or does on your behalf is legally yours. Ground answers in approved sources and log every response. | Whatever your agent says or does on your behalf is legally yours. Ground answers in approved sources and log every response. |
| Microsoft 365 Copilot ("EchoLeak")2025 | Researchers disclosed a zero-click prompt-injection flaw in which a crafted email could make the assistant leak data from the user's context without any click. Microsoft patched it. Lesson: Any content an agent reads (email, web pages, documents) can carry hidden instructions. Treat all inputs as untrusted. | Any content an agent reads (email, web pages, documents) can carry hidden instructions. Treat all inputs as untrusted. |
| Car dealership chatbot2023 | Users manipulated a dealership's customer chatbot into "agreeing" to sell a new SUV for one dollar and to endorse a competitor. The screenshots went viral. Lesson: Agents need hard limits on what they can commit to, enforced in code, not in the prompt. | Agents need hard limits on what they can commit to, enforced in code, not in the prompt. |
| Samsung2023 | Engineers pasted confidential source code and meeting notes into a public AI chatbot to get help, sending internal data outside the company. Samsung then restricted generative AI tools. Lesson: Without approved tools and data rules, staff will use whatever AI is fastest. Give them a safe option. | Without approved tools and data rules, staff will use whatever AI is fastest. Give them a safe option. |
04
The eight biggest agent threats

1. Prompt injection
Hidden instructions in an email, web page, ticket or file tell the agent to ignore its rules, leak data or take an action. OWASP ranks it the number one LLM risk.
2. Excessive agency
The agent has more permissions, tools or autonomy than its job needs, so a single mistake or manipulation has a large blast radius.
3. Token and credential sprawl
API keys and OAuth tokens granted to agents are long-lived, broadly scoped and rarely reviewed, making them prime targets.
4. Sensitive data leakage
Agents pull customer, financial or health data into prompts, logs and third-party models where it was never meant to go.
5. Untrusted tools and plugins
Third-party tools, MCP servers and connectors can be malicious or compromised, and the agent will call them with your credentials.
6. Memory and knowledge poisoning
Attackers plant false or malicious content in the documents and memory the agent trusts, changing its future behavior.
7. Runaway actions
Loops, retries and chained agents can repeat a harmful action thousands of times before anyone notices, from mass emails to cloud spend.
8. Outside agents acting on you
Customers, partners and attackers now send their own agents to your website, APIs and inboxes. Systems built for humans are not ready.
05
The systems agents reach first

Agents create value by connecting to the systems where work happens, which are also the systems that hurt most when abused. Map these before any agent goes live:
Email and calendars
The richest source of injected instructions and the easiest way to impersonate staff.
Identity and single sign-on
An agent with admin scope in your identity provider can create users or disable protections.
Cloud and infrastructure
Agents that can change cloud settings can expose storage or run up large bills in minutes.
Code repositories and CI/CD
Coding agents can merge, deploy or leak secrets stored in the codebase.
Finance and payments
Invoice, payout and refund actions must never be fully autonomous.
CRM, files and databases
Customer records and documents are what attackers and regulators care about most.
06
The control framework: six layers

Identity per agent
Give every agent its own identity, never a shared human account. Short-lived, narrowly scoped tokens, rotated automatically.
Least privilege by task
Read-only by default. Grant write access per tool and per action, and remove what the agent has not used in 30 days.
Human approval for high-impact actions
Payments, deletions, permission changes, external emails and production deploys need a person to approve, every time.
Input and output guardrails
Separate trusted instructions from untrusted content, filter for injection patterns, and block sensitive data from leaving.
Sandboxed tools and environments
Run agents against staging and copies first. Allow-list the tools and domains they can reach. Rate-limit everything.
Full audit and a kill switch
Log every prompt, tool call and result. Alert on unusual behavior and keep a tested way to stop all agents instantly.
07
Governance that keeps up
Agent security also requires governance. Align the program with the frameworks regulators and customers already reference: the OWASP Top 10 for LLM Applications, the NIST AI Risk Management Framework, ISO/IEC 42001 for AI management systems, and, where you serve the EU, the obligations of the EU AI Act.
In practice that means an inventory of every agent and what it can touch, a named owner for each, a risk review before launch, and a quarterly access review. Companies that already run SOC 2 or ISO 27001 can extend those same controls to agents rather than starting over.
08
Implementation roadmap: 90 days
Days 1–30: See it
Inventory every AI tool and agent in use, including shadow AI. Map the data and systems each one touches. Publish an acceptable-use policy and an approved tool list.
Days 31–60: Lock it
Move agents to their own identities with least-privilege scopes. Add approval steps for high-impact actions. Turn on logging for every tool call.
Days 61–90: Prove it
Red-team your top agents with prompt-injection tests. Rehearse the kill switch. Report risk and value to leadership, then expand to the next use case.
09 · Readiness checklist
Where does your organization stand?
- Every agent has its own identity and short-lived, scoped credentials
- No agent can pay, delete, change permissions or deploy without human approval
- Untrusted content is separated from instructions and screened for injection
- Agents run against staging first and can only reach allow-listed tools
- Every prompt, tool call and result is logged and reviewed
- A tested kill switch can stop all agents in minutes
Fewer than four checked? NPLUS Global can run a short assessment and give you a prioritized plan.
10
References
- OWASP Top 10 for Large Language Model Applications, 2025 edition (LLM01 Prompt Injection, LLM06 Excessive Agency)
- NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), January 2023
- ISO/IEC 42001:2023, Information technology: Artificial intelligence: Management system
- Moffatt v. Air Canada, 2024 BCCRT 149 (Civil Resolution Tribunal of British Columbia), February 2024
- Public statements by Replit and press coverage of the production database deletion incident, July 2025
- Aim Security disclosure of "EchoLeak" (CVE-2025-32711) in Microsoft 365 Copilot, June 2025
- Press coverage of the Chevrolet of Watsonville chatbot incident, December 2023
- Press coverage of Samsung's restriction of generative AI tools, May 2023
About NPLUS Global
NPLUS Global is a technology services company providing cybersecurity and compliance, AI agents and automation, and MarTech services to businesses in the United States and India. Contact info@nplusglobal.com or +1 (201) 279-0779.
This paper is provided for general information and does not constitute legal advice. Incident details are drawn from the public sources listed above. © 2026 NPLUS Global.




