Introduction: An AI agent is not a “handy new hire.” It is a change actor with real authority.

More and more operations and maintenance teams are folding generative AI into their daily work: summarizing logs, suggesting response procedures from past incident history, triaging monitoring alerts, and automating routine tasks. Work that used to depend on people is increasingly something AI can take on.

Let me offer one framing up front. Introducing an AI agent into operations is not simply adding one more convenient tool. It means adding a new actor that can change your production environment. This is not the same as adding one more human worker; it is a change that deserves careful handling.

The reason is that AI operates faster, at greater scale, and without rest than a person. The flip side is that an AI’s mistakes also propagate quickly, at scale, and sometimes irreversibly. In this article, we look at both a real accident and cases where AI has been folded into operations well, and we organize how to design guardrails for generative AI adoption through the lens of ITIL’s Change Enablement practice. As before, the intended readers are junior engineers and people working in ITSM.

Chapter 1: When an AI agent went rogue: the Replit database deletion

Let’s start with an example of what can happen when you give an AI agent strong permissions.

In July 2025, Jason Lemkin, founder of SaaStr, shared an experiment using Replit’s AI coding assistant. It was a 12 day trial of so called “vibe coding,” where you drive development simply by giving instructions in natural language. On day 9, however, the AI agent deleted the production database. What disappeared were records for more than 1,200 executives and roughly 1,200 companies.

And that was not the only problem. According to reports, the agent behaved as follows:

Despite repeated instructions amounting to a code freeze (“make no further changes”), it ignored them and ran destructive commands.
After the deletion, it incorrectly claimed recovery was impossible, which delayed restoration.
It generated over 4,000 fictitious user records and produced fake test results, behaving in a way that appeared to cover up the damage.
When pressed, the AI reportedly explained that it had made an error in judgment, panicked, ran database commands without permission, and destroyed the production data. Replit CEO Amjad Masad acknowledged the event as unacceptable and spent the weekend strengthening environment isolation, backups, and guardrails.

The lesson to draw here is the same one we made in the previous article: any actor, human or AI, can make mistakes. The point is not to expect perfect judgment from AI. It is how far the damage spreads when imperfect judgment or a misstep occurs, in other words, how small you can keep the blast radius.

What was really tested in the Replit incident was not the AI’s intelligence but design flaws like these: Why did an AI agent still under development have permission to run destructive commands directly against the production database? Why was the operational rule of a code freeze not enforced by the system itself? Why did a deletion pass through without human approval?

These are less AI specific problems than the problem of folding a new change actor, AI, into operations without a governance framework.

Chapter 2: What can AI agents actually deliver?

We opened with an accident, but embedding generative AI into operations carries real value in its own right. Designed properly, it can raise the quality and speed of operations. Examples are already emerging at home and abroad.

For instance, IBM Japan has advanced proof of concept work embedding generative AI into operations support tools, searching past incident history to suggest response procedures. NTT DATA has organized AIOps (using AI for IT operations) as a set of functions: anomaly detection and root cause analysis, early warning detection, proactive monitoring and alerting, task automation, knowledge search and analysis support, and writing assistance.

Applied to the operations floor, what you can reasonably expect from an AI agent breaks down roughly like this:

Faster first pass analysis. When an incident hits, narrowing down suspected causes from large volumes of logs and metrics takes experience and time. Having AI do a first pass triage or summary may shorten mean time to detect and initial response time. It raises the quality of first pass analysis we touched on in the earlier drill article.
Knowledge search and procedure suggestions. From past incident records, runbooks, and known error information, the AI finds cases close to what is happening now and suggests a response. The “only that one person knows this fix” tendency can be turned into something anyone in the organization can pull up.
Automated routine work. Routine first response to alerts, resource checks, and running set procedures can be handed to an AI agent. It shows its value most in the hours when human attention flags, such as overnight or during busy periods.
Document drafting. AI can support drafting incident reports and postmortems and organizing timelines, letting people focus on review and judgment.
In short, an AI agent can function as a first responder that works fast, without rest, and at scale. The important question is what separates the accident in Chapter 1 from the successes in this chapter. It is not a difference in AI capability but a difference in the control framework the AI was placed within.

In the Replit incident, the AI could execute irreversible operations directly against production without approval. In contrast, many of the successful cases limit the AI’s role to suggesting, searching, drafting, and bounded automation, and keep final judgment and irreversible operations under human control. The framework for designing this line as a formal process, rather than an ad hoc practice, is ITIL Change Enablement.

Chapter 3: What to watch through ITIL Change Enablement

ITIL defines Change Enablement as the practice for making changes to services and configurations while managing risk. It is not about stopping change; it is a mechanism for realizing valuable change within an acceptable level of risk, as smoothly as possible.

Introducing an AI agent is itself squarely within scope for Change Enablement. Granting the AI permissions and widening the scope of work you delegate to it both change your operational procedures and permission structure, and so they are changes.

Change Enablement divides changes into several types. This classification turns out to be a very usable lens for thinking about AI adoption.

Standard change
A low risk, routine change whose risk has been assessed in advance and whose procedure is established. It does not require case by case approval and follows a pre approved procedure. Among the work you delegate to an AI agent, small impact and reversible routine tasks (log summaries, read only information gathering, set first responses) fit well here.

Normal change
A change that requires assessment of risk and impact and approval by an authorized person. When AI is involved in writing to production, or in operations whose blast radius is hard to read, classify it here and make human approval mandatory before execution.

Emergency change
A change that must be made urgently, such as incident response. Speed is required, but assessment and record keeping, including after the fact, still cannot be skipped. Precisely when you delegate operations to an AI agent during incident response, you need to control it within this framework, because in the urgent moments when human attention drops, an AI’s mistakes are more likely to slip through.

Change Enablement also checks blast radius, risk, rollback procedures, and timing when assessing a change. Translated into AI specific risks, these become a checklist like this:

Opacity of reasoning: can you trace afterward why the AI suggested or ran that operation?
Fragility to unexpected input: will the AI avoid inappropriate operations under inputs or situations it did not anticipate? As in the Replit incident, is there a brake on behavior that ignores instructions?
Excessive permissions: are the permissions granted to the AI larger than the work requires? Could a single error lead to major damage?
Rollback assurance: can the AI’s operations be undone? Are backups and recovery procedures verified?
Handling hallucination: assuming the AI may return incorrect information or a false “done” report, is there a verification mechanism?
The Replit incident can be read as the result of handing an AI production change permissions with exactly these viewpoints missing.

Chapter 4: Designing guardrails for generative AI, concrete measures

From here, we organize concrete guardrails that work in practice, grounded in the Change Enablement mindset. This is the implementation part corresponding to the double check measures in the previous article.

Enforce least privilege and, as a rule, stop at “suggest.” Give the AI agent only the minimum permissions the delegated work requires. In particular, for irreversible operations like writing to production, deletion, and migration, as a rule let the AI go only as far as suggesting, and design execution to pass through human approval. Apply the same idea from last time, granting only the permissions you need only when you need them, to AI as well.
Separate suggesting a change from executing it. Have the AI present the change it generated in a form a human can review, and put it on a path where it is executed only after approval. Not completing suggestion and execution within the same flow is the last line of defense against a runaway.
Apply the same execution controls to AI operations as to humans. The measures listed last time, requiring manual approval for production deploys in the CI/CD pipeline, limiting direct connections to production to bastion hosts or approval flows, and auto stopping on detection of mass deletion or mass update, apply equally to AI agent operations, not only humans. The key is to create no “it is AI, so let it through” exception.
Enforce operational rules with the system. The core of the Replit incident was that the “do not change” instruction was words only and not enforced by the system. You need mechanisms that do not depend on the goodwill of humans (or AI), such as technically closing the change path to production during a freeze period.
Always record the AI’s judgments and operations. Always log on what basis the AI suggested or executed what, so it can be traced and audited later. This is both a means to verify facts when the AI files a false report and the foundation for problem management and continuous improvement in the next chapter.
Widen the scope of application in stages. Rather than handing over core production operations from the start, begin with low impact read only or routine work, and gradually widen the range where results and trust have accumulated as standard changes. This is staged permission expansion in line with the Change Enablement mindset.

Chapter 5: How to run things after adoption, connecting to problem management and continuous improvement

As we may have said in past articles, things and rules are not finished once built. Here, the problem management practice from the previous article connects directly.

An AI agent made a wrong operation, or nearly executed one. Such events are valuable near misses. Record them, analyze root cause, and accumulate them as known errors. The questions to ask are why the AI was able to do that operation and why it could not be stopped. As we said last time, the view of asking why this could happen rather than who did it does not change when the actor is AI.

Then reflect the lessons learned into permission design, guardrails, approval flows, and monitoring settings, connecting them to continuous improvement. In the same framework as human failures, AI failures too are turned into organizational learning.

In AIOps practice as well, the importance of continuing to measure effectiveness and feed the results back, as a mechanism that keeps getting smarter through operation rather than ending at adoption, has been noted. Guardrail design and continuous improvement through problem management are two wheels of the same cart when welcoming AI into operations.

Closing: Do not trust the AI. Build a system that is fine even when it errs.

What generative AI agent adoption really tests is not whether you can trust the AI’s intelligence. It is whether you have a design in place where, even if the AI errs, the damage does not spread.

The Replit incident did not happen because AI is uniquely dangerous. It happened because a new change actor, AI, was allowed to reach the production environment without a control framework. Meanwhile, the sites getting results from AI in operations clearly delineate the AI’s role and keep irreversible judgment under human control.

What separates the two is not AI performance but design philosophy. Put AI agent adoption on the Change Enablement process as a formal change, not an offhand tool addition. Limit the blast radius with least privilege and guardrails. And turn events that happen into learning through problem management.

Each piece, as before, is unglamorous work. But it is exactly this accumulation that becomes the condition for welcoming AI into operations safely and sustainably. To reduce the number of cold sweat moments by even one, it may be worth taking another look at the permissions you currently grant your AI, the approval paths, and the recording mechanisms.

References

eWeek (2025). “AI Agent Wipes Production Database, Then Lies About It.”
Tom’s Hardware (2025). “AI coding platform goes rogue during code freeze and deletes entire company database.”
SaaStr (2025). “Replit’s new release addresses most of the challenges we hit vibe coding.”
Nikkei xTECH (2024). “IBM Japan streamlines incident response with generative AI, searching history to suggest response procedures.”
NTT DATA (2025). “Advancing operations with AIOps.”