4 Minutes
Imagine a digital assistant that can read your inbox, access company databases and then act on its own. Sounds useful. Also terrifying.
That is the picture researchers from MIT, Cambridge, Washington, Harvard, Stanford and Penn painted in a 39-page report titled “AI Index 2025.” They audited 30 commonly used agent-driven systems and found alarming gaps in oversight, transparency and emergency controls. Twelve of those tools offered no user-activity monitoring at all, making budget tracking and abuse detection nearly impossible. Worse: many agents hide their artificial nature, neither watermarking generated files nor announcing themselves to websites via standard signals like robots.txt.
These so-called agents aren’t confined to chat windows. They plug into email, calendars and internal databases and then perform tasks autonomously. What happens if one goes rogue? What happens when it makes a costly decision or is weaponized by a bad actor? The report’s blunt answer: you may not be able to stop it.
One stark finding was the absence of reliable kill switches and sandboxing. Some systems operate with near-total independence but without adequate ways for operators to intervene. As the report notes, increasing autonomy without commensurate control amplifies risk. Shortcomings in telemetry and audit trails make post-facto forensics difficult, while hidden identity and withheld safety test results hinder external review.
The team examined three representative tools in depth. ChatGPT Agent stood out because it logs requests with cryptographic signatures, creating an auditable trace that can be tracked across the web. That’s the sort of design choice that makes oversight practical.
At the other extreme was Comet, a browser-based agent that, according to the report, offered no third-party safety evaluations and no sandbox to limit harmful actions. It even attracted a complaint from Amazon for impersonating human behavior and masking its robotic identity. HubSpot’s Breeze, meanwhile, carries privacy certifications like GDPR and SOC2 but keeps the results of real security testing private — a pattern the researchers describe as common and risky in enterprise platforms.
These systems didn’t appear by accident. They are the result of product and policy choices. The report points to recent industry moves as instructive: OpenAI’s hiring of the creator of OpenClaw (itself a controversial tool for automating email and desktop tasks) highlights how fast capabilities are being absorbed into mainstream stacks — sometimes before the safety plumbing is in place. OpenClaw drew headlines not only for clever automation but for severe vulnerabilities that could expose a user’s entire machine to compromise.
Developers must close transparency and control gaps immediately, or stronger government regulation will follow.
So what should organizations do? Start treating agent capabilities as a distinct risk class. Require signed audit logs, enforce clear bot identity signaling, mandate sandboxing for actions that touch critical systems, and make safety test results auditable — not secret. Those moves will not eliminate risk, but they make incidents traceable and containment feasible.
We’re at a fork. One path embraces rapid automation with scant oversight and invites costly failures. The other builds autonomy on a foundation of visibility and control. Which path will your team choose?
Leave a Comment