AI Agents Are Visiting Your Website: Access Control Guide

AI agents are no longer only tools that businesses build for themselves. They are also becoming a new kind of visitor. A customer may ask a personal agent to compare products, find a support answer, check availability or help complete a purchase. To your website, those requests can look like automation rather than a conventional human browser session.

That makes a blanket allow-or-block rule too crude. Blocking every agent can turn away legitimate customers. Trusting every automated request can expose accounts, inventory, payments and business workflows to abuse.

The issue became timely on 4 November 2026, when Cloudflare launched new personal-agent analytics and granular access controls. On 5 November, Cloudflare announced the Trusted Agent Delegation coalition to address the harder questions of identity, reputation, accountability and user authority. The practical message for Australian businesses is simple: observe agent traffic, classify website journeys by risk, and keep authorisation in your own systems.

This article focuses on inbound customer-directed agents. If your business is building its own automation, see VaniTech's guide to AI browser automation security. For commerce architecture, read agentic commerce readiness.

Personal agents are not the same as search crawlers

Search crawlers usually collect public pages for later indexing. Training crawlers may collect content to improve a model. A personal agent acts in real time because a person asked it to do something.

Cloudflare now separates these behaviours into Search, Agent and Training policies, with individual overrides for leading personal agents. That distinction matters because each behaviour creates a different business decision:

• Search: do you want public content discoverable later?
• Training: what reuse preference does the business want to express?
• Agent: what may an automated visitor do now on a customer's behalf?

An agent reading a public FAQ is low risk. The same agent trying passwords, changing an address, redeeming loyalty value or submitting a payment is not. The policy should follow the journey and action, not only the user-agent name.

Cloudflare documents Allow as the default for Search, Agent and Training on new domains. That is a starting state, not a business policy. Existing domains also need review because inherited rules, WAF exceptions and broad bot blocks can create unintended outcomes.

Use a journey-based access matrix

Start by mapping what an agent wants to do and what could go wrong. A practical matrix for many SME websites looks like this:

This is a starting framework, not a universal rule. A public support site and a healthcare portal should not use the same thresholds. The important point is that access becomes progressively stricter as data sensitivity, customer impact and reversibility increase.

Identity helps, but identity is not permission

Traditional bot detection often relies on a user-agent string or an IP range. Both are imperfect. A string can be copied, and cloud IP ownership can be difficult to interpret.

Web Bot Auth is an emerging approach based on HTTP Message Signatures. The current IETF draft lets an agent sign a request and include a Signature-Agent URL. A server or edge provider can resolve the published keys and verify that the request was signed by the operator associated with that identity. Cloudflare also supports Web Bot Auth within its verified-bot program.

That is useful because a website can log, allow, rate-limit or block a verified agent more reliably than it can a self-declared string. But it proves only part of the trust chain.

• Agent identity: which operator signed the request?
• User identity: which customer is represented?
• Delegation: did that customer authorise this action?
• Scope: what data, amount, time window and operation were allowed?
• Accountability: who resolves a mistake, dispute or compromised session?

Cloudflare's 5 November announcement explicitly frames identity, reputation and accountability as separate problems. The Web Bot Auth document is also an active Internet-Draft, not a final RFC. Treat signed identity as a stronger input to policy—not as permission to bypass normal authentication, authorisation or fraud controls.

Protect account and transaction endpoints separately

Cloudflare says its Account Abuse Protection can show whether a personal agent made a login or sign-up request and which agent was involved. That visibility is useful, but it does not replace account security.

For authenticated and transactional journeys, combine agent-aware signals with normal controls:

• phishing-resistant MFA or passkeys where appropriate;
• risk-based login and sign-up checks instead of a single permanent allowlist;
• short-lived, scoped sessions rather than broad shared credentials;
• object-level authorisation on every private record;
• server-side price, stock, tax, eligibility and policy validation;
• idempotency keys for orders, bookings, payments and refunds;
• explicit confirmation for high-impact or irreversible actions;
• rate limits that protect the service without locking out legitimate customers;
• central logs for successful and failed authentication and business actions.

ASD's 2026 guidance recommends inventories, least privilege, limited auditable third-party access, monitoring and tested rate limits for AI services. Its system-access guidance also supports MFA, risk-based lockout and central authentication logging. Those principles apply whether the automation belongs to your business or arrives on behalf of a customer.

Keep privacy and data sharing proportionate

An agent-mediated journey can introduce another party into the path between a customer and your systems. Before exposing private data, establish what the agent receives, where it is processed, how long it is retained and how the customer can revoke access.

OAIC guidance for Australian Privacy Principle 3 says collection must be reasonably necessary and proportionate, and organisations should take a data-minimisation approach. For agent access, practical questions include:

• Can the task be completed with a product ID instead of a full customer profile?
• Does an order-status request need the entire order history?
• Can a booking agent see only available slots rather than other customers' details?
• Is sensitive information excluded from logs, prompts and analytics?
• Does the privacy notice explain material automated access and disclosure?
• Can the customer review, cancel or dispute an action?

Public content access is different from private account delegation. Do not infer permission to disclose personal information merely because an agent can reach the website or verify its operator identity.

Measure customer value and security together

Do not report agent traffic as success simply because request volume increased. Build a scorecard that connects technical signals to business outcomes:

• requests by verified, known and unknown agent;
• endpoints and journey stages used;
• allow, rate-limit, challenge and block outcomes;
• authentication failures, abuse signals and policy exceptions;
• completed enquiries, bookings, purchases or support tasks;
• human handoffs, disputes, duplicate actions and reversals;
• latency, error rate and infrastructure cost;
• customer complaints or accessibility problems caused by controls.

Cloudflare's new population and event views can provide edge-level visibility for supported configurations. Application and analytics data are still required to understand whether the represented customer achieved a useful outcome. Use correlation IDs and privacy-conscious event design so teams can trace a journey without collecting unnecessary personal information.