Web User Agents

1. What is a web user agent

A web user agent is any software entity that interacts with websites outside the entity itself, on behalf of its user, including just to display the content of websites. In web specifications and the rest of this document, web user agents are usually referred to as just "user agents", but there are other kinds of user agents in other domains, for example "mail user agents" in the context of email. A person can use many different user agents in their day-to-day life.

The most common type of web user agent is the web browser, including in-app browsers that can follow cross-site links. However, user agents also include other tools like search engines, voice-driven assistants, and generative AI systems that present snippets or summaries of website content, or help people navigate and interact with websites.

User agent behavior is not completely defined by web standards or even by technical specifications in general. In particular, user agents choose which specifications to implement in order to best serve their users, and they implement proprietary user interfaces and other behavior around the specifications they do implement.

1.1. User agents as software components

As software entities, user agents can be parts of larger applications, and they can call libraries that implement the web platform or parts of it.

If the user of an application can distinguish the user agent inside that application from other parts of the application that act on the application’s author’s behalf ("first-party" parts of the application), then only the user agent parts, which browse "third-party" content, need to follow the user agent duties. Applications with embedded user agents should give their users clear expectations about what behavior they should expect from different parts of the app. If the distinction between third-party and first-party content is too small, the application should also follow the user agent duties for its first-party content.

Similarly a library that implements the web platform may or may not be a full user agent. Some, like SFSafariViewController and Android Custom Tabs take the responsibility of implementing the user agent duties and of making a clear distinction between the third-party content they browse and the first-party content controlled by the embedded application. These libraries are user agents.

Other libraries, like the Android, iOS, and Windows WebView libraries, are designed to support many kinds of applications. Because some of these applications only include first-party content, WebView libraries aren’t meant to be user agents on their own, and they don’t implement the user agent duties.

Ultimately, an embedding application is responsible for ensuring that it follows the user agent duties if it or part of it is a user agent. This can be nearly trivial if it only browses third-party content using a user agent library. Developers need to take more care when implementing an in-app browser using a non-user agent WebView.

2. How the ecosystem works

The structure of the web browser market helps ensure that browsers act as trustworthy agents, but it can also give them conflicts of interest. There are many browsers available for free, which allows users to switch away from a browser that violates their trust. Security and privacy researchers also actively investigate browser behavior and publicize any unexpected or user-hostile behavior, which helps users pick a browser and be confident that it’s trustworthy. These researchers benefit from the parts of browsers that are open source.

The trustworthiness of the available browsers is further aided by a culture of service to end-users among the people who develop browsers. However, we can’t just assume that culture will sustain itself: it’s important that browsers which lose that culture also lose in the marketplace, fast enough to prevent the culture-loss from spreading.

3. Duties of user agents

Each user agent serves its user (The Internet is for End Users), not any of the other constituencies. A user agent owes its user various duties, which should be established through collective discussions and embodied in the various standards that user agents implement.

3.1. Protection

It should be safe to visit a web page. Visiting a page must not by itself let the page change the user’s computer or environment, such as by installing software or accessing hardware.

Any data revealed to sites or other observers should be aligned with the user’s preferences and follow the principle of data minimization. In particular, user agents must act to limit the potential for sites to track user activity [unsanctioned-tracking].

Users may choose to share more information, such as by filling out forms or granting permissions. Even then, user agents must help users avoid deception and clearly signal when a page attempts to gain elevated access.

Access to local files or other sensitive resources must be limited to clear, intentional user actions, with appropriate warnings.

3.2. Honesty

A user agent is responsible to explain to its user what’s going on, in terms that user is likely to be able to understand. These explanations should happen at appropriate times and frequencies, so that they actually help the user understand instead of interrupting and distracting the user.

3.3. Loyalty

A user agent must serve its user’s interests over its implementer’s interests and over the interests of any other party.

A user agent can be loyal while also collecting payment for its services, as long as that payment is fair and was agreed ahead of time.

A user agent can also help its user make a credible commitment to a page in order to get that page’s services, and it’s not disloyal to maintain that commitment after getting the services.