Web of Things (WoT) Discovery

Abstract

The W3C Web of Things (WoT) is intended to enable interoperability across IoT platforms and application domains. One key mechanism for accomplishing this goal is the definition and use of metadata describing the interactions an IoT device or service makes available over the network at a suitable level of abstraction. The WoT Thing Description specification satisfies this objective.

However, in order to use a Thing its Thing Description first has to be obtained. The WoT Discovery process described in this document addresses this problem. WoT Discovery needs to support the distribution of WoT Thing Descriptions in a variety of use cases. This includes ad-hoc and engineered systems; during development and at runtime; and on both local and global networks. The process also needs to work with existing discovery mechanisms, be secure, protect private information, and be able to efficiently handle updates to WoT Thing Descriptions and the dynamic and diverse nature of the IoT ecosystem.

The WoT Discovery process is divided into two phases, Introduction, and Exploration. The Introduction phase leverages existing discovery mechanisms but does not directly expose metadata; they are simply used to discover Exploration services, which provide metadata but only after secure authentication and authorization. This document normatively defines two Exploration services, one for WoT Thing self-description with a single WoT Thing Description and a searchable WoT Thing Description Directory service for collections of Thing Descriptions. A variety of Introduction services are also described and where necessary normative definitions are given to support them.

6. Exploration Mechanisms

Editor's note : Exploration Mechanism Overview

To do: Description of supported explorations, and requirements for new exploration mechanisms.

Exploration mechanisms high-level class diagram — Figure 4 The high-level class diagram of the exploration mechanisms, depicting how Things expose TDs .

Figure 4 depicts the high-level information model for self-describing and directory services. A directory may contain TDs and at the same time provide a TD and act as a self-describing Thing. The exploration mechanisms are described in § 6.1 Self-description and § 6.2 Directory .

Ontology of TD in discovery context — Figure 5 The ontology of Thing Descriptions in the Discovery context.

Figure 5 illustrates the Discovery ontology as an extension of the Thing ontology.

The ontology includes a class for metadata that are associated with TDs stored in a directory. This class is called RegistrationInformation and described as part of the directory specification in § 6.2.1.1 Registration Information .

Moreover, the Discovery ontology defines two new Thing Description classes that may be used to model special exploratory metadata:

Issue 148 : Discovery context and namespace under w3.org/ns

The type URIs used below are tentative and subject to change.

Thing Directory

A TD which describes a Thing Description Directory instance MUST use type


ThingDirectory

from the discovery context or URI


https://www.w3.org/2021/wot/discovery#ThingDirectory

§ 8.1 Directory API Thing Description which describes the API of the Thing Description Directory is an example of this TD class.

Thing Link

A TD which describes a reference to another TD MUST use type


ThingLink

from the discovery context or URI


https://www.w3.org/2021/wot/discovery#ThingLink

. A Thing Link MUST define the referenced TD as a Link with


describedby

link relation type,


application/td+json

media type and


href

set to the target URL.

Example 2 is an example Thing Link.

Example 2 : Example of a Thing Link, referencing a remote TD

{
    "@context": [
        "http://www.w3.org/ns/td",
        "https://w3c.github.io/wot-discovery/context/discovery-context.jsonld"
    ],
    "@type": "ThingLink",
    "id": "urn:example:link",
    "links": [{
        "rel": "describedby",
        "href": "https://example.com/td.jsonld",
        "type": "application/td+json"
    }],
    "security": "nosec_sc",
    "securityDefinitions": {
        "nosec_sc": {
            "scheme": "nosec"
        }
    },
    "title": "Example TD referencing another"
}

Issue 148 : Discovery context and namespace under w3.org/ns

The context URIs are tentative and subject to change.

A Thing Link can be used in various scenarios. For example:

A self-describing Thing with limited computational resources intends to describe itself: host a minimal TD (Thing Link) locally and references a larger one with the full details hosted at a different URL, perhaps in a directory.
A self-describing Thing or proxy has a very large or dynamic description: registers a small or static TD (Thing Link) in a directory which references the actual TD hosted at the edge.
A device intends to publish an entire TD which contains private and public parts: publish one TD (Thing Link) with only the public information referencing another TD which contains the full description.

6.1 Self-description

The self-description is an exploration mechanism in which a Thing hosts its own TD and exposes it at a URL or through others means. If exposed at a URL (e.g. over HTTP or CoAP), the URL may be advertised via one of the § 5. Introduction Mechanisms . The hosted TD may also be registered inside a Thing Description Directory as prescribed in § 6.2 Directory .

The self-description using the following protocols must be according to the given specification:

HTTP

The HTTP-based self-description SHOULD be over HTTPS (HTTP Over TLS). The server SHOULD serve the requests after performing necessary authentication and authorization.

The HTTP server MUST serve the TD with a GET method. A successful response MUST have 200 (OK) status, contain application/td+json Content-Type header, and the TD in body. The server MAY provide alternative representations through server-driven content negotiation, that is by honouring the request's Accept header and responding with the supported TD serialization and equivalent Content-Type header.

The HTTP server MUST respond to HEAD requests by returning only the headers equivalent to those returned by a GET request to the same endpoint. This enables clients to retrieve HTTP headers such as the Content-Length in advance to know the size of the TD (in bytes) and decide on an efficient query strategy.

Error responses:

401 (Unauthorized): No authentication.
403 (Forbidden): Insufficient rights to the resource.

6.2 Directory

Editor's note : Directory Overview

To do: Describe mechanisms for TDs to be hosted in a searchable directory service.

6.2.1 Information Model

Editor's note : Directory Information Model

To Do: Formal definition of information contained in a directory and its organization.

As shown in Figure 4 , the Thing Description Directory can contain zero or more TDs . For every TD, the directory maintains additional metadata for bookkeeping and search purposes. These are described in § 6.2.1.1 Registration Information and § 6.2.1.3 Anonymous TD Identifiers . A TD that embeds such additional metadata as part of the interaction with the directory is called an Enriched TD .

6.2.1.1 Registration Information

The ontology of a TD in the Discovery context was introduced in Figure 5 . The


RegistrationInformation

class is associated with TDs that are stored in a directory. The following table lists the registration information attributes for use within TDs that embed or reference the Discovery context. Note that only an Enriched TD embeds the registration information. In this table, client refers to the producer or consumer of a TD and server refers to the Thing Description Directory .

Vocabulary term	Description	Client Assignment	Server Assignment	Type
`created`	Provides information when the TD instance was created inside the directory. This MAY be set by the directory and returned to consumers.	read-only	optional	`dateTime`
`modified`	Provides information when the TD instance was last modified inside the directory. This MAY be set by the directory and returned to consumers.	read-only	optional	`dateTime`
`expires`	Provides the absolute time for when the TD instance registration expires. The producer MAY set this to indicate the absolute expiry time during the registration. For servers that support expirable TDs: If `ttl` (relative expiry) is present, the server MUST ignore client assignments to `expires` and instead compute and set it internally.	optional	optional	`dateTime`
`ttl`	Time-to-live: relative amount of time in seconds from the registration time until when the TD instance registration expires. The producer MAY set this to indicate the relative expiry time during the registration. For servers that support expirable TDs: The server MUST use `ttl` to calculate the `expires` (relative expiry) value.	optional	read-only	`number`
`retrieved`	The time at which the TD was retrieved from the server. This is useful for clients that intend to process other absolute timestamps but do not have an internal clock or other means of acquiring the current time.	read-only	optional	`dateTime`

6.2.1.2 Registration Expiry

Section § 6.2.1.1 Registration Information introduces few attributes to specify and discover the expiry time of registered TDs .

Producers can set the expiry time to inform the directory and other consumers about the validity of the TD registrations. The expiry is also a useful indicator to inform the consumers about expiry of dynamic TDs , e.g. when changes to metadata such as geolocation or properties are expected to be valid for a limited period. Consumers may rely on the expiry time to know how long a retrieved TD will be valid and when they need to request a more recent one. Consumers who retrieve an expired TD may consider it as metadata of an inactive client.

For the servers, the expiry time is useful for implementing automatic removal of obsolete or accidental registrations. Servers SHOULD periodically purge TDs that are past their expiry times. Prescribing a global mandate or upper limit for the expiry time is application-specific and beyond the scope of this specification. The servers MAY mandate or set a configurable upper limit to expiry times and refuse incompliant requests. The purging by servers is particularly beneficial when interacting with clients (e.g. IoT devices) that are unable to explicitly deregister their TDs . This could be due to protocol-specific limitations, failure, destruction, or ungraceful decommissioning. Such clients should set a reasonably short expiry time and periodically extend it during the normal operation. If a client ceases to operate, a directory with purging capability will automatically remove its registration.

6.2.1.3 Anonymous TD Identifiers

The directory assigns local identifiers to Anonymous TDs to enable management and retrieval of such TDs from the directory. In situations where the server exposes an Anonymous TD (e.g. retrieval, listing, search), it MUST add the local identifier to the TD to allow local referencing. The local identifier SHOULD be a UUID Version 4, presented as a URN [ RFC4122 ]. UUID Version 4 is a random or pseudo-random number which does not carry unintended information about the host or the resource.

6.2.2 Directory Service API

The Directory APIs must use secure protocols guaranteeing System User Data authenticity and confidentiality (see [ WOT-SECURITY ]). The HTTP API MUST be exposed over HTTPS (HTTP Over TLS).

The HTTP API responses must use appropriate status codes described in this section for success and error responses. The HTTP API MUST use the Problem Details [ RFC7807 ] format to carry error details in HTTP client error (4xx) and server error (5xx) responses. This enables both machines and humans to know the high-level error class and fine-grained details.

Editor's note : WoT-specific error types

The Problem Details error type field is a URI reference which could used to map the occurred error to WoT-specific error class. There are few open issues raising the lack of WoT-specific error types: wot-discovery#44 , wot-thing-description#303 , wot-scripting-api#200 .
For now, type can be omitted which defaults to "about:blank", and title should be set to HTTP status text.

For each HTTP endpoint that responds to the GET method, the server MUST accept HEAD requests and return only the headers. This allows clients to retrieve headers such as the Content-Length without receiving the body and decide on a suitable strategy to query the information. For example, a constrained client can request only the necessary parts of an object (using an apprpriate search query) or retrieve a list of items in small subsets.

6.2.2.1 Registration

The Registration API is a RESTful HTTP API in accordance with the recommendations defined in [ RFC7231 ] and [ REST-IOT ]. The default serialization format for all request and success response bodies MUST be JSON, with JSON-LD 1.1 [ JSON-LD11 ] syntax to support extensions and semantic processing. Directories MAY accept additional representations based on request's indicated Content-Type or Content-Encoding, and provide additional representations through server-driven content negotiation.

The Registration API MUST provide create, retrieve, update, delete, and listing (CRUDL) interfaces. The operations are described below:

6.2.2.1.1 Creation

The API MUST allow registration of a TD object passed as request body. The request SHOULD contain application/td+json Content-Type header for JSON serialization of TD. The TD object is validated in accordance with § 6.2.2.1.6 Validation . Note that a TD may or may not be generated by the Thing it describes. For brownfield devices in particular a separate Discoverer process or service may be required that generates and registers a TD for a Thing on its behalf.

A TD which is identified with an id attribute MUST be handled differently with one that has no identifier ( Anonymous TD ). The create operations are elaborated below:

A TD MUST be submitted to the directory using an HTTP PUT request at /things/{id} endpoint, where id is the unique TD identifier. Upon successful processing, the server MUST respond with 201 (Created) status.
Note: If the target location corresponds to an existing TD, the request shall instead proceed as an Update operation and respond the appropriate status code (see Update section).

The create operation for TDs that have identifiers is specified as createThing action in § 8.1 Directory API Thing Description .
An Anonymous TD MUST be submitted to the directory using an HTTP POST request at /things endpoint. Upon successful processing, the server MUST respond with 201 (Created) status and a Location header containing a system-generated identifier for the TD. The scheme of the system-generated ID is described in § 6.2.1.3 Anonymous TD Identifiers .
The create operation for Anonymous TDs is specified as createAnonymousThing action in § 8.1 Directory API Thing Description .

A server that supports expirable TDs will realize such functionality as described in § 6.2.1.2 Registration Expiry . In particular, if ttl (relative expiry) is given during the creation, such servers will calculate and store the expires value.

Error responses:

400 (Bad Request): Invalid serialization or TD. This is accompanied by an appropriate response message.
401 (Unauthorized): No authentication.
403 (Forbidden): Insufficient rights to the resource.

Issue 48 : How to avoid duplicates in a directory Propose Closing

Registration of TDs using non-idempotent HTTP POST method enables creation of anonymous TDs (TDs without id attribute). The producer can distinguish between the created TDs using the unique-system generated IDs given in the response Location header.

A side-effect of this is that clients will be able to register duplicate TDs accidentally or on purpose.

Need to clarify:

What are the use cases where a duplicate TD is desired?
How can we mitigate accidental duplicates (deduplication)?

6.2.2.1.2 Retrieval

A TD MUST be retrieved from the directory using an HTTP GET request at /things/{id} endpoint, where id is the unique TD identifier. A successful response MUST have 200 (OK) status, contain application/td+json Content-Type header, and the requested TD in body.

The retrieve operation is specified as retrieveThing action in § 8.1 Directory API Thing Description .

Error responses:

404 (Not Found): TD with the given id not found.
401 (Unauthorized): No authentication.
403 (Forbidden): Insufficient rights to the resource.

The following is an example of a retrieved TD:

Example 3 : Example Enriched TD

{
    "@context": [
        "http://www.w3.org/ns/td",
        "https://w3c.github.io/wot-discovery/context/discovery-context.jsonld"
    ],
    "id": "urn:example:simple-td",
    "security": "basic_sc",
    "securityDefinitions": {
        "basic_sc": {
            "scheme": "basic"
        }
    },
    "registration": {
        "created": "2021-01-19T17:02:00Z",
        "modified": "2021-01-19T17:02:00Z"
    },
    "title": "Simple TD"
}

Issue 148 : Discovery context and namespace under w3.org/ns

The context URIs are tentative and subject to change.

This is an Enriched TD which includes the registration information such as the creation and modification time of the TD within the directory.

The example below shows a retrieved Anonymous TD that is in Enriched TD form and has local identifier urn:uuid:48951ff3-4019-4e67-b217-dbbf011873dc.

Example 4 : Example Enriched Anonymous TD

{
    "@context": [
        "http://www.w3.org/ns/td",
        "https://w3c.github.io/wot-discovery/context/discovery-context.jsonld"
    ],
    "id": "urn:uuid:48951ff3-4019-4e67-b217-dbbf011873dc",
    "security": "basic_sc",
    "securityDefinitions": {
        "basic_sc": {
            "scheme": "basic"
        }
    },
    "registration": {
        "created": "2021-01-19T17:02:00Z",
        "modified": "2021-01-19T17:02:00Z"
    },
    "title": "Anonymous TD"
}

Issue 148 : Discovery context and namespace under w3.org/ns

The context URIs are tentative and subject to change.

The following is an example of a retrieved TD that was registered with a relative expiry time of 3600 seconds (one hour). The server has calculated the absolute expiry time as one hour after the modification time.

Example 5 : Example Expirable Enriched TD

{
    "@context": [
        "http://www.w3.org/ns/td",
        "https://w3c.github.io/wot-discovery/context/discovery-context.jsonld"
    ],
    "id": "urn:example:expirable-td",
    "security": "basic_sc",
    "securityDefinitions": {
        "basic_sc": {
            "scheme": "basic"
        }
    },
    "registration": {
        "created": "2021-05-10T16:00:00Z",
        "modified": "2021-05-10T17:00:00Z",
        "expires": "2021-05-10T18:00:00Z",
        "ttl": 3600,
        "retrieved": "2021-05-10T17:35:00Z"
    },
    "title": "Expirable TD"
}

Issue 148 : Discovery context and namespace under w3.org/ns

The context URIs are tentative and subject to change.

For the sake of readability, the time values in this example are set to exact numbers. In realistic settings, time values may include fractions.

6.2.2.1.3 Update

The API MUST allow modifications to an existing TD as full replacement or partial updates. The update operations are described below:

A modified TD MUST replace an existing one when submitted using an HTTP PUT request at /things/{id} endpoint, where id is the identifier of the existing TD. The request SHOULD contain application/td+json Content-Type header for JSON serialization of TD. The TD object is validated in accordance with § 6.2.2.1.6 Validation . Upon success, the server MUST respond with 204 (No Content) status.
A server that supports expirable TDs will realize such functionality as described in § 6.2.1.2 Registration Expiry . If ttl (relative expiry) is set during the update operation, the server will calculate and set the expires (absolute expiry) value.

This operation is specified as updateThing property in § 8.1 Directory API Thing Description .

Note: If the target location does not correspond to an existing TD, the request shall instead proceed as a Create operation and respond the appropriate status code (see Create section). In other words, an HTTP PUT request acts as a create or update operation.
An existing TD MUST be partially modified when the modified parts are submitted using an HTTP PATCH request at /things/{id} endpoint, where id is the identifier of the existing TD. The partial update MUST be processed using the JSON merge patch format format described in [ RFC7396 ]. The request MUST contain application/merge-patch+json Content-Type header for JSON serialization of the merge patch document. The input MUST be in Partial TD form and conform to the original TD structure. If the input contains members that appear in the original TD, their values are replaced. If a member do not appear in the original TD, that member is added. If the member is set to null but appear in the original TD, that member is removed. Members with object values are processed recursively. After applying the modifications, the TD object is validated in accordance with § 6.2.2.1.6 Validation . Upon success, the server MUST respond with a 204 (No Content) status.
A server that supports expirable TDs will realize such functionality as described in § 6.2.1.2 Registration Expiry . During the partial update operation, if the resulting TD has ttl (relative expiry), the server will calculate and set a new expires (absolute expiry) value.

This operation is specified as partiallyUpdateThing property in § 8.1 Directory API Thing Description .

The following example is a merge patch document to update only the base and registration expires fields of a TD:
Example 6 : Example merge patch document
```
{
    "base": "http://192.168.0.2/",
    "registration": {
        "expires": "2021-01-20T17:02:00Z"
    }
}
```

Error responses:

400 (Bad Request): Invalid serialization or TD. This is accompanied by an appropriate response message.
404 (Not Found): TD with the given id not found (for PATCH only).
401 (Unauthorized): No authentication.
403 (Forbidden): Insufficient rights to the resource.

6.2.2.1.4 Deletion

A TD MUST be removed from the directory when an HTTP DELETE request is submitted at /things/{id}, where id is the identifier of the existing TD. A successful response MUST have 204 (No Content) status. The retrieve operation is specified as deleteThing property in § 8.1 Directory API Thing Description .

Error responses:

404 (Not Found): TD with the given id not found.
401 (Unauthorized): No authentication.
403 (Forbidden): Insufficient rights to the resource.

6.2.2.1.5 Listing

The listing endpoint provides different ways to query the collection of full TD objects from the directory.

In many scenarios, retrieving parts instead of full TD objects is preferred because only a subset of elements are needed (e.g. id and href of a property for all TDs) and to save networking resources. The Search API allows querying parts of TD objects; see § 6.2.2.4 Search .

The list of TDs MUST be retrieved from the directory using an HTTP GET request at /things endpoint. A successful response MUST have 200 (OK) status, contain application/ld+json Content-Type header, and an array of TDs in the body.

There may be scenarios in which clients need to retrieve the collection in small subsets of TDs. While the Search API ( § 6.2.2.4 Search ) does offer the ability to query a specific range, it may not be optimal, nor developer-friendly. The server MAY support pagination to return the collection in small subsets. The pagination must be based on the following rules:

When the limit query parameter is set to a positive integer, the server MAY respond with a subset of TDs totalling to less than or equal to the requested number.
When there are more TDs after a returned subset of the collection, the response MUST contain a next Link header [ RFC8288 ] with the URL of the next subset. The next link MUST have the same limit argument given on the initial request as well as a zero-based offset argument anchored at the beginning of the next subset. The link may be absolute or relative to directory API's base URL. Moreover, it may include additional arguments that are necessary for ordering or session management.
All paged responses MUST contain a canonical Link header [ RFC8288 ] pointing to the collection and include an etag parameter to represent the current state of the collection. The link may be absolute or relative to directory API's base URL. The etag value could be a revision number, timestamp, or UUID Version 4, set whenever the TD collection changes in a way that affects the ordering of the TDs. The clients may rely on the etag value to know whether the collection remains consistent across paginated retrieval of the collection. For example, creation or deletion of TDs or update of TD fields used for ordering may make shift the calculated paging window.
By default, the collection MUST be sorted alphanumerically by the unique identifier of TDs. The server MAY support sorting by other TD attributes using query arguments: sort_by to select a field (e.g. created ) and sort_order to choose the order (i.e. asc or desc for ascending and descending ordering). If the server does not support custom sorting, it MUST reject the request with 501 (Not Implemented) status. If sorting attributes are accepted, they MUST be added consistently to all next links.

This above specification follows a subset of Linked Data Paging [ LDP-Paging ] to allow optional pagination of the JSON-LD array. Additional parts of Linked Data Paging may be implemented for examples to honour client's query preference or to add other link relations for semantic annotation and alternative navigation links.

The following example provides a walk-through of the paginated retrieval of TDs:

Example 7 : Query all TDs with page size 10.

The client requests the list with a


limit

of 10:

GET /things?limit=10 HTTP/1.1   
Host:
tdd.example.com

Response:

HTTP/1.1 200 OK
Content-Type: application/ld+json
Link: </things?offset=10&limit=10>; rel="next"
Link: </things>; rel="canonical"; etag="v1"


[{
TD
},
...9
other
TDs...
]

The response includes two Link headers with relative URLs:


next

link to query the remaining TDs,


canonical

link referencing the full collection with


etag

value indicating the state of that collection.
The response body is an array of 10 TDs.

The presence of the


next

link implies that subsequent pages should be queried for the remaining TDs. The client requests the next page via the


next

link:

GET /things?offset=10&limit=10 HTTP/1.1   
Host:
tdd.example.com

Response:

HTTP/1.1 200 OK
Content-Type: application/ld+json
Link: </things>; rel="canonical"; etag="v1"


[{
TD
},
{
TD
}]

The server response has the remaining two TDs. There is no


next

link, because this is the last page. The


etag

value has not changed implying that the collection has remained consistent between the two requests.

The client has retrieved the whole collection, containing 12 TDs.

Alternative to a pure array of TDs as the body of the response, the server MAY send a more verbose payload allowing server-side information, like pagination information, to be put in addition to the actual data.

For pagination information the alternative format leans against the formats as described in Hydra Advanced Concepts , more concretely the Partial Collection View . Adapted to our purposes and using the members field to accomodate the array of TDs, it looks like as follows for the listing endpoint:

Example 8 : Alternative payload format including pagination information

{
	"@context": "https://w3c.github.io/wot-discovery/context/discovery-context.jsonld",
	"@type": "ThingCollection",
	"total: 12,
	"members": [
		{ TD },
		9 other TDs...
	],
	"@id": "/things?offset=0&limit=10&format=collection",
	"next": "/things?offset=10&limit=10&format=collection"
}

To tell the server which format to send, the additional query parameter ?format=array|collection can be added to the request. ?format=array is the default parameter, does not have to be provided explicitly, and yields to a server response of the pure array of TDs. ?format=collection should yield to a server response with the format as described in Example 8 .

Serializing and returning the full list of TDs may be burdensome to servers. As such, servers should serialize incrementally and utilize protocol-specific mechanisms to respond in chunks. HTTP/1.1 servers SHOULD perform chunked Transfer-Encoding [ RFC7230 ] to respond the data incrementally. Most HTTP/1.1 clients automatically process the data received with chunked transfer encoding. Memory-constrained applications which require the full list should consider processing the received data incrementally. Chunked transfer encoding is not supported in HTTP/2. HTTP/2 servers SHOULD respond the data incrementally using HTTP Frames [ RFC7540 ].

The listing operation is specified as things property in § 8.1 Directory API Thing Description .

Error responses:

401 (Unauthorized): No authentication.
403 (Forbidden): Insufficient rights to the resource.

6.2.2.1.6 Validation

The syntactic validation of TD objects before storage is RECOMMENDED to prevent common erroneous submissions. The server MAY use Thing Description JSON Schema to validate standard TD vocabulary, or a more comprehensive JSON Schema to also validate extensions.

If the server fails to validate the TD object, it MUST inform the client with necessary details to identify and resolve the errors. The validation error MUST be described as Problem Details [ RFC7807 ] with an extension field called validationErrors, set to an array of objects with field and description fields. This is necessary to represent the error in a machine-readable way.

Example 9 is an example error response with two validation errors.

Example 9 : Example validation error response

{
    "title": "Bad Request",
    "status": 400,
    "detail": "The input did not pass the JSON Schema validation",
    "instance": "/errors/30585584-cce2-4d04-8079-67b33ffdafbc",
    "validationErrors": [
        {
            "field": "(root)",
            "description": "security is required"
        },
        {
            "field": "properties.status.forms.0.href",
            "description": "Invalid type. Expected: string, given: integer"
        }
    ]
}

Editor's note : Validation error type

This example skips the type field due to the current lack of specific error types. See notes in § 6.2.2 Directory Service API .

Issue 99 : Define requirements and add section for validation

How much validation does a directory need to do?

If it only supports syntactic queries, is JSONSchema validation sufficient? Do we also use SHACL?
If we support SPARQL, do we need to do more to make sure the ontologies used are correct?
Is there a subset of TD assertions we should check?

6.2.2.2 Management

Editor's note : Management API

To do: Other administrative functions not having to do with CRUD of individual records, for example, security configuration. Also, administrator roles may expand the capabilities of administrators for management of records (for instance, the ability to delete a record they did not create).

6.2.2.3 Notification

The Notification API is to notify clients about the changes to Thing Descriptions maintained within the directory. The Notification API MUST follow the Server-Sent Events [ EVENTSOURCE ] specifications to serve events to clients at /events endpoint. In particular, the server responds to successful requests with 200 (OK) status and text/event-stream Content Type. Re-connecting clients may continue from the last event by providing the last event ID as Last-Event-ID header value. The server SHOULD provide an event ID as the id field in each event and respond to re-connecting clients by delivering all missed events.

Event Types

The server MUST produce events attributed to the lifecycle of the Thing Descriptions within the directory using


create


update

, and


delete

event types.

Event Filtering

The API enables server-side filtering of events to reduce resource consumption by delivering only the events required by clients. Client libraries may offer additional filtering capabilities on the client-side.

The server MUST support event filtering based on the event type given by the client upon subscription.

For example, given the URI Template /events{/type}:

/events/create instructs the server to only deliver events of type create
/events instructs the server to deliver all events

The clients need to subscribe separately to receive a subset of the events (e.g. only


create

and


delete

) from the server. When using HTTP/2, multiple subscriptions on the same domain (HTTP streams) get multiplexed on a single connection.

Issue 176 : Event payload filtering Stretch goal 2021-10 F2F

Event filtering based on the payload is work in progress.

Event Data

The event data MUST contain the JSON serialization of the event object. The event data object is a Partial TD or the whole TD object depending on the request:

The event data object MUST at least include the identifier of the TD created, updated, or deleted at that event in Partial TD form.
Example 10 : Creation and deletion SSE events for a TD
```
event: create
data: {"id": "urn:example:simple-td"}
id: event_1
event: delete
data: {"id": "urn:example:simple-td"}
id:
event_2
```

When


diff

query parameter is set to


true

and the event has


create

type, the server MAY return the whole TD object as event data.

Example 11 : Full creation SSE event for a TD

event: create
data: {
data:     "@context": [
data:         "https://www.w3.org/2019/wot/td/v1",
data:         "https://w3c.github.io/wot-discovery/context/discovery-context.jsonld"
data:     ],
data:     "description": "This is a new TD",
data:     "id": "urn:example:simple-td",
data:     "security": "basic_sc",
data:     "securityDefinitions": {
data:         "basic_sc": {
data:             "scheme": "basic"
data:         }
data:     },
data:     "registration": {
data:         "created": "2021-01-19T17:02:00Z",
data:         "modified": "2021-01-19T17:02:00Z"
data:     },
data:     "title": "Simple TD"
data: }
id:
event_1

Issue 148 : Discovery context and namespace under w3.org/ns

The context URIs are tentative and subject to change.

When diff query parameter is set to true and the event has update type, the server MAY inform the client about the updated parts following the JSON Merge Patch [ RFC7396 ] format. An update event data that is based on JSON Merge Patch [ RFC7396 ] MUST always include the identifier of the TD regardless of whether it is changed.
The following example shows the event triggered on update of the TD from Example 11 :
Example 12 : Full update SSE event for a TD with removed description and changed title
```
event: create
data: {
data:     "description": null,
data:     "id": "urn:example:simple-td",
data:     "title": "Updated TD"
data: }
id:
event_3
```
The diff query parameter MUST be ignored for delete events. In other words, the server does not need to include additional properties in the payload of delete events when diff is set to true.
When a server which does not support the diff query parameter is requested with such query parameter, it MUST reject the request with 501 (Not Implemented) status. This is to inform the clients about the lack of such functionality at the connection time to avoid runtime exceptions caused by missing event data attributes.

The Notification API is specified as three event affordances in § 8.1 Directory API Thing Description , namely: thingCreation, thingUpdate, and thingDeletion.

Editor's note : SSE Authorization Header

Some early SSE implementations (including HTML5 EventSource) do not allow setting custom headers in the initial HTTP request. Authorization header is required in few OAuth2 flows and passing it as a query parameter is not advised . There are polyfills for browsers and modern libraries which allow setting Authorization header.

6.2.2.4 Search

Editor's note : Search API Overview

Sub-API to search a directory, e.g. issue a query. There are different forms and levels of query possible, ~~for example,~~ including both syntactic ~~(JSONPath, XPath) vs.~~ and semantic ~~(SPARQL),~~ queries. In general, query types are optional and ~~the more advanced~~ all query types ~~may~~ do not need to be supported by all ~~directories. So this~~ directories - or at all. This API ~~will have further subsections, some of which will be optional.~~ has subsections for each query type. Search also includes a sub-API for managing listing the contents (eg returned by a query) including handling pagination, etc. ~~Note that~~ There is also one special form of query ~~will be~~ able to return ~~everything.~~ the entire contents of the directory. Results may be subject to the requestor's ~~authorization.~~ authorization, and this query form is only suitable for small directories. Larger directories (with, for instance, more than a hundred or so entries) should implement some form of query.
To discuss further: Federated queries to other TDDs, Spatial and network-limited queries, Links

The Directory has three search APIs: syntactic search with JSONPath [ JSONPATH ] or XPath [ xpath-31 ], and semantic search with SPARQL [ sparql11-overview ]. ~~The Directory MUST implement the syntactic search with JSONPath.~~ The Directory ~~SHOULD~~ MAY implement the syntactic search with XPath. The Directory MAY implement semantic search with SPARQL. In addition, in the future Directories may support syntactic search with JSONPath, but this its specification is still in a draft status. We document in an informative manner a JSONPath entry point.

6.2.2.4.1 Syntactic search: JSONPath

This section is non-normative.

To do: The JSONPath subAPI is optional and informative, so it is not appropriate to have RFC assertions here. They will have to be reworded. The JSONPath API MUST allow searching TDs using an HTTP

GET

request at


/search/jsonpath?query={query}

endpoint, where


query

is the JSONPath expression. The request MUST contain a valid JSONPath [ JSONPATH ] as searching parameter. A successful response MUST have 200 (OK) status, contain


application/json

Content-Type header, and in the body a set of complete TDs or a set of TD fragments. The syntactic search with JSONPath is specified as


searchJSONPath

action in § 8.1 Directory API Thing Description .

List of errors:

400 (Bad Request): JSONPath expression not provided or contains syntax errors.
401 (Unauthorized): No authentication.
403 (Forbidden): Insufficient rights to the resource.

Issue 56 : Reach out to JSONPath community T2TRG Propose Closing

The standardization of JSONPath expressions is in progress by an independent working group.

6.2.2.4.2 Syntactic search: XPath

The support for XPath Search API is recommended. If implemented, the XPath query API MUST allow searching TDs using an HTTP

GET

request at


/search/xpath?query={query}

endpoint, where


query

is the XPath expression. The request MUST contain a valid XPath [ xpath-31 ] as search parameter. A successful response MUST have 200 (OK) status, contain


application/json

Content-Type header, and in the body a set of complete TDs or a set of TD fragments. The syntactic search with XPath is specified as


searchXPath

action in § 8.1 Directory API Thing Description .

List of errors:

400 (Bad Request): XPath expression not provided or contains syntax errors.
401 (Unauthorized): No authentication.
403 (Forbidden): Insufficient rights to the resource.
501 (Not Implemented): XPath API not supported.

6.2.2.4.3 Semantic search: SPARQL

The support for SPARQL Search API is optional. If implemented, the SPARQL search API MUST allow searching TDs using the SPARQL protocol [ sparql11-overview ]. The SPARQL API MUST accept queries using HTTP

GET

requests at


/search/sparql?query={query}

endpoint, where


query

is the SPARQL expression. The support for SPARQL search using HTTP


POST

method at


/search/sparql

endpoint is OPTIONAL .


UPDATE

queries are out of the scope for the API. A successful response MUST have 200 (OK) status, and depending on the type of query contain by default as Content-Type header


application/ld+json

for


CONSTRUCT

and


DESCRIBE

queries or


application/json

for


SELECT

ASK

. The response body MAY contain TD fragments or a set of TDs depending on the query. The semantic search with SPARQL is specified as


searchSPARQL

action in § 8.1 Directory API Thing Description .

List of errors:

400 (Bad Request): SPARQL query not provided or contains syntax errors.
401 (Unauthorized): No authentication.
403 (Forbidden): Insufficient rights to the resource.
501 (Not Implemented): SPARQL API not supported.

A WoT Thing Description Directory MAY implement federation in its SPARQL query API. If implemented, the SPARQL API MUST implement the SPARQL 1.1 Federated Query standard [ sparql11-overview ].

6.2.3 Security and Privacy

Editor's note : Security Considerations Overview

Minimum security and privacy requirements for confidentiality, authentication, access control, etc.

8. Directory Service API Specifications

8.1 Directory API Thing Description

Below is a generic Thing Description for the Directory API with OAuth2 security. This specification only covers the HTTP and SSE protocol bindings. Other protocol bindings may be specified in future. The Thing Description alone should not be considered as the reference to implement or interact with a directory. The full specification is available as human-readable text in § 6.2.2 Directory Service API .

                
{
    "@context": [
        "http://www.w3.org/ns/td",
        "https://w3c.github.io/wot-discovery/context/discovery-context.jsonld"
    ],
    "@type": "ThingDirectory",
    "title": "Thing Description Directory (TDD)",
    "version": {
        "instance": "1.0.0-alpha"
    },
    "securityDefinitions": {
        "oauth2_code": {
            "scheme": "oauth2",
            "flow": "code",
            "authorization": "https://auth.example.com/authorization",
            "token": "https://auth.example.com/token",
            "scopes": [
                "write",
                "read",
                "readAll",
                "search",
                "notification"
            ]
        },
        "oauth2_client": {
            "scheme": "oauth2",
            "flow": "client",
            "token": "https://auth.example.com/token",
            "scopes": [
                "write",
                "read",
                "readAll",
                "search",
                "notification"
            ]
        },
        "oauth2_device": {
            "scheme": "oauth2",
            "flow": "device",
            "authorization": "https://auth.example.com/device_authorization",
            "token": "https://auth.example.com/token",
            "scopes": [
                "write",
                "read",
                "readAll",
                "search",
                "notification"
            ]
        },
        "combo_sc": {
            "scheme": "combo",
            "oneOf": [
                "oauth2_code",
                "oauth2_client",
                "oauth2_device"
            ]
        }
    },
    "security": "combo_sc",
    "base": "https://tdd.example.com",
    "properties": {
        "things": {
            "description": "Retrieve all Thing Descriptions",
            "uriVariables": {
                "offset": {
                    "title": "Number of TDs to skip before the page",
                    "type": "number",
                    "default": 0
                },
                "limit": {
                    "title": "Number of TDs in a page",
                    "type": "number"
                },
                "format": {
                    "title": "Payload format",
                    "type": "string",
                    "enum": ["array", "collection"],
                    "default": "array"
                },
                "sort_by": {
                    "title": "Comparator TD attribute for collection sorting",
                    "type": "string",
                    "default": "id"
                },
                "sort_order": {
                    "title": "Sorting order",
                    "type": "string",
                    "enum": ["asc", "desc"],
                    "default": "asc"
                }
            },
            "forms": [
                {
                    "href": "/things",
                    "htv:methodName": "GET",
                    "response": {
                        "description": "Success response",
                        "htv:statusCodeValue": 200,
                        "contentType": "application/ld+json"
                    },
                    "scopes": "readAll"
                },
                {
                    "href": "/things{?offset,limit,sort_by,sort_order}",
                    "htv:methodName": "GET",
                    "response": {
                        "description": "Success response",
                        "htv:statusCodeValue": 200,
                        "contentType": "application/ld+json",
                        "htv:headers": [
                            {
                                "htv:fieldName": "Link"
                            }
                        ]
                    },
                    "additionalResponses": [
                        {
                            "description": "Invalid query arguments",
                            "contentType": "application/problem+json",
                            "htv:statusCodeValue": 400
                        }
                    ],
                    "scopes": "readAll"
                }
            ]
        }
    },
    "actions": {
        "createThing": {
            "description": "Create a Thing Description",
            "uriVariables": {
                "id": {
                    "@type": "ThingID",
                    "title": "Thing Description ID",
                    "type": "string",
                    "format": "iri-reference"
                }
            },
            "input": {
                "description": "The schema is implied by the content type",
                "type": "object"
            },
            "forms": [
                {
                    "href": "/things/{id}",
                    "htv:methodName": "PUT",
                    "contentType": "application/td+json",
                    "response": {
                        "description": "Success response",
                        "htv:statusCodeValue": 201
                    },
                    "additionalResponses": [
                        {
                            "description": "Invalid serialization or TD",
                            "contentType": "application/problem+json",
                            "htv:statusCodeValue": 400
                        }
                    ],
                    "scopes": "write"
                }
            ]
        },
        "createAnonymousThing": {
            "description": "Create an anonymous Thing Description",
            "input": {
                "description": "The schema is implied by the content type",
                "type": "object"
            },
            "forms": [
                {
                    "href": "/things",
                    "htv:methodName": "POST",
                    "contentType": "application/td+json",
                    "response": {
                        "description": "Success response including the system-generated URI",
                        "htv:headers": [
                            {
                                "description": "System-generated URI",
                                "htv:fieldName": "Location"
                            }
                        ],
                        "htv:statusCodeValue": 201
                    },
                    "additionalResponses": [
                        {
                            "description": "Invalid serialization or TD",
                            "contentType": "application/problem+json",
                            "htv:statusCodeValue": 400
                        }
                    ],
                    "scopes": "write"
                }
            ]
        },
        "retrieveThing": {
            "description": "Retrieve a Thing Description",
            "uriVariables": {
                "id": {
                    "@type": "ThingID",
                    "title": "Thing Description ID",
                    "type": "string",
                    "format": "iri-reference"
                }
            },
            "output": {
                "description": "The schema is implied by the content type",
                "type": "object"
            },
            "safe": true,
            "idempotent": true,
            "forms": [
                {
                    "href": "/things/{id}",
                    "htv:methodName": "GET",
                    "response": {
                        "description": "Success response",
                        "htv:statusCodeValue": 200,
                        "contentType": "application/td+json"
                    },
                    "additionalResponses": [
                        {
                            "description": "TD with the given id not found",
                            "contentType": "application/problem+json",
                            "htv:statusCodeValue": 404
                        }
                    ],
                    "scopes": "read"
                }
            ]
        },
        "updateThing": {
            "description": "Update a Thing Description",
            "uriVariables": {
                "id": {
                    "@type": "ThingID",
                    "title": "Thing Description ID",
                    "type": "string",
                    "format": "iri-reference"
                }
            },
            "input": {
                "description": "The schema is implied by the content type",
                "type": "object"
            },
            "forms": [
                {
                    "href": "/things/{id}",
                    "htv:methodName": "PUT",
                    "contentType": "application/td+json",
                    "response": {
                        "description": "Success response",
                        "htv:statusCodeValue": 204
                    },
                    "additionalResponses": [
                        {
                            "description": "Invalid serialization or TD",
                            "contentType": "application/problem+json",
                            "htv:statusCodeValue": 400
                        }
                    ],
                    "scopes": "write"
                }
            ]
        },
        "partiallyUpdateThing": {
            "description": "Partially update a Thing Description",
            "uriVariables": {
                "id": {
                    "@type": "ThingID",
                    "title": "Thing Description ID",
                    "type": "string",
                    "format": "iri-reference"
                }
            },
            "input": {
                "description": "The schema is implied by the content type",
                "type": "object"
            },
            "forms": [
                {
                    "href": "/things/{id}",
                    "htv:methodName": "PATCH",
                    "contentType": "application/merge-patch+json",
                    "response": {
                        "description": "Success response",
                        "htv:statusCodeValue": 204
                    },
                    "additionalResponses": [
                        {
                            "description": "Invalid serialization or TD",
                            "contentType": "application/problem+json",
                            "htv:statusCodeValue": 400
                        },
                        {
                            "description": "TD with the given id not found",
                            "contentType": "application/problem+json",
                            "htv:statusCodeValue": 404
                        }
                    ],
                    "scopes": "write"
                }
            ]
        },
        "deleteThing": {
            "description": "Delete a Thing Description",
            "uriVariables": {
                "id": {
                    "@type": "ThingID",
                    "title": "Thing Description ID",
                    "type": "string",
                    "format": "iri-reference"
                }
            },
            "forms": [
                {
                    "href": "/things/{id}",
                    "htv:methodName": "DELETE",
                    "response": {
                        "description": "Success response",
                        "htv:statusCodeValue": 204
                    },
                    "additionalResponses": [
                        {
                            "description": "TD with the given id not found",
                            "contentType": "application/problem+json",
                            "htv:statusCodeValue": 404
                        }
                    ],
                    "scopes": "write"
                }
            ]
        },
        "searchJSONPath": {
            "description": "JSONPath syntactic search",
            "uriVariables": {
                "query": {
                    "title": "A valid JSONPath expression",
                    "type": "string"
                }
            },
            "output": {
                "description": "The schema depends on the given query",
                "type": "object"
            },
            "safe": true,
            "idempotent": true,
            "forms": [
                {
                    "href": "/search/jsonpath?query={query}",
                    "htv:methodName": "GET",
                    "response": {
                        "description": "Success response",
                        "contentType": "application/json",
                        "htv:statusCodeValue": 200
                    },
                    "additionalResponses": [
                        {
                            "description": "JSONPath expression not provided or contains syntax errors",
                            "contentType": "application/problem+json",
                            "htv:statusCodeValue": 400
                        }
                    ],
                    "scopes": "search"
                }
            ]
        },
        "searchXPath": {
            "description": "XPath syntactic search",
            "uriVariables": {
                "query": {
                    "title": "A valid XPath expression",
                    "type": "string"
                }
            },
            "output": {
                "description": "The schema depends on the given query",
                "type": "object"
            },
            "safe": true,
            "idempotent": true,
            "forms": [
                {
                    "href": "/search/xpath?query={query}",
                    "htv:methodName": "GET",
                    "response": {
                        "description": "Success response",
                        "contentType": "application/json",
                        "htv:statusCodeValue": 200
                    },
                    "additionalResponses": [
                        {
                            "description": "XPath expression not provided or contains syntax errors",
                            "contentType": "application/problem+json",
                            "htv:statusCodeValue": 400
                        }
                    ],
                    "scopes": "search"
                }
            ]
        },
        "searchSPARQL": {
            "description": "SPARQL semantic search",
            "uriVariables": {
                "query": {
                    "title": "A valid SPARQL 1.1. query",
                    "type": "string"
                }
            },
            "output": {
                "description": "The schema depends on the given query",
                "type": "object"
            },
            "safe": true,
            "idempotent": true,
            "forms": [
                {
                    "href": "/search/sparql?query={query}",
                    "htv:methodName": "GET",
                    "response": {
                        "description": "Success response",
                        "contentType": "application/json",
                        "htv:statusCodeValue": 200
                    },
                    "additionalResponses": [
                        {
                            "description": "SPARQL query not provided or contains syntax errors",
                            "contentType": "application/problem+json",
                            "htv:statusCodeValue": 400
                        }
                    ],
                    "scopes": "search"
                },
                {
                    "href": "/search/sparql",
                    "htv:methodName": "POST",
                    "response": {
                        "description": "Success response",
                        "contentType": "application/json",
                        "htv:statusCodeValue": 200
                    },
                    "additionalResponses": [
                        {
                            "description": "SPARQL query not provided or contains syntax errors",
                            "contentType": "application/problem+json",
                            "htv:statusCodeValue": 400
                        }
                    ],
                    "scopes": "search"
                }
            ]
        }
    },
    "events": {
        "thingCreation": {
          "description": "Registration of Thing Descriptions inside the directory",
          "uriVariables": {
            "diff": {
                "description": "Receive the full created TD as event data",
                "type": "boolean"
            }
          },
          "data": {
            "title": "Partial/Full TD",
            "type": "object"
          },
          "forms": [
              {
                  "op": "subscribeevent",
                  "href": "/events/create{?diff}",
                  "subprotocol": "sse",
                  "contentType": "text/event-stream",
                  "htv:headers": [
                      {
                          "description": "ID of the last event for reconnection",
                          "htv:fieldName": "Last-Event-ID"
                      }
                  ],
                  "scopes": "notification"
              }
          ]
        },
        "thingUpdate": {
          "description": "Updates to Thing Descriptions within the directory",
          "uriVariables": {
            "diff": {
                "description": "Include TD changes inside event data",
                "type": "boolean"
            }
          },
          "data": {
            "title": "Partial TD",
            "type": "object",
            "contentMediaType": "application/merge-patch+json"
          },
          "forms": [
              {
                  "op": "subscribeevent",
                  "href": "/events/update{?diff}",
                  "subprotocol": "sse",
                  "contentType": "text/event-stream",
                  "htv:headers": [
                      {
                          "description": "ID of the last event for reconnection",
                          "htv:fieldName": "Last-Event-ID"
                      }
                  ],
                  "scopes": "notification"
              }
          ]
        },
        "thingDeletion": {
          "description": "Deletion of Thing Descriptions from the directory",
          "data": {
            "title": "Partial TD",
            "type": "object"
          },
          "forms": [
              {
                  "op": "subscribeevent",
                  "href": "/events/delete",
                  "subprotocol": "sse",
                  "contentType": "text/event-stream",
                  "htv:headers": [
                      {
                          "description": "ID of the last event for reconnection",
                          "htv:fieldName": "Last-Event-ID"
                      }
                  ],
                  "scopes": "notification"
              }
           ]
        }
    }
}

Editor's note : Scopes

The scopes may need to be more concrete to able to allow updates but prevent creation.

Issue 82 : Creation of OpenAPI spec from Directory TD

Need to confirm if equivalent OpenAPI spec can be easily created out of the TD in § 8.1 Directory API Thing Description . If yes, a sentence may be added indicating this possibility.

Editor's note : Context URIs

The context URIs are tentative and subject to change.

Issue 86 : "Thing description" for Directory should be a "Thing Model"

The TD defined for a Directory service should really be a Thing Model. We should also "template" the "base" parameter (or omit it, and say it needs to be added when the Thing Model is instantiated).

Since we are talking about a class of things, we should have a Thing Model. However, the definition of Thing Model needs to be updated to allow specifying security scheme templates, form templates (including URL templates giving https), etc.

Value	Description	Reference
`wot.thing`	Thing Description of a Thing	§ 5.4 CoRE Link Format and CoRE Resource Directory
`wot.directory`	Directory Description of a Thing Description Directory	§ 5.4 CoRE Link Format and CoRE Resource Directory

Web of Things (WoT) Discovery

Abstract

Status of This Document

1. Introduction

2. Conformance

3. Terminology

4. Architecture

5. Introduction Mechanisms

5.1 Direct

5.2 Well-Known URIs

5.3 DNS-Based Service Discovery

5.4 CoRE Link Format and CoRE Resource Directory

5.5 DID Documents

6. Exploration Mechanisms

6.1 Self-description

6.2 Directory

6.2.1 Information Model

6.2.1.1 Registration Information

6.2.1.2 Registration Expiry

6.2.1.3 Anonymous TD Identifiers

6.2.2 Directory Service API

6.2.2.1 Registration

6.2.2.1.1 Creation

6.2.2.1.2 Retrieval

6.2.2.1.3 Update

6.2.2.1.4 Deletion

6.2.2.1.5 Listing

6.2.2.1.6 Validation

6.2.2.2 Management

6.2.2.3 Notification

6.2.2.4 Search

6.2.2.4.1 Syntactic search: JSONPath

6.2.2.4.2 Syntactic search: XPath

6.2.2.4.3 Semantic search: SPARQL

6.2.3 Security and Privacy

7. Security and Privacy Considerations

7.1 Denial of Service

7.2 Location Tracking

8. Directory Service API Specifications

8.1 Directory API Thing Description

9. IANA Considerations

9.1 Well-Known URI Registration

9.2 Service Name Registration

9.3 CoRE Resource Types Registration

A. Recent Specification Changes

Changes from First Draft

B. Acknowledgments

C. References

C.1 Normative references

C.2 Informative references