KV Storage

1. Introduction

<p>
  You have viewed this page
  <span id="count">an untold number of</span>
  time(s).
</p>
<script type="module">
  import { storage } from "std:kv-storage";
  (async () => {
    let pageLoadCount = await storage.get("pageLoadCount") || 0;
    ++pageLoadCount;
    document.querySelector("#count").textContent = pageLoadCount;
    await storage.set("pageLoadCount", pageLoadCount);
  })();
</script>

2. The std:kv-storage built-in module

import * as kvs from "std:kv-storage"

Imports the KV storage API’s namespace object as the variable kvs .

If the module is not imported in a secure context , the import statement will cause a " SecurityError " DOMException , as persistent storage is a powerful feature.

kvs . storage

Returns the default storage area. It is a pre-constructed instance of the StorageArea class, meant to be a convenience similar to localStorage .

kvs . StorageArea

Returns the constructor for the StorageArea class, to allow the creation of isolated storage areas.

This specification defines a new built-in module. Tentatively, depending on further discussions, we use the specifier " std:kv-storage " to denote it for now. This is not final and is subject to change depending on the details of how built-in modules end up working. [JSSTDLIB]

Its exports are the following:

storage

An instance of the StorageArea class, created as if by Construct ( StorageArea , « " default " »).

StorageArea

The StorageArea class

import { storage, StorageArea } from "std:kv-storage";

In addition to establishing its exports, evaluating the module must perform the following steps:

1. If the current settings object is not contextually secure , throw a " SecurityError " DOMException .

3. The StorageArea class

Upon evaluating the std:kv-storage module, the StorageArea class must be created in the current realm . The result must be equivalent to evaluating the following JavaScript code, with the following two exceptions:

• The constructor, method, and getter bodies must behave as specified below instead of being the no-ops shown in this code block.

• HostHasSourceTextAvailable must return false for all function objects (i.e. the constructor, methods, and getter) created.

class StorageArea {
  constructor(name)  { /* see below */ }
  set(key, value)    { /* see below */ }
  get(key)           { /* see below */ }
  delete(key)        { /* see below */ }
  clear()            { /* see below */ }
  keys()             { /* see below */ }
  values()           { /* see below */ }
  entries()          { /* see below */ }
  get backingStore() { /* see below */ }
}

The prototype property of StorageArea must additionally have a @@asyncIterator property, whose value is equal to the same function object as the original value of StorageArea.prototype.entries() .

Each StorageArea instance must also contain the [[DatabaseName]] , [[DatabasePromise]] , and [[BackingStoreObject]] internal slots. The following is a non-normative summary of their meaning:

[[DatabaseName]]

A string containing the name of the backing IndexedDB database.

[[DatabasePromise]]

A promise for an IDBDatabase object, lazily initialized when performing any database operation .

[[BackingStoreObject]]

The object returned by the backingStore getter, cached to ensure identity across gets.

A JavaScript value val brand checks as a StorageArea if Type ( val ) is Object, val has a [[DatabaseName]] internal slot, and val ’s relevant realm is equal to the current realm .

The realm check here gives us semantics identical to using JavaScript’s WeakMap or the proposed private class fields. This ensures StorageArea does not use any magic, like the platform’s usual cross-realm brand checks, which go beyond what can be implemented in JavaScript. [ECMA-262] [CLASS-FIELDS]

3.1. constructor( name )

storage = new StorageArea ( name )

Creates a new StorageArea that provides an async key/value store view onto an IndexedDB database named `kv-storage:${name}` .

This does not actually open or create the database yet; that is done lazily when other methods are called. This means that all other methods can reject with database-related exceptions in failure cases.

3.2. set( key , value )

await storage . set ( key , value )

Asynchronously stores the given value so that it can later be retrieved by the given key .

Keys have to follow the same restrictions as IndexedDB keys: roughly, a key can be a number, string, array, Date , ArrayBuffer , DataView , typed array , or an array of these. Invalid keys will cause the returned promise to reject with a " DataError " DOMException .

Values can be any value that can be structured-serialized for storage . Un-serializable values will cause a " DataCloneError " DOMException . The value undefined will cause the corresponding entry to be deleted.

The returned promise will fulfill with undefined on success.

3.3. get( key )

value = await storage . get ( key )

Asynchronously retrieves the value stored at the given key , or undefined if there is no value stored at key .

Values retrieved will be structured-deserialized from their original form.

3.4. delete( key )

await storage . delete ( key )

Asynchronously deletes the entry at the given key . This is equivalent to storage . set ( key , undefined).

The returned promise will fulfill with undefined on success.

3.5. clear()

await storage . clear ()

Asynchronously deletes all entries in this storage area.

This is done by actually deleting the underlying IndexedDB database. As such, it always can be used as a fail-safe to get a clean slate, as shown below .

The returned promise will fulfill with undefined on success.

To delete the database given a string name :

// This upgrade to version 100 breaks the "cats" storage area: since StorageAreas
// assume a version of 1, "cats" can no longer be used with KV storage.
const openRequest = indexedDB.open("kv-storage:cats", 100);
openRequest.onsuccess = () => {
  openRequest.onsuccess.close();
};
(async () => {
  const area = new StorageArea("cats");
  // Due to the above upgrade, all other methods will reject:
  try {
    await area.set("fluffy", new Cat());
  } catch (e) {
    // This will be reached and output a "VersionError" DOMException
    console.error(e);
  }
  // But clear() will delete the database entirely:
  await area.clear();
  // Now we can use it again!
  await area.set("fluffy", new Cat());
  await area.set("tigger", new Cat());
  // Also, the version is back down to 1:
  console.assert(area.backingStore.version === 1);
})();

3.6. keys()

for await (const key of storage . keys ()) { ... }

Retrieves an async iterator containing the keys of all entries in this storage area.

Keys will be yielded in ascending order; roughly, segregated by type, and then sorted within each type. They will be key round-tripped from their original form.

The iterator provides a live view onto the storage area: modifications made to entries sorted after the last-returned one will be reflected in the iteration.

await storage.set(10, "value 10");
await storage.set(20, "value 20");
await storage.set(30, "value 30");
const keysSeen = [];
for await (const key of storage.keys()) {
  if (key === 20) {
    await storage.set(15, "value 15");
    await storage.delete(20);
    await storage.set(25, "value 25");
  }
  keysSeen.push(key);
}
console.log(keysSeen);   // logs 10, 20, 25, 30

3.7. values()

for await (const value of storage . values ()) { ... }

Asynchronously retrieves an array containing the values of all entries in this storage area.

Values will be ordered as corresponding to their keys; see keys() . They will be structured-deserialized from their original form.

The iterator provides a live view onto the storage area: modifications made to entries sorted after the last-returned one will be reflected in the iteration.

3.8. entries()

for await (const [ key , value ] of storage . entries ()) { ... }

for await (const [ key , value ] of storage ) { ... }

Asynchronously retrieves an array of two-element [key, value] arrays, each of which corresponds to an entry in this storage area.

Entries will be ordered as corresponding to their keys; see keys() . Each key and value will be key round-tripped and structured-deserialized from its original form, respectively.

The iterator provides a live view onto the storage area: modifications made to entries sorted after the last-returned one will be reflected in the iteration.

const entries = [];
for await (const entry of storage.entries()) {
  entries.push(entry);
}
fetch("/storage-receiver", {
  method: "POST",
  body: entries,
  headers: {
    "Content-Type": "application/json"
  }
});

3.9. get backingStore()

{ database , store , version } = storage . backingStore

Asynchronously retrieves an an object containing all of the information necessary to manually interface with the IndexedDB backing store that underlies this storage area:

• database will be a string equal to " kv-storage: " concatenated with the database name passed to the constructor. (For the default storage area, it will be " kv-storage:default ".)

• store will be the string " store ".

• version will be the number 1.

It is good practice to use the backingStore property to retrieve this information, instead of memorizing the above factoids.

bulbasaur.onchange = () => storage.set("bulbasaur", bulbasaur.checked);
ivysaur.onchange = () => storage.set("ivysaur", ivysaur.checked);
venusaur.onchange = () => storage.set("venusaur", venusaur.checked);
// ...

bulbasaurEvolve.onclick = async () => {
  await storage.set("bulbasaur", false);
  await storage.set("ivysaur", true);
};

const { database, store, version } = storage.backingStore;
const request = indexedDB.open(database, version);
request.onsuccess = () => {
  const db = request.result;
  bulbasaurEvolve.onclick = () => {
    const transaction = db.transaction(store, "readwrite");
    const store = transaction.objectStore(store);
    store.put("bulbasaur", false);
    store.put("ivysaur", true);
    db.close();
  };
};

4. The storage area async iterator

Much of this section is amenable to being generalized into a reusable primitive, probably via Web IDL. See heycam/webidl#580 .

Upon evaluating the std:kv-storage module, let the storage area async iterator prototype object be object obtained via the following steps executed in the current realm :

1. Let proto be ObjectCreate ( %IteratorPrototype% ).

2. Let next be CreateBuiltinFunction (the steps of §4.2 next() ).

3. Perform CreateMethodProperty ( proto , " next ", next ).

4. Return proto .

4.1. Creation

To create a storage area async iterator , given a StorageArea area and a string mode which is one of either " keys ", " values ", or " entries ":

1. Let iter be ObjectCreate ( the storage area async iterator prototype object , « [[Area]] , [[Mode]] , [[LastKey]] , [[OngoingPromise]] »).

2. Set iter . [[Area]] to area .

3. Set iter . [[Mode]] to mode .

4. Set iter . [[LastKey]] to not yet started .

5. Set iter . [[OngoingPromise]] to undefined.

6. Return iter .

The following is a non-normative summary of the internal slots that get added to objects created in such a way:

[[Area]]

A pointer back to the originating StorageArea , used so that the async iterator can perform database operations .

[[Mode]]

One of " keys ", " values ", or " entries ", indicating the types of values that iteration will retrieve from the storage area.

[[LastKey]]

The key of the entry that was most recently iterated over, used to perform the next iteration. Or, if next() has not yet been called, it will be set to not yet started .

[[OngoingPromise]]

A reference to the promise that was returned by the most recent call to next() , if that promise has not yet settled, or undefined if it has. Used to prevent concurrent executions of the main get the next IterResult algorithm, which would be bad because that algorithm needs to complete in order for [[LastKey]] to be set correctly.

4.2. next()

1. Let iter be this object.

2. If Type ( iter ) is not Object, or iter ’s relevant realm is not equal to the current realm , or iter does not have a [[Area]] internal slot, then return a promise rejected with a TypeError exception.

3. Let currentOngoingPromise be iter . [[OngoingPromise]] .

4. Let resultPromise be undefined.

5. If currentOngoingPromise is not undefined, then set resultPromise to the result of transforming currentOngoingPromise by the result of getting the next IterResult given iter .

6. Otherwise, set resultPromise to the result of getting the next IterResult given iter .

7. Set iter . [[OngoingPromise]] to resultPromise .

8. Return resultPromise .

To get the next IterResult given iter :

1. Return the result of performing a database operation given iter . [[Area]] , " read ", and the following steps operating on transaction and store :

   1. Let lastKey be iter . [[LastKey]] .

   2. If lastKey is undefined, then return CreateIterResultObject (undefined, true).

   3. Let range be the result of getting the range for lastKey .

   4. Let key and iterResultValue be null.

   5. Let promise be a new promise .

   6. Switch on iter . [[Mode]] :

      " keys "

      1. Let request be the result of performing the steps listed in the description of IDBObjectStore 's getKey() method on store , given the argument range .

      2. Add a simple event listener to request for " success " that performs the following steps:

         1. Set key to request ’s result .

         2. Set iterResultValue to key .

         3. Finish up .

      3. Add a simple event listener to request for " error " that rejects promise with request ’s error .

      " values "

      1. Let keyRequest be the result of performing the steps listed in the description of IDBObjectStore 's getKey() method on store , given the argument range .

      2. Let valueRequest be the result of performing the steps listed in the description of IDBObjectStore 's get() method on store , given the argument range .

      3. Add a simple event listener to valueRequest for " success " that performs the following steps:

         1. Set key to keyRequest ’s result .

         2. Set iterResultValue to valueRequest ’s result .

         3. Finish up .

      4. Add a simple event listener to keyRequest for " error " that rejects promise with keyRequest ’s error .

      5. Add a simple event listener to valueRequest for " error " that rejects promise with valueRequest ’s error .

      " entries "

      1. Let keyRequest be the result of performing the steps listed in the description of IDBObjectStore 's getKey() method on store , given the argument range .

      2. Let valueRequest be the result of performing the steps listed in the description of IDBObjectStore 's get() method on store , given the argument range .

      3. Add a simple event listener to valueRequest for " success " that performs the following steps:

         1. Set key to keyRequest ’s result .

         2. Let value be valueRequest ’s result .

         3. Set iterResultValue to CreateArrayFromList (« key , value »).

         4. Finish up .

      4. Add a simple event listener to keyRequest for " error " that rejects promise with keyRequest ’s error .

      5. Add a simple event listener to valueRequest for " error " that rejects promise with valueRequest ’s error .

      When the above steps say to finish up , which they will do after having set key and iterResultValue appropriately, perform the following steps:

      1. Set iter . [[LastKey]] to key .

      2. Set iter . [[OngoingPromise]] to undefined.

      3. Let done be true if key is undefined, and false otherwise.

      4. Resolve promise with CreateIterResultObject ( iterResultValue , done ).

   7. Return promise .

5. Supporting operations and concepts

To add a simple event listener , given an EventTarget target , an event type string type , and a set of steps steps :

The current IDBFactory is the IDBFactory instance returned by the following steps:

To perform a database operation given a StorageArea area , a mode string mode , and a set of steps steps that operate on an IDBTransaction transaction and an IDBObjectStore store :

To initialize the database promise for a StorageArea area :

To check the database schema for an IDBDatabase database :

Check the database schema only needs to be called in the initial setup algorithm, initialize the database promise , since once the database connection has been opened, the schema cannot change.

A value value is allowed as a key if the following steps return true:

Key round-tripping refers to the way in which JavaScript values are processed by first being passed through IndexedDB’s convert a value to a key operation, then converted back through its convert a key to a value operation. Keys returned by the keys() or entries() methods will have gone through this process.

Notably, any typed arrays or DataView s will have been "unwrapped", and returned back as just ArrayBuffer s containing the same bytes. Also, similar to the structured-serialization / deserialization process, any "expando" properties or other modifications will not be preserved by key round-tripping .

For primitive string or number values, there’s no need to worry about key round-tripping ; the values are indistinguishable.

To get the range for a key key :

1. If key is not yet started , then return the result of performing the steps listed in the description of the IDBKeyRange.lowerBound() static method, given the argument −Infinity.

   The intent here is to get an unbounded key range , but this is the closest thing we can get that is representable as an IDBKeyRange object. It works equivalently for our purposes, but will behave incorrectly if Indexed DB ever adds keys that sort below −Infinity. See some discussion on potential future improvements .

2. Otherwise, return the result of performing the steps listed listed in the description of the IDBKeyRange.lowerBound() static method, given the arguments lastKey and true.

The special value not yet started can be taken to be any JavaScript value that is not equal to any other program-accessible JavaScript value (but is equal to itself). It is used exclusively as an argument to the get the range for a key algorithm.

A newly created object or symbol, e.g. const nys = {} or const nys = Symbol () , would satisfy this definition.

6. Appendix: is this API perfectly layered?

The APIs in this specification, being layered on top of Indexed DB as they are, are almost entirely well-layered, in the sense of building on low-level features in the way promoted by the Extensible Web Manifesto . (Indeed, the unusual class definition pattern was motivated by a desire to further improve this layering.) However, it fails in two ways, both around ensuring the encapsulation of the implementation: [EXTENSIBLE]

• By requiring censorship of the output of Function.prototype.toString() for the functions produced. See drufball/layered-apis#7 .

• By directly invoking the algorithms of various IDL operations and attributes, instead of going through the global, potentially-overridable JavaScript APIs. (E.g., in various algorithm steps that say "performing the steps listed in the description of", or the allowed as a key algorithm which uses IsArray directly instead of going through Array.isArray() .) See drufball/layered-apis#6 .

Eventually we hope to introduce the ability for web authors to write code that gets these same benefits, instead of locking them up so that only web platform APIs like KV storage can achieve this level of encapsulation. That’s a separate effort, however, which is best followed in the above-linked issue threads.

Acknowledgments

The editor would like to thank Andrew Sutherland, Kenneth Rohde Christiansen, Jake Archibald, Jan Varga, Joshua Bell, and Victor Costan for their contributions to this specification.

Conformance

This specification depends on the Infra Standard. [INFRA]