Translator and Language Detector APIs

1. If this ’s relevant global object is a Window whose associated Document is not fully active , then return a promise rejected with an " InvalidStateError " DOMException .

2. If options [" signal "] exists and is aborted , then return a promise rejected with options [" signal "]'s abort reason .

3. Validate and canonicalize translator options given options .

   This can mutate options .

4. Return the result of creating an AI model object given this ’s relevant realm , options , compute translator options availability , download the translation model , initialize the translation model , and create a translator object .

1. Validate and canonicalize language tags given options and " sourceLanguage ".

2. Validate and canonicalize language tags given options and " targetLanguage ".

1. Assert : these steps are running in parallel .

2. Initiate the download process for everything the user agent needs to translate text from options [" sourceLanguage "] to options [" targetLanguage "].

   This could include both a base translation model and specific language arc material, or perhaps material for multiple language arcs if an intermediate language is used.

3. If the download process cannot be started for any reason, then return false.

4. Return true.

1. Assert : these steps are running in parallel .

2. Perform any necessary initialization operations for the AI model backing the user agent’s capabilities for translating from options [" sourceLanguage "] to options [" targetLanguage "].

   This could include loading the model into memory, or loading any fine-tunings necessary to support the specific options in question.

3. If initialization failed for any reason, then return false.

4. Return true.

1. Assert : these steps are running on realm ’s surrounding agent ’s event loop .

2. Return a new AITranslator object, created in realm , with

   source language

   options [" sourceLanguage "]

   target language

   options [" targetLanguage "]

1. If this ’s relevant global object is a Window whose associated Document is not fully active , then return a promise rejected with an " InvalidStateError " DOMException .

2. Validate and canonicalize translator options given options .

3. Let promise be a new promise created in this ’s relevant realm .

4. In parallel :

   1. Let availability be the result of computing translator options availability given options .

   2. Queue a global task on the AI task source given this ’s relevant global object to perform the following steps:

      1. If availability is null, then reject promise with an " UnknownError " DOMException .

      2. Otherwise, resolve promise with availability .

1. Assert : this algorithm is running in parallel .

2. Let availabilities be the user agent’s translator language arc availabilities .

3. If availabilities is null, then return null.

4. For each languageArc → availability in availabilities :

   1. Let sourceLanguageBestFit be LookupMatchingLocaleByBestFit (« languageArc ’s source language », « options [" sourceLanguage "] »).

   2. Let targetLanguageBestFit be LookupMatchingLocaleByBestFit (« languageArc ’s target language », « options [" targetLanguage "] »).

   3. If sourceLanguageBestFit and targetLanguageBestFit are both not undefined, then:

      1. Set options [" sourceLanguage "] to sourceLanguageBestFit .[[locale]].

      2. Set options [" targetLanguage "] to targetLanguageBestFit .[[locale]].

      3. Return availability .

5. If ( options [" sourceLanguage "], options [" targetLanguage "]) can be fulfilled by the identity translation , then return " available ".

   Such cases could also return " downloadable ", " downloading ", or " available " because of the above steps, if the user agent has specific entries in its translator language arc availabilities for the given language arc. However, the identity translation is always available, so this step ensures that we never return " unavailable " for such cases.

   One language arc that can be fulfilled by the identity translation is ( "en-US" , "en-GB" ). It is conceivable that an implementation might support a specialized model for this translation, which would show up in the translator language arc availabilities .

   On the other hand, it’s pretty unlikely that an implementation has any specialized model for the language arc (" en-x-asdf ", " en-x-xyzw "). In such a case, this step takes over, and later calls to the translate algorithm will use the identity translation.

   Note that when this step takes over, options [" sourceLanguage "] and options [" targetLanguage "] are not modified, so if this algorithm is being called from create() , that means the resulting AITranslator object’s sourceLanguage and targetLanguage properties will return the original inputs, and not some canonicalized form.

6. Return " unavailable ".

1. Assert : this algorithm is running in parallel .

2. If there is some error attempting to determine what language arcs the user agent supports translating text between, which the user agent believes to be transient (such that re-querying the translator language arc availabilities could stop producing such an error), then return null.

3. Return a map from language arcs to AIAvailability values, where each key is a language arc that the user agent supports translating text between, filled according to the following constraints:

   • If the user agent supports translating text from the source language to the target language of the language arc without performing any downloading operations, then the map must contain an entry whose key is that language arc and whose value is " available ".

   • If the user agent supports translating text from the source language to the target language of the language arc , but only after finishing a currently-ongoing download, then the map must contain an entry whose key is that language arc and whose value is " downloading ".

   • If the user agent supports translating text from the source language to the target language of the language arc , but only after performing a not-currently ongoing download, then the map must contain an entry whose key is that language arc and whose value is " downloadable ".

   • The keys must not include any language arcs that overlap with the other keys .

1. Let sourceLanguages be the set composed of the source languages of each item in otherArcs .

2. If LookupMatchingLocaleByBestFit ( sourceLanguages , « arc ’s source language ») is not undefined, then return true.

3. Let targetLanguages be the set composed of the target languages of each item in otherArcs .

4. If LookupMatchingLocaleByBestFit ( targetLanguages , « arc ’s target language ») is not undefined, then return true.

5. Return false.

1. If LookupMatchingLocaleByBestFit (« arc ’s source language », « arc ’s target language ») is not undefined, then return true.

2. If LookupMatchingLocaleByBestFit (« arc ’s target language », « arc ’s source language ») is not undefined, then return true.

3. Return false.

1. Let operation be an algorithm step which takes arguments chunkProduced , done , error , and stopProducing , and translates input given this ’s source language , this ’s target language , chunkProduced , done , error , and stopProducing .

2. Return the result of getting an aggregated AI model result given this , options , and operation .

1. Let operation be an algorithm step which takes arguments chunkProduced , done , error , and stopProducing , and translates input given this ’s source language , this ’s target language , chunkProduced , done , error , and stopProducing .

2. Return the result of getting a streaming AI model result given this , options , and operation .

• a string input ,

• a Unicode canonicalized locale identifier sourceLanguage ,

• a Unicode canonicalized locale identifier targetLanguage ,

• an algorithm chunkProduced that takes a string and returns nothing,

• an algorithm done that takes no arguments and returns nothing,

• an algorithm error that takes error information and returns nothing, and

• an algorithm stopProducing that takes no arguments and returns a boolean,

perform the following steps:

1. Assert : this algorithm is running in parallel .

2. In an implementation-defined manner, subject to the following guidelines, begin the processs of translating input from sourceLanguage into targetLanguage .

   If input is the empty string, or otherwise consists of no translatable content (e.g., only contains whitespace, or control characters), then the resulting translation should be input . In such cases, sourceLanguage and targetLanguage should be ignored.

   If ( sourceLanguage , targetLanguage ) can be fulfilled by the identity translation , then the resulting translation should be input .

3. While true:

   1. Wait for the next chunk of translated text to be produced, for the translation process to finish, or for the result of calling stopProducing to become true.

   2. If such a chunk is successfully produced:

      1. Let it be represented as a string chunk .

      2. Perform chunkProduced given chunk .

   3. Otherwise, if the translation process has finished:

      1. Perform done .

      2. Break .

   4. Otherwise, if stopProducing returns true, then break .

   5. Otherwise, if an error occurred during translation:

      1. Let the error be represented as error information errorInfo according to the guidance in § 2.4.2 Errors .

      2. Perform error given errorInfo .

      3. Break .

1. If this ’s relevant global object is a Window whose associated Document is not fully active , then return a promise rejected with an " InvalidStateError " DOMException .

2. If options [" signal "] exists and is aborted , then return a promise rejected with options [" signal "]'s abort reason .

3. Validate and canonicalize language detector options given options .

   This can mutate options .

4. Return the result of creating an AI model object given this ’s relevant realm , options , compute language detector options availability , download the language detector model , initialize the language detector model , and create the language detector object .

1. Validate and canonicalize language tags given options and " expectedInputLanguages ".

1. Assert : these steps are running in parallel .

2. Initiate the download process for everything the user agent needs to detect the languages of input text, including all the languages in options [" expectedInputLanguages "].

   This could include both a base language detection model, and specific fine-tunings or other material to help with the languages identified in options [" expectedInputLanguages "].

3. If the download process cannot be started for any reason, then return false.

4. Return true.

1. Assert : these steps are running in parallel .

2. Perform any necessary initialization operations for the AI model backing the user agent’s capabilities for detecting the languages of input text.

   This could include loading the model into memory, or loading any fine-tunings necessary to support the languages identified in options [" expectedInputLanguages "].

3. If initialization failed for any reason, then return false.

4. Return true.

1. Assert : these steps are running on realm ’s surrounding agent ’s event loop .

2. Return a new AILanguageDetector object, created in realm , with

   expected input languages

   the result of creating a frozen array given options [" expectedInputLanguages "] if it is not empty ; otherwise null

1. If this ’s relevant global object is a Window whose associated Document is not fully active , then return a promise rejected with an " InvalidStateError " DOMException .

2. Validate and canonicalize language detector options given options .

3. Let promise be a new promise created in this ’s relevant realm .

4. In parallel :

   1. Let availability be the result of computing language detector options availability given options .

   2. Queue a global task on the AI task source given this ’s relevant global object to perform the following steps:

      1. If availability is null, then reject promise with an " UnknownError " DOMException .

      2. Otherwise, resolve promise with availability .

1. Assert : this algorithm is running in parallel .

2. If there is some error attempting to determine what languages the user agent supports detecting, which the user agent believes to be transient (such that re-querying could stop producing such an error), then return null.

3. Let availabilities be the result of getting language availabilities given the purpose of detecting text written in that language.

4. Let availability be " available ".

5. For each language in options [" expectedInputLanguages "]:

   1. For each availabilityToCheck in « " available ", " downloading ", " downloadable " »:

      1. Let languagesWithThisAvailability be availabilities [ availabilityToCheck ].

      2. Let bestMatch be LookupMatchingLocaleByBestFit ( languagesWithThisAvailability , « language »).

      3. If bestMatch is not undefined, then:

         1. Replace language with bestMatch .[[locale]] in options [" expectedInputLanguages "].

         2. Set availability to the minimum availability given availability and availabilityToCheck .

         3. Break .

   2. Return " unavailable ".

6. Return availability .

1. If this ’s relevant global object is a Window whose associated Document is not fully active , then return a promise rejected with an " InvalidStateError " DOMException .

2. Let signals be « this ’s destruction abort controller ’s signal ».

3. If options [" signal "] exists , then append it to signals .

4. Let compositeSignal be the result of creating a dependent abort signal given signals using AbortSignal and this ’s relevant realm .

5. If compositeSignal is aborted , then return a promise rejected with compositeSignal ’s abort reason .

6. Let abortedDuringOperation be false.

   This variable will be written to from the event loop , but read from in parallel .

7. Add the following abort steps to compositeSignal :

   1. Set abortedDuringOperation to true.

8. Let promise be a new promise created in this ’s relevant realm .

9. In parallel :

   1. Let stopProducing be the following steps:

      1. Return abortedDuringOperation .

   2. Let result be the result of detecting languages given input and stopProducing .

   3. Queue a global task on the AI task source given this ’s relevant global object to perform the following steps:

      1. If abortedDuringOperation is true, then reject promise with compositeSignal ’s abort reason .

      2. Otherwise, if result is an error information , then reject promise with the result of creating a DOMException with name given by errorInfo ’s error name , using errorInfo ’s error information to populate the message appropriately.

      3. Otherwise:

         1. Assert : result is a list of LanguageDetectionResult dictionaries. (It is not null, since in that case abortedDuringOperation would have been true.)

         2. Resolve promise with result .

1. Assert : this algorithm is running in parallel .

2. Let availabilities be the result of getting language availabilities given the purpose of detecting text written in that language.

3. Let currentlyAvailableLanguages be availabilities [" available "].

4. In an implementation-defined manner, subject to the following guidelines, let rawResult and unknown be the result of detecting the languages of input .

   rawResult must be a map which has a key for each language in currentlyAvailableLanguages . The value for each such key must be a number between 0 and 1. This value must represent the implementation’s confidence that input is written in that language.

   unknown must be a number between 0 and 1 that represents the implementation’s confidence that input is not written in any of the languages in currentlyAvailableLanguages .

   The values of rawResult , plus unknown , must sum to 1. Each such value, or unknown , may be 0.

   If the implementation believes input to be written in multiple languages, then it should attempt to apportion the values of rawResult and unknown such that they are proportionate to the amount of input written in each detected language. The exact scheme for apportioning input is implementation-defined .

   If input is " tacosを食べる ", the implementation might split this into " tacos " and " を食べる ", and then detect the languages of each separately. The first part might be detected as English with confidence 0.5 and Spanish with confidence 0.5, and the second part as Japanese with confidence 1. The resulting rawResult then might be «[ " en " → 0.25, " es " → 0.25, " ja " → 0.5 ]» (with unknown set to 0).

   The decision to split this into two parts, instead of e.g. the three parts " tacos ", " を ", and " 食べる ", was an implementation-defined choice. Similarly, the decision to treat each part as contributing to "half" of the result, instead of e.g. weighting by number of code points , was implementation-defined .

   (Realistically, we expect that implementations will split on larger chunks than this, as generally more than 4-5 code points are necessary for most language detection models.)

   If stopProducing returns true at any point during this process, then return null.

   If an error occurred during language detection, then return an error information according to the guidance in § 3.3.2 Errors .

5. Sort in descending order rawResult with a less than algorithm which given entries a and b , returns true if a ’s value is less than b ’s value .

6. Let results be an empty list .

7. Let cumulativeConfidence be 0.

8. For each key → value of rawResult :

   1. If value is 0, then break .

   2. If value is less than unknown , then break .

   3. Append «[ " detectedLanguage " → key , " confidence " → value ]» to results .

   4. Set cumulativeConfidence to cumulativeConfidence + value .

   5. If cumulativeConfidence is greater than or equal to 0.99, then break .

9. Assert : 1 − cumulativeConfidence is greater than or equal to unknown .

10. Append «[ " detectedLanguage " → " und ", " confidence " → 1 − cumulativeConfidence ]» to results .

11. Return results .

The post-processing of rawResult and unknown essentially consolidates all languages below a certain threshold into the " und " language. Languages which are less than 1% likely, or contribute to less than 1% of the text, are considered more likely to be noise than to be worth detecting. Similarly, if the implementation is less sure about a language than it is about the text not being in any of the languages it knows, that language is probably not worth returning to the web developer.