Intl for engines without one (niso-intl)
Generated from docs/engines/intl.md when the site was built at 7aa23e2.
QuickJS and Porffor have no Intl; VS Code’s workbench fails at startup with
“ReferenceError: Intl is not defined”. crates/niso-intl gives such an
engine ECMA-402 on ICU4X 2.4: getCanonicalLocales, supportedLocalesOf,
Intl.Locale, NumberFormat, DateTimeFormat, Collator, PluralRules, Segmenter,
RelativeTimeFormat, ListFormat and DisplayNames, and the toLocaleString,
toLocaleDateString, toLocaleTimeString, localeCompare and
toLocaleUpperCase/LowerCase methods of Number, BigInt, Array, TypedArray,
Date and String. DurationFormat is not built.
js/intl.jsis the spec’s algorithms in JS: CanonicalizeLocaleList, ResolveLocale and the lookup matcher, GetOption and the digit options, the constructors, brand checks, the legacy NumberFormat fallback symbol, and arrays built with defineProperty so a test that taints Array.prototype or Object.prototype sees nothing. The order in which options are read and converted is observable, and it is easier to match the spec text line by line in JS than through an engine’s C API. It captures the intrinsics it uses when it loads, so a page that replaces a global later changes nothing.- One native call does what needs locale data:
niso_intl_call(op, strings) -> strings(include/niso_intl.h). Its ops are canonicalize, maximize, minimize, lookup, numberingSystem, numberFormat, the DateTimeFormat ops, collatorLocale, compare, plural, pluralRange, pluralCategories, segment, relativeTime, listFormat, displayName and caseMap. numberFormat takes the resolved options and an exact decimal string and returns the formatted parts; rounding (all nine modes, increments, significant and fraction digits, morePrecision and lessPrecision) runs in Rust on fixed_decimal. This call is also the seam for a driver over a system’s own ICU, should a platform ever need one; there is none today, ICU4X runs everywhere. bindings/js/intl_hosts.cis the host function over that call, written againstniso_engine.h’s qe_* API, so every engine behind that interface can use it unchanged. It lives with the bindings, so niso-intl itself is only a C ABI.
Where it sits
Section titled “Where it sits”A new layer, engine builtins, between the C API composition and the
bindings (architecture.md 2.1, rule 8, cargo xtask layers). Intl is part
of the language, not the web platform, so it is not a web API crate, and
rule 6 forbids bindings from depending on web API crates anyway. It is not a
binding either: binding to binding is a sideways edge that would need a new
allowlist entry, and that list only shrinks. An engine builtin depends on no
niso crate, and only a binding, composition or program links it.
Engines
Section titled “Engines”- QuickJS (niso-quickjs): default feature
intl. build.rs splices intl.js into the glue at/*@INTL@*/and compiles intl_hosts.c withNISO_INTL, which appends niso-intl’s hosts toniso_js_host_at. The spliced code installs nothing whenG.Intlalready exists. - Porffor (niso-porffor): the same two steps and the same default feature, in crates/niso-porffor/build.rs. Porffor compiles intl.js ahead of time with the glue. A smoke test of NumberFormat, DateTimeFormat, Collator, PluralRules, Segmenter, RelativeTimeFormat, ListFormat and the toLocale methods prints the expected results.
- The host libraries, niso-app (iOS) and niso-android, take either engine
(the quickjs or porffor feature) and turn on
intlby default. Theirintlandintl-english,intl-full,intl-namesandintl-dictionariesfeatures pass on to whichever engine is built, so Porffor with Intl is--no-default-features --features porffor,intl, as hosts/apple/build.sh and hosts/android/native.sh build it. niso-jvm has the sameintlfeature, off by default; niso-android turns it on. - Node-API (V8 under Node and Deno, JavaScriptCore under Bun): skipped. The engines have their own Intl, and niso-napi’s build leaves the marker empty.
Locale data
Section titled “Locale data”Which data a program carries is the product composition’s choice, not
niso-intl’s: niso-quickjs’s features intl (curated), intl-english,
intl-full, intl-names and intl-dictionaries pass it through. The
default everywhere, iOS, Android and Linux included, is curated.
data/generate.sh builds a binary that lists the data markers niso-intl’s
code uses, then data/gen exports just those from CLDR 49 and ICU4X’s
segmentation models as postcard blobs, compressed with zstd -19 and
committed under data/. build.rs decompresses the chosen ones into the
binary. Rerun generate.sh when the code starts using new data.
| File | Holds | zst bytes |
|---|---|---|
| curated.postcard | 26 language families: en en-GB de fr es it pt pt-PT ja ko zh zh-Hant ru tr pl cs hu nl sv ar he hi id th vi uk, each with its regional variants; only the default zh and zh-Hant collations | 391,192 |
| english.postcard | the en family, without Chinese collation | 143,075 |
| names.postcard | display names, time zone names and currency long names for the curated locales | 494,907 |
| dictionaries.postcard | word segmentation dictionaries and LSTM models for Chinese, Japanese and Southeast Asian scripts | 2,035,645 |
full is ICU4X’s own compiled data, every locale CLDR has. names and
dictionaries are not compiled in by default. An app can opt in through the
features, or bundle either file as an asset and hand it over at run time
with niso_intl_load_data. Without dictionaries, word segmentation of
Chinese, Japanese, Thai, Lao, Khmer and Burmese falls back to UAX #29
rules.
Loading data at run time
Section titled “Loading data at run time”niso_intl_load_data(data, len) (include/niso_intl.h) is plain C, so any
host can call it: hosts/apple/build.sh copies niso_intl.h next to niso_app.h,
and the symbol is in niso-app’s static library and libniso_android.so. A
JNI method on the Android host and a Swift wrapper on iOS are follow-ups.
- A request goes to the newest blob that has both its data marker and its locale, then the older ones, then the build’s, so a blob can add locales as well as data (a blob with en’s decimal symbols does not hide the build’s de).
- Each load swaps in a new provider; formatters made from the one before own their data and hold on to just the blobs they came from, so nothing is leaked.
- The shim learns again what it had cached from the data (available locales, the default locale, calendars and hour cycles, collations) after a load: it checks a load count before using them.
- Blobs must come from a trusted source, like the app’s own assets. ICU4X checks a blob’s structure but trusts its contents.
- A full build has every locale already and returns 2 without loading.
ICU4X keeps no list of available locales, so a locale counts as available if its decimal symbols, plural rules or list patterns resolve to something other than root. The default locale is the system’s (sys-locale: the device’s language on iOS and Android, the locale environment on Linux), matched to the nearest locale the data has (fr-CA becomes fr when only fr is there), else en-US.
What it costs
Section titled “What it costs”Measured at af15174 with curated, against builds with the intl
feature off (the APK’s at a9089e3; the commits since touch only niso-intl
and tests):
| Program | Without Intl | With Intl | Added |
|---|---|---|---|
| niso-run, stripped (Linux x86-64) | 34,437,312 | 37,507,520 | 3,070,208 |
| niso-run, zstd -19 | 11,191,047 | 12,070,246 | 879,199 |
| radio APK (arm64) | 30,148,203 | 32,868,075 | 2,719,872 |
| radio APK, zstd -19 | 11,514,693 | 12,343,301 | 828,608 |
| libniso_android.so | 25,455,088 | 28,174,960 | 2,719,872 |
Installed on the device the APK grows by about 2.7 MB too, though two measurements of the same APK without Intl disagreed (29,384 KB and 32,556 KB by du), so that figure is approximate.
Where it goes, measured again at 93473a9 (niso-run, release profile,
Linux x86-64, by size -A on the unstripped binaries and nm symbol
sizes by crate):
| niso-run | Stripped | Added | zstd -19 | Added |
|---|---|---|---|---|
| no Intl | 38,103,512 | 12,249,492 | ||
english |
40,195,160 | 2,091,648 | 12,893,495 | 644,003 |
curated |
41,165,912 | 3,062,400 | 13,143,223 | 893,731 |
About half of it is data and half code, and the code is the same with either data set:
- Data. build.rs compiles the blob in decompressed: curated.postcard is
1,444,163 bytes in the binary (391,192 as the zst file in the
repository), english.postcard 472,287.
.rodatagrows by 1,627,776 with curated and 657,024 with english; the rest of that, about 184 KB in both, is the code’s own constants. - Code, about 1.6 MB:
.text1,180,048, unwind tables (.eh_frame,.eh_frame_hdr,.gcc_except_table) 178,768, and.data.rel.ro29,560. By crate (symbol sizes, generic instantiations counted under the crate that defines them): icu_datetime 166 KB, icu_provider 150 KB (the data loading and deserialization, once per data marker), niso_intl 114 KB, slice instantiations 107 KB, icu_experimental 73 KB, icu_locale_core 68 KB, jiff and jiff_core 119 KB, icu_segmenter 57 KB, yoke 45 KB, icu_collator 41 KB, icu_locale 27 KB, zerovec 22 KB, icu_decimal 21 KB, and under 20 KB each for icu_calendar, zerotrie, icu_plurals, fixed_decimal, icu_casemap and the rest.
Startup: a QuickJS context (Js::new, release, Linux x86-64, median of
12 interleaved runs of 40 contexts each, on a loaded machine) takes
42.2 ms with Intl and 38.8 ms without, so about 3.4 ms a context or
worker. Nearly all of it is QuickJS compiling the shim: with the shim
compiled but not run, a context takes 41.9 ms. Installing Intl lazily, as
an accessor that runs the shim on first use, would save the remaining
0.3 ms. It would also make globalThis.Intl an accessor where ECMA-402
has a data property, and capture the intrinsics at first use rather than
before the page runs. So Intl installs eagerly. Saving the compile time
means precompiling the glue to QuickJS bytecode, which would help the
rest of the glue as well.
Time zones
Section titled “Time zones”Offsets come from the system’s TZDB through jiff (/usr/share/zoneinfo on Linux, macOS and iOS), and the default zone from iana-time-zone, looked up again each time a DateTimeFormat is made. Only Windows and wasm, which have no TZDB, get jiff’s bundled copy. Zone names (for timeZoneName) come from names.postcard. Checked on the iOS 27 Simulator and an Android 36 emulator.
On Android, niso-intl reads /apex/com.android.tzdata/etc/tz/tzdata first:
the tzdata APEX module, which Google updates apart from the system image
(Android 10 and later). jiff reads only the system image’s copy and the
pre-APEX update path, so a device with an old system image but an updated
module gets the old rules from jiff alone; jiff issue #582 reports this
(Asia/Amman off by an hour on Android 11). Upstream note: the fix in jiff is
to try the APEX path ahead of the two it has, the order bionic’s
__bionic_open_tzdata uses. The issue was closed asking for a
reproduction with tauri-jiff; niso-intl’s datetime::tests::android_apex_first
covers the order on our side, and the emulator above has the APEX file
(tz_version, tzdata). Once jiff reads it, the APEX lookup here can go.
A system with no TZDB (a minimal Linux container) has only UTC, GMT and offset zones: a timeZone the TZDB has no rules for is a RangeError when the DateTimeFormat is made, as an unknown zone is, and the default zone is UTC.
Status
Section titled “Status”test262 intl402 (pinned in tests/test262/fetch.sh), subtests passed on QuickJS, the same on Linux and macOS:
| Area | Passed |
|---|---|
| All of intl402 | 2044/2564 (46 before niso-intl; Node 26.10.0: 2448/2560) |
| Array, BigInt, Date, Number, String, TypedArray prototype methods | 104/104 |
| Collator | 126/130 |
| DateTimeFormat | 308/360 |
| DisplayNames | 112/114 |
| ListFormat | 160/162 |
| NumberFormat | 458/502 |
| PluralRules | 102/106 |
| RelativeTimeFormat | 158/160 |
| Segmenter | 150/158 |
| Locale | 242/380 |
| Intl (getCanonicalLocales, supportedLocalesOf and the rest) | 82/132 |
| DurationFormat | 0/208, not built |
Porffor isn’t in this table: it compiles scripts ahead of time and cannot evaluate test262 files. These are the known gaps:
- DurationFormat.
- Units. ICU4X 2.4’s experimental unit formatter has only area, duration, length, mass and volume, with no speed, temperature, digital or “-per-” units, so unit style prints the identifier. icu4x-units.md is a write-up of what ICU4X would need.
- Localized NaN (zh-TW’s 非數值). ICU4X’s decimal symbols have no NaN.
- Per-locale range patterns (pt-PT’s “3 - 5 €”).
- Digits for numbering systems without data. ICU4X has symbols for some, and other simple-digit systems (ECMA-402 table 14) are transliterated from latn.
- Collator
usage: 'search'sorts as'sort'. - DisplayNames has no names for 5 to 8 letter language codes, and its names come only from names.postcard, which the default build leaves out.
- Intl.Locale’s locale-info methods (getCalendars, getCollations, getHourCycles, getNumberingSystems, getTextInfo, getTimeZones, getWeekInfo) fail in 68 files.
currencyDisplay: 'name'without names.postcard formats with the code (“USD 1.00”).- A year, month or era with timeZoneName and no day or time has no ICU4X field set, so the period and the zone are formatted apart and joined with what the locale puts between a date and its zone.
- dayPeriod. ICU4X’s field bag takes the flexible day periods but formats AM/PM, so dayPeriod: ‘short’ gives “2 PM” rather than “in the afternoon”, and resolvedOptions has no dayPeriod (the dayPeriod-*-en.js and order-dayPeriod.js tests).