WPT
Generated from docs/wpt.md when the site was built at 7aa23e2.
crates/niso-wpt runs web-platform-tests’ testharness.js tests headless, each in its
own process, from the pinned WPT checkout tests/wpt/fetch.sh puts in third_party/wpt
(all of WPT: 1.3 GB with its .git, under a minute to fetch). It takes its tests from
WPT’s own manifest for that revision, so only testharness.js tests run: reftests,
manual, crash, print and visual tests, and the helper pages tests load, never run
as if they were tests, and tests that use testdriver.js run with niso-wpt’s own
testdriver (below).
tests/wpt/fetch.sh # WPT, and the Baseline datacargo run --release -p niso-wpt # the Baseline suite, against the expectationscargo run --release -p niso-wpt -- --update # record it as the expectationscargo run --release -p niso-wpt -- dom/nodes # the tests under some directoriescargo run --release -p niso-wpt -- --features grid,fetch # the tests WPT maps to some featuresnode tests/wpt/baseline.mjs # score this system's expectationscargo run --release -p niso-wpt -- --engine node # the same tests on Node-API, into target/wpt-napi/tests/test262/fetch.sh # test262's harness and intl402 testscargo run --release -p niso-wpt -- --suite test262 # intl402, against tests/test262's expectationsThe suite
Section titled “The suite”By default niso-wpt runs the Baseline suite: every test WPT maps to a
web feature that is Baseline, widely or newly available, and every test under the
directories tests/wpt/suites.txt lists (what the suite was before, kept so
nothing that ran stops running). The mapping and the statuses come from the pinned
WPT and web-features (below), so the suite follows them when they move. Features
tests/wpt/excluded-features.txt lists, each with its reason, are left out of the
suite and the score.
ECMAScript itself is out of scope. The JavaScript group of web-features (arrays,
promises, iterators and the rest of the language and its library) is tested by
test262, the engines’ own conformance suite, not by WPT, and niso takes its
JavaScript from the engine it embeds (QuickJS-ng, Porffor) rather than
implementing it, so excluded-features.txt lists it. What WPT does test around
JavaScript (modules, workers, SharedArrayBuffer, transferable buffers) stays
in. Intl stays in too: ECMA-402 needs locale data niso has to supply, since the
engines don’t bring it, so the Intl features count in the score, by test262’s
intl402 tests (test262 intl402) rather than WPT’s.
tests/wpt/tested-elsewhere.txt names that suite as the reason a feature with
no result from it isn’t run. sourcemap-header is excluded until niso has
devtools.
A run’s results go to target/wpt/results.json (with how long each file took, in
ms) and its score to target/wpt/baseline.json. A plain run changes no tracked
file.
wptserve
Section titled “wptserve”Tests run against WPT’s own server, as in a browser’s run: niso-wpt starts
third_party/wpt/wpt serve for the run, at WPT’s usual ports (8000 and 8001 for
http, 8443 and 8444 for https, 8888 and 8889 for WebSockets), and stops it and the
servers it started at the end. It needs python3; the first run makes wptserve’s
virtualenv in target/wpt/venv, which downloads its packages. Its log is
target/wpt/wptserve.log.
Each run’s server listens on a loopback address of its own on Linux, where all of
127/8 is loopback (127.x.y.z), so runs side by side don’t collide; on macOS,
where only 127.0.0.1 is, it listens there. The tests’ processes reach it through
niso-core’s wpt-network feature, which only niso-wpt enables, so no product
build has it: net::use_wpt_server sends WPT’s hosts (web-platform.test,
not-web-platform.test and their subdomains) to that address, keeping their ports,
for fetch, resources, workers’ scripts and WebSockets, and trusts WPT’s test CA
(tools/certs/cacert.pem) and no other. That is what WPT’s own runners get from a
hosts file and an installed CA, without either.
Tests run at http://web-platform.test:8000/, or https://web-platform.test:8443/
for .https. ones. niso-wpt still serves the checkout’s plain files itself, as
they are on disk; what only wptserve makes goes to it: Python handlers, .asis
files, substitution (.sub., so those tests run too), pipes, and headers from a
.headers or __dir__.headers file beyond a type and caching. A test’s process
ends as soon as it reports its result, as a browser’s runner moves on, so a
WebSocket a test leaves open or a request no server answers doesn’t hold it.
niso-wpt --one <file> (a path relative to third_party/wpt, or absolute)
runs one test the same way: it starts its own wptserve and stops it at the end,
or uses the one at NISO_WPT_SERVER (its address) when that is set.
Expectations
Section titled “Expectations”tests/wpt/expectations.json records, for every test file, its harness status,
its subtest count and which subtests pass: their names, or when fewer the names
of those that fail, or neither when they all pass. Since a file that lists its
failures takes the rest of a run’s subtests to pass, a file that reports a
different number of subtests than recorded (a new subtest in WPT, or one that no
longer runs) isn’t compared but fails the gate until --update records it.
Results can differ by
system, mostly through fonts: text and math are laid out with the system’s fonts,
so a test that measures them can pass on one and fail on another. So
tests/wpt/expectations.linux.json and tests/wpt/expectations.macos.json hold
the subtests a system passes or fails besides the shared file, or, for a file
whose subtest count differs on that system, a whole expectation of its own; a
system’s expectations are the shared file with its own on top. macOS, for one, fails 23
subtests in four mathml/relations/css-styling files that Linux passes, all of
them measuring <mroot> and <msqrt>, whose radical sign comes from a font.
--update records a system’s results as its expectations. What differs from the
shared file goes into the system’s own file, and a difference every other
system’s file records too moves into the shared file, so the shared file holds
what all the systems agree on; a subtest count likewise, while a harness status,
which only the score reads, comes from the last update. A full
--update records the run alone, so files that left the suite drop out
everywhere, and rewrites the tracked score, tests/wpt/baseline.json, from the
new expectations (the committed one is Linux’s).
Every branch that runs --update rewrites these files, so git’s line merge
conflicts on them whenever two branches recorded results. .gitattributes gives
them a merge driver, tests/wpt/merge.mjs, which tests/wpt/setup-merge.sh sets
up in a checkout (tests/check.sh runs it). It merges an expectation file per
test file: a test one side changed takes that side’s entry, and a test both
sides changed differently takes the entry of the branch being merged in and is
printed, so the next gate run checks it. baseline.json, derived from the
expectations, keeps ours and is stale until node tests/wpt/baseline.mjs or
the next full --update rewrites it. node --test tests/wpt/merge.test.mjs
tests the driver.
Upstream failures
Section titled “Upstream failures”Some expected failures are bugs in the crates niso builds on, each with a repro
under repros/; they pass again once the crate is fixed and the pin moves:
css/filter-effects/animation/filter-interpolation-002.html, the four subtests at (2147483648): Stylo mixes the colours with f32 weights that cancel at 2^31 and keeps the start colour (repros/stylo-color-animation-weights-cancel).
The gate
Section titled “The gate”Some tests aren’t deterministic, so a result only changes when three runs agree:
a file whose results aren’t what its expectations say runs twice more. A run
exits 1 when a subtest the expectations list fails in all three runs, or a file
reports a different number of subtests in all three, unless
tests/wpt/flaky.txt lists its file. --update records a new pass, a new
failure, a harness status or a subtest count only when all three runs agree, and
leaves the expectations of files flaky.txt lists as they are. That list is a
list of bugs, each group with its suspected cause: fetch tests racing the virtual
clock against the real network, a test that reads Date.now() (the real clock)
across virtual-time timers, and tests that only fail on a busy machine. A file
leaves it once its results are stable, and its expectations are recorded again.
Bugs the suite found
Section titled “Bugs the suite found”- A response’s
statusTextis the status code’s standard reason phrase, not the server’s: ureq doesn’t keep the reason phrase (fetch/content-length/api-and-duplicate-headers.any.js,xhr/status-basic.htm,xhr/status-async.htm). - Header values with control bytes the Fetch standard allows (all but NUL, CR
and LF) fail in ureq’s
httptypes, so such a request is a network error (fetch/api/headers/header-values.any.js). - niso has no Content Security Policy.
script-src-wildcards-disallowed.htmlpassed itsdata:case only whiledata:scripts failed to load in niso-wpt; they load now, as fetch has them. - Mixed content: a client with an HTTPS state (an https page, or a blob: or data: worker it started) gets a network error for an http request, and an http image is upgraded to https. Requests to the machine itself are exempt. Opaque-response blocking checks only its never-sniffed MIME types, so it doesn’t sniff an opaque response’s body.
- For wptserve’s URLs niso-wpt’s build fetches a link’s resource synchronously, so
a response the server holds open stalls the page
(
preload/modulepreload-fires-before-dependencies.html). A product build fetches what only the network has in the background and fires the link’s event when it arrives. - ureq writes header names in lower case, so a server sees
content-typewhatever case the page gave (xhr/setrequestheader-*.htm,xhr/send-accept-language.htm,xhr/request-content-length.any.js). - ureq inflates a gzip or deflate body itself and drops its
Content-EncodingandContent-Length(xhr/response-data-gzip.htm,xhr/response-data-deflate.htm). - A response whose headers end where the connection closes, without the blank
line after them, is a network error in ureq (the
.asisresources ofxhr/getresponseheader.any.jsandxhr/getallresponseheaders.htm). - XMLHttpRequest:
- The body arrives whole, so a download fires one progress event
(
xhr/response-data-progress.htm). - A synchronous request blocks the thread until its response arrives, and its timeout is checked then. In a window it can have no timeout or responseType, as the standard has it; in a worker it can.
open()encodes a URL’s query in UTF-8, not in the document’s encoding (xhr/open-url-encoding.htm).- An image button that submits has the coordinates (0, 0) in a form’s
entry list, and a form’s submission fires no
formdataevent (onlynew FormData(form)does). - A shared
WebAssembly.Memory’s buffer isn’t a SharedArrayBuffer, soxhr/send-data-sharedarraybuffer.any.jscan’t make one. - Tests that open windows or iframes, navigate, or wait on the document’s
readiness time out:
xhr/open-url-multi-window*.htm,xhr/access-control-sandboxed-iframe-*.htm,xhr/send-entity-body-document.htm,xhr/sync-xhr-and-window-onload.html.
- The body arrives whole, so a download fires one progress event
(
- DOM Parsing:
- niso-wpt loads an
.xhtmltest as HTML, so the XHTML tests (domparsing/*.xhtml) fail; DOMParser parses XHTML as XML. - XMLSerializer follows WPT where it and the standard differ (the
standard’s issues 29, 44 and 52), except that an
xmlns=""that agrees with an element’s namespace is dropped, as the standard has it. - A document DOMParser makes is never the window’s, so its
<meta charset>changes nothing (domparsing/DOMParser-parseFromString-encoding.html).
- niso-wpt loads an
- niso-wpt reads a page’s script whose src has a query
(
/cors/support.js?pipe=sub) from the checkout, without the query’s pipes, so its{{…}}stay (xhr/send-conditional-cors.htm); about 660 files have such a script. - Before
niso-wptran against wptserve, these tests passed only because every request to a handler failed, or the in-process server answered a Python handler with its source; the expectations lost those passes then.
Run time
Section titled “Run time”A full run of the Baseline suite (19,734 files) against wptserve at the pinned revision, release build, with other work on both machines (their load averages are given):
| Machine | Jobs | Run | --update |
Load average |
|---|---|---|---|---|
| yachiyo: Core Ultra 7 265K, 20 cores, Linux | 20 (default) | 301 and 358 s | 203 and 644 s | 18 to 53 |
| beako: M4 Pro, 12 cores, macOS | 12 (default) | 316 and 341 s | 333 and 484 s | 3 to 22 |
Before niso-wpt ran against wptserve, the suite had 19,185 files and a run took
142 to 290 s on yachiyo (median 175 s over seven runs, load 15 to 30) and 300 s
on beako (load 35 to 46). Eight files take about 60 s each now: fetch tests that
time out (fetch/range/general, fetch/api/abort/general, each in a window and
a worker), two modulepreload tests, websockets/keeping-connection-open/001.html,
and resize-observer/notify.html, which hangs. Under load the pbkdf2 tests
come close.
Fetching all of WPT took 36 s on yachiyo and 63 s on beako; building niso-wpt
from cold took 68 s on beako. Reruns of failing files add a few seconds. Fewer jobs
don’t make a run steadier than the reruns and flaky.txt already do.
test262 intl402
Section titled “test262 intl402”niso-wpt --suite test262 runs test262’s ECMA-402 tests, test/intl402 of the
revision tests/test262/fetch.sh pins (a sparse checkout of the harness and
intl402 into third_party/test262), with WPT’s runner: a process per file, the
expectations in tests/test262 in the same compact form, a result that changes
only when three runs agree, --update, --filter, directories (relative to
the checkout) and --engine node (results and diff.md in
target/test262-napi/) work as they do for WPT. Tests of the test262 features
tests/test262/excluded-features.txt lists don’t run: Temporal, which is the
engine’s.
A file runs once in each mode its frontmatter allows (default and
strict mode unless its flags say onlyStrict, noStrict or raw), each in
a page of its own, and each mode is a subtest. The page runs a prelude with
print and what of $262 a page can give (global, evalScript as indirect
eval, detachArrayBuffer by transfer, no createRealm or agent), then
assert.js, sta.js, doneprintHandle.js for an async test and the files it
includes, then the test as a classic script ending in a statement that notes it
ran to its end. A test passes when it ran to its end and nothing threw, an
async one once it called $DONE without an error, and a negative test when it
didn’t run to its end and what threw, where the page says, is the error it
expects. niso fires no error event at a window for an uncaught exception
(glue.js’s report writes it out), so a failure’s message is the last
Test262Error the harness made, or just threw. intl402 has no negative or
async tests at this revision.
The Baseline score counts each intl402 file as a script test of the web feature
tests/test262/web-features.txt maps it to, by a test262 feature its
frontmatter lists or its directory, and reads this system’s test262
expectations, or a run’s results with --test262-results FILE. tests/check.sh
runs the suite after WPT.
Baseline score
Section titled “Baseline score”The Baseline score says how close niso is to the web’s
Baseline feature set, by how many of each feature’s
WPT tests it passes. tests/wpt/baseline.mjs computes it from a run’s results.
tests/wpt/fetch.sh downloads, into third_party/wpt-features and each checked
against its pinned SHA-256:
- web-features’
data.jsonfrom its npm package (WEB_FEATURES): every feature’s name, group, and Baseline status:high(widely available),low(newly available) orfalse(limited availability,limitedin the score). - WPT’s
MANIFEST.jsonandWEB_FEATURES_MANIFEST.jsonfor the pinned revision, the assets of its release (REV_RELEASE, themerge_pr_*release whose commit isREV). The test manifest lists every test and its kind; the web-features manifest maps features to test URLs. WPT generates it (./wpt web-features-manifest) from theWEB_FEATURES.ymlfiles in its directories, with their globs, exclusions and inheritance from parent directories, and wpt.fyi reads the same file, so the mapping is exactly wpt.fyi’s. - wpt.fyi’s summary of a Chrome stable run at the same revision (
CHROME_RUN), for each test’s subtest count.
To move to a newer WPT, pick a revision that has both a merge_pr_* release and a
Chrome run on wpt.fyi (https://wpt.fyi/api/runs?sha=REV&label=master), update
REV, REV_RELEASE, CHROME_RUN and the checksums, bump WEB_FEATURES to a
release from around then, and run --update. An id web-features doesn’t know (yet,
or any more) is left out and listed under unknownIds; a feature that moved counts
under its new id.
What it counts
Section titled “What it counts”A test is a test URL without its variant’s query: niso-wpt runs a file once,
without one, which runs every subtest the variants split between them. Each of a
feature’s tests is a script test (testharness.js without testdriver.js), a
testdriver test (testharness.js with it), a reftest (print reftests included), a
manual test, or other (crash, visual, WebDriver and AAM tests); script and
testdriver tests run. Per feature, baseline.json has its tests of each kind,
how many ran (ran, and of each kind scriptRan and testdriverRan), why each
test that didn’t run didn’t (notRun), and its subtests: in Chrome’s run over its
script and testdriver tests (wpt), in the tests that ran (ran) and passing
(passed), and those two in its script and its testdriver tests apart
(script, testdriver). The reasons a test doesn’t run are:
needs-reftests: the runner below doesn’t exist yet;manual,other: a person, a crash harness, WebDriver or an accessibility API;worker-variant: a shared or service worker or shadow realm variant, which niso has none of;tentative: in atentativedirectory;not-html: an XHTML or SVG test, whichniso-wptdoesn’t load;not-in-suite: a test of a feature outside the suite (limited availability);no-result: in the suite but missing from the results scored.
A feature tested-elsewhere.txt lists, with no WPT test that ran, is not run
for the reason it gives there (test262-intl402), not counted as having no tests.
A test whose harness didn’t finish OK (it errored, timed out or crashed, often
before registering most of its subtests) has as many subtests as Chrome’s run of
it if that’s more, so a test that dies early counts as all of its subtests
failing rather than as one; a test that reported none counts as one subtest. A
feature’s pass rate is its passing subtests over the subtests of its tests that
ran. Its coverage is ran (a script or testdriver test ran), not-run (it has tests, none of
which ran; notRunBecause names the reason most of them have) or unmapped (WPT maps
no test to it).
The score is one system’s: system names it (this system’s by default, or
--system linux|macos). Scoring the expectations, each feature that ran also
has systems, its subtests passed and run on every system with an
expectations.<os>.json, as [passed, ran], which the docs site’s Baseline
banners show per system.
For each Baseline status, without excluded features, the summary has how many
features there are, how many have tests, reftests and testdriver tests, how many
ran, how many pass all their subtests and at least 90% of them, the tests by kind
and the reasons the rest didn’t run, the subtests passed over all of them and in
script and testdriver tests apart, their rate (subtestPassRate), and the mean of the features’ own pass rates (meanPassRate).
The subtest rate is dominated by the features with the most subtests (text-encoding
alone has over a million in Chrome’s count), so the mean of the features’ rates is
the one to read as how much of each feature works.
Caveats
Section titled “Caveats”- Unknown isn’t failing. A feature no test of ran has no pass rate, whatever its tests would show; the summary counts those features by why.
- Features without WPT tests. Some features have no tests mapped at all, often
because they are tested elsewhere (test262) or haven’t been mapped yet. They
count as
unmapped, never as passing or failing. - Tests that need more than a headless page. Reftests compare rendered pixels, and testdriver tests get only the input calls below.
- Incomplete mappings.
WEB_FEATURES.ymlfiles cover only part of WPT, and a glob maps whole directories, so a feature can get tests that are about another one, and many tests map to none (most ofdom/nodes, which is whysuites.txtstays in the suite). A feature’s pass rate is only as good as its mapping. - The body’s
overflowdoesn’t reach the viewport. CSS gives the viewport the body’soverflowwhen the root’s isvisible, and leaves the body withvisible; blitz has no such propagation, so a body withoverflow: autoand a height scrolls itself, and the viewport doesn’t. The patch for blitz and its repro are inrepros/blitz-dom-overflow-propagation. - Subtest counts. Chrome’s count of a test’s subtests sums its variants and can differ from what niso’s run of the same file registers; it stands in only where niso’s count can’t be trusted (a harness that didn’t finish) and for tests that didn’t run.
testdriver
Section titled “testdriver”Tests that load /resources/testdriver.js (1,283 files in the suite) run with
niso-wpt’s own test_driver_internal, WPT’s vendor hook, which it puts in
place of testdriver-vendor.js (VENDOR in crates/niso-wpt). No WebDriver
or CDP is involved: click, send_keys, bless (a click) and
action_sequence queue their call, and the runner carries it out between the
page’s tasks, stepping virtual time a timer at a time so a call is done before
the clock moves on. Every other call (set_permission, virtual authenticators
and sensors, cookies, window management, accessibility, BiDi) rejects at once
with “niso-wpt’s testdriver doesn’t support …”, so those tests fail rather
than wait out their timeout. Calls aimed at another document or browsing
context reject the same way.
The input itself comes from crates/niso-actions: WebDriver’s actions
algorithm (ticks, pauses, pointer moves and scrolls stepped a frame at a time
over their duration, element origins, modifiers, Release Actions), Element
Click’s and Element Send Keys’ input, and WebDriver’s key table. It drives
niso through the core calls a host’s input reaches, as niso_dom_* makes them:
Dom::pointer, Dom::key, Dom::wheel, then what the host’s own views would
do with input nobody canceled (Dom::text_input and Dom::text_commit for a
key typed into the focused field, Dom::scroll for a wheel). It knows no
protocol and no JS engine, so niso-cdp’s Input domain and a BiDi
input.performActions can use it as is. As wptrunner does, each action
sequence starts by releasing what the one before left pressed.
At the pinned revision 490 of the testdriver tests’ 7,620 subtests pass, and 139 of the 1,283 files pass whole. These tests count in the baseline score since this driver landed; before it they were skipped as needing testdriver, so the headline numbers are not comparable with earlier baselines.
Element Click aims at the center of the element’s first client rect, and
getClientRects() gives a non-atomic inline a rect per line fragment, from
the inline layout. Neither it nor getBoundingClientRect() applies CSS
transforms yet, so a click on a transformed element lands where its untransformed
box would be. Most failures are
niso’s, not the driver’s (below). The testdriver tests add about 16 s to a run
on yachiyo.
What the testdriver tests fail on
Section titled “What the testdriver tests fail on”The 1,144 testdriver files that don’t pass whole, by what they need that niso
doesn’t have, most files first. A cause counts a file when its source uses the
feature (or, for the globals, its subtests fail with a ReferenceError), so
the counts overlap and are approximate, and a file can need several causes
fixed before it passes. “Unlocks” lists the Baseline features with the most
of those files.
| # | Cause | Files | Unlocks (files) |
|---|---|---|---|
| 1 | iframes and window.open: niso has no nested or second browsing context |
148 | scroll-to-text-fragment 19, pointer-events-api 17, draganddrop 11, user-activation 9, cookies 7 |
| 2 | testdriver calls the driver rejects: set_permission, virtual authenticators and sensors, cookies, accessibility, reports |
106 | media-capture 35, device-orientation-events 19, webauthn 14, screen-wake-lock 8 |
| 3 | What user activation is for: window.open, fullscreen, activation from another frame’s input (navigator.userActivation and transient activation from keys and presses work) |
85 | scroll-to-text-fragment 18, user-activation 16, screen-orientation 9, cookies 5 |
| 4 | Scrolling: smooth scrolling, scrolling from keys, scroll snapping, the body’s overflow reaching the viewport (scrollTo and scrollBy options and scrollend work) |
73 | scrollend 16, scroll-snap 12, touch-action 11 |
| 5 | attachShadow and declarative shadow roots |
66 | shadow-dom 30, popover 14, pointer-events-api 10 |
| 6 | Touch input and touch-action |
66 | pointer-events-api 34, touch-action 22 |
| 7 | Drag and drop: draggable, drag events, DataTransfer |
66 | draganddrop 56, pointer-events-api 6 |
| 8 | Popover (popover, showPopover, light dismiss) |
63 | popover 51, selectors 4, toggleevent-source 3 |
| 9 | PerformanceObserver and Event Timing entries |
48 | event-timing 44, largest-contentful-paint 3 |
| 10 | Pointer capture: setPointerCapture, gotpointercapture, lostpointercapture |
48 | pointer-events-api 44 |
| 11 | Selection: getSelection, text control selection |
44 | selection-api 23, shadow-dom 5, contenteditable 4 |
| 12 | <dialog> showModal and the modal top layer |
42 | dialog 27, popover 4, selectors 3 |
| 13 | Keyboard focus beyond Tab, Enter and space (which work): :focus-visible after key input, select and other controls’ keys |
27 | focus-visible 6, user-pseudos 2, select 2 |
| 14 | getCoalescedEvents and getPredictedEvents |
7 | pointer-events-api 7 |
| 15 | scrollIntoView (testdriver.js calls it for an element outside the viewport) |
6 | one each: content-visibility, scroll-snap, scrollend, vertical-form-controls |
| 16 | Elements by id as window globals (named access on Window) | 6 | one each: media-capture, pointer-events-api, web-audio, webauthn |
| 17 | click, auxclick and contextmenu as PointerEvents |
4 | pointer-events-api 4 |
| 18 | MouseEvent.getModifierState |
1 | pointer-events-api 1 |
Event handler content attributes, the first cause before (148 files), compile
now, and named access on Window, the third (103), is down to 6 files whose
ReferenceErrors name something else niso lacks. The 127 failing files that
use handler attributes fail for other causes: 59 of them matched no other row
of the old table, and they are mostly the scrolling, touch and pointer capture
tests that rows 4, 6 and 10 now count. Row 2 is the driver’s to widen, with a native
meaning behind most of it (permissions, sensors, passkeys; see Browser UI
features). Rows 1, 5 and 7 are the larger engine features. Rows 3, 4 and 13
still count every failing file that uses user activation, scrolling or keyboard
focus, though what they name first now works: the files left need the rest of
the row, or another row, to pass.
Not run yet
Section titled “Not run yet”- Reftests: 17,111 in the Baseline set (15,391 under
css/). 300 widely and 33 newly available features have them, and for 36 widely and 8 newly available features they are the only tests. They needniso-wptto paint a test and its references to pixels (a tiny-skia painter of niso’s render stream) and compare them with WPT’sfuzzyranges.
Browser UI features
Section titled “Browser UI features”Features about a browser’s own UI stay in scope, because each has a native meaning in niso. None of them is tested by WebDriver alone.
| Features | Native meaning |
|---|---|
alerts |
alert, confirm and prompt as the platform’s modal dialog |
download |
<a download> saves through the share sheet or Files on iOS, the system file picker or DownloadManager on Android, a save dialog on desktop |
print, print-events, page-setup, print-color-adjust |
the platform print APIs (UIPrintInteractionController, PrintManager), with beforeprint/afterprint around them and @page for the page |
pdf-viewer |
pdfViewerEnabled is true where the platform shows PDFs (PDFKit, PdfRenderer) |
window |
window.open and close as a new window or scene (UIScene, an Activity, a desktop window) |
title |
the window’s title and its label in the task switcher |
barprop |
an app has no location, menu or tool bars, so they report not visible, as in an installed web app |
history, location |
in-app navigation, with the system back gesture or button going back |
screen, screen-orientation, screen-wake-lock |
the display’s metrics, the orientation lock, keeping the screen on |
notifications-apps, push |
the platform’s notifications and push (UserNotifications and APNs, NotificationManager and FCM) |
clipboard-events, async-clipboard |
the system clipboard |
draganddrop |
the platform’s drag and drop, between apps too |
input-file, input-date-time, select |
the native file, date and option pickers |
geolocation, device-orientation-events, gamepad, speech-synthesis, media-capture |
the platform’s location, motion sensors, game controllers, text to speech, camera and microphone |
webauthn, credential-management |
passkeys and password AutoFill (AuthenticationServices, Credential Manager) |
prefers-color-scheme, prefers-contrast, prefers-reduced-motion, forced-colors, system-color, color-scheme |
the OS appearance and accessibility settings |
dynamic-range, color-gamut |
the display’s capabilities |
safe-area-inset, visual-viewport |
the safe area and the on-screen keyboard’s insets |
page-visibility |
the app going to the background and back |
spellcheck, autocorrect, enterkeyhint, inputmode, autofocus |
the native text field’s input traits and keyboard |
accesskey |
hardware keyboard shortcuts |
console |
the platform log (os_log, logcat) |
scrollbar-color, scrollbar-width, scrollbar-gutter |
the native scroll indicators, where the platform lets them be styled |
user-agent-sniffing, online, cookie-enabled, hardware-concurrency |
niso’s own answers: its user agent, the network’s reachability, its cookie store, the device’s cores |
meta-theme-color (the status bar or window colour), registerprotocolhandler and
app-protocol-handlers (URL schemes), share and app-share-targets (the share
sheet, both ways), manifest (the app’s name and icon), badging (the app icon’s
badge) and favicons (no web-features id) have native meanings too, but none is
Baseline in web-features 3.45.0, so they aren’t in the suite yet.