In-browser inference
Upscaling and background removal run on your own GPU or CPU. This page explains how the backend gets chosen, why cross-origin isolation is required, and where the first-load cost comes from.
The runtime
Both tools share a single copy of ONNX Runtime Web (1.30.0); the models are standard
ONNX graphs. The runtime files (about 25.5 MB) are deployed with the site under /ort/
rather than loaded from a CDN — no third-party dependency, and no exposure to CDN availability.
How the backend is chosen
The page probes device capabilities on load, then tries backends in this order:
| Backend | Condition | Speed |
|---|---|---|
| WebGPU | Browser and GPU driver both support it | Fastest. Runs on the GPU using fp16 weights. |
| WASM (multi-threaded) | No WebGPU, but the page is cross-origin isolated | Moderate. CPU-bound, parallelised across up to 4 threads. |
| WASM (single-threaded) | Neither of the above | Slowest. Still works, just takes longer. |
If WebGPU fails to initialise it falls back automatically to WASM rather than failing outright. The UI always shows which backend is actually in use.
Cross-origin isolation
Browsers do not hand out SharedArrayBuffer by default, and without it WASM cannot run
multi-threaded. The way to unlock it is to make the page "cross-origin isolated" via two response headers:
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corpThis is a performance switch, not a nice-to-have. Without them, every machine lacking WebGPU drops to single-threaded WASM and runs several times slower.
The headers live in public/_headers (a Cloudflare Pages format). Other hosts need their
own equivalent — see Deploying.
What COEP does and does not block
require-corp sounds alarming, but it only constrains no-cors subresources —
things loaded by <img>, <script> and <link>.
The site only loads its own assets today, which is why this is safe.
fetch() is the exception: it goes through the CORS channel and is not
subject to COEP. That is exactly how background removal pulls weights from HuggingFace — even though
HuggingFace's responses carry no Cross-Origin-Resource-Policy header at all.
fetch(), never via
<img> or <script> — switching the loading mechanism would get
them blocked by the browser.
The cost of a first visit
| Item | Size | When |
|---|---|---|
| Main JS + CSS | ~470 KB (132 KB gzipped) | First page load |
| ORT runtime | 25.5 MB → 6.3 MB over the wire | First use of inference |
| Upscaling model | 4.9 MB | First upscale |
| Removal model (lite) | 94 MB | First removal |
| Removal model (no-WebGPU variant) | 192 MB | First removal without WebGPU |
Downloaded models are stored in the browser's Cache Storage and not requested again.
Why 25.5 MB of runtime only transfers 6.3 MB
ORT's wasm binary is 25.5 MB, over Cloudflare's 25 MiB single-file limit, so a direct
deploy would be rejected. The build therefore stores it gzipped (~6.3 MB), and the browser decompresses
it via DecompressionStream into a Blob URL before handing it to ORT. That also cuts
first-load volume by three quarters.
Why model weights are not committed
The ORT runtime is generated at build time and does ship inside dist/.
Model weights are handled differently on each side:
- Upscaling model (4.9 MB) is committed to
public/models/. - Removal models (94–452 MB) are not committed; they are fetched from HuggingFace at runtime and cached.
The reason is size: the removal weights are far past what any static host will accept. The upscaling model is small enough to keep in the repo, which guarantees the build output is complete and does not depend on a third-party site being reachable.