Models

Every AI feature here is built on open-source models, all under permissive MIT / BSD / Apache licenses that allow commercial use.

Upscaling: Real-ESRGAN

The upscaler is Real-ESRGAN General x4v3, a 4× super-resolution model. Real-ESRGAN was proposed by Xintao Wang et al. It targets degraded images from the real world — compression artifacts, mild blur, noise — rather than super-resolution under ideal conditions only.

Item	Value
Model size	4.9 MB
Scale factor	4×
Input size	Fixed 128×128; larger images are split into tiles
License	BSD-3-Clause (model weights)
ONNX export	Qualcomm AI Hub

Because the input is a fixed 128×128, larger images are cut into 128-pixel tiles, inferred one by one, then stitched back together. The seams need some overlap and blending, otherwise a grid pattern shows up — this is the standard approach in super-resolution apps.

Cutout: MODNet and BiRefNet

Automatic background removal offers three tiers, selectable in the UI. The default is the smallest, MODNet — proposed by Zhanghan Ke et al. (MODNet), a lightweight network trained specifically for portrait matting. It is an order of magnitude smaller, yet handles portrait high-frequency edges such as hair and semi-transparency cleanly. BiRefNet (Bilateral Reference Network), proposed by Peng Zheng et al., is a general dichotomous image segmentation model — it does people and objects alike.

Choosing between the three tiers

This is not purely a quality difference — it is a trade between how much you download and what the model can cut:

Tier	When to use it	Size
MODNet Default	Use this one for portraits. It has two known limits. First, it is trained specifically for portraits, so it degrades badly on products, logos and other non-portrait subjects — switch to HD when that happens (the UI will prompt you). Second, it always runs on CPU: this model does not hold up on WebGPU, where testing dropped its foreground coverage from 47.4% to 35.9% with a large bite taken out of the subject's edges. So it is simply never given the GPU — see "About WebGPU" below.	24 MB (fp32)
BiRefNet Lite HD	Use this for non-portrait subjects, or when detail matters. It is not picky about the subject type, and stays solid on hair, fur and complex structures.	94 MB (fp16) 183 MB (fp32)
BiRefNet Fallback	Only used when your device has no WebGPU. Results are essentially the same as Lite, but the file is far larger and much slower.	452 MB

Why every tier has two sizes. Each tier ships two sets of weights, picked by backend: fp16 on WebGPU, fp32 on WASM. The UI shows the one your device will actually download — otherwise you would see "94 MB" on a CPU and download 183 MB, and reasonably assume something broke. The default tier is the exception: it runs on CPU either way, so it has a single size.

Why the default is the smallest. A 94 MB download is a steep first-run barrier, and portraits are by far the most common case. The default is therefore MODNet at 24 MB — a quarter of the HD tier: solve "will the first run ever finish?" before worrying about "is one pass perfect?". The 6 MB quantized build was deliberately rejected — testing showed it is lossless on portraits but falls apart on objects. Half the size for "completely unusable on non-portraits" is a bad trade.

Technical details

Item	MODNet (default)	BiRefNet (HD)
Input size	Aspect-preserving resize to a 512 short edge, then both sides aligned to a multiple of 32	Fixed 512×512, whole image in one pass (no tiling)
Input tensor name	`input`	`input_image`
Output tensor name	`output`	`output_image`
Output form	Sigmoid is already baked into the export; values are 0–1	Unactivated logits; sigmoid is applied to get a 0–1 alpha
Preprocessing	Normalise (x/255 − 0.5) / 0.5 → [−1, 1]	Direct resize to 512×512 (no letterbox), ImageNet mean/std normalization
License	Apache-2.0	MIT

MODNet's sides must be multiples of 32. That is not an arbitrary choice: the architecture downsamples and concatenates several times, and even multiples of 16 fail at a concat node with a mismatched-dimensions error. Both numbers are copied from the model's preprocessor_config.json and verified by testing.

The mask resolution caps at 512×512. When the source image is larger, edge accuracy is limited by the mask resolution rather than by the model's capability. A note under the result panel explains this.

The output tensor name was a gotcha. BiRefNet has multiple ONNX exports, and different exporters name the output differently. Testing confirmed both revisions here use output_image (not the more common logits). The code is defensive: if the registry name doesn't match, it falls back to the sole output or infers it from the shape.

About WebGPU: not every model belongs on the GPU

The bottom of the UI shows which backend is actually in use. There is a rule here that is not obvious, but matters a great deal.

WebGPU can compute the wrong answer silently. ONNX Runtime's WebGPU backend runs intermediate activations in half precision on the GPU. For networks dense in normalisation layers that introduces substantial error, and it does not raise anything — tensor shapes are right, values are finite, only the result itself is wrong. Measured on Intel Gen-12LP with ONNX Runtime 1.30.0, same 1440×2456 portrait:

MODNet: WebGPU foreground coverage 35.90% vs CPU 47.41% — a large bite out of the subject's edges
BiRefNet: WebGPU 47.08% vs CPU 47.08% — identical on both ends
MobileSAM encoder: output statistics bit-for-bit identical across both backends (WebGPU is about 4.6× faster)

Switching MODNet to fp32 weights does not help — on WebGPU, fp32 gives 35.90% and fp16 gives 35.85%, essentially the same. So the loss comes from the execution path, not from weight precision. Each model therefore carries a "is WebGPU trustworthy" flag; MODNet is explicitly excluded and always runs on CPU, while BiRefNet and MobileSAM are free to use the GPU.

This is also the general lesson this project paid for: checking that nothing threw an error is not enough — you have to check the actual pixels. The numbers above only surfaced by diffing the finished PNGs from both paths pixel by pixel. Over 27.8% of pixels differed by more than 32/255, and nothing was raised anywhere in the process.

Click to cut: MobileSAM

Click-to-cut takes the other route: it does not guess your subject, it listens to you. The model is MobileSAM — MobileSAM swaps Segment Anything's image encoder for a lightweight TinyViT, shrinking the whole thing from roughly 600 MB to just over 40 MB while keeping segmentation quality essentially level.

It is split into two models: an image encoder that turns the picture into features, and a mask decoder that turns prompt points into a mask. That split is what makes clicking feel fast — the image is encoded once, and every subsequent click runs only the decoder. The first click is slow; every click after it is quick.

Item	Value
Total size	42.6 MB (26.9 MiB encoder + 15.7 MiB decoder)
Encoder input	`input_image`, fixed 1024×1024, HWC layout (no batch dimension), pixel values 0–255
Non-square images	Aspect-preserving resize, then centred black padding to 1024×1024; the mask is mapped back to the original frame (the subject is never stretched)
Decoder inputs	Image features + point coordinates (×1024) + point labels + `orig_im_size`
Decoder outputs	`masks` (a native 1024×1024 mask) and `iou_predictions` (scores)
Normalisation	Baked into the graph; the caller does none
License	Apache-2.0

The decoder returns three candidate masks with three scores, and the app keeps the highest-scoring one. The more points you place, the better those scores express "this really is the subject" — which is why adding a point beats switching models.

Why we did not just use the ready-made sam-web component. It hardcodes ONNX Runtime's wasm path to the jsDelivr CDN and ships its own separate runtime: that would compile a second copy of wasm alongside our self-hosted ONNX Runtime and add an external request. Instead we wired the same tensor spec ourselves and reused the runtime already on the site — no request leaves the site, and nothing doubles in size.

Crop and rotate: no model at all

Cropping, free rotation, flipping and slicing are deterministic geometric transformations. Canvas alone is enough, so nothing needs downloading. One thing that is easy to misread: rotation does not paint the corners black — the canvas grows to the rotated bounding box, so the corners always contain content and nothing is cut or filled. The crop rectangle therefore lives in post-rotation coordinates, which keeps dragging what-you-see-is-what-you-get with no angle arithmetic.

Auto-straightening is pure algorithm too, no model: downscale to 480 pixels, go greyscale, compute Sobel gradients, keep only strong-gradient pixels (direction in flat regions is noise), then fold each edge's orientation into "how far off the nearest horizontal or vertical axis it is" and vote into a weighted histogram. The peak is the image's dominant skew.

Folding both axes into one interval is the key step: documents, desks and buildings carry plenty of both horizontal and vertical edges, and without folding the histogram shows two peaks, neither of which represents the overall tilt. That also marks the limit — with no clear straight lines in frame (a landscape, a product shot on a plain backdrop) the result is noise. So detection returns a confidence value and is only auto-applied when it is high enough; skews beyond 15° are rejected outright, since that is deliberate rotation rather than a crooked photo.

Where the weights come from

Model	Source
Real-ESRGAN x4v3	Committed under `public/models/` and deployed with the site
MODNet	Xenova/modnet on HuggingFace
BiRefNet Lite	studioludens/birefnet-lite-512 on HuggingFace
BiRefNet	naddy24/birefnet-512-webgpu on HuggingFace
MobileSAM (encoder + decoder)	Acly/MobileSAM on HuggingFace

HuggingFace paths are mutable, so the code pins an exact commit revision — the model that works today is still the same model tomorrow. Upgrading a model means changing that revision constant. Together these weights far exceed what any static host will accept, so none of them are committed; the browser fetches and caches them on first use instead (see "In-browser inference").

Download sources and network fallback

All of these live on huggingface.co, which is unreachable from mainland China on a direct connection — not with an error, but with a connection that simply hangs until it times out. For a tool where every computation happens locally, stalling on the first download is the worst possible place to lose, so fetching has three safety nets:

Multiple sources, tried in order. Every HuggingFace URL has a mirror twin (hf-mirror.com); the mirror goes first, the official host second. The mirror is a reverse proxy with 1:1 path mapping — swapping the hostname is all it takes, so a new model can never be "added without a mirror entry". If one is down, the other takes over.
Each source gets 10 seconds to produce response headers. Only the headers are timed, not the whole body — a multi-megabyte model legitimately takes minutes on a slow connection, and a blanket limit would kill healthy downloads. But some limit is required: an unreachable address neither errors nor returns, so without one the fallback would never get its turn.
The winning source is remembered. It is persisted to localStorage, so the next visit starts there instead of burning another timeout on the dead source for every file. If localStorage is unavailable (private mode), the code falls back to the default order — correctness is unaffected.

When every source fails, the page says so instead of spinning forever. The message is "Could not download the model weights — neither the official host nor the mirror responded. Check your network or proxy and try again." At that point your own network setup is the only thing left to inspect.

Caches are also separated per tool: upscaling, cutout and click-to-cut each get their own bucket, so clearing one never touches the other two. The upscaling model (4.9 MB) and the ONNX Runtime itself (/ort/) ship with the site and never touch an external host.

Home GitHub Blog 中文