Understanding TensorFlow.js
Models in the browser — WebGL or WASM, your choice.
What TFJS can run, how the model.json + weights.bin split works, and the three backends that decide whether inference fits the device.
Inference in the browser.
TensorFlow.js (Google, 2018) runs neural-network inference (and training) directly in the browser. Use cases: pose estimation on a webcam, image classification, voice activity detection, recommendation in client code. Three reasons to run in the browser: privacy (data never leaves the device), latency (no network round-trip), cost (no server GPUs). Three reasons not to: model size, battery, the user's hardware is unpredictable.
The model file layout.
A TFJS model is two files. model.json — small JSON with the architecture (layers, shapes) and a manifest listing the weight shards.group1-shard1of3.bin, ...shard2of3.bin,...shard3of3.bin — binary weight tensors split for parallel HTTP fetching. tf.loadGraphModel("/model.json") fetches all of them and builds the in-memory model. Total download for a ResNet-50: ~25 MB. For MobileNetV3-small: ~3 MB.
The three backends.
WebGL: the default. Uses the GPU via WebGL shaders. Fast on desktop, varying on mobile. CPU: pure JavaScript fallback when WebGL fails. Slow but reliable. WebAssembly (WASM): newer alternative — runs SIMD-optimised C++ in WASM. Often faster than WebGL on integer ops, more consistent across devices, smaller warmup time. Choose by benchmarking your model on your target devices;tf.setBackend("wasm") is the switch.
A worked load.
MobileNetV3-small (3 MB total) on a webcam frame. Load:const model = await tf.loadGraphModel("/mobilenet/model.json") — downloads 4 files in parallel, ~600 ms on a fast connection. Warmup: pass one dummy frame to compile shaders, ~200 ms first time. Inference: ~10-30 ms per frame on WebGL on a 2022 phone, ~50 ms on WASM. 30 fps achievable on most modern devices. Send to tf.browser.fromPixels() for camera input.
MobileNetV3 in the browser
3 MB model, WebGL
Load → warmup → infer.
600ms load + 200ms warmup + 20ms/frame
= 30+ fps on modern phones
Convert from Python.
Most models start their life in Python — TensorFlow, Keras, PyTorch.tensorflowjs_converter takes a SavedModel, TF Hub URL, or Keras .h5 and emits the model.json + shards. PyTorch models go via ONNX → TFJS or via ONNX Runtime Web (the modern alternative). Quantisation in the converter step (--quantize_uint8) shrinks the weights 4× with usually-imperceptible accuracy loss — worth doing for any browser deployment.
When not to use TFJS.
Server-side inference at scale — Python with batching is faster. Large language models — they don't fit in browser memory yet. Tasks where accuracy matters more than latency — the small models that fit in TFJS often trail server-side counterparts. Real-time tasks on low-end Android — WASM helps but the spread is wide. For the right shape — small model, real-time UX, privacy-sensitive — TFJS is unmatched.