LLM in the Browser

Step 1: Choose a model. The model files (ONNX format) are downloaded from Hugging Face directly to your browser cache. This is a one-time download per model.

Step 2: The model loads into memory. Transformers.js creates an inference pipeline using either WebGPU (GPU-accelerated) or WASM (CPU fallback), depending on your hardware.

Step 3: When you send a message, the model generates a response token-by-token inside a Web Worker. The worker runs off the main thread so the UI stays responsive. No data ever leaves your browser.

Large language models like GPT-4 have hundreds of billions of parameters and require data-centre-grade hardware. Small language models (SLMs) have from tens of millions up to a few billion parameters -small enough to run entirely in your browser.

The models here range from 82M parameters (Kokoro, ~92MB) to 3.8B parameters (Phi 3.5 Mini, ~2.3GB). For comparison, GPT-3 had 175 billion parameters -roughly 46 times larger than the biggest model on this page.

SLMs are surprisingly capable for their size. They can answer questions, follow instructions, and generate coherent text. They struggle with complex reasoning, factual accuracy, and long outputs -but they demonstrate the core mechanics of how language models work.

Causal (decoder-only) models generate text left-to-right, predicting one token at a time. Each token can only "see" the tokens before it. GPT, LLaMA, and SmolLM all use this architecture. They are good at open-ended generation and conversation.

Seq2Seq (encoder-decoder) models like T5 and FLAN-T5 first encode the entire input into a representation, then decode an output from it. This makes them better at tasks with clear input-output mappings like translation, summarisation, and question-answering.

On this page, SmolLM2 and Qwen are causal models that support multi-turn chat. FLAN-T5 is a seq2seq model -it processes each message independently without conversation history, which makes it better suited for single-turn Q&A.

Neural network weights are normally stored as 32-bit floating point numbers (FP32). A 360M-parameter model at FP32 would need about 1.4GB. Quantization reduces precision to shrink the model and speed up inference.

INT8 quantization uses 8-bit integers, cutting size by ~4x with minimal quality loss. Q4F16 (4-bit weights, 16-bit activations) cuts size by ~8x -the SmolLM2 360M model drops from ~1.4GB to ~273MB. The trade-off is a small reduction in output quality.

Most models on this page use Q4F16 quantization via Transformers.js (a few, like Kokoro and BART, use INT8). This is why a "360 million parameter" model only needs a ~273MB download instead of the ~1.4GB you might expect. The quality loss from quantization is barely noticeable at this scale.

WebGPU is a modern browser API that gives JavaScript direct access to GPU compute shaders. When available, Transformers.js runs matrix multiplications on the GPU, which can be 5-20x faster than CPU execution for model inference.

WASM (WebAssembly) is the fallback. The ONNX Runtime compiles to WASM and runs model inference on the CPU. It works in every modern browser but is significantly slower, especially for larger models.

This page automatically detects whether your browser supports WebGPU. Chrome 113+ and Edge 113+ have WebGPU enabled by default. Safari and Firefox have partial or experimental support. If WebGPU is unavailable, the models will still work -just more slowly.

This page includes two tiny abliterated chat models: Qwen 3.5 0.8B Uncensored (the Josiefied Qwen 3.5 export) and Qwen 3 0.6B Abliterated (Heretic), next to the official Qwen 3 0.6B chat row. Abliterated or gabliterated weights are modified to reduce refusal behaviour, often by editing internal refusal directions so the model is more likely to comply with prompts that aligned models would reject.

This can increase unsafe-compliance risk: the model may provide harmful, deceptive, explicit, or policy-violating instructions with less resistance. Treat outputs as untrusted text and verify everything independently.

Quality can also shift. Public evaluations report mixed capability trade-offs after refusal-direction edits, including potential regressions or behaviour drift depending on the model and method used.

Use these models only for controlled testing, research, and red-team style experimentation. Don't use them for medical, legal, financial, security-critical, or other high-stakes decisions.

Authorised security testing: less-restricted models can help defenders run realistic attack simulations against their own systems (prompt-injection tests, phishing simulations, exploit-chain reasoning, and weak-control discovery) before adversaries do.

Guardrail stress-testing: security teams can use these models to find where policy filters break, then harden downstream controls (sandboxing, tool permissions, network egress rules, approval steps, and output validation) using standards like OWASP LLM guidance.

Incident-response training: they are useful for tabletop exercises, SOC runbooks, and replaying historical attack scenarios with more adversarial dialogue than heavily restricted hosted assistants allow.

Local availability: when cloud-hosted APIs are unavailable due to network, cost, or data-residency constraints, local models can support lawful product exploration and engineering brainstorming. This doesn't mean safety guardrails are unnecessary. Always apply your own controls.

Research flexibility: teams can evaluate model behaviour under difficult prompts, compare alignment trade-offs, and build safer production wrappers around a transparent local baseline. Always keep use authorised, legal, and contained.

Privacy: Your messages never leave your device. There's no server, no API, and no logging. Everything runs in your browser tab.

No cost: Cloud-based LLM APIs charge per token. Running locally is completely free after the one-time model download.

No rate limits: There are no API quotas or throttling. Generate as much text as you want.

Offline capable: Once a model is cached in your browser, you can use it without an internet connection.

Educational value: You can see exactly how a language model behaves -its strengths, weaknesses, and failure modes -without any filtering or post-processing.

The trade-off is quality: these small models can't match the capabilities of cloud-hosted models with hundreds of billions of parameters. But for learning, experimentation, and privacy-sensitive tasks, local inference is a genuine alternative.