Glyph-1: Stateless, Reversible Link Compression

Most URL shorteners work the same way: you paste in a link, a server stores it in a database, and you get a short ID back. That ID only works for as long as the database entry and service behind it still exist (RIP goo.gl).

Unwrite's free tools are static files, so they can't write link mappings to a database. I also didn't want the tool to rely on people trusting Unwrite to keep those mappings alive indefinitely.

Glyph-1 compresses the URL into the token itself. Compression and expansion are exact inverse functions, with no lookup, account or server-side mapping involved. The token contains the URL in compressed form. Everything the codec depends on, including dictionaries, templates and formats, is fixed in the source once that codec version is released. I'll call that "frozen" throughout this post.

This post covers how version 1 works, the decisions behind it, and the main problems I ran into while building it. The implementation lives in the Unwrite Links tool at /links.

Design constraints

These were the requirements I set. Given any URL S up to 8192 characters, Glyph-1 must produce a URL-safe token T that:

1. 1expands back to the exact original string (expand(T) === S) - same case, same query order, including a dangling ? or #, and including a user:password@ prefix
2. 2only uses URL-safe characters (base64url, no padding)
3. 3never accidentally exposes the destination hostname or brand as readable text inside the token
4. 4returns an error when the token is mistyped instead of decoding to a different URL (the check is 8 bits, so this is typo detection, not a security feature)
5. 5stores nothing anywhere

Meet those five requirements and there is no database involved at all.

Why generic compression didn't work

My first instinct was gzip plus base64url. Generic compressors such as deflate, brotli and zstd are excellent on larger inputs, but a short URL gives them very little room to work. On something like a 180-character Amazon link, the format overhead and internal tables can use several bytes before useful compression has even started.

URLs are also far more structured than normal text. They use a small set of schemes, the same common hosts, repeated path words and tracking keys, then usually a relatively small identifier or value that actually changes. Glyph-1 is designed around that structure.

Four encoders

Glyph-1 runs four encoders and keeps whichever result is shortest, measured in bits. At this size, individual bits genuinely matter. A short mode tag at the front records which encoder won, so expansion only runs the matching decoder.

Identity guarantees there is always at least one valid result. Every candidate is also decoded and compared with the original before it can be selected. If I introduce a bug in one encoder, that candidate is rejected before it can produce a token that won't expand correctly.

The wrapper

The compressed body gets a small wrapper containing the mode, checksum and scramble step before the final base64url encoding.

The mode codes are variable length so the common case gets the shortest code. No valid code is the prefix of another, which means the decoder always knows where the tag ends. Template mode wins most often, so it gets a single bit. A template URL therefore spends 9 bits on the complete header: 1 mode bit and 8 checksum bits.

There is no version field in the token either. A new codec version ships with a new decoder. The tool isn't live yet, so there are no existing Glyph-1 tokens that need compatibility handling.

Once the header and body are written, everything except the leading two bits is XOR'd against a fixed public bit pattern. That's the Twist step described later. The result is then encoded as base64url.

The short form is unwr.dev/{T}, with unwr.io and unwr.link using the same codec. Each domain serves the preview itself, right at that address: the page decodes the token locally and shows the destination before anything is opened. The token never appears on any other origin. There is no automatic redirect, and there is no second decoder that tries to interpret arbitrary leftover paths.

The rebuild check

I stopped using new URL() inside the codec early on because it changes some valid input. It can lowercase the scheme, remove default ports and add slashes. That's useful browser behaviour, but it breaks byte-for-byte reversibility when the decoder has to reproduce the exact string the user pasted. The safety preview does use new URL() deliberately, because there I want the displayed host to match what the browser will actually open.

Glyph-1 instead splits the input using standard RFC 3986 punctuation and joins the pieces back together. Structured mode only runs if that split-and-rebuild process reproduces the input character for character. If the check fails, that URL skips structured mode and the other encoders still get a chance. Template mode is stricter again and only accepts an exact string match.

Templates

A YouTube watch URL is 43 characters long, but only about 11 characters vary between videos. The rest is repeated on every URL in that family, so there is no reason to store it each time.

Template IDs are variable length as well. The most common family, https://www.youtube.com/watch?v=, gets the cheapest ID. URLs with extra query parameters use a second YouTube template containing the same video ID followed by the query pairs. This is about as compact as Glyph-1 can get:

Amazon works the same way. The host and /dp/ path are implied by the template, so only the ASIN and any query need to be encoded. A 10-character ASIN is stored as one base-36 integer, which fits in 52 bits instead of the 60 bits needed to store each character separately. The complete token is around 11 characters. Amazon URLs with a query use a related template with the same ASIN representation followed by the query pairs.

GitHub owner/repo has its own template. Lowercase names can be stored as a single base-26 integer instead of 5 bits per letter, so something like unwrite/glyph comes out at around 13 characters.

Long numeric IDs are kept as digit strings with an explicit length and are never converted through floating-point numbers. X status IDs can exceed 32 bits, so preserving the digits exactly matters. The digits are packed as one large number using ceil(n log2 10) bits instead of 4 bits for every decimal digit.

Structured mode

When the rebuild check passes, Glyph-1 writes the URL field by field:

• Scheme. A short code for https or http. Other schemes are checked against a dictionary or stored as a leftover string.
• Authority. The user:password@ component if present, followed by the host and port.
• Host. An exact match in a fixed table of common hosts, an IPv4 address stored as four bytes, a www. flag plus a known ending such as .com or .co.uk, or a leftover string.
• Path. Empty, /, or /-separated segments. Each segment is checked against a dictionary of common path words.
• Query. Split on &. Common tracking-key layouts are frozen as shapes. A known UTM layout stores a small shape ID and only the values because the keys are implied by the shape. Unknown layouts still store their keys. Each value is then tested against several compact representations: dictionary match, integer, hyphen- or plus-separated words, mixed word/hex/digit chunks, a known click-ID prefix plus the remainder, or raw text. Capitalised words are stored lowercase with a short case marker, paying per-letter bits only when the pattern is irregular. Plain integers also have a dedicated form stored as one variable-length number.
• Fragment. Three states: missing, empty #, or present.

This is how a 220-character Amazon link with UTM fields and a click ID can drop into the mid-fifties. The query keys are represented by one shape ID, newsletter becomes one dictionary index, spring-sale-2026 becomes three word indexes, and the click ID can be split into known words and hex.

The host table is just a fixed list of exact strings such as www.youtube.com, github.com and en.wikipedia.org. A match is stored as an index, so the hostname never appears directly in the token. Hosts outside the table are stored as leftover strings.

Leftover strings: word splitting

Anything the dictionaries don't recognise, including an unusual path segment, host, query value or template argument, is stored as a leftover string. The code calls these residuals. Each residual is encoded using whichever of three forms is shortest: one large number, raw UTF-8, or a word split.

Word splitting is useful because many long URL slugs are ordinary words separated by punctuation. Glyph-1 splits around those separators, preserves them, and stores recognised words as indexes into a fixed 4096-word dictionary. URL, marketing and technical vocabulary comes first, followed by common English words. Each recognised word also carries a small case marker: lowercase and Title-case are one or two bits, anything irregular pays per letter.

A 12-bit dictionary index can replace a word that would otherwise cost roughly 6 bits per character. Runs of digits stay numeric, and unknown words fall back to the number representation.

This makes a large difference on blog, article and profile URLs where most of the path is readable language. The result is still completely deterministic and requires no lookup outside the codec itself.

Leftover strings: numbers

Identifiers need a different representation. Video IDs, hexadecimal hashes and UUIDs are often best treated as strings over a small known alphabet.

Glyph-1 stores these using a short alphabet ID and length, then treats the entire string as one large integer in that alphabet's base. The code calls these runes. A group of n characters from an alphabet of size A costs bit_length(A^n - 1) bits. That gives true base-36 packing instead of padding every character out to 6 bits. A lowercase UUID with hyphens is special-cased as its 16 raw bytes. Raw UTF-8 remains the fallback when no supported alphabet fits or when numeric packing would be larger.

This is why a YouTube ID is cheaper as a number than text, why a 10-character ASIN fits in 52 bits, and why an apparently random click ID can still compress once its alphabet is known.

The 6-bit problem

One issue took much longer to spot.

The final token is base64url, so every 6 payload bits become one visible character. If an internal string also uses the base64url alphabet and is stored as fixed 6-bit symbols, those internal bit groups can line up with the final output boundaries. When that happens, part of the supposedly encoded value can appear literally in the token.

For example, the slug Mechanical_Keyboard needs the full base64url alphabet because it mixes upper and lower case with an underscore. Stored as fixed 6-bit symbols, each group maps directly to one output character:

That matters because readable destination names inside a short-link path are useful for phishing. I wanted the token itself to stay opaque.

Lowercase slugs with hyphens don't have the same problem because a 37-symbol alphabet packed as one large number does not line up with 6-bit boundaries. The problem appears with 64-symbol alphabets, which unfortunately includes YouTube IDs, one of the most common values the codec stores. A length prefix can shift variable-length values out of alignment, but fixed-length identifiers such as video IDs and ASINs always start in the same place. Raw UTF-8 also avoids the issue because base64url encoding ASCII does not reproduce the ASCII text directly.

I first tried changing the character widths. Seven-bit symbols waste a bit per character and still don't solve the underlying problem because later fields can bring the stream back onto a 6-bit boundary. The fix needs to happen once at the outer layer after every encoder has finished. That's what Twist does.

Twist

Twist applies a fixed XOR pattern to the payload so the internal bit layout cannot line up with the visible base64url characters. It is public and reversible and provides no secrecy. Anyone with the source can undo it. Its only job is to prevent readable internal values appearing directly in the output token.

Let K be the first 16 bytes of SHA-256("glyph-1/twist"), stored as a frozen constant:

051a07a91176e3fd2bde6529a5cbfbe2

The leading two bits stay unchanged. Every later bit i is XOR'd against the pattern:

s[i] = p[i] XOR ((K[(i-2) mod 16] >> (i mod 8)) AND 1)

XOR is self-inverse, so the same operation applies and removes the Twist. Because the checksum is written before Twist runs, a damaged token still reaches the normal validation path after it is reversed and fails cleanly if the data no longer matches.

Dictionary LZ

Some URLs miss both the template and structured encoders but still contain plenty of common URL fragments. The fallback uses LZ77-style compression with matches taken from a fixed reference string of at most 4096 bytes instead of the input's own previous content.

That reference string contains common URL fragments such as https://www., /watch?v=, utm_source=, percent encodings, and the same hosts and keys already used by structured mode.

An index of 3-byte substrings keeps matching fast even at the 8 KiB input limit. Each token starts with a flag bit and then stores either one literal byte or a 12-bit offset plus a 6-bit match length. Matches range from 3 to 66 bytes. The body begins with the decoded length so the decoder knows exactly when to stop.

A literal costs 9 bits, so genuinely incompressible input is smaller in identity mode at 8 bits per byte. The encoder comparison picks identity automatically in those cases.

Integrity

Glyph-1 uses CRC-8/AUTOSAR over the original UTF-8 bytes. This is a full CRC-8 rather than a longer CRC truncated to 8 bits. With polynomial 0x2F, any single flipped bit and any odd number of flipped bits changes the checksum at any supported input length.

A damaged token can fail base64url validation, a range check or the checksum. In each case the tool returns an error instead of opening a different destination.

The checksum is only 8 bits because its purpose is catching typos. Anyone with the source can generate a valid token for any destination, which is inherent to a public reversible codec. That's also one of the reasons the product always shows a preview before navigation. A wider checksum would usually add another visible character, and 8 bits is enough for the error-detection job I wanted here.

Phishing, domains and SEO

A reversible codec still needs sensible behaviour around the link itself. I treated the token format, preview page and domains as part of the same design.

Readable destination names. If words such as paypal or github can appear directly in the short-link path, an attacker can make something like unwr.dev/paypal.xxxxx look more convincing. Known hosts are stored as dictionary indexes, and Twist prevents aligned internal values from appearing as readable text. The resulting tokens stay opaque.

Destination ownership. The preview presents the destination as exactly that, a destination, and never as Unwrite content. Search engines also should not index token preview pages such as unwr.dev/{token}. The canonical page for the tool is /links/.

Preview before opening. Automatic redirects make it much easier to hide the real destination. Glyph-1 shows the scheme, hostname and full URL first, then requires the user to click Continue. That link uses rel="noreferrer noopener", so the destination does not receive Unwrite as the referrer and a malicious page cannot read the token URL from document.referrer.

Scheme restrictions. http and https can be opened from the preview. Schemes such as javascript:, data:, vbscript:, file:, blob: and about: still round-trip because the codec preserves the original string, but the Continue button will not open them.

Credentials and lookalike hosts. The preview warns about user:password@ credentials, hosts that mix ASCII with lookalike non-ASCII characters, invisible or direction-changing characters hidden in the hostname, and punycode labels beginning with xn--. The codec preserves those inputs exactly. The warning logic stays in the preview layer.

Shareable link length. unwr.dev/ adds 9 characters to the token, or 17 including https://. A 43-character YouTube URL can therefore have a much shorter token without producing a shorter complete shareable URL. The UI shows both lengths so the user can see the actual result.

Domain reputation. URL shorteners can end up on blocklists, and I don't want that risk attached to the main unwrite.co origin. One Safe Browsing issue affecting short-link traffic could otherwise affect GPT, Images, PDF and Voice as well. Glyph-1 therefore uses separate domains: unwr.dev, unwr.io and unwr.link. They isolate the shortener reputation from the main site.

SEO. Unwrite only serves the preview and never serves a copy of the destination page or presents itself as the destination. The original site remains the canonical content source.

Scam, spam and malware URLs

A shortener that never says anything becomes free infrastructure for phishing campaigns. Glyph-1 also has no database, so I cannot delete a link that turns out to be malicious. The token keeps working because the URL is inside it. That moves all of the responsibility onto the preview, so the preview does real checking.

Blocklists. Every preview checks the destination against blocklists that rebuild daily from public feeds: URLhaus and Phishing Army for malware, phishing and scam domains, and the StevenBlack hosts list for spam and unwanted content. There are two levels of response. A spam or unwanted-content match shows an amber caution. A malware or phishing match shows a red warning and asks for one extra explicit click before the link is shown. Nothing is a hard block. A blocklist can be wrong, and the person in front of the screen gets the final say.

Checking without phoning home. The obvious implementation sends the URL to a reputation API. That would undo the privacy design of the tool, so the browser instead downloads the whole blocklist in compressed form (a Bloom filter, roughly 190 KB) and checks locally. A Bloom filter produces false positives at around 1%, so a hit is then confirmed against an exact list that also downloads to the browser and is checked locally. A refuted hit shows nothing, a confirmed hit shows the warning, and a hit that cannot be verified honestly says the destination "appears on" a blocklist. At no point does the URL, a hash of it, or any part of it leave the browser. The single outside signal is the download of the exact list itself, which tells the safety origin only that some filter matched, never which destination.

Mistyped and lookalike domains. Scammers rely on domains that look right: rnicrosoft.com with r-n standing in for m, paypa1.com with a digit, Cyrillic characters that render like Latin ones, or apple.com.verify-account.xyz where the real registrable domain hides at the end. The preview checks the destination against a curated list of heavily impersonated brands using confusable-character folding, the classic substitution tricks, single-typo distance and brand-in-subdomain detection. A one-letter slip such as microsfot.com gets a direct button to the genuine site, because that case is usually an honest mistake rather than an attack. These checks run entirely locally too.

The blocklists refresh themselves daily with no manual step, while the codec stays frozen. Like the credential and punycode warnings above, all of this lives in the preview layer, so the reversibility guarantee is never touched.

Tracking parameters

Removing UTM parameters changes the URL, so that cannot happen inside a codec whose main guarantee is exact reversibility. The tool offers tracking-parameter removal as a separate optional step, clearly labels that the URL is being changed, and then compresses the result the user chose.

Limits

The scope is deliberately narrow: reversible compression plus a preview page. An arbitrary 2 KB URL cannot become a six-character ID without some external mapping, because there is not enough information in six characters to reconstruct it. Custom cloaking and branded redirect subdomains are outside the design as well.

The complete round trip

Putting the pieces together, this is the whole algorithm in one picture. Compression is a tournament with a verification gate; expansion is a straight line with three places to fail closed:

Worked example

Take https://www.youtube.com/watch?v=dQw4w9WgXcQ.

The YouTube watch template matches and produces a 13-character token. The anatomy diagram above shows the bit layout. Expanding the token restores the exact same 43-character URL, including www and the original query.

A 220-character Amazon URL with UTM fields and a click ID usually selects the Amazon-plus-query template. The host and /dp/ are implied, the ASIN uses 52 bits, and the query stores the known shape plus only the values. Those values can then use dictionary indexes, hyphen-separated words, a plain integer for qid, and a known click-ID prefix followed by the remaining packed value. In this case, the complete shareable Unwrite URL is shorter than the original, which is exactly where the format is useful.

What happens if Unwrite disappears?

Everything needed to decode Glyph-1 is included in the source: dictionaries, templates, word lists and exact bit layouts. The TypeScript implementation is the source of truth for those layouts, so if this post ever disagrees with the code, the code wins. Any change to those frozen parts means shipping a new codec version with the corresponding decoder.

goo.gl links depended on Google's database continuing to resolve them. Glyph-1 tokens carry the information needed for decoding themselves. If Unwrite disappears, anyone with the source can still expand every token offline without access to an Unwrite database or account.

That persistence is one of the main reasons I built it this way. I haven't found another URL shortener with quite the same property.

If you want to try it, the tool is at /links. If you find a URL that does not round-trip exactly, I genuinely want to know about it.