URL Encoder / Decoder

RFC 3986, the URI standard, divides characters into exactly two meaningful sets. The unreserved set — letters, digits, and the four symbols - . _ ~ — may appear anywhere in a URL without encoding, and encoding them is legal but pointless: %41 and A identify the same thing, and URL normalizers are expected to decode them back. The reserved set is where the design lives: the gen-delims : / ? # [ ] @ mark the boundaries between URL components, and the sub-delims ! $ & ' ( ) * + , ; = carry meaning inside components. Everything else — spaces, quotes, angle brackets, every non-ASCII character — simply cannot appear raw and must be percent-encoded as %HH byte values. The crucial subtlety is that for reserved characters, encoded and raw forms are not equivalent. A raw / in a path separates segments; an encoded %2F is data that merely looks like a slash. That means whether a character "needs encoding" depends entirely on which component it sits in: a / is fine in a path but must become %2F inside a query parameter value that happens to contain a URL, and a ? is structure the first time it appears but plain data afterwards. This is also why the number-one practical rule of URL construction exists: never encode a whole assembled URL, because you cannot do it correctly — one pass will either miss the data characters or destroy the structural ones. Encode each dynamic value on its own, then assemble.

JavaScript ships three generations of encoding APIs, and picking the wrong one is the most common URL bug in front-end code. encodeURI treats its input as a complete URL: it escapes characters that can never appear (spaces, quotes, non-ASCII) but preserves all reserved delimiters — : / ? # & = pass through untouched. That makes it useless for encoding values: feed it a search term containing an ampersand and the ampersand survives, splitting your parameter in two. encodeURIComponent is the value encoder: it escapes everything except unreserved characters plus ! ' ( ) * — so &, =, /, ? and friends all become safe. The rule of thumb is simply that you almost always want encodeURIComponent, applied to each value before it is concatenated into a URL. Neither function is a perfect RFC 3986 citizen. encodeURIComponent leaves ! ' ( ) * raw even though the RFC classifies them as sub-delims; OAuth 1.0 signatures and other byte-exact protocols require patching those five with an extra replace pass. The ancient escape() function should never be used at all — it predates UTF-8 adoption and mangles non-Latin text. And decodeURIComponent throws a URIError on any malformed sequence (a lone %, %ZZ), so decoding untrusted input belongs inside try/catch. The modern answer skips manual encoding entirely: the URL and URLSearchParams classes parse, build, and serialize URLs component by component, encoding each part with the correct rules automatically.

const q = 'fish & chips?';

encodeURI(q)             // 'fish%20&%20chips?'   & and ? survive!
encodeURIComponent(q)    // 'fish%20%26%20chips%3F'  safe as a value

// The failure mode:
'https://x.io/s?q=' + encodeURI(q)
// → ?q=fish%20&%20chips?   server sees TWO params: q and ' chips?'

// The modern way — encoding handled per component:
const u = new URL('https://x.io/s');
u.searchParams.set('q', q);
u.toString()             // 'https://x.io/s?q=fish+%26+chips%3F'

The space character has two encodings because two different specifications claimed it. RFC 3986 percent-encoding says a space is %20, everywhere. But HTML forms predate that rule: the application/x-www-form-urlencoded format, defined for form submission in the early 1990s and still specified by the WHATWG URL standard today, encodes a space as + (and a literal plus as %2B). Both encodings are correct — in their own context. Query strings produced by form submission and by URLSearchParams use +; paths, and anything encoded with encodeURIComponent, use %20. The bugs come from crossing contexts. The classic casualty is data that legitimately contains a plus sign: phone numbers in international format (+8210...) and base64 strings. Send phone=+82101234567 unencoded and the server's form decoder turns the + into a space, yielding ' 82101234567' — a corruption that often survives all the way into the database before anyone notices. The plus must be sent as %2B. In the reverse direction, JavaScript's decodeURIComponent knows nothing about form encoding: it leaves + as a literal plus, so decoding a form-encoded query string with it silently loses every space. URLSearchParams decodes + correctly. Language libraries encode (pun intended) this split in their APIs: PHP has urlencode (form style, +) versus rawurlencode (RFC 3986, %20); Python's urllib.parse.quote uses %20 while quote_plus uses +; Java's misleadingly named URLEncoder is a form encoder and produces + — using it to build a path is a well-known mistake. When in doubt, %20 is the safer output: form decoders accept both, but path parsers only accept %20.

// The international phone number bug
fetch('/api/sms?to=+821012345678')
// server-side form decoding: '+' → ' '
// to = ' 821012345678'        ← corrupted, silently

fetch('/api/sms?to=%2B821012345678')
// to = '+821012345678'        ← correct

// Decoding mismatch:
decodeURIComponent('a+b%20c')            // 'a+b c'  (+ kept)
new URLSearchParams('x=a+b%20c').get('x') // 'a b c'  (+ = space)

Percent-encoding is not idempotent: running it twice produces different output, because the % character itself encodes to %25. Encode a space once and you get %20; encode that result again and the % becomes %25, yielding %2520. When a user eventually sees "New%20York" rendered in a page or an email subject, somewhere in the pipeline a value was encoded twice and decoded once. The usual suspects are hand-rolled code calling encodeURIComponent on a value that a framework, HTTP client, or template engine then encodes again — or a redirect chain where each hop re-encodes the already-encoded next-URL parameter. The mirror-image bug is more dangerous. If any layer decodes more than once, %2520 collapses to %20 and then to a real space — and attackers weaponize exactly this. A security filter that checks a path for ../ sequences can be bypassed with %252e%252e%252f: the filter inspects the once-decoded string, sees the harmless %2e%2e%2f, and approves it; a later component decodes again and the directory traversal comes back to life. Real CVEs of this shape exist across proxies, app servers, and frameworks, which is why security guidance is blunt: decode exactly once at the trust boundary, then validate the decoded value, and never decode again after validation. The engineering discipline that prevents both directions is ownership: exactly one layer owns encoding, at the last moment before the URL leaves your code, and exactly one layer owns decoding, at the moment the request enters. If you spot %25 followed by two hex digits in a URL you did not intend to nest, you are almost certainly looking at a double-encoding bug — decode step by step and count how many passes it takes to reach plain text.

'New York'
→ encode once:  'New%20York'      correct
→ encode twice: 'New%2520York'    bug: % became %25

// Path traversal hidden by double encoding:
'../etc/passwd'
→ once:  '%2E%2E%2Fetc%2Fpasswd'   filter catches %2E%2E%2F
→ twice: '%252E%252E%252Fetc...'   filter sees nothing;
                                   second decode revives ../

// Diagnosis: decode until stable, count the passes
decodeURIComponent('New%2520York') // 'New%20York'  (pass 1)
decodeURIComponent('New%20York')   // 'New York'    (pass 2 → double-encoded)

Percent-encoding operates on bytes, not characters, so before a non-ASCII character can be encoded it must first be serialized to bytes — and the choice of encoding is where history left a mess. The modern rule, stated in RFC 3986 and enforced by the WHATWG URL standard that browsers implement, is UTF-8: the Korean syllable 한 becomes the three bytes ED 95 9C and thus %ED%95%9C, and a single emoji can balloon into a dozen percent-escapes. Browsers hide this from users — the address bar displays the decoded Unicode while the wire request carries the escapes. Mojibake happens when the two ends disagree on the byte serialization. A value encoded from EUC-KR or Latin-1 bytes but decoded as UTF-8 produces garbage characters or outright decode errors; this was endemic in the pre-UTF-8 web and still surfaces with legacy systems and old Java servlet containers whose default URI charset was ISO-8859-1. A subtler variant is Unicode normalization: café can be serialized as one precomposed code point (NFC) or as e plus a combining accent (NFD), and the two forms percent-encode to different byte sequences that will not match on string comparison. Normalize to NFC before encoding. Hostnames play by an entirely different rulebook: percent-encoding is not allowed in the host component at all. Internationalized domain names go through IDNA instead, which transforms Unicode labels into the ASCII-compatible Punycode form — 한글.example becomes xn--bj0bj06e.example. That xn-- prefix in a certificate or log is not corruption; it is the DNS-safe spelling of a Unicode domain, and it is also why lookalike-character phishing (IDN homograph attacks) makes browsers render suspicious mixed-script hostnames in Punycode form.