WEP: Base64 Encoding API

Status update (AsByteSlice generalization)

The encode / decode API below has since been generalized over AsByteSlice, which supersedes the concrete signatures documented in this WEP:

encode / encode_url / encode_with now take &S for any S: AsByteSlice (ByteSlice, ByteList, ByteArray, or a String's UTF-8 bytes), not &List<u8>.
decode now takes the same &S and returns Option<ByteList>, absorbing the former decode_bytes(&List<u8>) — which has been removed.

The lenient-decode and alphabet/padding behaviour described below is unchanged. See core:base64 for the current signatures.

Context

Base64 is a fundamental encoding used across web, security, and data interchange contexts — JWT tokens, data URIs, HTTP authentication, S3 object metadata, PEM certificates, and more. A standard library without Base64 support forces every project to implement its own, leading to subtle bugs (padding handling, alphabet confusion) and unnecessary dependency overhead.

Design Goals

Simple things simple: The common cases (standard Base64 encode/decode) should be one function call with no configuration
Complex things possible: Edge cases (URL-safe alphabet, padding control) should be expressible without separate function names for every combination
Lenient decode: Decoding should accept all reasonable inputs — padding or no padding, either alphabet or both mixed
Wasm-friendly: Pure computation, no I/O, no effects — suitable for any world

Prior Art

Language	API Shape	Standard vs URL-safe	Decode Padding	Decode Alphabet
Rust (`base64` crate)	`Engine` object with `Config`	4 engine constants: `STANDARD`, `STANDARD_NO_PAD`, `URL_SAFE`, `URL_SAFE_NO_PAD`	Configurable: `Indifferent` or `RequireCanonical` (strict by default)	Must match engine
Go (`encoding/base64`)	`*Encoding` object	4 globals: `StdEncoding`, `URLEncoding`, `RawStdEncoding`, `RawURLEncoding`	Lenient by default; `.Strict()` for canonical	Must match encoding
Python (`base64`)	Module functions	Separate functions: `b64encode`/`urlsafe_b64encode`	Lenient (auto-pads); `validate=True` for strict	Must call correct function
JavaScript (TC39 2025)	`Uint8Array` methods + options bag	`{ alphabet: "base64url" }` option	`lastChunkHandling`: `"loose"` (default) / `"strict"` / `"stop-before-partial"`	Must specify option
Swift (`Foundation`)	`Data` methods + `OptionSet`	No built-in URL-safe (pitch pending)	Lenient by default	N/A

Observations:

The 2×2 matrix (standard/URL-safe × padded/unpadded) is the fundamental design space. Go covers it with 4 globals, Rust with 4 engine constants — both lead to combinatorial explosion of named objects
JavaScript's modern TC39 API (Uint8Array.toBase64(), 2025) uses an options bag — closest to Wado's flags approach, but still requires the caller to specify the alphabet for decoding
Python keeps it simple with module functions but requires calling the correct decode function for the alphabet
Rust (base64 v0.22+) uses an Engine trait with GeneralPurpose config — powerful but verbose for simple cases (BASE64_STANDARD.encode(b"hello"))
No mainstream language auto-detects the alphabet on decode, even though the character set is unambiguous: +/ never appears in URL-safe and -_ never appears in standard

Key Insight: Asymmetric API

Encoding requires the caller to specify the output format — you must choose between +/ and -_, padding or no padding. But decoding needs no configuration: the decode table maps both + and - to value 62, and both / and _ to value 63. The two alphabets share the remaining 62 characters. This means a single decode table handles all valid Base64 input — standard, URL-safe, or even mixed — with zero detection overhead.

Padding is equally simple — just strip trailing = and compute the output length from remaining characters.

Decision

Module: `core:base64`

use { encode, decode, Encoding } from "core:base64";

`Encoding` Flags

Use Wado's flags type to control encoding variants:

pub flags Encoding {
    UrlSafe,    // use -_ alphabet instead of +/
    NoPadding,  // omit trailing = padding
}

Flags	Alphabet	Padding	Use Case
`Encoding::none()`	`+/`	Yes	PEM, MIME, email (RFC 4648 §4)
`UrlSafe \| NoPadding`	`-_`	No	JWT, URL query params (RFC 4648 §5)
`NoPadding`	`+/`	No	Rare; some binary protocols
`UrlSafe`	`-_`	Yes	Rare; some storage systems

Encode Functions

Encoding requires explicit format specification — the caller must decide the output format:

/// Standard Base64 (RFC 4648 §4, padded).
pub fn encode(data: &List<u8>) -> String

/// URL-safe Base64 (RFC 4648 §5, no padding).
/// Equivalent to `encode_with(data, Encoding::UrlSafe | Encoding::NoPadding)`.
pub fn encode_url(data: &List<u8>) -> String

/// Encode with custom flags.
pub fn encode_with(data: &List<u8>, encoding: Encoding) -> String

Decode Functions

Decoding is always lenient — auto-detects alphabet and accepts padding regardless:

/// Decode Base64 from a string.
/// Accepts both standard (+/) and URL-safe (-_) alphabets, with or without padding.
/// Returns null on invalid input.
pub fn decode(encoded: String) -> Option<List<u8>>

/// Decode Base64 from raw bytes (e.g., HTTP body, file content).
/// Same lenient behavior as `decode`.
pub fn decode_bytes(encoded: &List<u8>) -> Option<List<u8>>

No Encoding parameter on decode — there is no configuration to pass.

Complete API Summary

Function	Input	Output	Description
`encode`	`&List<u8>`	`String`	Standard, padded
`encode_url`	`&List<u8>`	`String`	URL-safe, no padding
`encode_with`	`&List<u8>`, `Encoding`	`String`	Custom flags
`decode`	`String`	`Option<List<u8>>`	Auto-detect, lenient
`decode_bytes`	`&List<u8>`	`Option<List<u8>>`	Auto-detect, lenient (from bytes)

5 public functions. 1 flags type.

Usage Examples

use { encode, decode, encode_url, encode_with, decode_bytes, Encoding } from "core:base64";

export fn run() {
    let data: List<u8> = [72, 101, 108, 108, 111] as List<u8>;

    // Standard encode/decode
    let b64 = encode(&data);
    assert b64 == "SGVsbG8=";

    if let Some(decoded) = decode(b64) {
        assert decoded.len() == 5;
        assert decoded[0] == 72 as u8;
    }

    // Padding-less input works too
    if let Some(decoded) = decode("SGVsbG8") {
        assert decoded.len() == 5;
    }

    // URL-safe encode
    let url = encode_url(&data);
    // decode auto-detects — no separate decode_url needed
    if let Some(decoded) = decode(url) {
        assert decoded.len() == 5;
    }

    // Decode from raw bytes (e.g., HTTP response body)
    let raw: List<u8> = [83, 71, 86, 115, 98, 71, 56, 61] as List<u8>;
    if let Some(decoded) = decode_bytes(&raw) {
        assert decoded.len() == 5;
    }

    // Custom: standard alphabet, no padding
    let no_pad = encode_with(&data, Encoding::NoPadding);
    assert no_pad == "SGVsbG8";

    // Invalid input returns null
    assert decode("!!!") matches { None };
    // Mixed alphabets accepted (lenient)
    assert decode("abc+def_ghi") matches { Some(_) };
}

Error Cases

decode / decode_bytes return None for:

Non-Base64 characters (control chars, spaces, symbols outside the alphabet)
Truncated input (length mod 4 == 1 after stripping padding, which is structurally invalid)

Whitespace (newlines, spaces) is not stripped automatically. Callers handling MIME-formatted Base64 (76-character lines) should strip whitespace before calling decode. This keeps the core implementation simple and predictable.

Implementation Notes

The implementation will be pure Wado, following the same pattern as core:zlib and core:collections. No new compiler features are required.

File	Description
`wado-compiler/lib/core/base64.wado`	Implementation — `Encoding` flags, encode/decode functions
`wado-compiler/lib/core/base64_test.wado`	Tests — encode/decode round-trips, edge cases, error cases

Internally, the encode/decode loop operates on List<u8> with the standard 3-byte-to-4-character mapping. Lookup tables are global arrays initialized at module load.

Consequences

Positive

5 functions cover all use cases — compared to Go's 4 encoding objects or Rust's engine configuration
Lenient decode eliminates a class of bugs — callers never need to worry about padding, alphabet choice, or alphabet mixing
flags type is idiomatic — reuses Wado's existing flags feature, bitwise combinable, zero overhead
decode_bytes serves real-world needs — HTTP bodies, file contents, and network buffers arrive as bytes, not strings
No effects — pure functions, usable in any context

Negative

No alphabet validation on decode — mixed standard and URL-safe characters are silently accepted. In practice this is harmless since both map to the same values
No streaming API — the entire input must be in memory. Streaming Base64 (e.g., for multi-gigabyte files) is out of scope; it can be added later as core:base64/stream if needed
Whitespace not stripped — MIME Base64 users must pre-strip newlines. This is intentional (explicit > implicit) but may surprise users coming from lenient implementations like Python's

Comparison

Feature	Wado	Go	Rust (`base64`)	JS (TC39)	Python
Encode standard	`encode(&data)`	`StdEncoding.EncodeToString(data)`	`STANDARD.encode(&data)`	`arr.toBase64()`	`b64encode(data)`
Encode URL-safe	`encode_url(&data)`	`RawURLEncoding.EncodeToString(data)`	`URL_SAFE_NO_PAD.encode(&data)`	`arr.toBase64({alphabet:"base64url",omitPadding:true})`	`urlsafe_b64encode(data)`
Custom encode	`encode_with(&data, flags)`	Create new `Encoding`	Build `GeneralPurpose` engine	Options bag	N/A
Decode (any)	`decode(s)`	Must pick correct `Encoding`	Must pick correct `Engine`	Must specify `alphabet` option	Must pick correct function
Decode lenient	Always	Lenient by default	Strict by default	`"loose"` by default	Always
Auto-detect alphabet	Yes	No	No	No	No
From bytes	`decode_bytes(&b)`	`Decode(dst, src)`	`decode(&data)` (takes `AsRef<[u8]>`)	N/A (string only)	`b64decode(s)` (takes bytes)

Related WEPs

Tuple and List Literal Syntax — List<u8> usage
Global Variables — lookup tables as module globals

References

RFC 4648 — The Base16, Base32, and Base64 Data Encodings
RFC 7515 — JSON Web Signature (JWS) — uses Base64url without padding

Navigation Menu

Uh oh!

Uh oh!

WEP: Base64 Encoding API

Status update (AsByteSlice generalization)

Context

Design Goals

Prior Art

Key Insight: Asymmetric API

Decision

Module: `core:base64`

`Encoding` Flags

Encode Functions

Decode Functions

Complete API Summary

Usage Examples

Error Cases

Implementation Notes

Consequences

Positive

Negative

Comparison

Related WEPs

References

Uh oh!

Uh oh!

FilesExpand file tree

wep-2026-02-27-base64-api.md

Latest commit

History

wep-2026-02-27-base64-api.md

File metadata and controls

WEP: Base64 Encoding API

Status update (AsByteSlice generalization)

Context

Design Goals

Prior Art

Key Insight: Asymmetric API

Decision

Module: core:base64

Encoding Flags

Encode Functions

Decode Functions

Complete API Summary

Usage Examples

Error Cases

Implementation Notes

Consequences

Positive

Negative

Comparison

Related WEPs

References

Module: `core:base64`

`Encoding` Flags