Skip to content

feat(iterator): add es-toolkit/iterator lazy iterator module (Phase 1)#1815

Open
raon0211 wants to merge 3 commits into
mainfrom
work/bold-euclid-4d4dec
Open

feat(iterator): add es-toolkit/iterator lazy iterator module (Phase 1)#1815
raon0211 wants to merge 3 commits into
mainfrom
work/bold-euclid-4d4dec

Conversation

@raon0211

@raon0211 raon0211 commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator

Overview

Adds es-toolkit/iterator — a lazy-by-default utility module for iterators — plus es-toolkit/fp/iterator for use inside pipe. This is Phase 1: a representative slice that locks the design pattern before expanding.

The module mirrors native TC39 Iterator Helper semantics: functions take the single-shot iterators returned by .values() and return lazy IteratorObjects. The result objects are created with Object.create(Iterator.prototype), so they are iterable, single-shot, and chain with native .map()/.filter()/.take()/.toArray() — measured to add no overhead over a plain object literal.

Design decisions

  • Match native, don't replace it. Functions already in the Iterator Helpers proposal (map, filter, take, drop, flatMap, reduce, toArray, forEach, some, every, find) are not reimplemented in the data-first module — per es-toolkit's "don't implement Stage 3+ proposals" principle. Only operators the proposal lacks are added. For the same reason toSet is not provided (new Set(iterator) already does it).
  • es-toolkit/fp/iterator provides the data-last/pipe surface. Inside a pipe you can't chain native .map() ergonomically, so this module exposes thin data-last adapters: native delegation (map(fn) => it => Iterator.from(it).map(fn)) for the standard helpers, and wrappers around the data-first functions for the rest. They carry no .lazy metadata, so they compose through the existing pipe while staying fully lazy — no changes to the lazy machinery.
  • Hand-rolled next() over generators. A quick benchmark (map → filter → consume, 1M elements) showed manual iterator objects ~2× faster than yield-based generators, and faster than native helpers. The internal iterator() helper absorbs the boilerplate.
  • Iterator.from only where a method is called (the fp native-delegation adapters), not in every data-first function — those only call .next(), which works on any iterator.

Phase 1 surface

es-toolkit/iterator (data-first): range, iterate, takeWhile, dropWhile, chunk, zip, scan, uniqBy, head, count, partition

es-toolkit/fp/iterator (data-last): the above (except the range/iterate sources) + native-delegating map, filter, take, drop, flatMap, toArray, reduce, forEach, find, some, every

import { range, iterate, chunk } from 'es-toolkit/iterator';

range(0, Infinity).take(5).toArray(); // [0, 1, 2, 3, 4]
iterate(1, x => x * 2).take(5).toArray(); // [1, 2, 4, 8, 16]
chunk([1, 2, 3, 4, 5].values(), 2).toArray(); // [[1, 2], [3, 4], [5]]

import { pipe } from 'es-toolkit/fp';
import { filter, map, toArray } from 'es-toolkit/fp/iterator';

pipe(
  [1, 2, 3, 4, 5, 6].values(),
  filter(x => x % 2 === 0),
  map(x => x * 10),
  toArray()
); // [20, 40, 60]

Notable behavior choices

  • zip stops at the shortest source (rather than padding to the longest like array zip), so it stays safe with infinite iterators.
  • scan emits the initial value first (scan-left), so output length is n + 1.
  • head is provided as a named terminal even though iterator.next().value covers it, because it can't be chained ergonomically inside pipe.
  • iterate / range are infinite-capable sources and must be bounded by a short-circuiting helper (take, takeWhile) before being consumed.

Testing

  • 71 tests across src/iterator and src/fp/iterator, including laziness proofs (side-effect counters), early-termination on infinite sources, single-shot contract, and native-helper chaining.
  • tsc --noEmit and eslint clean.

Note: a local yarn build fails in this sandbox due to a pre-existing tsdown + Yarn PnP ESM-loader issue (reproduces on a clean checkout without these changes); typecheck, lint, and tests all pass.

Follow-ups (next phases)

  • Phase 2: remaining lazy transforms (windowed, flatten, intersperse, concat, pairwise, tail, compact, …) and terminals (last, min/max, sum/sumBy, groupBy, …).
  • Phase 3: inherently-eager operators (sortBy, reverse, shuffle, …) and docs in all 4 languages. Per the fp precedent, per-function docs are deferred from this PR.

Introduce `es-toolkit/iterator` (data-first) and `es-toolkit/fp/iterator`
(data-last, pipe-friendly), built on native Iterator Helper semantics:
functions operate on the single-shot iterators returned by `.values()` and
return lazy `IteratorObject`s whose prototype is `Iterator.prototype`, so
results chain with native `.map()/.take()/.toArray()` etc.

Functions already covered by the TC39 Iterator Helpers (map/filter/take/drop/
flatMap/reduce/toArray/...) are intentionally NOT reimplemented in the
data-first module — only operators the proposal lacks are added. The pipe
module exposes thin data-last adapters (native delegation for the standard
helpers, wrappers for the rest) so they compose in `pipe` without changes to
the existing lazy machinery.

Phase 1 surface:
- iterator (data-first): range, iterate, takeWhile, dropWhile, chunk, zip,
  scan, uniqBy, toSet, count, partition
- fp/iterator (data-last): the above + native-delegating map, filter, take,
  drop, flatMap, toArray, reduce, forEach, find, some, every

Implementation uses hand-rolled `next()` iterator objects rather than
generators (measured ~2x faster for the same transforms). Packaging mirrors
the fp module: dev + publishConfig exports, files entries, and postbuild
root-export shims (including a nested fp/iterator shim).
@raon0211 raon0211 requested a review from dayongkr as a code owner June 27, 2026 07:54
@vercel

vercel Bot commented Jun 27, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
es-toolkit Ready Ready Preview, Comment Jun 29, 2026 1:05am

Request Review

raon0211 added 2 commits June 28, 2026 10:09
- Remove `iterate`: niche infinite generator outside the module's pragmatic
  scope; numeric sequences are covered by `range`.
- Remove `toSet`: redundant with native `new Set(iterator)` since the lazy
  iterators are iterable.
- Add `head` (data-first + fp/iterator): returns the first element or
  undefined, reading a single element so it is safe on infinite iterators and
  useful as a pipe terminal where native `.next().value` is awkward.
Restore `iterate(seed, getNext)`, a lazy infinite source that yields
`seed, getNext(seed), getNext(getNext(seed)), ...` — the standard
"iterate" of Haskell/Clojure/Kotlin generateSequence/Java Stream.iterate.
Data-first only (it is a source used as a pipe's first argument), so no
fp/iterator wrapper.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant