A polyfill for Intl.Segmenter.

ECMA-402 Spec Compliance#

This package implements Intl.Segmenter. See the conformance report for tested coverage and known limitations.

Rust and TypeScript implementations share an independent ICU4C/ICU4J test matrix for UTF-16 boundaries and word classification. Hermes runs both implementations in source and bytecode modes, including containing() and record property checks. Dictionary segmentation and sentence abbreviation suppression can differ between the implementations.

Specification Details#

• TC39 Proposal: Intl.Segmenter
• Stage: Stage 4 (Finalized)
• Spec: ECMA-402 Intl.Segmenter

✅ Implemented Features#

Core Methods#

• segment(string) - Returns an iterable Segments object for the string
• resolvedOptions() - Returns resolved options
• supportedLocalesOf(locales) - Returns supported locales

Granularity Options#

All 3 segmentation granularities are supported:

• 'grapheme' - Grapheme cluster boundaries (user-perceived characters)
  • Handles combining marks, emoji, etc.
  • Example: "👨‍👩‍👧‍👦" is one grapheme
• 'word' - Word boundaries with word/punctuation classification
  • Identifies words, spaces, punctuation
  • Provides isWordLike property
• 'sentence' - Sentence boundaries
  • Handles abbreviations, numbers, quotes
  • Locale-aware sentence breaks

Segments Object#

The Segments object returned by segment() is:

• Iterable - Can be used with for...of loops
• containing(index) - Finds the segment containing a UTF-16 code-unit index

Segment Object Properties#

Each segment has:

• segment - The text of the segment
• index - Start index in the original string
• input - The original input string
• isWordLike - (word granularity only) Whether segment is a word

Example Usage#

Global import#

import '@formatjs/intl-segmenter/polyfill.js'

// Grapheme segmentation (user-perceived characters)
const graphemeSegmenter = new Intl.Segmenter('en', {granularity: 'grapheme'})
const graphemes = [...graphemeSegmenter.segment('Hello👋')]
// [
//   {segment: 'H', index: 0, input: 'Hello👋'},
//   {segment: 'e', index: 1, input: 'Hello👋'},
//   {segment: 'l', index: 2, input: 'Hello👋'},
//   {segment: 'l', index: 3, input: 'Hello👋'},
//   {segment: 'o', index: 4, input: 'Hello👋'},
//   {segment: '👋', index: 5, input: 'Hello👋'}  // Emoji as one grapheme
// ]

// Word segmentation
const wordSegmenter = new Intl.Segmenter('en', {granularity: 'word'})
const words = [...wordSegmenter.segment('Hello, world!')]
// [
//   {segment: 'Hello', index: 0, isWordLike: true},
//   {segment: ',', index: 5, isWordLike: false},
//   {segment: ' ', index: 6, isWordLike: false},
//   {segment: 'world', index: 7, isWordLike: true},
//   {segment: '!', index: 12, isWordLike: false}
// ]

// Filter to only word-like segments
const onlyWords = words.filter(s => s.isWordLike)
// [{segment: 'Hello', ...}, {segment: 'world', ...}]

// Sentence segmentation
const sentenceSegmenter = new Intl.Segmenter('en', {granularity: 'sentence'})
const sentences = [
  ...sentenceSegmenter.segment('Hello! How are you? I am fine.'),
]
// [
//   {segment: 'Hello! ', index: 0, input: '...'},
//   {segment: 'How are you? ', index: 7, input: '...'},
//   {segment: 'I am fine.', index: 20, input: '...'}
// ]

// containing() method - find segment at specific index
const segments = wordSegmenter.segment('Hello, world!')
segments.containing(7)
// {segment: 'world', index: 7, isWordLike: true}

// Locale-aware segmentation
const thaiSegmenter = new Intl.Segmenter('th', {granularity: 'word'})
const thaiWords = [...thaiSegmenter.segment('สวัสดีครับ')]
// Correctly segments Thai text without spaces

// Complex emoji handling
const emojiSegmenter = new Intl.Segmenter('en', {granularity: 'grapheme'})
const emojis = [...emojiSegmenter.segment('👨‍👩‍👧‍👦🏴󠁧󠁢󠁳󠁣󠁴󠁿')]
// [
//   {segment: '👨‍👩‍👧‍👦', index: 0},  // Family emoji (ZWJ sequence)
//   {segment: '🏴󠁧󠁢󠁳󠁣󠁴󠁿', index: ...}   // Scotland flag
// ]

ES Modules#

import {Segmenter} from '@formatjs/intl-segmenter'

// Grapheme segmentation (user-perceived characters)
const graphemeSegmenter = new Segmenter('en', {granularity: 'grapheme'})
const graphemes = [...graphemeSegmenter.segment('Hello👋')]

Use Cases#

• Character counting: Get accurate character count including complex emoji
• Word counting: Count words across different scripts and languages
• Text truncation: Safely truncate at grapheme boundaries
• Syntax highlighting: Break code into word segments
• Search indexing: Segment text for full-text search
• Text analysis: Analyze sentence structure

Installation#

npm i @formatjs/intl-segmenter

Features#

Everything in intl-segmenter proposal

Usage#

Simple#

import '@formatjs/intl-segmenter/polyfill.js'

Dynamic import + capability detection#

async function polyfill(locale: string) {
  if (shouldPolyfill()) {
    await import('@formatjs/intl-segmenter/polyfill-force.js')
  }
}

Iteration and default locale#

segment() returns a reusable iterable with containing(). Each call to its Symbol.iterator creates an independent iterator. An iterator returns itself from Symbol.iterator and resumes from its current position. next() belongs to the iterator, not the iterable returned by segment(). The fallback locale is en, backed by the root segmentation rules.

containing(index) coerces its index to a number and truncates toward zero. BigInt values throw TypeError, including boxed BigInts and objects that coerce to BigInt. Errors thrown during coercion propagate unchanged.

Locales without dedicated CLDR tailoring use root Unicode segmentation rules. Regional locales retain their language's tailoring, including sentence suppressions.

Rust Hermes extension#

The experimental Rust port uses ICU4X grapheme, word and sentence models. Install SDK target :segmenter after canonicalization, following the native SDK guide. Register js/36-IntlSegmenter.js for precompiled Hermes extension setup. The SDK includes public headers, adapter source, compiled ICU data, and a typed consumer example. Desktop and Android archives are covered in the SDK guide.

ICU dictionary/LSTM models recognize complex-script words. Original UTF-16 input and indices remain exact; cursors advance independently. containing truncates before bounds checks, so -0.5 selects index zero; BigInt throws TypeError.

Word/sentence boundaries and word-like status are implementation-dependent. Compared with TS, ICU groups dictionary words, applies different abbreviation profiles, and excludes isolated marks/emoji from word-like segments. Tests cross-check ICU4C/J/X, both implementations, and the complete Unicode 17 grapheme, word, and sentence corpora. Segmenter keeps static data, matching the existing package's loading model.

Measured performance#

TypeScript and native JavaScript#

Measured 2026-10-31 on Apple M4 Max, macOS arm64, Node 24.14.0 / ICU 78.2; source 56d15e997. Median of seven warmed rounds.

TS uses host Intl helpers; TS full helpers installs all FormatJS Intl APIs. Module loading and data registration are excluded. Construction rows include resolvedOptions(). These are desktop microbenchmarks; native locale data may differ. † marks noisy rows (round max/min > 1.5). See workloads, commands, and raw results.

Rust core#

Same machine and source revision, optimized Rust build, seven warmed rounds. Rust core timings exclude Hermes/JSI and JS boundary work; inputs are prepared Rust strings/UTF-16. Data and API-boundary differences prevent direct speedup claims against TS/native JS. † marks round max/min > 1.5. Workload details and raw results.