Skip to content

test: add 36 edge case tests for all tokenizer variants

6b1cf73
Select commit
Loading
Failed to load commit list.
Sign in for the full log view
Closed

feat: register Hebrew and Chinese units in Tokenizer API #17

test: add 36 edge case tests for all tokenizer variants
6b1cf73
Select commit
Loading
Failed to load commit list.

Annotations

1 error and 1 warning
Test
failed Apr 8, 2026 in 23s