tokenization-sentencepiece
Language-independent tokenizer treating text as raw Unicode. Supports BPE and Unigram algorithms. Fast (50k sentences/sec), lightweight (6MB memory), deterministic vocabulary. Used by T5, ALBERT, XLNet, mBART. Train on raw text without pre-tokenization. Use when you need multilingual support, CJK languages, or reproducible tokenization.
[TokenizationSentencePieceLanguage-IndependentBPE
Orchestra Research
33,1943,891
ascii-art
ASCII art: pyfiglet, cowsay, boxes, image-to-ascii.
[ASCIIArtBannersCreative
delorenj
176
ascii-art
ASCII art: pyfiglet, cowsay, boxes, image-to-ascii.
[ASCIIArtBannersCreative
poorrican
0
ascii-art
ASCII art: pyfiglet, cowsay, boxes, image-to-ascii.
[ASCIIArtBannersCreative
PoorRican
0