huggingface-accelerate

Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.

[DistributedTrainingHuggingFaceAccelerate

PoorRican

0

axolotl

Axolotl: YAML LLM fine-tuning (LoRA, DPO, GRPO).

[Fine-TuningAxolotlLLMLoRA

PoorRican

0

obliteratus

OBLITERATUS: abliterate LLM refusals (diff-in-means).

[AbliterationUncensoringRefusal-RemovalLLM

PoorRican

0

huggingface-tokenizers

Fast tokenizers optimized for research and production. Rust-based implementation tokenizes 1GB in <20 seconds. Supports BPE, WordPiece, and Unigram algorithms. Train custom vocabularies, track alignments, handle padding/truncation. Integrates seamlessly with transformers. Use when you need high-performance tokenization or custom tokenizer training.

[TokenizationHuggingFaceBPEWordPiece

PoorRican

0