optimization-awq
Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.
[OptimizationAWQQuantization4-Bit
Orchestra Research
33,1943,891
outlines
Outlines: structured JSON/regex/Pydantic LLM generation.
[PromptEngineeringOutlinesStructured
poorrican
0
outlines
Outlines: structured JSON/regex/Pydantic LLM generation.
[PromptEngineeringOutlinesStructured
PoorRican
0