multimodal-audiocraft
PyTorch library for audio generation including text-to-music (MusicGen) and text-to-sound (AudioGen). Use when you need to generate music from text descriptions, create sound effects, or perform melody-conditioned music generation.
multimodal-blip-2
Vision-language pre-training framework bridging frozen image encoders and LLMs. Use when you need image captioning, visual question answering, image-text retrieval, or multimodal chat with state-of-the-art zero-shot performance.
multimodal-clip
OpenAI's model connecting vision and language. Enables zero-shot image classification, image-text matching, and cross-modal retrieval. Trained on 400M image-text pairs. Use for image search, content moderation, or vision-language tasks without fine-tuning. Best for general-purpose image understanding.
multimodal-segment-anything
Foundation model for image segmentation with zero-shot transfer. Use when you need to segment any object in images using points, boxes, or masks as prompts, or automatically generate all object masks in an image.
segment-anything-model
SAM: zero-shot image segmentation via points, boxes, masks.
audiocraft-audio-generation
AudioCraft: MusicGen text-to-music, AudioGen text-to-sound.
audiocraft-audio-generation
AudioCraft: MusicGen text-to-music, AudioGen text-to-sound.
clip
OpenAI's model connecting vision and language. Enables zero-shot image classification, image-text matching, and cross-modal retrieval. Trained on 400M image-text pairs. Use for image search, content moderation, or vision-language tasks without fine-tuning. Best for general-purpose image understanding.
segment-anything-model
SAM: zero-shot image segmentation via points, boxes, masks.
audiocraft-audio-generation
AudioCraft: MusicGen text-to-music, AudioGen text-to-sound.
clip
OpenAI's model connecting vision and language. Enables zero-shot image classification, image-text matching, and cross-modal retrieval. Trained on 400M image-text pairs. Use for image search, content moderation, or vision-language tasks without fine-tuning. Best for general-purpose image understanding.
segment-anything-model
SAM: zero-shot image segmentation via points, boxes, masks.