maestro - maestro-1.0.0

🚀 Added

• The VLMs field is evolving rapidly, making it tough to keep up. Maestro currently supports Florence-2, PaliGemma 2, and Qwen2.5-VL, and we'll strive to add key VLMs ASAP.

• VLMs fine-tuning can be costly. Maestro's built-in support for LoRA, QLoRA, and graph freezing allows training larger models even on less powerful hardware.

• VLMs fine-tuning requires lots of code. Maestro simplifies this complexity with single CLI/SDK calls.

• VLMs lack a unified approach. Maestro uses a consistent input format (JSONL now, COCO and YOLO coming soon) to minimize data formatting headaches.

🏆 Contributors

@onuralpszr (Onuralp SEZER), @SkalskiP (Piotr Skalski)

- Python
Published by SkalskiP about 1 year ago

maestro - multimodal-maestro-0.1.0

multimodal-maesto is out 🔥 🔥 🔥

🚀 Added

• SegmentAnythingMarkGenerator allowing the generation of segmentation marks.
• MarkVisualizer allowing to visualize the generated marks.
• prompt_image allowing for convenient GPT-4 Vision API querying.
• 🤗 Hugging Face Set-of-Mark space.

```python

import cv2 import torch import multimodalmaesto as mm

image = cv2.imread("...")

generator = mm.SegmentAnythingMarkGenerator() visualizer = mm.MarkVisualizer()

marks = generator.generate(image=image) marks = mm.refine_marks(marks=marks)

imageprompt = visualizer.visualize(image=image, marks=marks) textprompt = "Find dog."

response = mm.promptimage(apikey=apikey, image=imageprompt, prompt=text_prompt) response

"The dog is prominently featured in the center of the image with the label [9]."

masks = mm.extractrelevantmasks(text=response, detections=refined_marks)

{'6': array([ [False, False, False, ..., False, False, False], [False, False, False, ..., False, False, False], [False, False, False, ..., False, False, False], ..., [ True, True, True, ..., False, False, False], [ True, True, True, ..., False, False, False], [ True, True, True, ..., False, False, False]]) } ```

🏆 Contributors

@SkalskiP @deependujha

- Python
Published by SkalskiP about 2 years ago