evaluation_generated_images - Multimodal T2I Evaluation Framework v1.0.0

Multimodal Benchmarking & Recommendation of Text-to-Image Models

Version: v1.0.0
Release date: 2025-05-11

This is the first official release of the Evaluationgeneratedimages repository. It includes:

• 📦 Core Scripts

  • text2image_generation.py: inference against 12 text-to-image models
    
  • evaluation_metrics.py: computes CLIP Score, LPIPS, FID, MRR, Recall@3, and the aggregated Weighted Score
    
  • evaluation_pipeline.py: end-to-end benchmarking pipeline
    
  • visualization_app.py: Streamlit UI for interactive result exploration
    

• 📊 Precomputed Results

  • Baseline evaluation results for each model on the DeepFashion-MultiModal dataset (see results/)
    
  • Sample plots and charts under README_files/
    

• 📚 Documentation & Examples

  • Detailed setup and usage instructions in README.md
    
  • Citation instructions and CITATION.cff for seamless attribution
    

How to Get Started

1. Clone the repo and install dependencies:
   ```bash git clone https://github.com/kapilw25/Evaluationgeneratedimages.git cd Evaluationgeneratedimages pip install -r requirements.txt

2. Run the demo Streamlit app: bash streamlit run visualization_app.py

Citation

If you use this codebase, please cite: bibtex @misc{wanaskar2025multimodalbenchmarkingrecommendationtexttoimage, title={Multimodal Benchmarking and Recommendation of Text-to-Image Generation Models}, author={Kapil Wanaskar and Gaytri Jena and Magdalini Eirinaki}, year={2025}, eprint={2505.04650}, archivePrefix={arXiv}, primaryClass={cs.GR}, url={https://arxiv.org/abs/2505.04650} }

- Python
Published by kapilw25 about 1 year ago