Paper: Image Cropping under Design Constraints

Takumi Nishiyasu¹, Wataru Shimoda², Yoichi Sato^1
1Tokyo University, ² CyberAgent.Inc

Accepted to ACMMM Asia 2023. [Publication] [Arxiv] [project-page]

Requirements

• Python3.9+
• Install the following packages: bash pip install numpy pandas

Evaluation

This repository contains evaluation codes for "Image Cropping under Design Constraints".

• Evaluate the performance of the heatmap-based approach (iter_size = 100) by:

  bash python test.py --pred_file data/predictions.csv

• predictions.csv is a file for evaluation. The structure of the file is as follows:

  • image file name, predicted bounding box [x1,y1,x2,y2], ground truth bounding box [x1,y1,x2,y2], layout bounding box [x1,y1,x2,y2] ``` 1011366896_Large.jpg,"[[0, 0, 768, 768]]","[[0.0, 0.0, 756.0, 768.0]]","[[0.0, 0.0, 512.0, 384.0]]"

  1011366896_Large.jpg,"[[0, 0, 1024, 757]]","[[105.0, 0.0, 1024.0, 690.0]]","[[512.0, 0.0, 1024.0, 384.0]]"

  ...

  IMG_6266.jpg,"[[0, 96, 941, 736]]","[[0.0, 77.0, 911.0, 697.0]]","[[0.0, 170.0, 128.0, 682.0]]" ```

Evaluation data

• The extended evaluation dataset is here

• The JSON file contains the design constraints for preparing the ground truth bounding box and layout bounding box with image information of FLMS

• The structure of FLMSblankaspect.json is as follows: ``` { "1011366896Large-0-10158730158730158.png": { "imagename": "1011366896Large.jpg", "aspectratio": "1.0158730158730158", "blankspaceindex": 0, "blankspace": [ 0, 0, 512, 384 ], "blankaspectbbox": [ 0, 0, 756, 768 ] }, "1011366896Large-1-0750816104461371.png": {...},

  ...
  
  "IMG_6266-4-0_6805708013172338.png": {
  "image_name": "IMG_6266.jpg",
  "aspect_ratio": "0.6805708013172338",
  "blank_space_index": 4,
  "blank_space": [
      0,
      170,
      128,
      682
  ],
  "blank_aspect_bbox": [
      0,
      77,
      911,
      697
  ]
  }
  

  }

  ```

  Reference

  bibtex @inproceedings{flmsfang2014automatic, title={Automatic Image Cropping Using Visual Composition, Boundary Simplicity and Content Preservation Models}, author={Fang, Chen and Lin, Zhe and Mech, Radomir and Shen, Xiaohui}, booktitle={Proc. ACM International Conference on Multimedia (ACMMM)}, pages={1105--1108}, year={2014}, }

Citation

If you use this code or find our work is helpful, please consider citing our work:

bibtex @inproceedings{nishiyasu2023cropping, title={Image Cropping under Design Constraints}, author={Takumi, Nishiyasu and Wataru, Shimoda and Yoichi, Sato}, booktitle={Proc. ACM International Conference on Multimedia Asia (ACMMMAsia)}, year={2023}, }

Contact

If you have any requests or need to get in touch, please feel free to raise issues or send a Pull Request.