GAN Evaluator for Inception Score (IS) and Frechet Inception Distance (FID) in PyTorch

Please kindly *Star** this repo for better reach if you find it useful. Let's help out the community!*

Main Contributions

1. We created a GAN evaluator for IS and FID that
   • is easy to use,
   • accepts data as either dataloaders or individual batches, and
   • supports on-the-fly evaluation during training.
2. We provided a simple demo script to demonstrate one common use case.

NEWS

[Feb 18, 2023]

Now available on PyPI! Now you can pip install it to your desired environment via: pip install gan-evaluator And in your Python project, wherever you need the GAN_Evaluator, you can import via: from gan_evaluator import GAN_Evaluator NOTE 1: You no longer need to copy any code from this repo in order to use GAN_Evalutor! At this point, the primary purpose of this repo is description and demonstration. With that said, you surely can clone this repo and try out the demo script. Also, you may find it easier to copy and modify the code if you want slightly different behaviors.

NOTE 2: During pip install gan-evaluator, the dependencies of GAN_Evaluator (but not of the demo script) are also installed.

Demo Script: Use DCGAN to generate SVHN digits

The script can be found in src/train_dcgan_svhn.py

• Usage from the demo script, to give you a taste.

  Declaration evaluator = GAN_Evaluator(device=device, num_images_real=len(train_loader.dataset), num_images_fake=len(train_loader.dataset))

  Before traing loop evaluator.load_all_real_imgs(real_loader=train_loader, idx_in_loader=0)

  Inside traing loop if shall_plot: IS_mean, IS_std, FID = evaluator.fill_fake_img_batch(fake_batch=x_fake) else: evaluator.fill_fake_img_batch(fake_batch=x_fake, return_results=False)

  After each epoch of training evaluator.clear_fake_imgs()

• Some visualizations of the demo script:

  • Real (top) and Generated (bottom) images.
  • IS and FID curves.

Details: The Evaluator for IS and FID

Repository Hierarchy

GAN-evaluator ├── config | └── `dcgan_svhn.yaml` ├── data (*) ├── debug_plot (*) ├── logs (*) └── src ├── utils | ├── `gan_evaluator.py`: THIS CONTAINS OUR `GAN_Evaluator`. | └── other utility files. └── `train_dcgan_svhn.py`: our demo script. Folders marked with (*), if not exist, will be created automatically when you run train_dcgan_svhn.py.

Usage

• To integrate our evaluator into your existing project, you can simply install via pip!

• You can refer to the demo script to see how to interface with the evaluator. Briefly, you need to declare the evaluator, feed in the real images, and feed in the fake images. In most cases, you would feed in the real images in one pass, whereas feed in the fake image on-the-fly as the model is trained and fake images are generated. Each time you feed in the fake images, you can choose whether or not to compute the metrics. Lastly, don't forget to remove the fake images at the end of each epoch.

• To run our demo script, do the following after activating the proper environment. git clone git@github.com:ChenLiu-1996/GAN-evaluator.git cd src python train_dcgan_svhn.py --config ../config/dcgan_svhn.yaml

Citation

To be added.

Environement Setup

Acknowledgements

1. The code for the GAN_Evaluator (specifically, the computation of IS and FID) is inspired by:
   • https://github.com/sbarratt/inception-score-pytorch/blob/master/inception_score.py
   • https://www.kaggle.com/code/ibtesama/gan-in-pytorch-with-fid
   • https://github.com/mseitzer/pytorch-fid/blob/master/src/pytorchfid/fidscore.py
   • Note: We did not validate the "mathmetical correctness" in their computations. Please use mindfully.
2. The code for the demo script (specifically, architecture and training of DCGAN) is inspired by:
   • https://pytorch.org/tutorials/beginner/dcganfacestutorial