| No | Publication | Model | Paper | Benchmark | Version | |:---:|:-----------------:|:----------------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |:---------------------------------------------------------------------------------------------------------------:|:-------------:| |:openfilefolder: Feature Interaction Models| | 1 | WWW'07 | LR | Predicting Clicks: Estimating the Click-Through Rate for New Ads :triangularflagonpost:Microsoft | :arrowupperright: | torch | | 2 | ICDM'10 | FM | Factorization Machines | :arrowupperright: | torch | | 3 | CIKM'13 | DSSM | Learning Deep Structured Semantic Models for Web Search using Clickthrough Data :triangularflagonpost:Microsoft | :arrowupperright: | torch | | 4 | CIKM'15 | CCPM | A Convolutional Click Prediction Model | :arrowupperright: | torch | | 5 | RecSys'16 | FFM | Field-aware Factorization Machines for CTR Prediction :triangularflagonpost:Criteo | :arrowupperright: | torch | | 6 | RecSys'16 | DNN | Deep Neural Networks for YouTube Recommendations :triangularflagonpost:Google | :arrowupperright: | torch, tf | | 7 | DLRS'16 | Wide&Deep | Wide & Deep Learning for Recommender Systems :triangularflagonpost:Google | :arrowupperright: | torch, tf | | 8 | ICDM'16 | PNN | Product-based Neural Networks for User Response Prediction | :arrowupperright: | torch | | 9 | KDD'16 | DeepCrossing | Deep Crossing: Web-Scale Modeling without Manually Crafted Combinatorial Features :triangularflagonpost:Microsoft | :arrowupperright: | torch | | 10 | NIPS'16 | HOFM | Higher-Order Factorization Machines | :arrowupperright: | torch | | 11 | IJCAI'17 | DeepFM | DeepFM: A Factorization-Machine based Neural Network for CTR Prediction :triangularflagonpost:Huawei | :arrowupperright: | torch, tf | | 12 | SIGIR'17 | NFM | Neural Factorization Machines for Sparse Predictive Analytics | :arrowupperright: | torch | | 13 | IJCAI'17 | AFM | Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks | :arrowupperright: | torch | | 14 | ADKDD'17 | DCN | Deep & Cross Network for Ad Click Predictions :triangularflagonpost:Google | :arrowupperright: | torch, tf | | 15 | WWW'18 | FwFM | Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising :triangularflagonpost:Oath, TouchPal, LinkedIn, Alibaba | :arrowupperright: | torch | | 16 | KDD'18 | xDeepFM | xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems :triangularflagonpost:Microsoft | :arrowupperright: | torch | | 17 | CIKM'19 | FiGNN | FiGNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction | :arrowupperright: | torch | | 18 | CIKM'19 | AutoInt/AutoInt+ | AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks | :arrowupperright: | torch | | 19 | RecSys'19 | FiBiNET | FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction :triangularflagonpost:Sina Weibo | :arrowupperright: | torch | | 20 | WWW'19 | FGCNN | Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction :triangularflagonpost:Huawei | :arrowupperright: | torch | | 21 | AAAI'19 | HFM/HFM+ | Holographic Factorization Machines for Recommendation | :arrowupperright: | torch | | 22 | Arxiv'19 | DLRM | Deep Learning Recommendation Model for Personalization and Recommendation Systems :triangularflagonpost:Facebook | :arrowupperright: | torch | | 23 | NeuralNetworks'20 | ONN | Operation-aware Neural Networks for User Response Prediction | :arrowupperright: | torch, tf | | 24 | AAAI'20 | AFN/AFN+ | Adaptive Factorization Network: Learning Adaptive-Order Feature Interactions | :arrowupperright: | torch | | 25 | AAAI'20 | LorentzFM | Learning Feature Interactions with Lorentzian Factorization :triangularflagonpost:eBay | :arrowupperright: | torch | | 26 | WSDM'20 | InterHAt | Interpretable Click-through Rate Prediction through Hierarchical Attention :triangularflagonpost:NEC Labs, Google | :arrowupperright: | torch | | 27 | DLP-KDD'20 | FLEN | FLEN: Leveraging Field for Scalable CTR Prediction :triangularflagonpost:Tencent | :arrowupperright: | torch | | 28 | CIKM'20 | DeepIM | Deep Interaction Machine: A Simple but Effective Model for High-order Feature Interactions :triangularflagonpost:Alibaba, RealAI | :arrowupperright: | torch | | 29 | WWW'21 | FmFM | FM^2: Field-matrixed Factorization Machines for Recommender Systems :triangularflagonpost:Yahoo | :arrowupperright: | torch | | 30 | WWW'21 | DCN-V2 | DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems :triangularflagonpost:Google | :arrowupperright: | torch | | 31 | CIKM'21 | DESTINE | Disentangled Self-Attentive Neural Networks for Click-Through Rate Prediction :triangularflagonpost:Alibaba | :arrowupperright: | torch | | 32 | CIKM'21 | EDCN | Enhancing Explicit and Implicit Feature Interactions via Information Sharing for Parallel Deep CTR Models :triangularflagonpost:Huawei | :arrowupperright: | torch | | 33 | DLP-KDD'21 | MaskNet | MaskNet: Introducing Feature-Wise Multiplication to CTR Ranking Models by Instance-Guided Mask :triangularflagonpost:Sina Weibo | :arrowupperright: | torch | | 34 | SIGIR'21 | SAM | Looking at CTR Prediction Again: Is Attention All You Need? :triangularflagonpost:BOSS Zhipin | :arrowupperright: | torch | | 35 | KDD'21 | AOANet | Architecture and Operation Adaptive Network for Online Recommendations :triangularflagonpost:Didi Chuxing | :arrowupperright: | torch | | 36 | AAAI'23 | FinalMLP | FinalMLP: An Enhanced Two-Stream MLP Model for CTR Prediction :triangularflagonpost:Huawei | :arrowupperright: | torch | | 37 | SIGIR'23 | FinalNet | FINAL: Factorized Interaction Layer for CTR Prediction :triangularflagonpost:Huawei | :arrowupperright: | torch | | 38 | SIGIR'23 | EulerNet | EulerNet: Adaptive Feature Interaction Learning via Euler's Formula for CTR Prediction :triangularflagonpost:Huawei | :arrowupperright: | torch | | 39 | CIKM'23 | GDCN | Towards Deeper, Lighter and Interpretable Cross Network for CTR Prediction :triangularflagonpost:Microsoft | | torch | | 40 | ICML'24 | WuKong | Wukong: Towards a Scaling Law for Large-Scale Recommendation :triangularflagonpost:Meta | :arrowupperright: | torch | |:openfilefolder: Behavior Sequence Modeling| | 42 | KDD'18 | DIN | Deep Interest Network for Click-Through Rate Prediction :triangularflagonpost:Alibaba | :arrowupperright: | torch | | 43 | AAAI'19 | DIEN | Deep Interest Evolution Network for Click-Through Rate Prediction :triangularflagonpost:Alibaba | :arrowupperright: | torch | | 44 | DLP-KDD'19 | BST | Behavior Sequence Transformer for E-commerce Recommendation in Alibaba :triangularflagonpost:Alibaba | :arrowupperright: | torch | | 45 | CIKM'20 | DMIN | Deep Multi-Interest Network for Click-through Rate Prediction :triangularflagonpost:Alibaba | :arrowupperright: | torch | | 46 | AAAI'20 | DMR | Deep Match to Rank Model for Personalized Click-Through Rate Prediction :triangularflagonpost:Alibaba | :arrowupperright: | torch | | 47 | KDD'23 | TransAct | TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterest :triangularflagonpost:Pinterest | :arrowupperright: | torch | |:openfilefolder: Long Sequence Modeling| | 48 | CIKM'20 | SIM | Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction :triangularflagonpost:Alibaba | | torch | | 49 | DLP-KDD'22 | ETA | Efficient Long Sequential User Data Modeling for Click-Through Rate Prediction :triangularflagonpost:Alibaba | | torch | | 50 | CIKM'22 | SDIM | Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction :triangularflagonpost:Meituan | | torch | | 51 | KDD'23 | TWIN | TWIN: TWo-stage Interest Network for Lifelong User Behavior Modeling in CTR Prediction at Kuaishou :triangularflagonpost:KuaiShou | | torch | | 52 | KDD'25 | MIRRN | Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction :triangularflagonpost:Huawei | | torch | |:openfilefolder: Dynamic Weight Network| | 53 | NeurIPS'22 | APG | APG: Adaptive Parameter Generation Network for Click-Through Rate Prediction :triangularflagonpost:Alibaba | :arrowupperright: | torch | | 54 | KDD'23 | PPNet | PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information :triangularflagonpost:KuaiShou | :arrowupperright: | torch | |:openfilefolder: Multi-Task Modeling| | 55 | Arxiv'17 | ShareBottom | An Overview of Multi-Task Learning in Deep Neural Networks | | torch | | 56 | KDD'18 | MMoE | Modeling Task Relationships in Multi-task Learning with Multi-Gate Mixture-of-Experts :triangularflagonpost:Google | | torch | | 57 | RecSys'20 | PLE | Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations :triangularflagonpost:Tencent | | torch |

Benchmarking

We have benchmarked FuxiCTR models on a set of open datasets as follows:

• :star: Benchmark datasets for CTR prediction
• :star: Benchmark settings and running steps
• :star: Benchmark leaderboard for CTR prediction

Dependencies

FuxiCTR has the following dependencies:

• python 3.9+
• pytorch 1.10.0--2.1.2 (if using for torch models)
• tensorflow 2.1 (if using for tensorflow models)

Please install other required packages via pip install -r requirements.txt.

Quick Start

1. Run the demo examples

   Examples are provided in the demo directory to show some basic usage of FuxiCTR. Users can run the examples for quick start and to understand the workflow.

cd demo python example1_build_dataset_to_parquet.py python example2_DeepFM_with_parquet_input.py

1. Run a model on tiny data

   Users can easily run each model in the model zoo following the commands below, which is a demo for running DCN. In addition, users can modify the dataset config and model config files to run on their own datasets or with new hyper-parameters. More details can be found in the README.

``` cd modelzoo/DCN/DCNtorch python runexpid.py --expid DCNtest --gpu 0

# Change MODEL according to the target model name cd modelzoo/MODEL python runexpid.py --expid MODEL_test --gpu 0 ```

1. Run a model on benchmark datasets (e.g., Criteo)

Users can follow the benchmark section to get benchmark datasets and running steps for reproducing the existing results. Please see an example here: https://github.com/reczoo/BARS/tree/main/ranking/ctr/DCNv2/DCNv2criteox1

1. Implement a new model

The FuxiCTR library is designed to be modularized, so that every component can be overwritten by users according to their needs. In many cases, only the model class needs to be implemented for a new customized model. If data preprocessing or data loader is not directly applicable, one can also overwrite a new one through the core APIs. We show a concrete example which implements our new model FinalMLP that has been recently published in AAAI 2023.

1. Tune hyper-parameters of a model

FuxiCTR currently support fast grid search of hyper-parameters of a model using multiple GPUs. The following example shows the grid search of 8 experiments with 4 GPUs.

cd experiment python run_param_tuner.py --config config/DCN_tiny_parquet_tuner_config.yaml --gpu 0 1 2 3 0 1 2 3

🔥 Citation

If you use our code or benchmarks in your public research, please cite the following two papers.

• Jieming Zhu, Quanyu Dai, Liangcai Su, Rong Ma, Jinyang Liu, Guohao Cai, Xi Xiao, Rui Zhang. BARS: Towards Open Benchmarking for Recommender Systems. The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2022. [Bibtex]
• Jieming Zhu, Jinyang Liu, Shuai Yang, Qi Zhang, Xiuqiang He. BARS-CTR: Open Benchmarking for Click-Through Rate Prediction. The 30th ACM International Conference on Information and Knowledge Management (CIKM), 2021. [Bibtex]

🙋 Discussion

Welcome to join our WeChat group for any question and discussion. If you are interested in research and practice in recommender systems, please reach out via our WeChat group.