Awesome Prompt Engineering 🧙‍♂️

This repository contains a hand-curated resources for Prompt Engineering with a focus on Generative Pre-trained Transformer (GPT), ChatGPT, PaLM etc

Prompt Engineering Course is coming soon..

Table of Contents

• Papers
• Tools & Code
• Apis
• Datasets
• Models
• AI Content Detectors
• Educational
  • Courses
  • Tutorials
• Videos
• Books
• Communities
• How to Contribute

Papers

📄 - Prompt Engineering Techniques: - A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT 2023 - Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery 2023 - Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models 2023 - Progressive Prompts: Continual Learning for Language Models 2023 - Batch Prompting: Efficient Inference with LLM APIs 2023 - Successive Prompting for Decompleting Complex Questions 2022 - Structured Prompting: Scaling In-Context Learning to 1,000 Examples 2022 - Large Language Models Are Human-Level Prompt Engineers 2022 - Ask Me Anything: A simple strategy for prompting language models 2022 - Prompting GPT-3 To Be Reliable 2022 - Decomposed Prompting: A Modular Approach for Solving Complex Tasks 2022 - PromptChainer: Chaining Large Language Model Prompts through Visual Programming 2022 - Investigating Prompt Engineering in Diffusion Models 2022 - Show Your Work: Scratchpads for Intermediate Computation with Language Models 2021 - Reframing Instructional Prompts to GPTk's Language 2021 - Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity 2021 - The Power of Scale for Parameter-Efficient Prompt Tuning 2021 - Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm 2021 - Prefix-Tuning: Optimizing Continuous Prompts for Generation 2021

• Reasoning and In-Context Learning:

  • Multimodal Chain-of-Thought Reasoning in Language Models 2023
  • On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning 2022
  • ReAct: Synergizing Reasoning and Acting in Language Models 2022
  • Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought 2022
  • On the Advance of Making Language Models Better Reasoners 2022
  • Large Language Models are Zero-Shot Reasoners 2022
  • Reasoning Like Program Executors 2022
  • Self-Consistency Improves Chain of Thought Reasoning in Language Models 2022
  • Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? 2022
  • Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering 2022
  • Chain of Thought Prompting Elicits Reasoning in Large Language Models 2021
  • Generated Knowledge Prompting for Commonsense Reasoning 2021
  • BERTese: Learning to Speak to BERT 2021

• Evaluating and Improving Language Models:

  • Large Language Models Can Be Easily Distracted by Irrelevant Context 2023
  • Crawling the Internal Knowledge-Base of Language Models 2023
  • Discovering Language Model Behaviors with Model-Written Evaluations 2022
  • Calibrate Before Use: Improving Few-Shot Performance of Language Models 2021

• Applications of Language Models:

  • Prompting for Multimodal Hateful Meme Classification 2023
  • PLACES: Prompting Language Models for Social Conversation Synthesis 2023
  • Commonsense-Aware Prompting for Controllable Empathetic Dialogue Generation 2023
  • PAL: Program-aided Language Models 2023
  • Legal Prompt Engineering for Multilingual Legal Judgement Prediction 2023
  • Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language 2022
  • Plot Writing From Scratch Pre-Trained Language Models 2022
  • AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts 2020

• Threat Detection and Adversarial Examples:

  • Constitutional AI: Harmlessness from AI Feedback 2022
  • Ignore Previous Prompt: Attack Techniques For Language Models 2022
  • Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods 2022
  • Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examples 2022
  • Toxicity Detection with Generative Prompt-based Inference 2022
  • How Can We Know What Language Models Know? 2020

• Few-shot Learning and Performance Optimization:

  • Promptagator: Few-shot Dense Retrieval From 8 Examples 2022
  • The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning 2022
  • Making Pre-trained Language Models Better Few-shot Learners 2021
  • Language Models are Few-Shot Learners 2020

• Text to Image Generation:

  • A Taxonomy of Prompt Modifiers for Text-To-Image Generation 2022
  • Design Guidelines for Prompt Engineering Text-to-Image Generative Models 2021
  • High-Resolution Image Synthesis with Latent Diffusion Models 2021
  • DALL·E: Creating Images from Text 2021

• Text to Music/Sound Generation:

  • MusicLM: Generating Music From Text 2023
  • ERNIE-Music: Text-to-Waveform Music Generation with Diffusion Models 2023
  • Noise2Music: Text-conditioned Music Generation with Diffusion Models [2023) (Arxiv)
  • AudioLM: a Language Modeling Approach to Audio Generation 2023
  • Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models 2023

• Text to Video Generation:

  • Dreamix: Video Diffusion Models are General Video Editors 2023
  • Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation 2022
  • Noise2Music: Text-conditioned Music Generation with Diffusion Models [2023) (Arxiv)
  • AudioLM: a Language Modeling Approach to Audio Generation 2023

• Overviews:

  • Piloting Copilot and Codex: Hot Temperature, Cold Prompts, or Black Magic? 2022

Tools & Code

🔧

| Name | Description | Url | | :-------------------- | :----------: | :----------: | | GPT Index | GPT Index is a project consisting of a set of data structures designed to make it easier to use large external knowledge bases with LLMs. | [Github] | | Promptify | Solve NLP Problems with LLM's & Easily generate different NLP Task prompts for popular generative models like GPT, PaLM, and more with Promptify | [Github] | | Arize-Phoenix | Open-source tool for ML observability that runs in your notebook environment. Monitor and fine tune LLM, CV and Tabular Models. | [Github] | | Better Prompt | Test suite for LLM prompts before pushing them to PROD | [Github] | | Embedchain | Framework to create ChatGPT like bots over your dataset | [Github] | | Interactive Composition Explorerx | ICE is a Python library and trace visualizer for language model programs. | [Github] | | Haystack | Open source NLP framework to interact with your data using LLMs and Transformers. | [Github] | | LangChainx | Building applications with LLMs through composability | [Github] | | OpenPrompt | An Open-Source Framework for Prompt-learning | [Github] | | Prompt Engine | This repo contains an NPM utility library for creating and maintaining prompts for Large Language Models (LLMs). | [Github] | | PromptInject | PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. | [Github] | | Prompts AI | Advanced playground for GPT-3 | [Github] | | Prompt Source | PromptSource is a toolkit for creating, sharing and using natural language prompts. | [Github] | | ThoughtSource | A framework for the science of machine thinking | [Github] | | PROMPTMETHEUS | One-shot Prompt Engineering Toolkit | [Tool] |

Apis

💻

| Name | Description | Url | Paid or Open-Source | | :-------------------- | :----------: | :----------: | :----------: | | OpenAI | GPT-n for natural language tasks, Codex for translates natural language to code, and DALL·E for creates and edits original images | [OpenAI] | Paid | | CohereAI | Cohere provides access to advanced Large Language Models and NLP tools through one API | [CohereAI] | Paid | | Anthropic | Coming soon | [Anthropic] | Paid | | FLAN-T5 XXL | Coming soon | [HuggingFace] | Open-Source |

Datasets

💾

| Name | Description | Url | | :-------------------- | :----------: | :----------: | | P3 (Public Pool of Prompts) | P3 (Public Pool of Prompts) is a collection of prompted English datasets covering a diverse set of NLP tasks. | [HuggingFace] | | Awesome ChatGPT Prompts | Repo includes ChatGPT prompt curation to use ChatGPT better. | [Github] | | Writing Prompts | Collection of a large dataset of 300K human-written stories paired with writing prompts from an online forum(reddit) | [Kaggle] | | Midjourney Prompts | Text prompts and image URLs scraped from MidJourney's public Discord server | [HuggingFace] |

Models

🧠

| Name | Description | Url | | :-------------------- | :----------: | :----------: | | ChatGPT | ChatGPT | [OpenAI] | | Codex | The Codex models are descendants of our GPT-3 models that can understand and generate code. Their training data contains both natural language and billions of lines of public code from GitHub | [Github] | | Bloom | BigScience Large Open-science Open-access Multilingual Language Model | [HuggingFace] | | Facebook LLM | OPT-175B is a GPT-3 equivalent model trained by Meta. It is by far the largest pretrained language model available with 175 billion parameters. | [Alpa] | | GPT-NeoX | GPT-NeoX-20B, a 20 billion parameter autoregressive language model trained on the Pile | [HuggingFace] | | FLAN-T5 XXL | Flan-T5 is an instruction-tuned model, meaning that it exhibits zero-shot-like behavior when given instructions as part of the prompt. | [HuggingFace/Google] | | XLM-RoBERTa-XL | XLM-RoBERTa-XL model pre-trained on 2.5TB of filtered CommonCrawl data containing 100 languages. | [HuggingFace] | | GPT-J | It is a GPT-2-like causal language model trained on the Pile dataset | [HuggingFace] | | PaLM-rlhf-pytorch | Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM | [Github] | GPT-Neo | An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library. | [Github] | | LaMDA-rlhf-pytorch | Open-source pre-training implementation of Google's LaMDA in PyTorch. Adding RLHF similar to ChatGPT. | [Github] | | RLHF | Implementation of Reinforcement Learning from Human Feedback (RLHF) | [Github] | | GLM-130B | GLM-130B: An Open Bilingual Pre-Trained Model | [Github] |

AI Content Detectors

🔎

| Name | Description | Url | | :-------------------- | :----------: | :----------: | | AI Text Classifier | The AI Text Classifier is a fine-tuned GPT model that predicts how likely it is that a piece of text was generated by AI from a variety of sources, such as ChatGPT. | [OpenAI] | | GPT-2 Output Detector | This is an online demo of the GPT-2 output detector model, based on the 🤗/Transformers implementation of RoBERTa. | [HuggingFace] | | Openai Detector | AI classifier for indicating AI-written text (OpenAI Detector Python wrapper) | [GitHub] |

Courses

👩‍🏫

• ChatGPT Prompt Engineering for Developers, by deeplearning.ai

Tutorials

📚

• Introduction to Prompt Engineering

  • Prompt Engineering 101 - Introduction and resources
  • Prompt Engineering 101
  • Prompt Engineering Guide by SudalaiRajkumar

• Beginner's Guide to Generative Language Models

  • A beginner-friendly guide to generative language models - LaMBDA guide
  • Generative AI with Cohere: Part 1 - Model Prompting

• Best Practices for Prompt Engineering

  • Best practices for prompt engineering with OpenAI API
  • How to write good prompts

• Complete Guide to Prompt Engineering

  • A Complete Introduction to Prompt Engineering for Large Language Models
  • Prompt Engineering Guide: How to Engineer the Perfect Prompts

• Technical Aspects of Prompt Engineering

  • 3 Principles for prompt engineering with GPT-3
  • A Generic Framework for ChatGPT Prompt Engineering
  • Methods of prompt programming

• Resources for Prompt Engineering

  • Awesome ChatGPT Prompts
  • Best 100+ Stable Diffusion Prompts
  • DALLE Prompt Book
  • OpenAI Cookbook
  • Prompt Engineering by Microsoft

Videos

🎥

• Advanced ChatGPT Prompt Engineering
• ChatGPT: 5 Prompt Engineering Secrets For Beginners
• CMU Advanced NLP 2022: Prompting
• Prompt Engineering - A new profession ?
• ChatGPT Guide: 10x Your Results with Better Prompts
• Language Models and Prompt Engineering: Systematic Survey of Prompting Methods in NLP
• Prompt Engineering 101: Autocomplete, Zero-shot, One-shot, and Few-shot prompting

Communities

🤝

• OpenAI Discord
• PromptsLab Discord
• Learn Prompting
• r/ChatGPT Discord
• MidJourney Discord

How to Contribute

We welcome contributions to this list! In fact, that's the main reason why I created it - to encourage contributions and encourage people to subscribe to changes in order to stay informed about new and exciting developments in the world of Large Language Models(LLMs) & Prompt-Engineering.

Before contributing, please take a moment to review our contribution guidelines. These guidelines will help ensure that your contributions align with our objectives and meet our standards for quality and relevance. Thank you for your interest in contributing to this project!

Image Source: docs.cohere.ai