Data

Tools

Indexes

Applications

Experiments

Open Source Science

fuzzgen

https://github.com/snoopycomp/fuzzgen

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (14.0%) to scientific vocabulary
Last synced: 10 months ago · JSON representation ·

Repository

Basic Info
  • Host: GitHub
  • Owner: SnoopyComp
  • License: apache-2.0
  • Language: HTML
  • Default Branch: main
  • Size: 5.96 MB
Statistics
  • Stars: 0
  • Watchers: 1
  • Forks: 0
  • Open Issues: 1
  • Releases: 0
Created almost 2 years ago · Last pushed over 1 year ago
Metadata Files
Readme Contributing License Citation

README.md

A Framework for Fuzz Target Generation and Evaluation

This framework generates fuzz targets for real-world C/C++/Java/Python projects with various Large Language Models (LLM) and benchmarks them via the OSS-Fuzz platform.

More details available in AI-Powered Fuzzing: Breaking the Bug Hunting Barrier: Alt text

Current supported models are: - Vertex AI code-bison - Vertex AI code-bison-32k - Gemini Pro - Gemini Ultra - Gemini Experimental - Gemini 1.5 - OpenAI GPT-3.5-turbo - OpenAI GPT-4 - OpenAI GPT-4o - OpenAI GPT-3.5-turbo (Azure) - OpenAI GPT-4 (Azure) - OpenAI GPT-4o (Azure)

Generated fuzz targets are evaluated with four metrics against the most up-to-date data from production environment: - Compilability - Runtime crashes - Runtime coverage - Runtime line coverage diff against existing human-written fuzz targets in OSS-Fuzz.

Here is a sample experiment result from 2024 Jan 31. The experiment included 1300+ benchmarks from 297 open-source projects.

image

Overall, this framework manages to successfully leverage LLMs to generate valid fuzz targets (which generate non-zero coverage increase) for 160 C/C++ projects. The maximum line coverage increase is 29% from the existing human-written targets.

Note that these reports are not public as they may contain undisclosed vulnerabilities.

Usage

Check our detailed usage guide for instructions on how to run this framework and generate reports based on the results.

Collaborations

Interested in research or open-source community collaborations? Please feel free to create an issue or email us: oss-fuzz-team@google.com.

Bugs Discovered

So far, we have reported 26 new bugs/vulnerabilities found by automatically generated targets built by this framework: | Project | Bug | LLM | Prompt Builder | Target oracle | | ------- | --------- | --------- | --------------- | ------- | | cJSON | OOB read | Vertex AI | Default | Far reach, low coverage | | libplist | OOB read | Vertex AI | Default | Far reach, low coverage | | hunspell | OOB read | Vertex AI | default | Far reach, low coverage | | zstd | OOB write | Vertex AI | default | Far reach, low coverage | | gdbm | Stack buffer underflow | Vertex AI | default | Far reach, low coverage | | hoextdown | Use of uninitialised memory | Vertex AI | default | Far reach, low coverage | | pjsip | OOB read | Vertex AI | Default | Low coverage with fuzz keyword + easy params far reach | | pjsip | OOB read | Vertex AI | Default | Low coverage with fuzz keyword + easy params far reach | | gpac | OOB read | Vertex AI | Default | Low coverage with fuzz keyword + easy params far reach | | gpac | OOB read/write | Vertex AI | Default | All | | gpac | OOB read | Vertex AI | Default | All | | gpac | OOB read | Vertex AI | Default | All | | sqlite3 | OOB read | Vertex AI | Default | All | | htslib | OOB read | Vertex AI | Default | All | | libical | OOB read | Vertex AI | Default | All | | croaring | OOB read | Vertex AI | Test-to-harness | All | | Undisclosed | Java RCE (pending maintainer triage) | Vertex AI | Default | Far reach, low coverage | | Undisclosed | Regexp DoS (pending maintainer triage) | Vertex AI | Default | Far reach, low coverage | | Undisclosed | Use of uninitialised memory | Vertex AI | Test-to-harness | Test identifier | | Undisclosed | OOB read | Vertex AI | Default | Low coverage with fuzz keyword + easy params far reach | | Undisclosed | Use after free | Vertex AI | Default | Low coverage with fuzz keyword + easy params far reach | | Undisclosed | OOB read | Vertex AI | Default | All | | Undisclosed | OOB read/write | Vertex AI | Default | All | | Undisclosed | OOB read | Vertex AI | Default | All | | Undisclosed | OOB read | Vertex AI | Default | All | | Undisclosed | OOB read | Vertex AI | Test-to-harness | Test identifier |

These bugs could only have been discovered with newly generated targets. They were not reachable with existing OSS-Fuzz targets.

Current top coverage improvements by project

| Project | Coverage increase % * | |----------|-------------------| | tinyxml2 | 29.84 | | inih | 29.67 |
| lodepng | 26.21 | | libarchive | 23.39 | | cmark | 21.61 | | fribidi | 18.20 |
| lighttpd | 17.56 | | libmodbus | 16.59 | | valijson | 16.21 | | libiec61850 | 13.53 | | hiredis | 13.50 | | cmake | 12.62 | | pugixml | 12.43 | | meshoptimizer | 12.23 | | libusb | 11.12 | | json | 10.84 |

* Percentage coverage is calculated using a denominator of the total lines of source code compiled during the OSS-Fuzz build process for the entire project.

Citing This Work

Please click on the 'Cite this repository' button located on the right-hand side of this GitHub page for citation details.

Owner

  • Login: SnoopyComp
  • Kind: user

Citation (CITATION.cff) 

cff-version: 1.2.0
title: 'OSS-Fuzz-Gen: Automated Fuzz Target Generation'
message: >-
  If you use this software, please cite it using the
  metadata from this file.
type: software
authors:
  - given-names: Dongge
    family-names: Liu
    email: donggeliu@google.com
    affiliation: Google LLC
    orcid: 'https://orcid.org/0000-0003-4821-7033'
  - given-names: Oliver
    family-names: Chang
    email: ochang@google.com
    affiliation: Google LLC
    orcid: 'https://orcid.org/0009-0006-3181-4551'
  - given-names: Jonathan
    family-names: metzman
    email: metzman@google.com
    affiliation: Google LLC
    orcid: 'https://orcid.org/0000-0002-7042-0444'
  - given-names: Martin
    family-names: Sablotny
    email: msablotny@nvidia.com
    affiliation: NVIDIA
    orcid: 'https://orcid.org/0000-0002-9836-8254'
  - given-names: Mihai
    family-names: Maruseac
    email: mihaimaruseac@google.com
    affiliation: Google LLC
    orcid: 'https://orcid.org/0000-0002-6225-1206'
repository-code: 'https://github.com/google/oss-fuzz-gen'
url: >-
  https://security.googleblog.com/2023/08/ai-powered-fuzzing-breaking-bug-hunting.html
abstract: >-
  OSS-Fuzz-Gen, an innovative open-source project developed
  by Google, automates fuzz target generation to enhance
  software security and reliability. Utilizing advanced
  techniques, including large language models (LLM), static
  code analysis, and runtime crash diagnosis, this project
  efficiently creates and optimizes fuzz targets. These
  efforts increase code coverage and identify
  vulnerabilities within open-source projects. We actively
  encourage and support collaborations with the research and
  open-source communities, offering our services at no cost.
keywords:
  - Fuzzing
  - Fuzz target generation
  - Large Language Models
  - Open-source
  - Code analysis
  - Software security
license: Apache-2.0
version: 'https://github.com/google/oss-fuzz-gen/tree/v1.0'
date-released: '2024-05-02'

GitHub Events

Total
Last Year

Dependencies

.github/workflows/lint.yaml actions
  • actions/checkout v4 composite
  • actions/setup-python v5 composite
.github/workflows/osv-scanner-pr.yml actions
.github/workflows/osv-scanner-scheduled-push.yml actions
.github/workflows/push-pr-to-gcloud.yml actions
  • actions/checkout v4 composite
  • google-github-actions/auth v2 composite
  • google-github-actions/setup-gcloud v2 composite
.github/workflows/push-to-gcloud.yml actions
  • actions/checkout v4 composite
  • google-github-actions/auth v2 composite
  • google-github-actions/setup-gcloud v2 composite
.github/workflows/push-weekly-image-to-gcloud.yml actions
  • actions/checkout v4 composite
  • google-github-actions/auth v2 composite
  • google-github-actions/setup-gcloud v2 composite
Dockerfile docker
  • debian 12 build
ci/Dockerfile docker
  • debian 12 build
ci/requirements.txt pypi
  • PyGithub ==1.51
prompts/template_xml/jvm_requirement.txt pypi
  • Donotimportthesameclassmultipletimes. *
  • Eachiteminthislisthastwoattributes ,<class_name>tagcontainstheclassnameand<full_class_name>
  • Forexample ,ifthefullqualifiednameofaclassis<code>abc.def.ghi<
  • Hereisalistofclassesandtheirfullyqualifiedname.Youmustimportallclassesbytheirfullyqualifiedname. *
  • Stringtest =
  • WRONG *
  • containsthefullyqualifiednameofthegivenclass. *
  • importcom.code_intelligence.jazzer.api.FuzzedDataProvider *
  • publicclass *
  • publicstaticvoidfuzzerInitialize *
  • publicstaticvoidfuzzerTearDown *
  • publicstaticvoidfuzzerTestOneInput *
  • publicstaticvoidtesting *
  • statement <code>importabc.def.ghi
requirements.in pypi
  • GitPython ==3.1.43
  • Jinja2 ==3.1.4
  • PyYAML ==6.0.1
  • anthropic ==0.31.2
  • chardet ==5.2.0
  • cxxfilt ==0.3.0
  • google-cloud-aiplatform ==1.51.0
  • google-cloud-storage ==2.9.0
  • openai ==1.16.2
  • pandas ==2.2.2
  • pylint ==3.2.5
  • pyright ==1.1.345
  • requests ==2.32.0
  • tiktoken ==0.7.0
  • yapf ==0.40.1
requirements.txt pypi
  • annotated-types ==0.7.0
  • anthropic ==0.31.2
  • anyio ==4.4.0
  • astroid ==3.2.2
  • cachetools ==5.3.3
  • certifi ==2024.7.4
  • chardet ==5.2.0
  • charset-normalizer ==3.3.2
  • cxxfilt ==0.3.0
  • dill ==0.3.8
  • distro ==1.9.0
  • docstring-parser ==0.16
  • filelock ==3.15.4
  • fsspec ==2024.6.1
  • gitdb ==4.0.11
  • gitpython ==3.1.43
  • google-api-core ==2.19.1
  • google-auth ==2.31.0
  • google-cloud-aiplatform ==1.51.0
  • google-cloud-bigquery ==3.25.0
  • google-cloud-core ==2.4.1
  • google-cloud-resource-manager ==1.12.3
  • google-cloud-storage ==2.9.0
  • google-crc32c ==1.5.0
  • google-resumable-media ==2.7.1
  • googleapis-common-protos ==1.63.2
  • grpc-google-iam-v1 ==0.13.1
  • grpcio ==1.64.1
  • grpcio-status ==1.62.2
  • h11 ==0.14.0
  • httpcore ==1.0.5
  • httpx ==0.27.0
  • huggingface-hub ==0.24.1
  • idna ==3.7
  • importlib-metadata ==8.0.0
  • isort ==5.13.2
  • jinja2 ==3.1.4
  • jiter ==0.5.0
  • markupsafe ==2.1.5
  • mccabe ==0.7.0
  • nodeenv ==1.9.1
  • numpy ==2.0.0
  • openai ==1.16.2
  • packaging ==24.1
  • pandas ==2.2.2
  • platformdirs ==4.2.2
  • proto-plus ==1.24.0
  • protobuf ==4.25.3
  • pyasn1 ==0.6.0
  • pyasn1-modules ==0.4.0
  • pydantic ==2.8.2
  • pydantic-core ==2.20.1
  • pylint ==3.2.5
  • pyright ==1.1.345
  • python-dateutil ==2.9.0.post0
  • pytz ==2024.1
  • pyyaml ==6.0.1
  • regex ==2024.5.15
  • requests ==2.32.0
  • rsa ==4.9
  • shapely ==2.0.4
  • six ==1.16.0
  • smmap ==5.0.1
  • sniffio ==1.3.1
  • tiktoken ==0.7.0
  • tokenizers ==0.19.1
  • tomli ==2.0.1
  • tomlkit ==0.12.5
  • tqdm ==4.66.4
  • typing-extensions ==4.12.2
  • tzdata ==2024.1
  • urllib3 ==2.2.2
  • yapf ==0.40.1
  • zipp ==3.19.2