towards-efficient-complementary-security-analysis-using-large-language-models - v1.0.3 – Update CITATION.cff for standardized citation

This patch updates the CITATION.cff file to conform to CFF 1.2.0, fixes YAML syntax issues (quoting fields and removing invalid characters), and adds the type: dataset field for proper citation metadata.

- Python
Published by zumpious about 1 year ago

towards-efficient-complementary-security-analysis-using-large-language-models - Add CITATION.cff

This release adds a CITATION.cff file to provide standardized citation metadata for the repository.

- Python
Published by zumpious about 1 year ago

towards-efficient-complementary-security-analysis-using-large-language-models - Zenodo Release

This release is being created now to trigger Zenodo’s GitHub integration—since our initial tag was made before syncing the repo, Zenodo didn’t automatically import it.

- Python
Published by zumpious about 1 year ago

towards-efficient-complementary-security-analysis-using-large-language-models - v1.0.0 – Initial release of experimental results & notebooks

• Data (/data)

  • OWASP Benchmark (spotbugs_dataset.pkl)
    
  • Source: OWASP Benchmark
    
  • 2,740+ test cases across 11 vulnerability areas, analyzed with SpotBugs + FindSecBugs
    
  • Train split (~80% of findings) for preliminary studies & few-shot example generation
    
  • Test split (~20% of findings) for validation and cross-model comparison
    
  • See datasets.md for full dataset details
    

• Preliminary Studies (/data/preliminary_study)

  1. Contextual Information Analysis
     
     • Impact of SpotBugs report vs. CWE database context on LLM assessments
       
     • Conducted on the train split
       
     • README & details
       
  2. Prompting Techniques Comparison
     
     • Few-Shot, Chain-of-Thought (CoT), and Self-Consistency (SC) with GPT-3.5 Turbo
       
     • Conducted on the train split
       
     • README & details
       

• Main Research Findings (/data/towards_efficient_complementary_security_analysis)

  • Evaluation of multiple LLM families (Qwen, GPT, Phi, Llama) on:
    
  • OWASP test split (403 findings)
    
  • Real-world Mnestix dataset (114 findings)
    
  • Full write-up & data
    

• Supporting Code (/src)

  • Few-shot example scripts: fewshotexamples.py
    
  • Prompt templates: prompt_templates.py
    

Feel free to download the JSON files and open the notebooks to reproduce our analyses!

- Python
Published by zumpious about 1 year ago