Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/aida-ugent/evalne-robustness

Code for robustness evaluation using EvalNE (will be integrated in the main framework at a later stage)

https://github.com/aida-ugent/evalne-robustness

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (12.1%) to scientific vocabulary
Last synced: 9 months ago · JSON representation

Repository

Code for robustness evaluation using EvalNE (will be integrated in the main framework at a later stage)

Basic Info
  • Host: GitHub
  • Owner: aida-ugent
  • License: mit
  • Language: Python
  • Default Branch: master
  • Size: 723 KB
Statistics
  • Stars: 0
  • Watchers: 4
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created over 3 years ago · Last pushed over 3 years ago
Metadata Files
Readme

README.md

Code for reproducing the experients of "A Systematic Evaluation of Node Embedding Robustness", LoG 2022

This folder contains the EvalNE framework as well as our proposed extension towards robustness evaluation. This code together with the config files included in the experiments folder can be used to replicate the experimental results reported in our manuscript. See the Experiments section below for more details.

Installation

The library requires Python 2.7. Other package dependencies are listed in the requirements.txt file.

Before installing the library, ensure that some necessary packages are present: bash sudo apt-get install python-pip sudo apt-get install python-tk

Install the requirements and the EvalNE package: bash pip install -r requirements.txt sudo python setup.py install

Experiments

In order to replicate our experiments we provide 4 EvalNE configuration files. Below we show how to use these config files and later explain the relations between them and the experiments in our manuscript.

Running an evaluation in EvalNE with a given config file simply requires the file path to be provided as a parameter, e.g.: bash python -m evalne ./experiments/exp_nc_rand.ini

Config files are named based on the downstream task and poison attack type, thus, we have: * Random attacks on node classification experiment: exp_nc_rand.ini * Adversarial attacks on node classification experiment: exp_nc_adv.ini * Random attacks on network reconstruction experiment: exp_nr_rand.ini * Adversarial attacks on network reconstruction experiment: exp_nr_adv.ini

The evaluations must be run individually for each attack type and random seed. For instance, one should run exp_nc_rand.ini once with attack strategy addedgesrand and seed 42. Then the config file can be edited to change e.g. the attack type to deledgesrandno_ and rerun the evaluation. In each config file we have added the attack strategies and seeds used as values that can be easily commented or uncommented when running the evaluations. See the config files for more details.

Methods and Data

In order to simplify code and experiment checking we have included the data and methods used in the paper in two separate folder.

NOTE: The methods should each be each installed individually in one virtual environment (preferably named venv) to ensure correct evaluation. Follow the instructions in the respective readmes for installation.

Owner

  • Name: Ghent University Artificial Intelligence & Data Analytics Group
  • Login: aida-ugent
  • Kind: organization
  • Email: tijl.debie@ugent.be
  • Location: Ghent

GitHub Events

Total
Last Year

Issues and Pull Requests

Last synced: over 1 year ago

All Time
  • Total issues: 0
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 0
  • Total pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels