wiki-ores-feedback
ORES-Inspect is a web app for auditing machine learning models used on Wikipedia.
Science Score: 67.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
✓DOI references
Found 2 DOI reference(s) in README -
✓Academic publication links
Links to: arxiv.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (9.8%) to scientific vocabulary
Keywords
Repository
ORES-Inspect is a web app for auditing machine learning models used on Wikipedia.
Basic Info
- Host: GitHub
- Owner: levon003
- License: mit
- Language: Jupyter Notebook
- Default Branch: master
- Homepage: https://meta.wikimedia.org/wiki/Research:ORES_Inspect:_A_technology_probe_for_machine_learning_audits_on_enwiki
- Size: 10.8 MB
Statistics
- Stars: 1
- Watchers: 4
- Forks: 0
- Open Issues: 8
- Releases: 2
Topics
Metadata Files
README.md
ORES-Inspect
Try ORES-Inspect on Toolforge: https://ores-inspect.toolforge.org/
Read our summary paper, presented at Wiki Workshop 2024: https://arxiv.org/abs/2406.08453
Repository containing code for auditing the ORES edit quality models, including the ORES-Inspect tool.
Read more about this project and leave feedback at Research:ORES Inspect: A technology probe for machine learning audits on enwiki.
Repository contents:
audit_web_client- All code for the front-end and back-end of ORES-Inspect. See the README in that directory for development details, including how to run ORES-Inspect locally and how to contribute.notebook- Jupyter Notebooks analyzing static Wikipedia data dumps.scripts- Scripts for downloading dumps from dumps.wikimedia.orgscripts-*-SHW- Code for predicting reverts.sql- SQL for accessing the ToolsDB OIDB. Should be moved toaudit_web_client.src- Python code for processing downloaded dumps.figures- Matplotlib or other figures generated by analysis code.
Timeline
An initial version of this project started in January 2020. A revised version of this project, and development of ORES-Inspect, started in January 2021. ORES-Inspect was presented at the Wiki Workshop in June 2024.
Citation
To cite this work, cite our Wiki Workshop 2024 paper:
Zachary Levonian, Lauren Hagen, Lu Li, Jada Lilleboe, Solvejg Wastvedt, Aaron Halfaker, Loren Terveen. 2023. ORES-Inspect: A technology probe for machine learning audits on enwiki. In Wiki Workshop 2024, Online. DOI:https://doi.org/10.48550/arXiv.2406.08453
Owner
- Name: Zachary Levonian
- Login: levon003
- Kind: user
- Location: New York, NY
- Website: http://www-users.cs.umn.edu/~levon003/
- Twitter: zwlevonian
- Repositories: 6
- Profile: https://github.com/levon003
ML Engineer @ ConcertAI. PhD from @grouplens. Previously: Amazon, CaringBridge, MITRE
Citation (CITATION.cff)
cff-version: 1.2.0
message: "If you use this software, please cite the paper as below."
authors:
- family-names: Levonian
given-names: Zachary
orcid: https://orcid.org/0000-0002-8932-1489
- family-names: Hagen
given-names: Lauren
- family-names: Li
given-names: Lu
- family-names: Lilleboe
given-names: Jada
- family-names: Wastvedt
given-names: Solvejg
- family-names: Halfaker
given-names: Aaron
- family-names: Terveen
given-names: Loren
date-released: 2024-06-12
repository-code: "https://github.com/levon003/wiki-ores-feedback"
preferred-citation:
type: conference-paper
title: "ORES-Inspect: A technology probe for machine learning audits on enwiki"
abstract: "Auditing the machine learning (ML) models used on Wikipedia is important for ensuring that vandalism-detection processes remain fair and effective. However, conducting audits is challenging because stakeholders have diverse priorities and assembling evidence for a model's [in]efficacy is technically complex. We designed an interface to enable editors to learn about and audit the performance of the ORES edit quality model. ORES-Inspect is an open-source web tool and a provocative technology probe for researching how editors think about auditing the many ML models used on Wikipedia. We describe the design of ORES-Inspect and our plans for further research with this system."
doi: 10.48550/arXiv.2406.08453
year: 2024
conference:
name: "Wiki Workshop 2024"
date-start: "2024-06-20"
date-end: "2024-06-20"
authors:
- family-names: Levonian
given-names: Zachary
orcid: https://orcid.org/0000-0002-8932-1489
- family-names: Hagen
given-names: Lauren
- family-names: Li
given-names: Lu
- family-names: Lilleboe
given-names: Jada
- family-names: Wastvedt
given-names: Solvejg
- family-names: Halfaker
given-names: Aaron
- family-names: Terveen
given-names: Loren
GitHub Events
Total
Last Year
Dependencies
- 1764 dependencies
- eslint ^6.8.0 development
- eslint-config-airbnb ^18.2.1 development
- eslint-config-prettier ^6.15.0 development
- eslint-plugin-import ^2.22.1 development
- eslint-plugin-jsx-a11y ^6.4.1 development
- eslint-plugin-prettier ^3.3.1 development
- eslint-plugin-react ^7.22.0 development
- eslint-plugin-react-hooks ^2.5.1 development
- prettier ^1.19.1 development
- @material-ui/core ^4.11.3
- @material-ui/icons ^4.11.2
- @material-ui/lab ^4.0.0-alpha.57
- @material-ui/styles ^4.11.3
- @nivo/bar ^0.67.0
- @nivo/core ^0.67.0
- @nivo/line ^0.67.0
- @nivo/sankey ^0.67.0
- autosuggest-highlight ^3.1.1
- chart.js ^2.9.4
- clsx ^1.1.1
- cors ^2.8.5
- d3 ^6.5.0
- d3-sankey ^0.12.3
- fontsource-roboto ^4.0.0
- formik ^2.2.6
- history ^5.0.0
- http-proxy-middleware ^0.19.1
- lodash ^4.17.21
- moment ^2.29.1
- nprogress ^0.2.0
- prop-types ^15.7.2
- react ^16.14.0
- react-chartjs-2 ^2.11.1
- react-dom ^16.14.0
- react-feather ^2.0.9
- react-helmet ^6.1.0
- react-loading-icons ^1.0.6
- react-perfect-scrollbar ^1.5.8
- react-router ^6.0.0-beta.0
- react-router-dom ^6.0.0-beta.0
- react-scripts ^3.4.4
- react-svg-tooltip 0.0.11
- uuid ^8.3.2
- yup ^0.29.3
- Werkzeug *
- click *
- flask *
- mwoauth *
- mysqlclient *
- numpy *
- pyopenssl *
- pytz *
- sqlalchemy ==1.4.
- tqdm *