mturk-templates
A collection of MTurk templates designed to make complex tasks easier for human annotators.
Science Score: 54.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (9.9%) to scientific vocabulary
Keywords
Repository
A collection of MTurk templates designed to make complex tasks easier for human annotators.
Basic Info
Statistics
- Stars: 1
- Watchers: 1
- Forks: 1
- Open Issues: 0
- Releases: 0
Topics
Metadata Files
README.md
A collection of complex Amazon Mechanical Turk templates I created for my work. These templates are designed to make complex annotations tasks easier for human annotators to evaluate by breaking down each HIT into several subtasks. I will add templates and references here as I work on them.
Compare Recipes
This MTurk Task asks Turkers to compare two recipes by displaying the entirety of one and providing the other one step at a time. The ingredients list is shown to the far left to help ground the task in background information. Turkers answer questions about each step of the second recipe, followed by questions about both recipes in general. This template could be adapted for any task that involves comparing two long texts, like summarization, style transfer, machine translation, or question answering.
Used in Large Language Models as Sous Chefs: Revising Recipes with GPT-3 (Hwang et al., 2023).
Use and Citation
Feel free to fork this repository, submit pull requests, or use these templates for your work. If you use any of these templates for your work or inspiration, please cite this repository:
@software{Hwang_MTurk_Templates_for_2023,
author = {Hwang, Alyssa},
month = jun,
title = {{MTurk Templates for Complex Human Evaluation Tasks}},
url = {https://github.com/ahwang16/mturk-templates},
version = {1.0.0},
year = {2023}
}
Owner
- Name: Alyssa Hwang
- Login: ahwang16
- Kind: user
- Company: University of Pennsylvania
- Website: alyssahwang.com
- Repositories: 2
- Profile: https://github.com/ahwang16
Citation (CITATION.cff)
cff-version: 1.0.0 message: "If you use any of these MTurk templates, please cite us like this:" authors: - family-names: "Hwang" given-names: "Alyssa" orcid: "https://orcid.org/0009-0006-4827-8505" title: "MTurk Templates for Complex Human Evaluation Tasks" version: 1.0.0 date-released: 2023-06-23 url: "https://github.com/ahwang16/mturk-templates"
GitHub Events
Total
- Watch event: 1
Last Year
- Watch event: 1
Issues and Pull Requests
Last synced: 10 months ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0