heuristics-for-test-tool-design
A set of heuristics for use when designing or choosing a test tool, plus explanations. Based on research by Isabel Evans, University of Malta.
https://github.com/hci-lab-um/heuristics-for-test-tool-design
Science Score: 44.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (10.8%) to scientific vocabulary
Repository
A set of heuristics for use when designing or choosing a test tool, plus explanations. Based on research by Isabel Evans, University of Malta.
Basic Info
Statistics
- Stars: 13
- Watchers: 2
- Forks: 0
- Open Issues: 8
- Releases: 0
Metadata Files
README.md
idea-t: A Framework of Heuristics for Test Tool Design v1.7
Developed by: Isabel Evans - University of Malta
the idea-t framework contains 12 heuristic questions and supporting information which are intended to provoke thought and ideas when designing or choosing a test tool. Elsewhere in this repository are explanations and action lists for each heuristic, plus background to the research.
A quick-start / recap page and a downloadable meeting agenda using the heuristics as prompts are available, as well as a 2-page pdf with a brief summary, and any of these these can be used to start a conversation before deep diving into this repository. Review all the heuristics, and then decide where you need to deep-dive. You are unlikely to use the heuristics in a linear 1-12 way; you are more likely to iterate between the heuristics, finding out more about "Why?" as you explore "Who?" and "Context?"
A summary poster can be downloaded. Here is a thumbnail of the poster and a flowchart of the heuristics, clickable and accessible links are after the two illustrations.
(poster.jpg)
Click for example iterative flow through the heuristics
![flowchart iterating between the H01 Why?, H02-H06 Who? and H07-H12 Context? questions with feedback loops between each group][iterativeflow](interconnecting-heuristics.jpg)The heuristics have been developed iteratively based on research data collected 2018 to 2024, and are being evaluated in industry case studies and expert reviews. They may/will continue to change based on those case studies and reviews.
Heuristic questions
The Heuristics and links to explanations, mini usage cases and activities to help you answer the questions:
H01. [Why do we need this tool?](H01-Why-do-we-need-this-tool.md) H02. [Who will use or be affected by the tool?](H02-Who-will-use-or-be-affected-by-this-tool.md) H03. [What previous experiences do people bring to the tool?](H03-What-previous-experiences-do-people-bring-to-the-tool.md) H04. [What communication needs or preferences do people have?](H04-What-communication-needs-or-preferences-do-those-people-have.md) H05. [Is the learning goal "tool mastery" or "task completion"?](H05-personal-learning-goal-mastery-or-task-based.md) H06. [What learning preferences do people have?](H06-What-learning-preferences-do-those-people-have.md) H07. [Where will the tool be used?](H07-Where-will-the-tool-be-used.md) H08. [What workflows will the tool be part of?](H08-What-workflows-will-the-tool-be-part-of.md) H09. [What risks are associated with those workflows?](H09-What-risks-are-associated-with-those-workflows.md) H10. [What autonomy of work styles is allowed in those workflows and teams?](H10-What-autonomy-of-work-styles.md) H11. [When will the tool be used?](H11-When-will-the-tool-be-used.md) H12. [How long will the tool be used?](H12-How-long-will-the-tool-be-used.md)Click for more information about how to use the heuristics
These questions deliberately have multiple interpretations and a myriad of answers. They are intended to help you to think widely and come up with ideas to consider, who to involve, questions to ask. There are no right answers – by asking the questions and following the corresponding activities, you will find interpretations and answers for your context. **You can start to use the heuristics as an agenda for a planning meeting, or as a checklist to aid fast insights, in a tight timebox, to help you decide which heuristics you want to use for a deep dive.** Each question is linked to an explanation, with things to think about, research points and evidence from industry contributors, key questions to ask yourself, plus pointers to quality attributes to consider in your specific context. The information we provide tells you *why* each heuristic is important to consider. To help you think about *how* to answer the questions, we have suggested activities, with links to external sources in case you need more information. [More information on how to use the heuristics](How-to-use-the-heuristics.md).Click for Flowchart showing how to find your way around this repository
![flowchart shows that from README you can jump to each heuristic's desciption with explanations and activities, and to information about the researchers, the licence, and research daa on quality attributes, then from those return to the README][flow1](h-flow1.jpg)Click for information about the Research
The heuristic questions and explanations are grounded in research data collected from surveys, workshops, interviews, and industry and academic literature in the period 2018-2023. They have been developed iteratively through a series of reviews with UX, accessibility, and testing industry experts and practitioners. This process is ongoing. [Find out more about the research and the researcher](About-the-research-and-researcher.md). [See a list of links to preprints of research papers leading to the heuristics](preprints.md) NOTE: both the heuristics and the underlying explanations and activities are still draft. They are being developed in an iterative process which is looking at the content and the format of the materials. Example activities and links to further information are indicative, and to be replaced later. Format and medium will change. American spellings have been adopted. During evaluation of the heuristics, the repository will be frozen at the start of each case study/evaluation, and then changes made based on data from the evaluations, and from previous research to enrich the contents. Starting from a simple basis, the repository will grow over time. August 2025: following the final case studies and expert reviews, suggestions and edits have been made to this V1.x repository, bringing it to v1.7. Work will start shortly on v2.0 in a separate branch.Versions
| Version | Description | Tag | Link |
| ---------------- | --------------------------------------------- | ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- |
| v2 (coming soon) | Major update with new structure and filenames | v2 (not yet live) |
| v1.7 | Original published heuristics | v1.7 | View Files |
Citation
If you use these heuristics please cite our work:
Evans, I., Porter, C., & Micallef, M. (2024). Heuristics-for-test-tool-design (Version 1.7.0) [Computer software]. https://github.com/hci-lab-um/heuristics-for-test-tool-design
Note: use the "Cite this repository" function for more options (e.g. bibtex).
Licence
This material is GPL-3.0 licenced.
Owner
- Name: hci-lab-um
- Login: hci-lab-um
- Kind: organization
- Repositories: 6
- Profile: https://github.com/hci-lab-um
Citation (CITATION.cff)
cff-version: 1.2.0 message: "If you use this software, please cite it as below." authors: - family-names: "Evans" given-names: "Isabel" orcid: "https://orcid.org/0000-0003-3913-4463" - family-names: "Porter" given-names: "Chris" orcid: "https://orcid.org/0000-0003-3813-2941" - family-names: "Micallef" given-names: "Mark" orcid: "https://orcid.org/0000-0003-4159-6923" title: "Heuristics-for-test-tool-design" version: 1.5.0 date-released: 2024-11-26 url: "https://github.com/hci-lab-um/heuristics-for-test-tool-design"
GitHub Events
Total
- Issues event: 8
- Watch event: 13
- Issue comment event: 7
- Push event: 291
- Create event: 1
Last Year
- Issues event: 8
- Watch event: 13
- Issue comment event: 7
- Push event: 291
- Create event: 1
Issues and Pull Requests
Last synced: 7 months ago
All Time
- Total issues: 2
- Total pull requests: 0
- Average time to close issues: 6 days
- Average time to close pull requests: N/A
- Total issue authors: 1
- Total pull request authors: 0
- Average comments per issue: 1.0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 2
- Pull requests: 0
- Average time to close issues: 6 days
- Average time to close pull requests: N/A
- Issue authors: 1
- Pull request authors: 0
- Average comments per issue: 1.0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- IsabelEvans (6)