https://github.com/cdcgov/reportvision

Intelligent Data Workflow Automation

https://github.com/cdcgov/reportvision

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
  • Academic publication links
  • Committers with academic emails
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (12.3%) to scientific vocabulary

Keywords from Contributors

projection archival interactive generic sequences dibbs embedded genomics observability autograding
Last synced: 11 months ago · JSON representation

Repository

Intelligent Data Workflow Automation

Basic Info
  • Host: GitHub
  • Owner: CDCgov
  • Language: TypeScript
  • Default Branch: main
  • Homepage:
  • Size: 81.9 MB
Statistics
  • Stars: 1
  • Watchers: 3
  • Forks: 4
  • Open Issues: 34
  • Releases: 55
Created over 2 years ago · Last pushed over 1 year ago
Metadata Files
Readme Contributing Code of conduct

README.md

Table of Contents

1. Overview - The Problem - The Solution - Future Considerations

2. Infrastructure Architecture

3. Notices - 3.1 Privacy Standard Notice - 3.2 Records Management Standard Notice - 3.3 Domestic Copyright Protection Notice - 3.4 Public Domain Standard Notice - 3.5 Open Source Notice - 3.6 License Standard Notice - 3.7 Github Notice - 3.8 Contributing Standard Notice

4. General Disclaimer

5. Other Related Documents

1. Overview

The Intelligent Data Workflow Automation (IDWA) ReportVision Project aims to support the Office of Public Health Data, Surveillance, and Technology (OPHDST) in enhancing the ability of state, local, territorial, and tribal public health departments to manage, search, and secure critical data. As a key division of the CDC, OPHDST plays a vital role in public health infrastructure.

Please see the UserGuide to get a technical overview of this project.

The Problem

The exchange of public health data is hindered by outdated, manual processes. Some state, local, tribal, and territorial health departments still rely on fax, email, and physical mail to receive case data, requiring staff to manually review and re-enter information from lab reports. This labor-intensive process can take up to 20 minutes per report, and electronic data extraction remains cumbersome and error-prone, particularly when handling multiple documents. As a result, low accuracy in data ingestion impedes the ability of public health departments to efficiently process and utilize critical health data.

The Solution

ReportVision is a powerful tool designed to automate the reading and extracting of data from lab reports, helping public health departments streamline their workflows. Leveraging the power of the Tesseract engine and Microsoft Azure Cloud Platform, ReportVision allows teams to create customizable, data-driven templates for automatic extraction and annotation of multiple datasets—delivering notable accuracy and speed.

The goal is simple yet powerful: to provide jurisdictions with a "starter kit" that empowers them to rapidly build their own resources, provision scalable Azure infrastructure, or seamlessly replicate similar configurations in Amazon Web Services (AWS) or Google Cloud Platform (GCP).

With ReportVision, public health departments can move from cumbersome, error-prone processes to a highly efficient, automated workflow that supports critical decision-making with fast, reliable data.

This application offers a robust framework for public health departments and personnel to efficiently extract relevant data from lab reports utilizing an advanced Optical Character Recognition (OCR) model. This OCR technology significantly enhances both the speed and accuracy of data extraction, taking your data processing capabilities to the next level.

Check out the following video which demos the ReportVision application including how the updated OCR model works in action, and witness firsthand how ReportVision enhances both the speed and accuracy of data extraction!

https://github.com/user-attachments/assets/5b707ec2-5d11-436b-a270-98000ef21e8f

Future Considerations

The current version of the application is optimized only for PDF-based lab reports. However, as demand from public health departments and personnel continues to grow, we see significant potential to expand support for additional file formats in future updates.

2. Infrastructure Architecture

The following provides a High-Level Architectural Design of the ReportVision Azure Architecture, with OCR powered by Tesseract to enhance efficiency, user experience, and throughput.

rv-azure-architecture-diagram

3 Notices

3.1 Privacy Standard Notice

This repository contains only non-sensitive, publicly available data and information. All material and community participation is covered by the Disclaimer and Code of Conduct.

For more information about CDC's privacy policy, please visit http://www.cdc.gov/other/privacy.html.

3.2 Records Management Standard Notice

This repository is not a source of government records, but is a copy to increase collaboration and collaborative potential. All government records will be published through the CDC web site.

3.3 Domestic Copyright Protection Notice

This repository is a work of the United States Government and is not subject to domestic copyright protection under 17 U.S.C. § 105. If published in the public domain within the United States, copyright and related rights worldwide will be waived through the CC0 1.0 Universal public domain dedication.

3.4 Public Domain Standard Notice

This repository constitutes a work of the United States Government and is not subject to domestic copyright protection under 17 USC § 105. This repository is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication. All contributions to this repository will be released under the CC0 dedication. By submitting a pull request you are agreeing to comply with this waiver of copyright interest.

3.5 Open Source Notice

This repository is open source and follows open practices. Contributors are expected to adhere to the organization's rules of behavior.

3.6 License Standard Notice

The code in this repository is licensed under the Apache License 2.0 (ASL v2), or any later version at your discretion.

You are free to use, redistribute, and modify the source code under the terms of the Apache License 2.0. However, this software is distributed "as is", without any warranties of any kind, either express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose, or non-infringement. In no event shall the authors or copyright holders be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the software or the use or other dealings in the software.

For full licensing details, refer to the Apache License 2.0.

Additionally, any code forked from this open-source project will retain its original license.

3.7 Github Notice

GitHub is not hosted by the CDC, but is a third party website used by CDC and its partners to share information and collaborate on software. CDC use of GitHub does not imply an endorsement of any one particular service, product, or enterprise. If you are new to GitHub, we recommend starting with this basic tutorial to familiarize yourself with version control and collaboration.

3.8 Contributing Standard Notice

While we encourage continuous development of this repository's codebase, there is currently no designated department overseeing its management. If you'd like to contribute, you have two options:

  1. Clone the repository and create a new repository in your organization's codebase with the changes you wish to implement.
    • This option allows you to manage the changes independently within your own organization's environment.
  2. Submit a pull request and contact the CDC to inquire whether a department has been assigned to manage the repository.
    • If a CDC department is designated, you can coordinate with them for further changes.
    • Note: All comments, messages, pull requests, and other submissions received through CDC including this GitHub page may be subject to applicable federal law, including but not limited to the Federal Records Act, and may be archived. Learn more at http://www.cdc.gov/other/privacy.html.
    • Also see CONTRIBUTING.md and CDC Managed Repository Guidance.

4. General Disclaimer

This repository was created for use by CDC programs to collaborate on public health related projects in support of the CDC mission.

5. Other Related Documents

Owner

  • Name: Centers for Disease Control and Prevention
  • Login: CDCgov
  • Kind: organization
  • Email: data@cdc.gov
  • Location: Atlanta, GA

CDC's collaborative software projects to protect America from health, safety, and security threats, both foreign and in the U.S.

GitHub Events

Total
  • Create event: 189
  • Issues event: 187
  • Delete event: 187
  • Member event: 2
  • Issue comment event: 113
  • Push event: 486
  • Gollum event: 1
  • Pull request review comment event: 113
  • Pull request review event: 232
  • Pull request event: 188
  • Fork event: 3
Last Year
  • Create event: 189
  • Issues event: 187
  • Delete event: 187
  • Member event: 2
  • Issue comment event: 113
  • Push event: 486
  • Gollum event: 1
  • Pull request review comment event: 113
  • Pull request review event: 232
  • Pull request event: 188
  • Fork event: 3

Committers

Last synced: over 1 year ago

All Time
  • Total Commits: 238
  • Total Committers: 13
  • Avg Commits per committer: 18.308
  • Development Distribution Score (DDS): 0.782
Past Year
  • Commits: 199
  • Committers: 10
  • Avg Commits per committer: 19.9
  • Development Distribution Score (DDS): 0.769
Top Committers
Name Email Commits
Derek A Dombek 5****k 52
knguyenrise8 1****8 44
Zedd Shmais z****r@g****m 27
Arin Kulshi 1****t 26
Andrew Schreiber s****j 22
Jonathan Chang j****n@s****l 16
Derek Dombek d****m 16
dependabot[bot] 4****] 12
Mary Crawford 2****d 8
Bryan 8****n 7
Eric Buckley e****y@g****m 4
Arindam Kulshi a****4@g****m 3
Erik Knudsen 3****6 1
Committer Domains (Top 20 + Academic)

Issues and Pull Requests

Last synced: 11 months ago

All Time
  • Total issues: 168
  • Total pull requests: 136
  • Average time to close issues: 29 days
  • Average time to close pull requests: 7 days
  • Total issue authors: 16
  • Total pull request authors: 11
  • Average comments per issue: 0.56
  • Average comments per pull request: 0.32
  • Merged pull requests: 109
  • Bot issues: 0
  • Bot pull requests: 9
Past Year
  • Issues: 91
  • Pull requests: 109
  • Average time to close issues: 22 days
  • Average time to close pull requests: 4 days
  • Issue authors: 10
  • Pull request authors: 8
  • Average comments per issue: 0.64
  • Average comments per pull request: 0.27
  • Merged pull requests: 88
  • Bot issues: 0
  • Bot pull requests: 7
Top Authors
Issue Authors
  • derekadombek (29)
  • bora-skylight (28)
  • prikiser (24)
  • arinkulshi-skylight (22)
  • zdeveloper (11)
  • knguyenrise8 (10)
  • marycrawford (9)
  • jonchang (7)
  • Miringuk (6)
  • schreiaj (3)
  • dependabot[bot] (1)
Pull Request Authors
  • derekadombek (65)
  • knguyenrise8 (55)
  • arinkulshi-skylight (27)
  • marycrawford (22)
  • schreiaj (20)
  • zdeveloper (19)
  • dependabot[bot] (13)
  • jonchang (12)
  • ericbuckley (5)
  • bora-skylight (2)
  • eileenruberto (1)
  • bryanbritten (1)
  • akaday (1)
Top Labels
Issue Labels
OCR (35) ops (22) Engineering (21) Design (17) frontend (13) bug (12) MDE (10) enhancement (10) Design chore (8) Closeout (2) ops MDE (2) documentation (1) Research (1) Product (1) dependencies (1) python (1)
Pull Request Labels
ops (37) dependencies (13) Engineering (7) python (7) documentation (6) OCR (4) javascript (4) MDE (3) Design (1)

Dependencies

.github/workflows/issues-assign-project.yml actions
  • actions/add-to-project v0.5.0 composite
.github/workflows/issues-close-project-done.yaml actions
  • dasmerlon/project-issue-state-sync v2 composite
OCR/poetry.lock pypi
  • black 24.3.0
  • click 8.1.7
  • colorama 0.4.6
  • exceptiongroup 1.2.0
  • iniconfig 2.0.0
  • mypy-extensions 1.0.0
  • numpy 1.26.4
  • opencv-python 4.9.0.80
  • packaging 24.0
  • pathspec 0.12.1
  • platformdirs 4.2.0
  • pluggy 1.4.0
  • pytest 8.1.1
  • python-dotenv 1.0.1
  • tomli 2.0.1
  • typing-extensions 4.10.0
OCR/pyproject.toml pypi
  • black ^24.3.0 develop
  • pytest ^8.1.1 develop
  • numpy ^1.26.4
  • opencv-python ^4.9.0.80
  • python ^3.10
  • python-dotenv ^1.0.1
.github/workflows/ocr-tests.yml actions
  • actions/checkout v4 composite
  • actions/setup-python v5 composite
.github/workflows/lint.yaml actions
  • actions/checkout v4 composite
  • actions/setup-python v5 composite
.github/workflows/ocr-benchmarks.yml actions
  • actions/checkout v4 composite
  • actions/setup-python v5 composite
  • benchmark-action/github-action-benchmark v1 composite
.github/workflows/projects-auto-move-new-sprint.yml actions
  • blombard/move-to-next-iteration master composite