COHORT REQUESTER / SPARK JOB SERVER

The Cohort Requester is a spark application server for querying FHIR data with a set of criteria and return a count or list of patients that match the criteria.

Quick Start

Fill in the configuration file src/main/resources/application.conf with the appropriate values. Or use the following minimal needed environment variables: ```bash

The URL of the FHIR server

export FHIR_URL=http://localhost:8080/fhir ```

Build the project with maven: bash mvn clean package

Run the application server: bash java \ --add-opens=java.base/java.lang=ALL-UNNAMED \ --add-opens=java.base/java.lang.invoke=ALL-UNNAMED \ --add-opens=java.base/java.lang.reflect=ALL-UNNAMED \ --add-opens=java.base/java.io=ALL-UNNAMED \ --add-opens=java.base/java.net=ALL-UNNAMED \ --add-opens=java.base/java.nio=ALL-UNNAMED \ --add-opens=java.base/java.util=ALL-UNNAMED \ --add-opens=java.base/java.util.concurrent=ALL-UNNAMED \ --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED \ --add-opens=java.base/sun.nio.ch=ALL-UNNAMED \ --add-opens=java.base/sun.nio.cs=ALL-UNNAMED \ --add-opens=java.base/sun.security.action=ALL-UNNAMED \ --add-opens=java.base/sun.util.calendar=ALL-UNNAMED \ --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED \ -jar target/cohort-requester.jar

Launch a query with the following command: bash curl -X POST http://localhost:8090/jobs -H "Content-Type: application/json" -d '{ "input": { "cohortDefinitionSyntax": "{\"sourcePopulation\":{\"caresiteCohortList\":[118]},\"_type\":\"request\",\"request\":{\"_type\":\"andGroup\",\"_id\":0,\"isInclusive\":true,\"criteria\":[{\"_type\":\"basicResource\",\"_id\":1,\"isInclusive\":true,\"resourceType\":\"Patient\",\"filterFhir\":\"active=true&gender=female\",\"criteria\":[],\"dateRangeList\":[],\"temporalConstraints\":[]},{\"_type\":\"basicResource\",\"_id\":2,\"isInclusive\":true,\"resourceType\":\"Observation\",\"filterFhir\":\"code=http://snomed.info/sct|224299000\",\"criteria\":[],\"occurrence\":{\"n\":1,\"operator\":\">=\"},\"dateRangeList\":[],\"temporalConstraints\":[]}],\"dateRangeList\":[],\"temporalConstraints\":[]},\"temporalConstraints\":[]}", "mode": "count" } }'

Job Queries

The job query format is as follows : json { "input": { "cohortDefinitionSyntax": "<cohort definition syntax>", "mode": "<mode>", "modeOptions": { // optional mode options // optional list of criteria ids separated by commas or "all", this will activate a detailed count of the patients per criteria // or "ratio", this will activate a detailed count of final matched patients per criteria "details": "<details>", // optional sampling ratio value between 0.0 and 1.0 to limit the number of patients of the cohort to create (it can be used to sample an existing cohort) "sampling": "<sampling>", // optional cohort id to use as a base for the "createDiff" mode "baseCohortId": "<base cohort id>" }, "callbackUrl": "<callback url>" // optional callback url to retrieve the result } }

with mode being one of the following values: - count : Return the number of patients that match the criteria of the cohortDefinitionSyntax - count_all: Return the number of patients that match the criteria of the cohortDefinitionSyntax and an estimation minimum and maximum bounds - count_with_details: Return the number of patients that match the criteria of the cohortDefinitionSyntax with a detailed count per organization - create: Create a cohort of patients that match the criteria of the cohortDefinitionSyntax - create_diff: Create a change list from a base cohort of patients (defined in modeOptions) and the new/deleted ones that match the criteria of the cohortDefinitionSyntax

and cohortDefinitionSyntax being a JSON string that represents the criteria described in the query format section.

Job Response

If a callbackUrl is provided (else the result will be printed in the logs), the response will be sent to the provided URL with the following format: typescript type RESPONSE = { _type: string; // the job mode (count or create) message: string; // the message of the response (usually the job status) groupId: string; // the cohort id in case of a cohort creation job count: number; // the count of patients in case of a count job extra: Map<string, string>; // extra information about the count job (like details count per organization or per criteria) stack: string; // the stack trace in case of an error }

Query Format

The query format of the cohortDefinitionSyntax field is as follows (described in typescript):

``typescript type REQUEST = { sourcePopulation: { caresiteCohortList?: Array<Integer> // Optional list of caresite ids that will be used to filter the resource with the fhir param_list` }, _type: "request", request: CRITERIA, resourceType?: string // The fhir resource type of the cohort (Patient being the default) }

type CRITERIA = { _type: "andGroup" | "orGroup" | "nAmongM" | "basicResource", _id: Integer, // Unique identifier of the criteria isInclusive: Boolean // If true, the criteria is inclusive, if false, the criteria is exclusive }

type GROUPCRITERIA = CRITERIA & { _type: "andGroup" | "orGroup" | "nAmongM", // the type of the group "and" or "or" or "at least n matches among subcriterias" criteria: Array, // list of subcriterias (can be basicResource or group) temporalConstraints: Array<TEMPORALCONSTRAINT>, // list of temporal constraints between the subcriterias nAmongMOptions: Array // list of occurrence constraints for the nAmongM type }

type BASICRESOURCE = CRITERIA & { _type: "basicResource", resourceType: string, // The fhir resource type to filter filterFhir: string, // The fhir query string to filter the resource filterSolr?: string, // The solr query string to filter the resource (if using solr resolver) occurrence?: OCCURENCECONSTRAINT, // The occurrence constraint of the resource patientAge?: PATIENTAGE, // The age constraint of the patient dateRangeList?: Array<DATERANGE>, // The date range constraint of the resource encounterDateRange?: DATERANGE // The date range constraint of the related encounter uniqueFields?: Array<UNIQUEFIELD> // The unique fields of the resource to count the patients }

type PATIENT_AGE = { minAge?: string, // with the format {year}-{month}-{day}, eg. for someone of 21 years old this would be 21-0-0 maxAge?: string, // with the format {year}-{month}-{day}, eg. for someone of 21 years old this would be 21-0-0 datePreference?: string[], // resource reference field of the date to calculate the age dateIsNotNull?: boolean }

type DATE_RANGE = { minDate?: string, // with the standard format {year}-{month}-{day}, eg. for 2024-01-25 maxDate?: string, // with the standard format {year}-{month}-{day}, eg. for 2024-01-25 datePreference?: string[], // resource reference field of the date to filter dateIsNotNull?: boolean }

type UNIQUE_FIELD = { name?: string, // The fhir resource field to verify, for now only codes are supported and it is not need to fill this field which is ignored operator: string, // The operator to use for the verification n: number // The number of unique values }

type TEMPORALCONSTRAINT = { idList: string | Array, constraintType: "sameEncounter" | "differentEncounter" | "directChronologicalOrdering" | "sameEpisodeOfCare", occurrenceChoiceList?: Array<{id: number, occurrenceChoice: string}>, timeRelationMinDuration?: TEMPORALCONSTRAINTDURATION, timeRelationMaxDuration?: TEMPORALCONSTRAINTDURATION, datePreferenceList?: Array<{id: number, occurrenceChoice: string}>, filteredCriteriaIdList?: string | Array, dateIsNotNullList?: boolean | {id: number, dateIsNotNull: boolean} }

type TEMPORALCONSTRAINTDURATION = { years?: number, months?: number, days?: number, hours?: number, minutes?: number, seconds?: number }

type OCCURENCE_CONSTRAINT = { n: number, operator: string, sameEncounter?: boolean, sameDay?: boolean, timeDelayMin?: boolean, timeDelayMax?: boolean } ```