MEIT: Multi-Modal Electrocardiogram Instruction Tuning on Large Language Models for Report Generation (ACL 2025 Findings🔥)

Easy steps for efficient implementations

Step 1: download data and preprocess data

• 1.download data from google drive to your linux device: Google drive link: #############.

• 2.proprocess data: get into the 'config.yaml' file to set up the link of downloaded data:

``` if mimic: dataset: datasetname: 'mimic' ## this is for mimic dataset 21k ecgpath: 'xxxx' # add your image file path here textpath: 'xxxtrain.csv'

if ptbxl: dataset: datasetname: 'ptbxl' ## this is for PTB-XL dataset 21k ecgpath: '/fs/scratch/PAS2473/ptb-xl-a-large-publicly-available-electrocardiography-dataset-1.0.3/' # add your image file path here textpath: '/users/PAS2473/brucewan666/ECG/ECG/instructdata/RGenptbxl_train.csv' ```

• 3.run preprocess data:

``` get into preprocess_ecg.py set the path of yours (an example of ptbxl):

buildinstructdataset(ecgname='ptbxl',savepath='/users/PAS2473/brucewan666/ECG/ECG/instructdata/ptbxlecg_train.jsonl') # mimic

```

• 4.set up environment：

pip install -r requirements.txt

• 5.run ecginstructiontuning data and inference with only one ECG_instruction, give an example of mimic-ecg data:

``` export CUDAVISIBLEDEVICES=0

MODELSIZE=7B NUMGPUS=1 BATCHSIZEPERGPU=16 TOTALBATCHSIZE=64 # 144 50277 GRADIENTACCSTEPS=$(($TOTALBATCHSIZE/$NUMGPUS/$BATCHSIZEPERGPU)) echo "Training llama model ${MODELSIZE} using $NUMGPUS GPUs, $BATCHSIZEPERGPU batch size per GPU, $GRADIENTACCSTEPS gradient accumulation steps" # --usedeepspeed \ # --deepspeedconfigfile /home/wan.512/ECGLLMs/open-instruct/dsconfigs/stage3nooffloadingaccelerate.conf \

Lora training

accelerate launch --mainprocessport 31225 \ --mixedprecision bf16 \ --nummachines 1 \ --numprocesses $NUMGPUS \ /users/PAS2473/brucewan666/ECG/ECG/finetuneecgllmwithloramimic.py \ --modelnameorpath meta-llama/Llama-2-7b-hf \ --uselora \ --lorarank 64 \ --loraalpha 128 \ --loradropout 0.1 \ --tokenizername meta-llama/Llama-2-7b-hf \ --useslowtokenizer \ --trainfile /users/PAS2473/brucewan666/ECG/ECG/instructdata/mimicecg.jsonl \ --maxseqlength 128 \ --preprocessingnumworkers 16 \ --checkpointingsteps epoch \ --perdevicetrainbatchsize $BATCHSIZEPERGPU \ --gradientaccumulationsteps $GRADIENTACCSTEPS \ --learningrate 2e-5 \ --lrschedulertype linear \ --warmupratio 0.03 \ --weightdecay 0. \ --numtrainepochs 5 \ --outputdir /fs/scratch/PAS2473/zhongweisaveckpt/gpt2largelorackpt \ --withtracking \ --reportto tensorboard \ --useecgllm \ --devratio 0.1 \ --valtestratio 0.1 \ --loggingsteps 100 \ --evalstep 3200 \ --teststep 4000 \ --llmtype llama2 \ --cachedir /fs/scratch/PAS2473/zhongweimodels

```