0
\$\begingroup\$

I'm working on a project involving ECG data classification using a Random Forest model. Unfortunately, my model's performance is significantly lower than expected, and I'm struggling to understand why. Here are the details: my data organisation:

data_organisation

Each folder for classes contains csv file (13 columns) Validation Accuracy: 0.22705314009661837

Test Accuracy: 0.1746987951807229 As you can see, the performance metrics are quite poor. Here is the code I used to train and evaluate the model:

import os
import numpy as np
import pandas as pd

def load_data(folder):
    X = []
    Y = []
    classes = [d for d in os.listdir(folder) if os.path.isdir(os.path.join(folder, d))]
    
    for class_name in classes:
        class_path = os.path.join(folder, class_name)
        for file_name in os.listdir(class_path):
            if file_name.endswith('.csv'):
                file_path = os.path.join(class_path, file_name)
                try:
                    data = pd.read_csv(file_path)
                    X.append(data.values)  # Ajouter les données du fichier
                    Y.append(class_name)   # Ajouter le nom de la classe pour ce fichier
                except Exception as e:
                    print(f"Error reading {file_path}: {e}")
    
    return X, Y
New contributor
MEJRI Rawaa is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
\$\endgroup\$
2
  • \$\begingroup\$ here is the whole of script link \$\endgroup\$ Commented 2 days ago
  • \$\begingroup\$ Why not just paste all the code into this post, just like you did on Stack Overflow? In fact I believe it's a requirement here on Code Review. \$\endgroup\$
    – tdy
    Commented 2 days ago

1 Answer 1

3
\$\begingroup\$

technical communication

The first thing you should worry about, before even being concerned with the machine, is communicating technical ideas to collaborators. As written, it is unclear what you started with and what you hoped would happen during training.

cite your reference

The ECG is a mature technology, with many clinical and signal processing details to attend to. You described none of them, nor did you let some cited author do most of the heavy lifting for you. Be like Newton: stand upon the shoulders of giants, that you may see a little farther.

describe your input

Is this a voltage time-series from a Holter monitor? From a 12-lead ECG? From a smartwatch? Was the subject drugged or physically active during measurements?

Did you use a standard comparison dataset such as PTB-XL or one provided by the 2017 PhysioNet/CINC challenge?

describe signal processing

Have the waveforms been low-pass filtered to denoise? Has max voltage been normalized to a standard value? Did an FFT or similar generate synthetic signals such a boolean heart rate indicator?

If you devote some care to better technical communication, you will have more fruitful collaboration with colleagues.

output classifications

You appear to be going for several clinical conditions:

  • conduction disturbance / branch block
  • hypertrophy
  • myocardial infarction
  • normal
  • ST/T changes

Four clinical diagnoses appears to be too ambitious a goal at the moment. Consider focusing on Normal plus just a single diagnosis. Additionally, each of the listed diagnoses has several specialized conditions within it. Consider filtering e.g. HYP down to just Left Ventricular Hypertrophy. After your model is performing well and you have a good understanding of the situation, then you'll want to revisit the topic and tackle more ambitious goals.

biological underpinnings

Before you even get to a binary classifier for one specific diagnosis, it seems like you would want to augment the input waveform with at least three one-hot encoded synthetic signals:

  • P wave
  • QRS complex
  • T wave

Each corresponds to a distinct physical phenomenon occurring in the organism under study, which will be directly related to forming a diagnosis that a specific bit of anatomy is misbehaving. You could train a model without them; perhaps the model would infer the structure given enough input. But why force the model to recapitulate so much hard work that has already been done for you in the literature?

To focus on just one of your proposed diagnoses, I can't imagine how a clinician or a model could decide on STTC without having some representation of ST depression or elevation. It seems pretty fundamental, yet your preprocessing procedure makes no mention of it.

model selection

You went through a process of choosing one modeling technique from a whole zoo of available ML models. You didn't write down any details of that process, indicating why you thought one or another candidate was more or less suitable. A maintenance engineer will not be able to revisit that thought process a year from now when requirements have changed, since there is no record of it.

Any advice on how to improve the performance of my Random Forest model on this ECG dataset

It's not clear that RF is even a relevant choice. Yes, it is "impedance matched" in the sense that it accepts inputs of the form you have, and produces classifications of the desired form.

But how did you expect it to make those decisions? When you manually inspect a few of the top decision trees, what feature is it deciding on? Does it make intuitive sense to you? Would it make sense to a clinician? Asking RF to interpret a raw waveform isn't always a good match.

We can easily find the heart rate mean & SD.

I'm no heart expert, but for RF to succeed I would expect it would need carefully curated synthetic features such as counts of exceptional events per hundred heart beats, maybe weak beat, skipped beat, too long by one SD, that sort of thing. Those I can see RF using for good diagnoses.

error

There are several explanations for poor training performance, such as low quality / noisy input data. But a big one is bias induced by inappropriate choice of model. Currently your hypothesis is that there's a Generating Process in the world which resembles RF decisions and so will be well modeled by it. Now, that may be true for the default rate on auto loans, where a credit score decision tree can numerically capture aspects of the economy relevant to loan repayment. But in the present biophysical situation, I'm afraid I'm just not yet seeing it.

Recommend you review the literature, examine reported error measures, and use that to inform a new choice of model for this project.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.