Multi-Modal ML Framework: Supporting Automated Seizure Detection

Seizure detection in epilepsy research faces significant challenges due to class imbalances, where non-seizure periods vastly outnumber seizure events, leading to high false-positive rates with traditional methods. To address this, CAAI developed a multi-modal machine learning framework that integrates data from multiple sources to enhance detection accuracy:

• Electrocorticography (ECoG) readings
• Piezoelectric motion sensor data
• Video recordings

Leveraging diverse data sources, it reduces false positives and enhances the reliability of seizure detection. Additionally, the system enables automated, continuous monitoring of laboratory animals, improving the efficiency of epilepsy studies and supporting more robust research outcomes. This benefits researchers by mitigating the need to manually watch weeks’ worth of footage and identify when seizures are occurring to get an accurate count.

After postprocessing and combination techniques, classification accuracy is improved with this multi-modal system as compared to the individual data sources.

Our system utilizes the ensemble technique to train a unique model on each data source and join the predictions through postprocessing techniques best suited to this particular use case. However, the system can be generalized and modified for different environments and data types. This system uses multiple data sources:

• Electrocorticography (ECoG) readings 
• Piezoelectric motion sensor data 
• Video recordings

Each of these three data sources is used to train an individual model, where given time frames are classified as either containing a seizure event or not. All three models generate predictions on the same time span, and these predictions are aligned by timestamps to join the results. Finally, postprocessing techniques are used to weigh each model’s predictions appropriately, reduce noise, and generate a summary of results containing the beginning timestamp of each predicted seizure and the duration of that seizure.

Due to the rarity of seizure events in the data, false positives are a common problem. Combining multiple data sources helps to alleviate this, allowing for customized weighing of data sources to determine the optimal combination of predictions that catches the most seizures while reducing the number of false positives. For example, many false positives can be filtered out by limiting the final seizure classifications to only time frames in which both models agree that a seizure took place. However, stricter thresholds on seizure classifications can lead to increased false negatives, so a balance is important to maximize both precision and recall.

In practice, the models trained on each of these data sources typically have high recall, meaning they correctly identify when a seizure takes place. However, they also have very low precision, meaning they produce a large number of false positives. Our research has found that when combining the results from these models and performing postprocessing techniques, the proportion of false positives can be significantly reduced while maintaining high recall.

All performance metrics can be found in the arXiv preprint. This includes individual ECoG results, individual Piezo results, individual video results, combined results for all three, false positives, and inference times.

This project has been completed.

Mullen, A., Armstrong, S. E., Perdeh, J., Bauer, B., Talbert, J., & Bumgardner, V. K. (2024). Multi-Modal Machine Learning Framework for Automated Seizure Detection in Laboratory Rats. arXiv preprint arXiv:2402.00965.

This research was supported in part by the University of Kentucky College of Pharmacy Team Science Pilot Award, as well as the National Institutes of Health under award numbers NS079507, NS131903, AG075583, 5R21NS131903-02, 5R01NS079507-10, and 5R01NS079507-09. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the University of Kentucky College of Pharmacy.

Aaron Mullen (Lead Developer), Sam Armstrong, and Dr. Cody Bumgardner worked on Multi-Modal ML Framework: Supporting Automated Seizure Detection.