Alireza Amirshahi
Publications
Infoscience publications
Publications
2024
* M2SKD: Multi-to-Single Knowledge Distillation of Real-Time Epileptic Seizure Detection for Low-Power Wearable Systems
Integrating low-power wearable systems into routine health monitoring is an ongoing challenge. Recent advances in the computation capabilities of wearables make it possible to target complex scenarios by exploiting multiple biosignals and using high-performance algorithms, such as Deep Neural Networks (DNNs). However, there is a trade-off between the algorithms' performance and the low-power requirements of platforms with limited resources. Besides, physically larger and multi-biosignal-based wearables bring significant discomfort to the patients. Consequently, reducing power consumption and discomfort is necessary for patients to use wearable devices continuously during everyday life. To overcome these challenges, in the context of epileptic seizure detection, we propose the M2SKD (Multi-to-Single Knowledge Distillation) approach targeting single-biosignal processing in wearable systems. The starting point is to train a highly-accurate multi-biosignal DNN, then apply M2SKD to develop a single-biosignal DNN solution for wearable systems that achieves an accuracy comparable to the original multi-biosignal DNN. To assess the practicality of our approach to real-life scenarios, we perform a comprehensive simulation experiment analysis on several edge computing platforms.
ACM Transactions on Intelligent Systems and Technology. 2024-05-29.* FETCH: A Fast and Efficient Technique for Channel Selection in EEG Wearable Systems
The rapid development of wearable biomedical systems now enables real-time monitoring of electroencephalography (EEG) signals. Acquisition of these signals relies on electrodes. These systems must meet the design challenge of selecting an optimal set of electrodes that balances performance and usability constraints. The search for the optimal subset of electrodes from a larger set is a problem with combinatorial complexity. While existing research has primarily focused on search strategies that only explore limited combinations, our methodology proposes a computationally efficient way to explore all combinations. To avoid the computational burden associated with training the model for each combination, we leverage an innovative approach inspired by few-shot learning. Remarkably, this strategy covers all the wearable electrode combinations while significantly reducing training time compared to retraining the network on each possible combination. In the context of an epileptic seizure detection task, the proposed method achieves an AUC value of 0.917 with configurations using eight electrodes. This performance matches that of prior research but is achieved in significantly less time, transforming a process that would span months into a matter of hours on a single GPU device. Our work allows comprehensive exploration of electrode configurations in wearable biomedical device design, yielding insights that enhance performance and real-world feasibility.
2024-04-04. Conference on Health, Inference, and Learning, NewYork, US, June 27-28, 2024.* Accelerator-driven Data Arrangement to Minimize Transformers Run-time on Multi-core Architectures
The increasing complexity of transformer models in artificial intelligence expands their computational costs, memory usage, and energy consumption. Hardware acceleration tackles the ensuing challenges by designing processors and accelerators tailored for transformer models, supporting their computation hotspots with high efficiency. However, memory bandwidth can hinder improvements in hardware accelerators. Against this backdrop, in this paper we propose a novel memory arrangement strategy, governed by the hardware accelerator's kernel size, which effectively minimizes off-chip data access. This arrangement is particularly beneficial for end-to-end transformer model inference, where most of the computation is based on general matrix multiplication (GEMM) operations. Additionally, we address the overhead of non-GEMM operations in transformer models within the scope of this memory data arrangement. Our study explores the implementation and effectiveness of the proposed accelerator-driven data arrangement approach in both single- and multi-core systems. Our evaluation demonstrates that our approach can achieve up to a 2.7x speed increase when executing inferences employing state-of-the-art transformers.
2024-01-18. 15th Workshop on Parallel Programming and Run-Time Management Techniques for Many-Core Architectures and 13th Workshop on Design Tools and Architectures for Multicore Embedded Computing Platforms (PARMA-DITAM 2024), Munich, Germany, January 18, 2024. DOI : 10.4230/OASIcs.PARMA-DITAM.2024.3.* SzCORE: A Seizure Community Open-source Research Evaluation framework for the validation of EEG-based automated seizure detection algorithms
The need for high-quality automated seizure detection algorithms based on electroencephalography (EEG) becomes ever more pressing with the increasing use of ambulatory and long-term EEG monitoring. Heterogeneity in validation methods of these algorithms influences the reported results and makes comprehensive evaluation and comparison challenging. This heterogeneity concerns in particular the choice of datasets, evaluation methodologies, and performance metrics. In this paper, we propose a unified framework designed to establish standardization in the validation of EEG-based seizure detection algorithms. Based on existing guidelines and recommendations, the framework introduces a set of recommendations and standards related to datasets, file formats, EEG data input content, seizure annotation input and output, cross-validation strategies, and performance metrics. We also propose the 10-20 seizure detection benchmark, a machine-learning benchmark based on public datasets converted to a standardized format. This benchmark defines the machine-learning task as well as reporting metrics. We illustrate the use of the benchmark by evaluating a set of existing seizure detection algorithms. The SzCORE (Seizure Community Open-source Research Evaluation) framework and benchmark are made publicly available along with an open-source software library to facilitate research use, while enabling rigorous evaluation of the clinical significance of the algorithms, fostering a collective effort to more optimally detect seizures to improve the lives of people with epilepsy.
2024. DOI : 10.48550/arxiv.2402.13005.2023
* Layer-Wise Learning Framework for Efficient DNN Deployment in Biomedical Wearable Systems
The development of low-power wearable systems requires specialized techniques to accommodate their unique requirements and constraints. While significant advancements have been made in the inference phase of artificial intelligence, the training phase remains a challenge, particularly for biomedical wearable systems. Traditional training algorithms might not be suitable for these applications due to the substantial memory requirements and high computational costs associated with processing the large number of bits involved in neural network operations. In this paper, we introduce a novel learning procedure specifically designed for low-power wearable systems, dubbed Bio-BPfree (deep neural network training without backpropagation for low-power wearable systems). Using a two-class classification task, Bio-BPfree replaces conventional forward and backward backpropagation passes with four forward passes, two for data of the positive class and two for data of the negative class. Each layer is equipped with a unique objective function aimed at minimizing the distance between data points within the same class while maximizing the distance between data points from different classes. Our experimental results, which were obtained by conducting rigorous evaluations on the MIT-BIH dataset that features electrocardiogram (ECG) signals, effectively demonstrate the superior performance and suitability of Bio-BPfree for two-class classification tasks, particularly within the challenging environment of low-power wearable systems designed for continuous health monitoring and assessment.
2023-10-09. IEEE-EMBS International Conference on Body Sensor Networks: Sensors and Systems for Digital Health (IEEE BSN) 2023, Cambridge, MA, US, October 9-11, 2023. DOI : 10.1109/BSN58485.2023.10331334.* TiC-SAT: Tightly-coupled Systolic Accelerator for Transformers
Transformer models have achieved impressive results in various AI scenarios, ranging from vision to natural language processing. However, their computational complexity and their vast number of parameters hinder their implementations on resource-constrained platforms. Furthermore, while loosely-coupled hardware accelerators have been proposed in the literature, data transfer costs limit their speed-up potential. We address this challenge along two axes. First, we introduce tightly-coupled, small-scale systolic arrays (TiC-SATs), governed by dedicated ISA extensions, as dedicated functional units to speed up execution. Then, thanks to the tightly-coupled architecture, we employ software optimizations to maximize data reuse, thus lowering miss rates across cache hierarchies. Full system simulations across various BERT and VisionTransformer models are employed to validate our strategy, resulting in substantial application-wide speed-ups (e.g., up to 89.5X for BERT-large). TiC-SAT is available as an open-source framework.
2023-01-16. ASP-DAC 2023, Tokyo, Japan, January 16-19, 2023. DOI : 10.1145/3566097.3567867.* Predicting Survey Response with Quotation-based Modeling: A Case Study on Favorability towards the United States
The acquisition of survey responses is a crucial component in conducting research aimed at comprehending public opinion. However, survey data collection can be arduous, time-consuming, and expensive, with no assurance of an adequate response rate. In this paper, we propose a pioneering approach for predicting survey responses by examining quotations using machine learning. Our investigation focuses on evaluating the degree of favorability towards the United States, a topic of interest to many organizations and governments. We leverage a vast corpus of quotations from individuals across different nationalities and time periods to extract their level of favorability. We employ a combination of natural language processing techniques and machine learning algorithms to construct a predictive model for survey responses. We investigate two scenarios: first, when no surveys have been conducted in a country, and second when surveys have been conducted but in specific years and do not cover all the years. Our experimental results demonstrate that our proposed approach can predict survey responses with high accuracy. Furthermore, we provide an exhaustive analysis of the crucial features that contributed to the model's performance. This study has the potential to impact survey research in the field of data science by substantially decreasing the cost and time required to conduct surveys while simultaneously providing accurate predictions of public opinion.
2023-01-01. 10th IEEE Swiss Conference on Data Science (SDS), Zurich, SWITZERLAND, Jun 22-23, 2023. p. 1-8. DOI : 10.1109/SDS57534.2023.00008.2022
* M2D2: Maximum-Mean-Discrepancy Decoder for Temporal Localization of Epileptic Brain Activities
Recent years have seen growing interest in leveraging deep learning models for monitoring epilepsy patients based on electroencephalographic (EEG) signals. However, these approaches often exhibit poor generalization when applied outside of the setting in which training data was collected. Furthermore, manual labeling of EEG signals is a time-consuming process requiring expert analysis, making fine-tuning patient-specific models to new settings a costly proposition. In this work, we propose the Maximum-Mean-Discrepancy Decoder (M2D2) for automatic temporal localization and labeling of seizures in long EEG recordings to assist medical experts. We show that M2D2 achieves 76.0% and 70.4% of F1-score for temporal localization when evaluated on EEG data gathered in a different clinical setting than the training data. The results demonstrate that M2D2 yields substantially higher generalization performance than other state-of-the-art deep learning-based approaches.
IEEE Journal of Biomedical and Health Informatics (JBHI). 2022-09-22. DOI : 10.1109/jbhi.2022.3208780.2021
* EpilepsyGAN: Synthetic Epileptic Brain Activities with Privacy Preservation
Epilepsy is a chronic neurological disorder affecting more than 65 million people worldwide and manifested by recurrent unprovoked seizures. The unpredictability of seizures not only degrades the quality of life of the patients, but it can also be life-threatening. Modern systems monitoring electroencephalography (EEG) signals are being currently developed with the view to detect epileptic seizures in order to alert caregivers and reduce the impact of seizures on patients' quality of life. Such seizure detection systems employ state-of-the-art machine learning algorithms that require a large amount of labeled personal data for training. However, acquiring EEG signals during epileptic seizures is a costly and time-consuming process for medical experts and patients. Furthermore, this data often contains sensitive personal information, presenting privacy concerns. In this work, we generate synthetic seizure-like brain electrical activities, i.e., EEG signals, that can be used to train seizure detection algorithms, alleviating the need for sensitive recorded data. Our experiments show that the synthetic seizure data generated with our GAN model succeeds at preserving the privacy of the patients without producing any degradation in performance during seizure monitoring.
IEEE Transactions on Biomedical Engineering. 2021. DOI : 10.1109/TBME.2020.3042574.