Oversampling Approach Using Radius-SMOTE for Imbalance Electroencephalography Datasets

Several studies related to emotion recognition based on Electroencephalogram signals have been carried out in feature extraction, feature representation, and classification. However, emotion recognition is strongly influenced by the distribution or balance of Electroencephalogram data. On the other hand, the limited data obtained significantly affects the imbalance condition of the resulting Electroencephalogram signal data. It has an impact on the low accuracy of emotion recognition. Therefore, based on these problems, the contribution of this research is to propose the Radius SMOTE method to overcome the imbalance of the DEAP dataset in the emotion recognition process. In addition to the EEG data oversampling process, there are several vital processes in emotion recognition based on EEG signals, including the feature extraction process and the emotion classification process. This study uses the Differential Entropy (DE) method in the EEG feature extraction process. The classification process in this study compares two classification methods, namely the Decision Tree method and the Convolutional Neural Network method. Based on the classification process using the Decision Tree method, the application of oversampling with the Radius SMOTE method resulted in the accuracy of recognizing arousal and valence emotions of 78.78% and 75.14%, respectively. Meanwhile, the Convolutional Neural Network method can accurately identify the arousal and valence emotions of 82.10% and 78.99%, respectively.


2-Literature Review
The oversampling process is used to create new synthetic data for the minority class due to its ability to improve classification accuracy than the under-sampling process [34]. There are two oversampling strategies, namely random and synthetic. Random oversampling is a non-heuristic method used to add data to a small portion of minor classes [33,35,36]. Ding et al. (2021) [36] conducted an oversampling study using the Random Oversampling method for the DEAP dataset. This method increased the accuracy of the recognition of arousal and valence emotions. However, it tends to experience overfitting problems [33]; hence it is imperative to generate new synthetic data from minority classes based on neighboring locations. Making synthetic data can use the Synthetic Minority Oversampling Technique (SMOTE) to overcome the overfitting problem found in the Random Oversampling method.
Several studies have been carried out using the SMOTE method to overcome data imbalance. For instance, the study by Sanguanmak and Hanskunatai (2016) [37] on using the SMOTE method for oversampling minor class data by combining oversampling and under-sampling techniques. Morales et al. (2013) [38] studied the use of Synthetic Oversampling of Instance Clustering (SOI-C) and Synthetic Oversampling of Instance Jittering (SOI-J). This approach compares the minor class data in each cluster during the synthetic data creation process with the MWMOTE method used to select sample data developed by Barua et al. (2014) [39]. In this study, each minor class data was given a weighted value based on the number of k Nearest Neighbor of the majority class data. The grouping process was carried out the Safe-Level-SMOTE method proposed by Bunkhumpornpat et al. (2009) [40] to avoid overlapping synthetic data in the minority class. This strategy was used to modify the SMOTE method by adding an initial selection process before creating new synthetic data.
However, several inconsistencies are associated with the SMOTE method, such as overlapping, small disjunct, and noise. Overlap is a condition in which some minority and majority class data distributions have the same area. Small disjunct is a condition where the majority class mainly surrounds the distribution of the minority. Meanwhile, noise is the process whereby the majority class covers the sample of the minority data. This condition can complicate the classification method responsible for determining the decision limit for each category [32,41]. Therefore, based on this problem, the Radius SMOTE method can overcome its weaknesses due to its ability to produce synthetic data from minor classes on the image of the fetal umbilical cord. The oversampling process can improve classification accuracy on fetal umbilical cord image data [33]. Based on these problems, this study proposes the Radius SMOTE method for the imbalance oversampling process in the DEAP dataset. In addition to the EEG data oversampling process, there are several vital processes in emotion recognition, such as the feature extraction and the emotion classification processes using the Differential Entropy (DE) method. According to [42], the method is capable of characterizing spatial data from EEG signals with the highlights feature comprising foremost exact and steady features [28,[43][44][45][46]. The classification process in this study compares two methods, namely the Decision Tree and the Convolutional Neural Network. The purpose of applying these two classification methods is to measure the accuracy of EEG data oversampling performance on a machine and deep learning.

3-Methodology
This chapter discusses several stages of the oversampling approach using the Radius SMOTE method for emotion recognition, as shown in Figure 1. Figure 1 shows seven stages in emotion recognition, namely the preprocessing, feature extraction, oversampling (which is the contribution of this study), feature representation, classification, validation, and accuracy calculation stages.

3-1-DEAP Dataset
The dataset used in this study is the DEAP. This dataset is publicly accessible via the web https://www.eecs.qmul.ac.uk/mmv/datasets/deap/. The following is a description of the DEAP dataset [20]:  The EEG signal data collected in this dataset consists of thirty-two participants with an equal number of males and females within the age of 19-37 years.
 These emotional reactions are recorded using an EEG device called Biosemi, where the number of channels used amounted to thirty-two.
 A total of 40 experiments with stimulus media for each participant were used to evoke their emotional reactions.
 The duration for each experiment is 1 minute (60 seconds), while the total time of the investigation per participant was 2400 seconds (40 experiments × 60 seconds).
 Every second for each channel of the EEG device produces an EEG sampling rate of 128 Hz.
In the DEAP dataset, the EEG signal acquisition process is carried out by placing thirty-two channels on the scalp. The position of the thirty-two channels on the scalp is presented in Figure 2. In Figure 2, NASION represents the skull's front, precisely at the top center of the forehead, while INION denotes the lower back. Meanwhile, Fp, F, T, P, O, and C represents the prefrontal, frontal, temporal, parietal, occipital, and central head. In addition, the following channels represent the development of several existing channels:  AF is a channel placed between Fp and F;  FC is a channel placed between F and C;  CP is a channel placed between C and P;  PO is a channel placed between P and O.
The position of channel placement in this DEAP dataset uses the 10 -20 International standard system. These values represent the percentage (%) of the distance between NASION and INION channels. Standard procedures have been carried out in the DEAP dataset acquisition process, starting from determining stimulus media, proper presentation setup, and standardization of experimental protocols. However, this dataset has data imbalance conditions found in participants S01, S02, S03, S04, S07, S09, S11, S12, S13, S14, S17, S18, S19, S20, S21, S22, S23, S24, S25, S27, S29, and S32. These unbalanced data are relatively high (40% < not balanced between high and low classes) on arousal emotion. Participants S04, S05, S06, S07, S11, S16, S18, S23, S26, S27, S28, and S30 had unbalanced data conditions, which were relatively high (40% < not balanced between high and low classes) on valence emotions. However, participants S16 had a balanced dataset condition for arousal emotion, while S09, S10, S14, S15, and S32 are associated with valence emotion. In addition to these participants, some participants had an imbalance condition that was not too high [20]. Therefore, this study proposes the Radius SMOTE method for oversampling the imbalance data in the DEAP dataset.

3-2-Preprocessing
At this stage, the decomposition process using a bandpass filter is carried out to determine the four frequencies of the EEG signal for the 32 channels. A bandpass filter is used to decompose the EEG signal on each channel into four frequency bands, namely Theta, Alpha, Beta, and Gamma frequencies. The decomposition process is carried out by determining each frequency band's Low and High Pass values. Table 1 shows the respective Low Pass and Band Pass values for each frequency band [28,46,47]. In general, the EEG signal has five frequency bands, out of which four, namely Theta, Alpha, Beta, and Gamma, are correlated with emotional reactions [46,48]. Figure 3 shows the decomposition process of the EEG signal into four frequency bands for the Fp1 and O2 channels. However, this process is carried out on all 32 channels, followed by the segmentation process in all sixty segments for each frequency band consisting of 32 participants. In the DEAP dataset, a participant was expected to conduct forty experiments. Figure 4 shows the segmentation process on the Frontopolar 1 (Fp1) and Occipital 2 (O2) channels for the first experiment.  The Fp1 and O2 channels produce sixty segments (Sg1 -Sg60) for one experiment. Each segment/piece consists of a 128 Hz sampling rate, while each participant is expected to possess 7680 segments (60 segments x 32 channels x 4 frequency bands).

3-3-Feature Extraction
After the segmentation process, the feature extraction process is carried out for each segment using the DE method. Each participant will generate 7680 DE feature data (32 channels x 4 frequency bands x 60 segments). Therefore, for the overall experiment, a total of 307200 DE feature data (7680 feature x 40 experiments) is obtained. The following is the formula for the Differential Entropy (DE) method [28,46]: where denotes Euler's constant (2.71828), δ 2 represents variance, h i is the Differential Entropy (DE) value corresponding to the EEG signal in each frequency band.

3-4-Oversampling
Radius-SMOTE is a method of making synthetic data by changing several steps. It is used to overcome problems, such as overlapping and noise, and also to decrease the accuracy performance in the classification process. Furthermore, it is also used to determine the imbalanced data, noise, and overlapping conditions in determining decision boundaries for each class in the dataset. In general, there are two stages of oversampling the EEG signal feature data from the minority class, namely the filtering and the synthetic data formation stages [33]:  Filtering stage. At this stage, the selection process is carried out to obtain data from the right EEG feature (SAFE) using a radius approach divided into SAFE and NOISE data using the KNN algorithm. Furthermore, data in the SAFE category is used as a reference in oversampling new/synthetic data to reduce its occurrence and create new noise data. Data oversampling is limited to this circular area to avoid overlapping conditions to other class areas.
Radius is used to determine the distance of the nearest majority data point from the sample and use it as the radius value. All new data points are created only within that radius constraint.
where ( 1 , 2 , 3 , … ) is the center point of the circle in the minority sample, while ( 1 , 2 , 3 , … ) is the new data point in the radius. Next, the calculation process of 2 is calculated to determine the value of the distance between and as in Equation 5. The illustration of this proposed model is shown in Figure 5. where ( 1 , 2 , 3 , … ) is the closest majority point to the center of the circle ( ). Furthermore, the distance of each minority sample is calculated from the majority class using the Euclidean distance method. The closest majority data point has a minimum distance to the minority data point, as shown in Equation 6.
In this study, the Radius-SMOTE parameter uses the k value of 5 in the KNN method to perform the filtering process of sample data.
 EEG feature data creation stage. Making this synthetic data is based on the concept of radius, where the determination of the safe radius value is obtained from the circle equation. Its diameter is the distance between the EEG feature data of the SAFE category and the closest majority. Where is the smallest distance between the minority (j) and majority (i). After determining the majority of the data points, the formation of synthetic data is carried out by interpolating the two points. Synthetic data formation is carried out in two directions, namely (positive) and − (negative), with Equations 7 and 8: Limiting the area of creating new data reduces the occurrence of overlapping data in the SMOTE method [33]. Figure 5 shows the process of oversampling data using the Radius SMOTE method.
Furthermore, the amount of synthetic data is made based on the imbalanced ratio value in each dataset. Therefore, the higher the imbalanced ratio value, the greater the number of synthetic data formed in one sample data.

3-5-Feature Representation
Feature values for each experiment in four frequency bands are represented in a 9 × 9 matrix. The blue, green, yellow, and red matrix denotes the theta, alpha, beta, and gamma-band frequencies. The combination of the four matrices is called the 3D Cube [46]. The DE feature values for all channels in each frequency band in one segment are represented in a 9 × 9 matrix. Furthermore, the obtained matrix from the four frequency bands in one segment forms a 3D Cube representation. Figure 6 shows the 3D Cube representation method.

3-6-Classification Process
This study applied the CNN and the Decision Tree methods to measure the accuracy of EEG data oversampling performance on the machine and deep learning processes. The CNN method uses a 3D cube in each segment as input data, producing high or low emotion outputs for each arousal and valence. Its architecture in this study adopted the study by Yang et al. (2017) [46], as shown in Figure 7. In the CNN method, each participant (independent subject) is carried out in one stage for the Arousal and Valence classification processes. Figure 7 shows that there are four processes in the CNN method for emotion classification, namely the convolution, flatten, fully connected, and output stages as follows [46]:  The convolution stage. It is divided into four: the 1 st , 2 nd , 3 rd , and 4 th convolutions. The 1 st uses a 4 × 4 × 64 filter, with the stride value, activation function, and zero padding is 1, ReLU, and SAME, respectively. This is in addition to a resulting feature map of 9 × 9 × 64. The 2 nd convolution uses a 4 × 4 × 128 filter, with a stride value, activation function, and zero padding is 1, ReLU, and SAME, respectively. This is in addition to the resulting feature map of 9 × 9 × 128. The third convolution uses a 4 x 4 x 256 filter, with a stride value, activation function, and zero padding is 1, ReLU, and SAME, respectively. In this convolution, the resulting feature map is 9 × 9 × 256. Finally, the fourth convolution uses a 1 × 1 × 64 filter, with a stride value, activation function, and zero padding is 1, ReLU, and SAME, respectively. In this convolution, the resulting feature map is 9 × 9 × 64. The following is Equation 9 of the convolution process: The variable [ ] represents the matrix of feature map at the i th index, where F, N, Bf, j &k, and m& n denotes the filter matrix, input matrix, the bias on the filter, the feature map locations in the input matrix, and the location of the filter matrix.
 Flatten stage. The feature map generated from the 4 th convolution is reshaped at this stage, thereby measuring 5184 neurons (9 × 9 × 64).
 Fully connected stage. In this process, 5184 neurons are fully connected to 1024 hidden layers. This hidden layer uses a dropout operation of 0.5 to prevent overfitting and speed up the learning process. Dropout is carried out by deactivating the neurons connected to the hidden layer. The neuron to be deactivated is randomly chosen at a probability value of 0.5. Furthermore, weighted addition is carried out on the active neurons. Equation 10 is used to determine the weighted addition from input to the hidden layer.
The variable _ represents the output value resulting from the weighted summation process in the i th output layer. Meanwhile, , , 1, , and n denote the node value of the j th input layer, weight value from the input to the hidden layer, bias value from the input to the hidden layer, number of nodes from the input to the hidden layer. Furthermore, the value of the variable _ is activated using the softmax method, as shown in Equation 12.
 Outputs stage. Furthermore, 1024 neurons in the hidden layer will be connected to two output layers representing high or low for arousal and valence, respectively. At this stage, a weighted summation process is also carried out from the hidden layer to the output layer. Equation 11 is used to determine the weighted addition from hidden to the output layer.
The variable _ represents the output value resulting from the weighted summation process in the i th output layer. Meanwhile, , , , 2, , and n denote the node value of the j th hidden layer, weight value from the hidden to the output layer, bias value from the hidden to the output layer, number of nodes from the hidden to the output layer. Furthermore, the value of the variable _ is activated using the softmax method, as shown in Equation 12.
where , ⃗ , , and represent the softmax activation value, the value of the input vector, the standard exponential function of vector input, number of emotion classes, and the standard exponential function of vector output, respectively. The activation results will produce an output value that represents the high or low class for each Arousal and Valence emotion In this model, the loss value calculation and update processes use the cross-entropy loss and the Adam Optimizer methods. Furthermore, several parameters such as the learning rate (1e-4), the epoch (75), and the batch size (128) were determined. The second experiment in this study used the Decision Tree method. The max_depth parameter consists of a Decision Tree method with a value of 20 without using 3D Cube as input data. This method was implemented using the python programming language obtained from https://github.com/ynulonger/DE_CNN [46].

3-7-Validation Process
The accuracy measurement process is carried out at this stage using the K-Fold Cross Validation method with a Kvalue of 10. Measurement of the accuracy of emotion recognition of arousal and valence is also carried out for all 32 participants [46]. According to Figure 8, a participant has 2400 data divided into ten sections. The first part is used for the validation process (K=1), where the first 240 data are used as test data (in the orange block), and from the 241st to 2400th data (2160 data in the blue block) used as data training. The second part is used for the second validation process (K=2), where the 241st to 480th data (240th data in the orange colour block) are used as a test, while the first 240 (blue colour block) and from the 481st to the 2400th (1920 data in the blue colour block) used as training data. This validation process is repeated ten times (K=10), where the last 240 data were used as testing (in the orange colour block) and the first 2160 data as training (in the blue block). This process is used to validate the model responsible for recognizing categories of arousal emotions.

3-8 Emotion Accuracy
The emotion category in this study refers to the Russell Circumplex model, where it can be grouped into arousal and valence, with each class consisting of high or low value. In theory, valence is an individual's emotion towards something or an event. Meanwhile, arousal is an individual's excitement to behave or express their emotions [49,50]. Figure 9 presents an emotional representation based on the Russell Circumplex model.

Figure 9. Russell circumplex model [51]
Each validation process produces an accuracy value for both arousal and valence emotions for the High/Low emotion category. The accuracy value of this study will be compared with emotion recognition from several preliminary studies. This comparison of accuracy aims to examine the proposals of this study.

4-Results and Discussion
This chapter presents the proposed Radius SMOTE method used for oversampling imbalanced data on the DEAP dataset. In Table 2, the DEAP dataset is presented for each imbalanced participant [20]. The DEAP dataset consists of balanced data for several participants. This data is obtained assuming there are high and low classes of arousal/valence with the same value of 1200 data, culminating in 2400 data. The distribution of the DEAP dataset after the oversampling process using Radius-SMOTE is shown in Table 3.  Table 3 indicates that the Radius-SMOTE method will generate synthetic data on the minor class. However, the addition of synthetic data exceeds the majority class, while data in the minority class becomes the majority after oversampling using Radius-SMOTE. This is followed by the classification process of deep and machine learning, using the CNN and Decision Tree methods. The use of these two methods aims to measure the results of emotion classification using deep and machine learning approaches. Based on the validation and classification processes using the K-Fold Cross-validation and the CNN methods, the accuracy values for recognizing arousal and valence emotions are 82.11% and 78.99%, respectively. Meanwhile, the accuracy value of using the Decision Tree method for the classification process, arousal, and valence accuracy is 78.78% and 75.14%, respectively. Figure 10 shows the accuracy of arousal emotion recognition for each participant using the CNN and the Radius SMOTE methods.

Figure 10. Comparison of arousal accuracy for using CNN and with or without the Radius SMOTE method
In Figure 10, the Radius SMOTE method for oversampling data and the CNN method for the classification process have the ability to increase the accuracy of recognizing arousal emotions. However, one participant with ID s16 did not show an increase in accuracy because the data was balanced, hence there was no oversampling. Subsequently, the Radius SMOTE and the CNN method are used to recognize valence emotions, as shown in Figure 11.

Figure 11. Comparison of valence accuracy for using CNN and with or without the Radius SMOTE method
Generally, using the Radius SMOTE method can improve the accuracy of recognizing valence emotions. However, some participants, such as ID s09, s10, s14, s15, and s32, did not experience an increase. However, the oversampling process was not carried out on these participants because the data was balanced. Apart from using the CNN method, this study also examined the use of the Decision Tree method combined with the Radius SMOTE. Figure 12 compares the accuracy with and without the Radius SMOTE method for recognizing arousal emotions. Figure 12. Comparison of arousal accuracy for using Decision Tree and with or without the Radius SMOTE method Figure 12 shows that the Radius SMOTE with the Decision Tree method increases the accuracy of recognizing arousal emotions. However, one participant with ID s16 did not experience an increase because the data were balanced without oversampling process. In the same way, the use of the Radius SMOTE method with the Decision Tree method can also increase the accuracy of recognizing valence emotions, as shown in Figure 13.

Figure 13. Comparison of valence accuracy for using Decision Tree and with or without the Radius SMOTE method
The Radius SMOTE method can improve recognizing valence emotions. However, some participants, such as ID s09, s10, s14, s15, and s32, did not experience an increase because of these data without oversampling process. In general, using Radius-SMOTE to oversampling on the DEAP dataset can improve emotion recognition accuracy. This conclusion indicates that the imbalanced data conditions in the DEAP dataset can reduce the accuracy of emotion recognition. Furthermore, the accuracy results obtained from this study are compared with several others, as shown in Table 4. The Radius SMOTE method for the oversampling process produces a higher arousal and valence emotion recognition accuracy than those proposed by Yang et al. in the DEAP data set [46]. Although the emotion classification process has been improved using the Capsule Network method and the combination of Convolutional Neural Network and Long Short Term Memory methods; however, the accuracy is achieved still lower than this research proposal [47,48,52]. On the other hand, the oversampling process on imbalanced data using the Radius SMOTE method produces higher accuracy than the Random Oversampling method [36]. In general, the application of the Differential Entropy, Radius SMOTE, 3D Cube, and Convolutional Neural Network for feature extraction, imbalanced data, representation, and classification in this study led to higher accuracy compared to some of the previous studies [36,[46][47][48]52].

5-Conclusion
The problem of data imbalance in the DEAP dataset was solved by applying an oversampling approach using the Radius-SMOTE method. The oversampling process produces new synthetic data on the minority class. This study conducted two experiments to measure the Radius SMOTE method's ability to increase emotion recognition accuracy. The feature extraction process used the Differential Entropy and the Decision Tree methods for the classification process in the first experiment. The second experiment used the Differential Entropy, 3D Cube, and Convolution Neural Network for feature extraction, representation, and emotion classification processes. Based on these two experiments, the oversampling approach using the Radius SMOTE method increases the accuracy of recognizing arousal and valence emotions. In essence, the Radius SMOTE is an oversampling method used to create new synthetic data based on secure Radius data. However, making synthetic data does not consider the amount of data in the majority class, thus causing the previously in the minority class to become the majority class. So the data will still experience an unbalanced condition. In the future, it is necessary to determine the suitable method to handle unbalanced data, both from the undersampling approach and other approaches.
Conversely, though emotion recognition accuracy has increased, it is still below 85% in arousal and valence. This problem becomes a challenge in future studies in recognizing emotions based on EEG signals. Therefore, emotion recognition based on EEG signals is strongly influenced by participant characteristics. Combining the baseline reduction approach to characterize participant characteristics is essential to examine the oversampling approach.

5-1-Limitations
The Radius SMOTE method is an oversampling method used to create new synthetic data based on SAFE radius data. However, its development does not consider the amount of data in the majority class; hence the minority class becomes the majority. In the future, it is necessary to determine the suitable method to handle unbalanced data, both from the under-sampling approach and other approaches. On the other hand, though emotion recognition accuracy has increased, the average accuracy is still below 85% for emotional arousal and valence. This problem becomes a challenge in future studies in its recognition based on EEG signals, which is strongly influenced by participant characteristics, such as personality traits, intellectual abilities, and gender [28,53]. Therefore, further study needs to be conducted to examine the oversampling approach and allow the combination with the baseline reduction approach to characterize participant characteristics.

6-2-Data Availability Statement
The data used in this study is the DEAP dataset. This dataset is freely accessible at [https://anaxagoras.eecs.qmul.ac.uk/request.php?dataset=DEAP].

6-3-Funding
The authors received no financial support for the research, authorship, and/or publication of this article.