The classification performance of the existing multi-classification tasks of finger motor imagery EEG signals is difficult to achieve usable performance. On the basis of detailed analysis of multiple components on the time scale of EEG signals, this paper designed a self-supervised sub-network for signal sub-segment extraction, and then input the sub-segment into the next sub-network for signal classification, it synthesized the two sub-networks into a self-supervised hybrid multi-task deep network. In the training phase, the sub-segment extraction sub-network extracted different sub-segments for each EEG signal, and the subsequent classification sub-network judged whether the sub-segment was the best and automatically adjusted the position of the sub-segment. It weighted the total loss function by two loss functions of two sub-networks, and extracted the best sub-segment signal and obtained the best classification effect through the overall network learning algorithm. In the verification and testing phase, the sub-segment extraction sub-network automatically extracted the corresponding sub-segment input classification sub-network according to the parameters of the training for classification. It verified the network performance on the largest SCP data of Motor-Imagery data set and the Data sets 4 of BCI Competition IV. On the SCP dataset, the average test classification accuracy of all subjects' three finger classification tasks is more than 70%, the average test classification accuracy of four finger classification tasks is about 60%, and the average test classification accuracy of five finger classification tasks is about 50%, which is significantly improved compared with the existing reports. It is confirmed that the network can effectively extract sub-segments of motor imagery EEG signals, and has good classification effect and generalization performance.