Aiming at the problem that the existing target detection framework based on big data and deep learning is difficult to realize real-time video target detection on low-power mobile and embedded devices, this paper improved the target detection framework SSD based on deep learning, and put forward an improved multi-target detection framework LSTM-SSD which was dedicated to multi-target detection of traffic scenes video. Combining single image detection frame with recursive neural network LSTM network to form an interleaved circular convolution structure, it realized the temporal association of network frame-level information by extracting the feature map between propagation frames by adopting a Bottleneck-LSTM layer, which greatly reduced the network calculation cost. Combining the time-aware information with the improved dynamic Kalman filtering algorithm, the tracking and identification of the targets which were influenced by strong interference such as light change and large-area occlusion in the video could be realized. Experimental results show that the improved LSTM-SSD can achieve good results when dealing with the difficult detection situations such as multi-targets, cluttered background, light changes, fuzziness and large-area occlusion. Compared with other target detection frameworks based on deep learning, the average accuracy rate of all kinds of target identification is increased by 5%~16%, the average accuracy rate is increased by 4%~10%, the multi-target detection rate is increased by 4%~19%, and the detection frame rate reaches 43 fps, basically meeting the requirements of real-time. The balance between the accuracy of the algorithm and the running speed is achieved, and a good detection and identification effect is achieved.