In the current production process, the feeding control of fish is still based on artificial experience judgment and time sequence, which is easy to cause bait waste and environmental pollution. The real-time detection of the feeding intensity of fish can be used to guide feeding, thus improving the bait utilization rate and reducing residual bait pollution. Based on this, this paper proposes a real-time detection algorithm of fish feeding intensity based on machine vision and a lightweight S3D algorithm, which can accurately locate the four intensity levels of "strong, medium, weak, and none" in the video. Firstly, the 3D spatiotemporal Sep-Inc module is proposed using the I3D network as the benchmark, the inception module, and the depth separable convolution. Secondly, a lightweight S3D network is formed by alternately building a 3D  Spatio-temporal SEP Inc module, pooling layer, and 3D convolution layer. Finally, the feeding intensity identification system of golden trout was developed by using PyQt5. The experimental results show that the identification accuracy of the S3D algorithm for four types of feeding intensity reaches 92.68%, which is 9.75% and 14.15% higher than that of C3d and R2 + 1D algorithms respectively. Meanwhile, the parameters and GFLOPs parameters are also greatly reduced, and the feeding intensity tags identified per second reach 17F / s. Research shows that the method is not only applicable to golden trout but more importantly, it is expected to apply to the breeding of other swimming fish and provide feeding decision suggestions for intelligent feeding systems.