In order to realize the automatic counting of the number of fish pumping through, a counting system of fish suction pump based on machine vision was designed. The system is mainly composed of fish pipe, camera, light source and industrial tablet computer. It adopts full shading design. The fish pipe is made of transparent plexiglass, and the fish contour information is obtained by backlight irradiation. Based on the working characteristics of counting equipment, a Double line-of-interest (DLOI) counting method was proposed, and the variation characteristics of video slice area were constructed by combining the analysis of projected area of fishes. By comparing the classification results of feature vectors by different learning algorithms, the SVM (Support Vector Machine) algorithm was selected to realize the counting of the number of fish pumping through. The results show that the counting system can realize the counting of the number of fish pumping through, with the average counting accuracy ≥ 90%. The counting algorithm can distinguish the overlapping and adhesion states effectively, which improves the robustness of the system.