In intensive aquaculture, real-time monitoring of fish feeding status is one of the important bases for developing scientific feeding strategies. In order to achieve temporal detection of fish feeding states , and to solve the problem of high error in detecting feeding state switching by existing methods, this paper proposes a fish state detection network BMN-Fish based on an improved BMN (Boundary Matching Network), which introduces an efficient depth module and consists of an Efficient Channel Attention Module Net (ECANet) and a Base Residual Module (BRM). Among them, the Efficient Channel Attention Module can expand the perceptual field of the temporal dimensional features, so as to improve the ability of the model to extract local information; the Base Residual Module enables the algorithm to focus on the regions of interest in the feature map, which can strengthen the global perception ability of the algorithm. The experimental results showed that the AUC and AR@100 of BMN-Fish were 93.32% and 95.28%, which were 2.17% and 1.95% higher than BMN, respectively. BMN-Fish could effectively detect the sequential feeding movements of fish, and provide important information for the formulation of intelligent feeding strategies for intensive aquaculture.