Fish object detection is of great significance for precision aquaculture, production automation, resource investigation and fish behavior research. In order to get the position and category of fish object quickly and accurately, a fish object detection method based on improved model YOLO V4 is proposed, Based on the CIoU(complete intersection over union) loss function, a new loss term is constructed. The improved loss function makes the real box and the intersecting box regress in the same aspect ratio. At the same time, the detection effect on a specific size and area is enhanced by setting a multi anchor box mode. The results show that the mAP (mean average precision) of the improved model YOLO V4 is greatly improved compared with the original model. The mAP on the self built data set, data set Fish4 knowledge and data set NCFM reaches 94.22%, 99.52% and 92.16% respectively. The research shows that the improved model YOLO v4 can quickly and accurately detect the position and category of fish, and the detection speed meets the real-time requirements, which can provide a reference for precision aquaculture of fishery.