This study proposes a target detection and tracking method based on the improved YOLOv11 model—YOLOv11n-DFM. It aims to evaluate losses during the fishing process by detecting the number of crab traps being lifted or lowered and to assess the normalcy of the trap mechanism by detecting the number of traps in key areas. The method integrates DyHead, FocalModulation, and CCFM modules into the YOLOv11n model to enhance multi-scale feature fusion, improve detection accuracy for traps of different scales, and reduce computational and memory costs. Additionally, the ByteTrack algorithm is employed to ensure precise tracking of the traps. Experimental results demonstrate that the YOLOv11n-DFM model improves detection accuracy by 1%, increases mAP@50-95 by 0.8%, while mAP@50 and recall remain unchanged. Compared to the YOLOv11n model, the detection performance is enhanced while maintaining the same detection efficiency. The study indicates that the YOLOv11n-DFM model excels in detecting and tracking the crabs' traps, consumes fewer computational resources, and is suitable for deployment in environments with limited computing power. It provides valuable references for fishery monitoring, resource management, and the future automation of crab trap deployment and collection.