Aiming at the problem that the traditional deep learning fish recognition method has a low accuracy rate and the parameters cannot be determined adaptively during the model training process. This paper proposes an improved ShuffleNet fish identification method based on Particle Swarm Optimization (PSO) and Transfer Learning (TL). The research takes 20 species of fish as the object, uses the particle swarm algorithm to take the model's loss function as the fitness function, optimizes the two hyperparameters of batch size and learning rate, and uses the transfer learning method for training to construct TL-PSO-ShuffleNet model. The research shows that compared with the models of AlexNet, MobileNet, and ShuffleNet, the recognition accuracy rate is increased by 57.89%, 30.43%, and 23.28%, respectively. The fish identification method proposed in this paper has the characteristics of high accuracy and self-adaptive parameter setting, which provides a reference for the research on automatic fish identification.