For the existing difficulty that underwater fish can’t be recognized quickly and accurately, a transfer learning method with automatic image enhancement function was proposed. In this method, after the color pattern of fish image switched from RGB to Lab, central peripheral operator was used to calculate the significant value of the entire input image, thereby providing the potential region of the fish target, then the GrabCut algorithm was combined to segment the fish target, and finally the original image of the merged segment was sent to the optimized residual network for training. The recognition results of 23 kinds of fishes show that fixing of the conv1 and conv2 layer parameters of the ResNet-50 pre-trained model on the ImageNet data set and fine-tuning of the high-level parameters can achieve the best recognition results, and on the public Fish4Knowledge data set, the model achieved the highest recognition accuracy, with an average recognition accuracy of 99.63%. The recognition accuracy and time performance of the proposed method are better than those of other convolutional neural network methods both on Fish4Knowledge and Fish30Image data sets, and the recognition accuracy is improved by at least 4.98%. Experiments on multiple data sets demonstrate the validity of the model.