Aiming at the difficulty in removing motion shadow regions in the process of fish behavior quantification, with goldfish as research object, the method of goldfish shadow removal and image segmentation based on improved K-means clustering algorithm is proposed to optimize traditional K-means clustering algorithm in such aspects as removing noisy points and isolated data points, using mahalanobis distance as distance measurement method, defining the number of clustering and choosing the initial clustering center. In the normal indoor environment, the camera is used to collect the images of goldfish in the glass fish tank. First, after 10 times of uniform compression, we use the median filtering method to preprocess the sample images, and then the sample images are converted from RGB color space to Lab color space. Finally, the components a and b are extracted and the improved K-means algorithm is used for clustering. The test results show that compared with the traditional K-means clustering algorithm and FCM (Fuzzy c-means) clustering algorithm, the improved algorithm has better effect on image shadow removal and segmentation. The average misclassification pixel ratio and the average running time based on improved K-means clustering algorithm are 2.48% and 0.875s respectively in 200 goldfish sample images with different shadows, which can satisfy the requirement of image preprocessing in the process of off-line fish behavior quantification.