It is necessary to count the shrimp eggs in vitro accurately in the artificial breeding of red claw crayfish. Due to the small size and high density of shrimp eggs, the artificial counting of shrimp eggs has low efficiency and may damage the eggs. We propose an end-to-end shrimp eggs counting model that is based on the residual building unit. This network model integrates a custom residual building unit in the front-end of the encoder network and performs feature extraction and down-sampling of the shrimp eggs’ position information. The back-end of the decoder network uses the transposed convolution instead of bilinear interpolation, so that the model can learn upsampling parameters automatically. Experiments on the shrimp eggs dataset, compared with VGG-16, CSRNet, MCNN, SANet, the proposed counting method based on the residual building unit can reach the counting accuracy of 98.88%, and has the lowest mean absolute error, the lowest mean square error, and the best quality of the predicted density map. These test results show that the proposed counting method of red claw crayfish’s eggs based on a residual building unit can effectively solve the difficult counting problems caused by the small size, high density and easy adhesion of the shrimp eggs, and provide a new method for the counting of red claw crayfish’s eggs. The computer vision automatic counting device for the shrimp eggs was based on the proposed counting model, which had the characteristics of simple structure and accurate counting, and could be applied to real scenes.