The invention provides an image rain removing method and a device, comprising: an image to be detected with raindrops is separated into a high-frequency component image and a low-frequency component image; the high-frequency component image is input to the trained convolution neural network with residual structure, and the rain-free image is outputted; the rain-free image is synthesized with the image to be detected, and the rain-free image is obtained; the rain-free image is inputted to the trained convolution neural network with discriminant structure, and the category of the rain-free imageand the raindrop-free image corresponding to the image to be detected are outputted; the category is inputted to the convolution neural network with a residual structure, and the parameters of the convolution neural network with residual structure are updated to obtain the final rain-free image. The final rain-free image obtained by the invention retains the texture details of the rain-free areain the image to be detected, so that the final rain-free image is close to the rain-free image corresponding to the image to be detected, and the important factors in the image to be detected are wellretained.