Single image rain/snow removal using distortion type information

Rainy or snowy weather conditions can severely impair the visual quality of images and videos. The rain streaks or snow particles that may vary in shape and size can also affect high-level computer vision system performance. Therefore, pre-processing of these distorted images prior to any other task is necessary. Moreover, due to the lack of temporal information in single images, removal of these artifacts becomes more challenging. In this paper, both the de-raining and de-snowing problems within a single algorithmic framework using a data-driven approach are addressed. In this method, the spatial characteristics of rain streaks and snow particles are investigated and two maps, namely, direction map and intensity map, are generated and exploited in the removal process. Using these two maps, the type of the distortion is classified using a convolutional neural network (CNN) and this information is used in the removal step, where the input image along with the two extracted maps and the information about the distortion type are used to train a deep fully convolutional rain/snow removal network (RSRNet). This network is trained such that it separates the important background scene edges from rain streaks or snow particles and uses the extracted edge map to augment the quality of the output image. Moreover, single images usually suffer from atmospheric haze in the presence of heavy rain or snow. Therefore, a simple dehazing method based on the dark channel prior (DCP) algorithm, which uses the edge map extracted in the RSRNet, is proposed to build a transmission map for the haze removal task. The experimental results on both the real and synthetic single rainy/snowy images demonstrate the superiority of the proposed method compared to the other rain/snow removal methods.