Detection of sealed and unsealed cracks with complex backgrounds using deep convolutional neural network

Crack Deep Network (CrackDN) is proposed in this research with the purpose of detecting sealed and unsealed cracks with complex road backgrounds. CrackDN is based on Faster Region Convolutional Neural Network (Fast-RCNN) architecture by embedding a sensitivity detection network parallel to the feature extraction Convolutional Neural Network (CNN), both of which are then connected to the Region Proposal Refinement Network (RPRN) for classification and regression. The state-of-the-art aspect of this research lies in the fusion of sensitivity detection network, which facilitated the CrackDN of being able to detect sealed and unsealed cracks with sever complex background. Four kinds of background conditions are considered for both sealed and unsealed crack analysis: normal and unbalanced illuminations, with markings and shadings. The raw pavement images are first processed simultaneously by the sensitivity extraction network, which is formed by a batch of line filters that each differ in angles for sensitive region extraction, and the CNN, which utilized ZF-Net. Then the extracted sensitive maps and feature maps are applied as the input of RPRN, thereby the prediction scores together with the bounding box can be obtained. The performance of CrackDN is compared with Faster-RCNN and SSD300 architectures for sealed and unsealed crack detection. Results demonstrate that CrackDN can achieve the detection mean average precision of higher than 0.90, which outperforms both Faster-RCNN and SSD300. The detection speed of CrackDN (around 6 fps) are slightly lower than SSD300 but significantly higher than Faster-RCNN. Meanwhile, the performance of sealed crack detection is better than unsealed crack detection for most background conditions. Moreover, sealed and unsealed cracks on the markings are the most difficult conditions for detection. However, CrackDN still can obtain the detection accuracy of above 0.85.