Convolutional neural networks, are one of the most representative deep learning models. CNNs were extensively used in many aspects of medical image analysis, allowing for great progress in computer-aided diagnosis in recent years. In this paper, we provide a survey on convolutional neural networks in medical image analysis. First, we review the commonly used CNNs in medical image processing, including AlexNet, GoogleNet, ResNet, R-CNN, and FCNN. Then, we present an overview of the use of CNNs, for image classification, segmentation, detection, and other tasks such as registration, content-based image retrieval, image generation and enhancement, in some typical medical diagnosis areas such as brain, breast, and abdominal. Finally, we discuss the remaining challenges of CNNs in medical image analysis, and accordingly we present some ideas for future research directions.