Recently, deep learning (DL) is an innovative machine learning (ML) technique that has gained the outstanding achievements in computer vision and natural language processing. This work takes advantage of DL for effectively handling automatic modulation classification (AMC), which is the fundamental function of numerous cognitive radio-based and spectrum sensing-based applications in many modern communication systems. Concretely, a novel deep convolutional neural network (DCNN) is proposed for learning a classification model from a massive amount of modulated signals, in which the network architecture has several convolutional blocks specialized to simultaneously capture the temporal intra-signal correlations and the spatial inter-signal relations. To this end, each block comprises various convolutional layers of asymmetric convolution kernels, whose outputs are gathered via a concatenation layer. For the enrichment of multi-scale deep feature and the prevention of gradient vanishing problem, these blocks are associated by skip connections to take into account the useful residual information. In experiments, the proposed CNN-based AMC method achieves the overall 24-modulation classification rate of 88.22% at 10dB SNR on the well-known DeepSig dataset.