With the rapid emergence of advanced technologies for wireless communications, automatic modulation classification (AMC) has been deployed in the physical layer to blindly identify the modulation fashion of an incoming signal at the receiver and consequently improve the efficiency of spectrum utilization and management. Although recent works on AMC have adopted deep learning with convolutional neural networks (CNNs) to deal with large-confusing signal data, they have shown to be vulnerable to channel deterioration by primitive architectures. In this letter, we design a high-performance CNN architecture, namely Residual-attention Convolutional Network (RanNet), that mainly involves multiple advanced processing blocks to learn intrinsic features of combined waveform data (including in-phase, quadrature, amplitude, and phase components). Each block incorporates attention connection and skip connection in a sophisticated-designed structure to strengthen relevant features and weaken irrelevant features while preventing the network from vanishing gradient. Simulation results on the RadioML 2018.01A dataset show that RanNet is robust to different channel impairments and outperforms state-of-the-art deep networks in terms of accuracy while having a reasonable complexity.