Data acquisition in the large-scale Internet of Things (IoT) demands high-cost sensing and communication. In this paper, we develop a data-driven deep learning method for low-complexity sensor selection and high-accuracy data reconstruction in large-scale IoT. The proposed deep learning model is designed with a sampling network for sensor selection and a deep neural network (DNN) for data reconstruction, where the Gumbel-max trick is applied to ensure that the loss function of the model is differentiable. With the design of an interconnected sampling network and reconstruction network, the sampling matrix for sensor selection and its corresponding data reconstruction can be trained simultaneously. The proposed method is data driven and does not require the graph structure in advance; hence, it can avoid the high-complexity computation required by graph sampling-based methods. Experiments on both synthetic and real-world datasets reveal the performance improvement of the proposed method in computational complexity and data reconstruction accuracy.