The emulation or layout in the study of processing-in-memory (PIM) is a highly time-consuming process. Especially, the processing-using-memory (PUM), a subset of PIM, is much more complex due to the positioning of the processing unit in the high-density data array. Because of this reason, it is important to efficiently verify PIM hardware using simulation to activate the PIM study. To this end, we modify the DRAMsim3, a memory simulator, to implement a PUM system, and propose a PIM operation compiler in the Zsim, a CPU simulator. The PIM operation compiler performs the role of tracing instructions from various precision deep neural network (DNN) workloads and generating PIM operation commands. Finally, we propose an architecture-level PUM simulation framework that can simulate the PUM system with DNN workloads based on the PIM command generated by the compiler.

The accuracy of convolutional neural networks (CNNs) has significantly improved over the years. Meanwhile, due to the high portability and usefulness of edge devices, the demand for artificial intelligence (AI) based applications on edge computing devices has been soaring recently. Accordingly, CNN inference has become one of the mainstream AI applications on edge devices. However, the continually increasing leakage power of edge devices drags down the wide deployment of CNN inference applications, as the technology node scales down. In this work, we focus on reducing the power consumption in main memory, which consumes considerable power in CNN inference. Particularly, we observed that the idle state of memory is dominant in computationally intensive CNN inference. To achieve low-power CNN inference on edge devices, we first utilize next-generation nonvolatile memory (NVM) as the main memory device rather than dynamic random-access memory (DRAM) only for CNN inference tasks. To mitigate the increased latency caused by NVM, we propose a novel commercial NVM system models; it is designed to predictably manage the locality-specific demands of CNN models while smartly leveraging existing resources in a modern NVM system. Furthermore, utilizing a prefetcher-based approach, we optimize the write allocation to enhance the data reuse and energy efficiency in CNN workloads. Based on simulation, our design improves the energy efficiency by 50% with a negligible impact on the performance compared with conventional DRAM-based platforms.

Processing-in-Memory (PIM) is an emerging computing architecture that has gained significant attention in recent times. It aims to maximize data movement efficiency by moving away from the traditional von Neumann architecture. PIM is particularly well-suited for handling deep neural networks (DNNs) that require significant data movement between the processing unit and the memory device. As a result, there has been substantial research in this area. To optimally handle DNNs with diverse structures and inductive biases, such as convolutional neural networks, graph convolutional networks, recurrent neural networks, and transformers, within a PIM architecture, careful consideration should be given to how data mapping and data flow are processed in PIM. This paper aims to provide insight into these aspects by analyzing the characteristics of various DNNs and providing detailed explanations of how they have been implemented with PIM architectures using commercially available memory technologies like DRAM and next-generation memory technologies like ReRAM.

Convolutional neural network (CNN) models require deeper networks and more training data for better performance, which in turn results in greater computational and memory requirements. In this paper, we analyze the memory access patterns that occur in main memory during the training processes of various CNN models. CNN training is a linear procedure consisting of a forward pass (FP) and a backward pass (BP). As a result of the analysis, we found that BP accounted for 83.4% of the total main memory accesses on average. Therefore, CNN training including FP and BP is much more memory-intensive than CNN inference using only FP. This demonstrates that CNN training is a suitable application for near-data processing to reduce memory bottlenecks and conserve computational resources.

In this paper, we propose a spatio-temporal switchable data prefetcher that can adapt to the locality characteristics of CNN models. The proposed prefetcher records the recent delta history by leveraging two tables. The first table predicts spatial address patterns by comparing the delta score with the last delta, while the second table predicts temporal address patterns by recording and reordering the delta sequence from the delta history. Consequently, the proposed prefetcher is capable of appropriately switching between these two prediction methodologies based on spatial and temporal localities. The experimental results on CNN inference workloads show that we achieved high average accuracy of 83.8% and coverage of 81.6%, and hence the proposed prefetcher improves system performance by 33.8% over a baseline with no data prefetcher and 21% over the best-performing prior spatio-temporal prefetcher.

In this paper, we analyze the suitability of convolutional neural network (CNN) inference workloads on a phase-change memory (PCM) platform. CNN inference has an average of $14\times$ more read requests than write requests (i.e., read dominant) and a significantly low last-level cache misses per kilo instructions (LLC MPKI) of 2 on average (i.e., computation intensive). In addition, to compare the latency and energy of PCM and DRAM systems, we evaluate CNN inference workloads on two memory systems through a memory simulator. As a result, compared to DRAM, PCM can save total energy by 54% on average, but instruction per cycle (IPC) of PCM is reduced by an average of 28%. In conclusion, CNN inference is a workload suitable for PCM in terms of energy efficiency, but it must be accompanied by a scheme to improve IPC for practical use.

In recent times, numerous convolutional neural network (CNN) based detection models have been proposed and have shown excellent performance. However, because these models are generally developed to detect objects in class units (e.g., person, car), additional training processes with numerous datasets are required to find a specific object. This paper proposes a model that accurately detects specific persons by using top clothing color information without any additional training processes. The proposed method combines CNN-based instance segmentation and pose estimation, utilizing all the advantages of each technique. To avoid redundant computations, these two schemes are implemented as a filtering-based sequential operation structure. As a result, the proposed method has a 92.57% of accuracy in detecting a specific person with only a slight processing speed decrease. Furthermore, in this paper, the proposed model is efficiently ported on the heterogeneous embedded platform (i.e., NVIDIA Jetson AGX Xavier) with a parallel processing technique to maximize the hardware utilization.