This monograph provides a comprehensive, example-driven introduction to differential privacy through the lens of practical programming. Bridging the gap between theory and implementation, it walks readers through core mechanisms, such as the Laplace and Exponential mechanisms, smooth sensitivity, sample-and-aggregate, and the sparse vector technique, all while grounding each in executable Python examples. With an emphasis on hands-on learning and reproducibility, the material is designed for learners who wish to move beyond abstract definitions and understand how differential privacy is applied to real datasets, models, and systems. The sections of this monograph are organized around both foundational theory and practical concerns: sensitivity and composition are introduced early, followed by in-depth treatments of de-identification, synthetic data generation, and private machine learning. The monograph also includes discussion of subtle implementation challenges such as efficiency, numerical stability, privacy accounting, and optimizations like ghost clipping. Wherever possible, examples use familiar tools like pandas, numpy, and matplotlib to increase clarity with data scientists. By making privacy-preserving algorithms concrete and programmable, the monograph aims to lower the barrier to entry for researchers, engineers, and educators working with sensitive data. It is intended as both a self-contained reference and a foundation for further exploration into formal privacy and responsible AI.
Deep neural networks (DNNs) now dominate the AI landscape and have shown impressive performance in diverse application domains, including vision, natural language processing (NLP), and healthcare. However, both public and private entities have been increasingly expressing significant concern about the potential of state-of-the-art AI models to cause societal and financial harm. This lack of trust arises from their black-box construction and vulnerability against natural and adversarial noise. As a result, researchers have spent considerable time developing automated methods for building safe and trustworthy DNNs. Abstract interpretation has emerged as the most popular framework for efficiently analyzing realistic DNNs among the various approaches. However, due to fundamental differences in the computational structure (e.g., high nonlinearity) of DNNs compared to traditional programs, developing efficient DNN analyzers has required tackling significantly different research challenges than encountered for programs. In this monograph, we describe state-of-the-art approaches based on abstract interpretation for analyzing DNNs. These approaches include the design of new abstract domains, synthesis of novel abstract transformers, abstraction refinement, and incremental analysis. We will discuss how the analysis results can be used to: (i) formally check whether a trained DNN satisfies desired output and gradient-based safety properties, (ii) guide the model updates during training towards satisfying safety properties, and (iii) reliably explain and interpret the black-box workings of DNNs.
Incremental computation aims to compute more efficiently on changed input by reusing previously computed results. We give a high-level overview of works on incremental computation, and highlight the essence underlying all of them, which we call incrementalization -- the discrete counterpart of differentiation in calculus. We review the gist of a systematic method for incrementalization, and a systematic method centered around it, called Iterate-Incrementalize-Implement, for program design and optimization, as well as algorithm design and optimization. At a meta-level, with historical contexts and for future directions, we stress the power of high-level data, control, and module abstractions in developing new and better algorithms and programs as well as their precise complexities.
Neurosymbolic programming combines the otherwise complementary worlds of deep learning and symbolic reasoning. It thereby enables more accurate, interpretable, and domain-aware solutions to AI tasks. We introduce Scallop, a general-purpose language and compiler toolchain for developing neurosymbolic applications. A Scallop program specifies a suitable decomposition of an AI task's computation into separate learning and reasoning modules. Learning modules are built using existing machine learning frameworks and range from custom neural models to foundation models for language, vision, and multi-modal data. Reasoning modules are specified in a declarative logic programming language based on Datalog which supports expressive features such as recursion, aggregation, negation, and probabilistic programming over structured relations. Scallop's compiler enables to automatically train neurosymbolic programs in a data- and compute-efficient manner using an end-to-end differentiable reasoning framework. Scallop also supports features useful for building real-world applications such as user-defined data types, and foreign interfaces. We demonstrate programming in Scallop for applications that span the domains of image and video processing, natural language processing, planning, and information retrieval in a variety of learning settings such as supervised learning, reinforcement learning, rule learning, contrastive learning, and in-context learning.
Computational visual perception seeks to reproduce human visionthrough the combination of visual sensors, artificial intelligence andcomputing. To this end, computer vision tasks are often reformulatedas mathematical inference problems where the objective is to determinethe set of parameters corresponding to the lowest potential of a taskspecificobjective function. Graphical models have been the most popularformulation in the field over the past two decades where the problemis viewed as a discrete assignment labeling one. Modularity, scalabilityand portability are the main strengths of these methods which oncecombined with efficient inference algorithms they could lead to state ofthe art results. In this tutorial we focus on the inference component ofthe problem and in particular we discuss in a systematic manner themost commonly used optimization principles in the context of graphicalmodels. Our study concerns inference over low rank models interactionsbetween variables are constrained to pairs as well as higher orderones arbitrary set of variables determine hyper-cliques on which constraintsare introduced and seeks a concise, self-contained presentationof prior art as well as the presentation of the current state of the artmethods in the field.
