The concept of subband transforms and multiresolution transforms was presented with ideal filters for the sake of conceptual simplicity. However, ideal filters with zero transition length between pass band and stop band are not physically realizable. Normally, use of realizable filters will cause aliasing (spectral overlap) due to the finite length in this transition region. However, there is a class of filters with frequency response in the transition region that cancels aliasing. We begin with deriving the alias cancellation and perfect reconstruction relationships for two-channel orthogonal filters. We then derive from first principles the wavelet filter relationships that produce the orthogonal wavelet transform. We then do the analogous, but more complicated derivations with the biorthogonal filters that produce the biorthogonal wavelet transform. We then present the Lifting Scheme consisting of sequential so-called lifting steps that guarantee perfect reconstruction with the output filters with savings of additions and multiplications compared to direct filtering using the output filter taps. In conclusion, we present commonly used wavelet filters by their filter taps and their lifting steps.
The development so far has treated only transformations of one-dimensional sources. Images and two-dimensional data also need compression and two-dimensional (2-D) transformations. The one-dimensional transformations generalize in a logical way to two dimensions. We show how to implement a general and separable two-dimensional transform and specifically the important 2-D Discrete Cosine Transform (DCT).
The discrete Karhunen-Loève transform (KLT) requires knowledge of the covariance function of the source and a solution for the eigenvectors of the covariance matrix. In general, especially for large N, the solution and the transform are computationally burdensome procedures with no fast algorithms for their execution. Instead, one almost always uses a source-independent transform with a fast execution algorithm. We present several useful sub-optimal transforms and calculate coding gains of two of them.
Transforms that decompose the source into non-overlapping and contiguous frequency ranges called subbands are called subband transforms. A wavelet transform, as we shall see, is just a particular kind of subband transform. The source sequence is fed to a bank of bandpass filters which are contiguous and cover the full frequency range. The set of output signals are the subband signals and can be recombined without degradation to produce the original signal. For the sake of illustration, we start with ideal “brick-wall” filters in their frequency ranges. We develop the subband rate allocation formulas and calculate coding gain using both ideal and the realizable Haar filters.
This book teaches the fundamentals and mathematical formulas of reversible transformations used in many source coding and signal processing systems.
Input signals are usually transformed sequentially in blocks of a given length N. In these circumstances, it is convenient to realize the sequence of transformed blocks using a bank of filters followed by downsampling. The reconstruction is realized with a filter bank performing upsampling and filtering. Here we present the details of how this is done.
In this chapter we explain the workings of an optimal transform coding system. Its components are forward and inverse transformations of blocks of source samples and encoders and decoders of the transformed blocks. We derive the optimal allocation of rate and distortion for encoding a Gaussian source using the mean squared error distortion criterion. We derive a formula for Transform Coding Gain for an arbitrary transform and the optimal Karhunen-Loève Transform.
In this chapter, a brief introduction to the field of artificial neural networks is provided with a focus on deep learning [9], neural network training, and different architectures. Artificial neural networks are powerful pattern recognition machines, and they have proved to be the most successful. Neural networks and deep learning are quite successful at end-to-end learning, and they do not require feature engineering as in traditional machine learning techniques such as SVM and decision trees. Deep learning has achieved unprecedented results in various fields, including signal processing [143, 144], computer vision [145], speech and audio processing [146], and natural language processing [147, 148].
This book introduces basic machine learning concepts and applications for a broad audience that includes students, faculty, and industry practitioners. We begin by describing how machine learning prov
The supervised learning paradigm [34] is perhaps the most popular method in the machine learning community. In supervised learning, one has access to the ground truth for samples contained in the training, validation, and test data sets. Ground truth represents “true” or “correct” labels for the input dataset. Expert help may be needed to obtain the correct labels for the data (medical image labeling, for example). The ML model is “trained” using a labeled input dataset termed training data. Once the model achieves the desired performance on training data, the trained model is then used to perform inference on unseen data. The data that has not been used for training and thus unseen by the model is termed test data.
This book is designed for use as a textbook for a one semester Signals and Systems class. It is sufficiently user friendly to be used for self study as well. It begins with a gentle introduction to the idea of abstraction by looking at numbers—the one highly abstract concept we use all the time. It then introduces some special functions that are useful for analyzing signals and systems. It then spends some time discussing some of the properties of systems; the goal being to introduce the idea of a linear time-invariant system which is the focus of the rest of the book. Fourier series, discrete and continuous time Fourier transforms are introduced as tools for the analysis of signals. The concepts of sampling and modulation which are very much a part of everyday life are discussed as applications of the these tools. Laplace transform and Z transform are then introduced as tools to analyze systems. The notions of stability of systems and feedback are analyzed using these tools. The book is divided into thirty bite-sized modules. Each module also links up with a video lecture through a QR code in each module. The video lectures are approximately thirty minutes long. There are a set of self study questions at the end of each module along with answers to help the reader reinforce the concepts in the module.
We turn our attention now to the response of continuous time linear time invariant systems. Given the impulse response h (t) of a continuous time linear time invariant system (Figure 9.1) we can find the output y (t) for any input x (t) by solving the convolution integral.
The Fourier transform is an immensely useful tool for understanding signals. However, when it comes to analyzing systems there is a limitation to the use of the Fourier transform. Recall that the Fourier transform exists for all signals satisfying the Dirichlet conditions. Namely:
