Dictionary learning algorithms have received widespread acceptance when it comes to data analysis and signal representations problems. These algorithms consist of two stages: the sparse coding stage and dictionary update stage. This latter stage can be achieved sequentially or in parallel. In this work, the maximum likelihood approach is used to derive a new approach to dictionary learning. The proposed method differs from recent dictionary learning algorithms for sparse representation by updating all the dictionary atoms in parallel using only one eigen-decomposition. The effectiveness of the proposed method is tested on two different image processing applications: filling-in missing pixels and noise removal.
In the synthesis model signals are represented as a sparse combinations of atoms from a dictionary. Dictionary learning describes the acquisition process of the underlying dictionary for a given set of training samples. While ideally this would be achieved by optimizing the expectation of the factors over the underlying distribution of the training data, in practice the necessary information about the distribution is not available. Therefore, in real world applications it is achieved by minimizing an empirical average over the available samples. The main goal of this paper is to provide a sample complexity estimate that controls to what extent the empirical average deviates from the cost function. This estimate then provides a suitable estimate to the accuracy of the representation of the learned dictionary. The presented approach exemplifies the general results proposed by the authors in [1] and gives more concrete bounds of the sample complexity of dictionary learning. We cover a variety of sparsity measures employed in the learning procedure.
PAC-Bayes generalization bounds offer a theoretical foundation for learning classifiers with low generalization error and predicting their performance on unseen data. Current formulations implicitly assume that the relative cost of misclassifying a positive or negative example is reflected by the class skew in the training dataset. We present a learning approach based on minimizing an asymmetric generalization bound that enables PAC-Bayesian model selection under a class-specific performance constraint.
Recently, a large amount of signal processing technology research on applications associated with music is being carried out. Sound synthesis, in particular, is one of the most interesting research themes. In this paper we propose a new approach to mathematically modeling harmonic-timbre structure with multi-beta-distribution (MBD). This probabilistic distribution has the advantage of enabling one to easily express varied timbre-structure with only a few parameters. We will define MBD itself, and present a method of estimating MBD parameters. Some experimental results are presented to discuss the performance of this method.
In this work we show that one can construct in an algorithmic way the normal Morse data for Morse functions on singularities of complex analytic varieties. For this construction we rewrite stratified Morse theory using the construction for getting morphing data of multidimensional rational functions.
Novelty detection, one-class classification, or outlier detection, is typically employed for analysing signals when few examples of "abnormal" data are available, such that a multi class approach cannot be taken. Multivariate, multimodal density estimation can be used to construct a model of the distribution of normal data. However, setting a decision boundary such that test data can be classified "normal" or "abnormal" with respect to the model of normality is typically performed using heuristic methods, such as thresholding the unconditional data density, p(x). This paper describes two principled methods of setting a decision boundary based on extreme value statistics: (i) a numerical method that produces an "optimal" solution, and (ii) an analytical approximation in closed form. We compare the performance of both approaches using large datasets; from biomedical patient monitoring and jet engine health monitoring, and conclude that the analytical approach performs novelty detection as successfully as the "optimal" numerical approach, both of which outperform the conventional method.
In this contribution, we provide a theoretical study of two hypothesis tests allowing to detect the presence of an unknown transmitter using several sensors. Both tests are based on the analysis of the eigenvalues of the sampled covariance matrix of the received signal. The generalized likelihood ratio test (GLRT) derived is analyzed under the assumption that both the number K of sensors and the length N of the observation window tend to infinity at the same rate: K/N rarr c isin (0, 1). The GLRT is compared with a test based on the condition number used which is used in cognitive radio applications. Using results of random matrix theory for spiked models and tools of large deviations, we provide the error exponent curve associated with both test and prove that the GLRT outperforms the test based on the condition number.
A major problem in applying Minimum Variance Distortionless Response (MVDR) beamformer, also known as Capon beamformer, in operational systems is the potential sensitivity to mismatch between actual and assumed models for the array response and/or the received signals used to derive the beamformer. Typical mismatches that have been addressed are DOA mismatch, sensor gain and phase perturbation, and sensor position perturbation. The subject of mutual coupling, for the most part, has been ignored in the development of algorithms for robust beamforming. In this paper, we present an extension of a previously developed robust MVDR beamformer that takes into account the mutual coupling between elements of the antenna array and the uncertainties in the array response and in the mutual coupling explicitly. We present several experimental results using measurements that were carried out in an anechoic chamber.
This paper evaluates the spatial focusing performance of the Time Reversal (TR) method in a multiple input single output (MISO) situation with experimental data. For the target user, the received signal is related to the channel link from the transmitter to the target receiver. The received power increases as we enlarge the array size, and it increases by 15.6dB with 16 transmit antennas, compared to the single link. For the eavesdropper, the received signal is determined by both the channel link from the transmitter to the target user and the channel link from the transmitter to the eavesdropper. In terms of unbalanced branch power, the minimum average Eavesdropping Margin (EM) and Peak Eavesdropping Margin (EMPeak) are 14dB and 13dB from 16 transmit elements.
Energy-based detection and estimation are crucial in sensor networks for sensor localization, target tracking, etc. In this paper, we present novel Gaussian approximations that are applicable to general energy-based source detection and localization problems in sensor networks. Using our approximations, we derive receiver operating characteristics curves and Cramer-Rao bounds, and we provide a factorized variational Bayes approximation to the location and source energy posterior for centralized or decentralized estimation. When the source signal and the sensor noise have uncorrelated Gaussian distributions, we demonstrate that the envelope of the sensor output can be accurately modeled with a multiplicative Gaussian noise model, which results in smaller estimation biases than the other Gaussian models typically used in the literature. We also prove that additive Gaussian noise models result in negatively biased speed estimates under the same signal assumptions, which can be circumvented by the proposed approximations.
This paper investigates how to choose the optimum tap-length and decision delay for the decision feedback equalizer (DFE). Although the feedback filter length can be set as the channel memory, there is no closed-form expression for the feedforward filter length and decision delay. In this paper, first we analytically show that the two dimensional search for the optimum feedforward filter length and decision delay can be simplified to a one dimensional search, and then describe a new adaptive DFE where the optimum structural parameters can be self-adapted
This paper deals with minimal bounds in the Bayesian context. We express the minimum mean square error of the conditional mean estimator as the solution of a continuum constrained optimization problem and, by relaxing these constraints, we obtain the bounds of the Weiss-Weinstein family. Moreover, this method enables us to derive new bounds as the Bayesian version of the deterministic Abel bound
The Kullback information criterion (KIC) is a recently developed tool for statistical model selection. KIC serves as an asymptotically unbiased estimator of the Kullback symmetric divergence, known as J-divergence. A corrected version for KIC denoted by KIC C have been also proposed to correct the bias of KIC. This version tends to overfit when the sample size increases. In this paper we propose an alternative to KIC C , the KIC U criterion which is unbiased estimator of the Kullback's symmetric divergence. It provides better model choice than KIC C for moderate to large sample size
Sequential tests outperform fixed sample size tests by requiring fewer samples on average to achieve the same level of error performance. The sequential probability ratio test (SPRT) has been suggested by Wald (1947) for sequential binary hypothesis testing problems. SPRT recursively calculates the likelihood of an observed data stream and requires this likelihood to be stored in memory between samples. In this paper we study the design of sequential detection tests under memory constraints. We derive the optimal sequential test in the case where only a quantized version of the likelihood can be stored in memory. An application of the proposed techniques is large scale sensor networks where price and communication constraints dictate limited complexity devices, which store and transmit concise representations of the state of nature.
We consider the general problem of sampling from a sequence of distributions that is defined on a union of sub-spaces. We will illustrate the general approach on the problem of sequential radial basis function (RBF) regression where the number of kernels is variable and unknown. Our approach, which we term trans-dimensional sequential Monte Carlo (TD-SMC), is based on a generalisation of importance sampling to spaces of variable dimension. In the spirit of P. Del Moral and A. Doucet (2002) we augment the target parameter space at the current time step with an auxiliary space corresponding to the parameters at the previous time step. This facilitates the design of efficient proposal distributions, which can then be formulated as moves from the auxiliary parameter space to the target parameter space, lending our algorithm its sequential character. These proposals are very general, and may include within model moves to update parameters, and trans-dimensional birth or death moves to add or remove parameters when appropriate. From this perspective our approach is reminiscent of the reversible jump Markov Chain Monte Carlo (RJ-MCMC) algorithm [P.J. Green, 1995].
A new method for target tracking of multiple points on an object by using particle filter with its novel importance function is proposed. The assumptions are such that the number of points is fixed and known, and the association between points of object and observed points are unknown. There exists missing and clutter in observation process where which observation corresponds to them are also unknown. The main difficulty of this problem is the formidable number of combinations in the association. The novel importance function using an idea of soft gating makes the problem tractable in a proper framework of particle filter. Simulation experiment illustrates the performance of the method.
The spatial time-frequency distributions (STFDs) have been developed and successfully applied to high-resolution direction-of-arrival (DOA) estimations and blind recovery of the source waveforms. In [Y. Zhang et al., Aug. 2003], the spatial polarimetric time-frequency distribution (SPTFD) was introduced as a platform for space-time processing for nonstationary source signals with different polarization properties. Based on SPTFD, polarimetric time-frequency MUSIC (PTF-MUSIC) has been proposed for DOA estimation of non-coherent signals. In this paper, the use of the PTF-MUSIC is extended to the coherent signal environment using spatial and polarization averaging. Simulation results are presented to show the effectiveness of the aforementioned decorrelation techniques.
In a time-hopping ultra-wideband (UWB) system, a receiver is required to operate over a large bandwidth. Conventional receivers consist of sliding correlators to correlate received signal with a template signal. Such a time-domain waveform detection scheme is essentially a single-user detection method, whose performance has been observed to degrade in a multiple access environment. In this paper, estimation of all users' signals is carried out in the frequency domain. First, Fourier transform (FT) is applied to the received time-domain signal. Then users' information in terms of modulation delays is estimated by frequency-domain multiuser detection (MUD) methods. Frequency diversity at the receiver can be explored for performance enhancement by generating multiple sinusoidal waveforms at different frequencies and processing transformed data in parallel. Its promising feature for combating narrow band interference is expected.
Linear representations and linear dimension reduction techniques are very common in signal and image processing. Many such applications reduce to solving problems of stochastic optimizations or statistical inferences on the set of all subspaces, i.e. a Grassmann manifold. Central to solving them is the computation of an exponential map (for constructing geodesies) and its inverse on a Grassmannian. Here we suggest efficient techniques for these two steps and illustrate two applications: (i) For image-based object recognition, we define and seek an optimal linear representation using a Metropolis-Hastings type, stochastic search algorithm on a Grassmann manifold, (ii) For statistical inferences, we illustrate computation of sample statistics, such as mean and variances, on a Grassmann manifold.
In the uplink of a long-code CDMA system, base station knows spreading codes of all serviced users. Given propagation delays, a blind adaptive CMA-based approach, with the aid of a set of MMSE-like constraints parameterized by channel-like vectors, is proposed to detect all users' symbols simultaneously. As by-products, the channel-like vectors are also obtained for all users. Since the constraints involve covariance of the received data, which is time varying and can not be obtained by traditional sample average, we thus propose to approximate it using estimated signature matrix and noise power. Simulation results show satisfactory performance of the proposed method.
