Predictive coding (PC) networks are a biologically interesting class of neural networks. Their layered hierarchy mimics the reciprocal connectivity pattern observed in the mammalian cortex, and they can be trained using local learning rules that approximate backpropagation (Bogacz, 2017). However, despite having feedback connections that enable information to flow down the network hierarchy, discriminative PC networks are not typically generative. Clamping the output class and running the network to equilibrium yields an input sample that usually does not resemble the training input. This letter studies this phenomenon and proposes a simple solution that promotes the generation of input samples that resemble the training inputs. Simple decay, a technique already in wide use in neural networks, pushes the PC network toward a unique minimum two-norm solution, and that unique solution provably (for linear networks) matches the training inputs. The method also vastly improves the samples generated for nonlinear networks, as we demonstrate on MNIST.

Predictive coding (PC) networks are a biologically interesting class of neural networks. Their layered hierarchy mimics the reciprocal connectivity pattern observed in the mammalian cortex, and they can be trained using local learning rules that approximate backpropagation [Bogacz, 2017]. However, despite having feedback connections that enable information to flow down the network hierarchy, discriminative PC networks are not generative. Clamping the output class and running the network to equilibrium yields an input sample that typically does not resemble the training input. This paper studies this phenomenon, and proposes a simple solution that promotes the generation of input samples that resemble the training inputs. Simple decay, a technique already in wide use in neural networks, pushes the PC network toward a unique minimum 2-norm solution, and that unique solution provably (for linear networks) matches the training inputs. The method also vastly improves the samples generated for nonlinear networks, as we demonstrate on MNIST.

Analog Gaussian watermarking systems in which transmitted watermarks are reproduced with certain distortion at receivers are addressed in this paper. Achievable distortion regions of private analog Gaussian watermarking systems and public analog Gaussian watermarking system are determined with respect to the mean-squared error distortion measure, with and without joint compression, respectively. The results show that, interestingly, if watermark sources and host signal sources are independent, then the achievable distortion regions of private analog Gaussian watermarking systems and public analog Gaussian watermarking systems coincide.

In this correspondence, a new digital watermarking scenario is studied, where watermarks and covertexts are correlated with each other. A necessary and sufficient condition is derived under which a watermark is first embedded, and can later be recovered with high probability at the output of a public watermark decoder after the watermarked signal is disturbed by a fixed memoryless attack channel. Interestingly, the condition also implies that the Shannon separation theorem does not hold in this scenario.

Reversible watermarking system with correlated watermarks and covertexts is studied. Necessary and sufficient conditions are derived such that the receiver can reproduce watermarks and covertexts within some distortions to original ones respectively, even watermarked signals are disturbed by an attacker. Furthermore, some interesting special cases are explored.

In order to accommodate high data-rate applications, there is a significant interest in extending space–time (ST) coding, originally proposed for frequency-nonselective channels assumed known to the receiver, to unknown ISI (inter-symbol interference) channels. In this paper, we consider the problem of blind channel identification and linear multiuser receiver design for ST-coded CDMA (code-division multiple-access) systems operating in frequency-selective fading channels. We investigate the identifiability conditions and present a subspace-based blind channel estimator for such systems. To assess the performance of the proposed and other potential estimators, we derive an unconditional Cramér–Rao bound (CRB) which, similarly to the proposed blind channel estimator, does not assume the knowledge of the transmitted information symbols. We discuss various linear multiuser detection schemes that can be used in conjunction with the proposed channel estimator for symbol demodulation in ST-coded CDMA systems. Numerical examples are presented to illustrate the performance of the proposed channel estimator and linear detectors in multipath Rayleigh fading channels.

In digital watermarking, a watermark is embedded into a covertext in such a way that the resulting watermarked signal is robust to certain distortion caused by either standard data processing in a friendly environment or malicious attacks in an unfriendly environment. The watermarked signal can then be used for different purposes ranging from copyright protection, data authentication, fingerprinting, to information hiding. In this thesis, digital watermarking will be investigated from both an information theoretic viewpoint and a numerical computation viewpoint. From the information theoretic viewpoint, we first study a new digital watermarking scenario, in which watermarks and covertexts are generated from a joint memoryless watermark and covertext source. The configuration of this scenario is different from that treated in existing digital watermarking works, where watermarks are assumed independent of covertexts. In the case of public watermarking where the covertext is not accessible to the watermark decoder, a necessary and sufficient condition is determined under which the watermark can be fully recovered with high probability at the end of watermark decoding after the watermarked signal is disturbed by a fixed memoryless attack channel. Moreover, by using similar techniques, a combined source coding and Gel'fand-Pinsker channel coding theorem is established, and an open problem proposed recently by Cox et al is solved. Interestingly, from the sufficient and necessary condition we can show that, in light of the correlation between the watermark and covertext, watermarks still can be fully recovered with high probability even if the entropy of the watermark source is strictly above the standard public watermarking capacity. We then extend the above watermarking scenario to a case of joint compression and watermarking, where the watermark and covertext are correlated, and the watermarked signal has to be further compressed. Given an additional constraint of the compression rate of the watermarked signals, a necessary and sufficient condition is determined again under which the watermark can be fully recovered with high probability at the end of public watermark decoding after the watermarked signal is disturbed by a fixed memoryless attack channel. The above two joint compression and watermarking models are further investigated under a less stringent environment where the reproduced watermark at the end of decoding is allowed to be within certain distortion of the original watermark. Sufficient conditions are determined in both cases, under which the original watermark can be reproduced with distortion less than a given distortion level after the watermarked signal is disturbed by a fixed memoryless attack channel and the covertext is not available to the watermark decoder. Watermarking capacities and joint compression and watermarking rate regions are often characterized and/or presented as optimization problems in information theoretic research. However, it does not mean that they can be calculated easily. In this thesis we first derive closed forms of watermarking capacities of private Laplacian watermarking systems with the magnitude-error distortion measure under a fixed additive Laplacian attack and a fixed arbitrary additive attack, respectively. Then, based on the idea of the Blahut-Arimoto algorithm for computing channel capacities and rate distortion functions, two iterative algorithms are proposed for calculating private watermarking capacities and compression and watermarking rate regions of joint compression and private watermarking systems with finite alphabets. Finally, iterative algorithms are developed for calculating public watermarking capacities and compression and watermarking rate regions of joint compression and public watermarking systems with finite alphabets based on the Blahut-Arimoto algorithm and the Shannon's strategy.

In this project, the researcher successfully developed a highway speed vehicle-mounted thermoplastic pavement marking material (TPMM) thickness measurement device based on the research results in Project 0-4882, Refinement of a Non-Contact Method to Determine the Thickness and Uniformity of Application for Thermoplastic Pavement Marking Material. The device can be used in routine Texas Department of Transportation (TxDOT) project monitoring practice. In Project 0-4882, a compact, lightweight pushcart laser device was successfully developed for measuring TPMM thickness. In comparison with the laser device developed in Project 0-4882, two new versions of vehicle mount laser thickness measurement devices have been developed in this project. The first new version, a point laser device, uses three synchronized point laser units to solve the inaccuracy problem caused by the slope of the road. The second new version, a scanning laser device, was developed based on the auto-synchronized laser scanning principle. This new scanning device can scan in the transverse direction and measure the average cross-section TPMM thickness. Since the scanning laser covers an area wider than the TPMM, the driver doesn't need to strictly follow the stripe, and the measurement accuracy will be increased on a bumpy or sloping road. In this report, the laser triangulation and auto-synchronized scanning laser method is briefly reviewed. Then, the specifications of the laser devices and field test results are presented.

In project 7-3969, the researchers had successfully developed a new, cheaper high speed texture laser for texture data collection. Based on the project 7-3969, 15 laser-based units which are capable of measuring pavement macro-texture at highway speed are delivered in this project. Compared with the old laser device developed in Project 7-3969, the new version developed in this project has significantly reduced the size, weight, and noise levels, and has improved the standoff distance, measurement range and accuracy. In this report, the laser triangulation method is briefly reviewed. Then, the improvement over the old laser device and the calibration procedures are also presented.

Based on the idea of the Blahut-Arimoto algorithm for computing channel capacities and rate-distortion functions, two iterative algorithms are developed for numerical computation of the compression and watermarking rate regions of joint compression and private watermarking systems with finite alphabets. The convergence of the algorithms developed is also proved

A joint compression and public digital watermarking system with correlated watermark source and covertext source (M, S) is investigated in this paper. More specifically, for a given distortion level D between S and a watermarked signal X and a given compression rate R c for X, the watermark source M can be fully recovered with high probability at the end of a public watermark decoder after the watermarked signal is disturbed by a fixed memoryless attack channel p(y\x) if and only if H(M) les maxp(x,u\m,s) min{R c - I(M, S; U, X) + I(M; U, Y), I(U; Y) - I(U; M, S) + I(M; U, Y)}, where the maximization is taken over all auxiliary random variables U (with finite alphabet U) and X jointly distributed with (M, S, X, Y) according to p(m, s, u, x, y) = p(m, s)p(u, x\m, s)p(y\x) such that Ed(S, X) les D

A new problem termed as combined source coding and watermarking is investigated, where an independently and identically distributed (iid) source M correlated with an iid host signal S is to be transmitted by embedding M into S. A necessary and sufficient condition is derived under which the source M can be recovered with high probability from a watermarked signal disturbed by a fixed memoryless attack channel p(y|x). Specifically, it is shown that M can be recovered with high probability if and only if H(M) is less than or equal to max{I(U;Y) - I(U;M,S) + I(M;U,Y) : U,X}, where the maximization is taken over all auxiliary random variables U and X such that the distortion between S and X is less than or equal to a prescribed distortion level D. In particular, the result implies that the Shannon separation theorem can not be extended to this case, that is, it is still possible to transmit M reliably even when H(M) is strictly greater than the watermarking capacity. A similar result is also established in the case of combined source coding and Gelfand-Pinsker channel coding

Thermo-plastic tape (TPT) provides delineation on highways around the world. The thickness of TPT on pavement is a very important parameter to control the quality of TPT, calculate durability of TPT, and provide information for the maintenance and replacement of TPT. Traditionally, the thickness measurement is conducted by using pre-embedded plates and measuring the thickness of TPT after spraying of the TPT marking materials. This method is labor intensive and cannot obtain a continuous-thickness profile. Developing an automatic thickness measurement system for TPT marking materials is critical to pavement management and public safety. The measurement system developed in this paper uses laser triangulation technique to detect the thickness of TPT. A dedicated digital laser signal processing circuit is developed to restore thickness information. The thickness measurement system provides continuous real-time thickness measurement of TPT. Lab and field tests under various conditions with TPT marking materials on real pavement surfaces were conducted. The test results showed that the measurement system is capable of reaching the resolution of 5 mils on pavement. The developed system for thickness measurement of TPT has a 267 KHz working frequency, which is the highest among similar devices. The high speed allows the system to provide higher accuracy and more flexibility in various applications.

The developed thermoplastic pavement marking material thickness detector is a vehicle-mounted device that measures the thickness of marking materials on the pavement surface by using point lasers and scanning laser technology. The device consists of two parts: a hardware system to measure the pavement marking by using the laser triangulation technique and the auto-synchronized scanning technique, and a software package to analyze and process the measured data. Part 1 of the user guide gives a general description of the system, describes the components of the point laser system and hardware operation procedures, discusses the software operations and data interpretation, and concludes with comments on troubleshooting. Part 2 of the user guide gives a general description of the system, describes the configuration of the scanning laser system and hardware operation procedures, discusses the software operations and data interpretation, and concludes with comments on troubleshooting.

Pavement marking materials provide delineation on highways around the world. The condition of the marking materials is very important for the driver's safety as well as the comfort and the driving expenses. Currently thermoplastic pavement marking materials (TPMM) are widely used in states. Measuring the thickness of TPMM on pavement is an essential index to monitor the contractors, calculate durability of marking materials, and provide better information for the pavement marking evaluation. In recent years to measure the thickness of TPMM, a procedure involving pre-embedded plates sprayed with the marking materials has been widely accepted. This method is labor intensive, and cannot obtain a continuous-thickness profile. Therefore there are demands to develop a high-speed automatic measuring system for determining the thickness and uniformity of marking materials. In this paper, a laser range sensor based on auto-synchronized laser scanning is proposed for the thermoplastic pavement marking material thickness measurement. Compare to classical triangulation method, this approach doesn't scarify the system resolution for large field of view and it is more suitable for highway speed measurement. To achieve high speed measurement, PSD (Position Sensitive Detector) is used in the prototype system instead of CCD (Charge Couple Device) in traditional auto-synchronized system. The standoff distance and transverse scan range of the prototype system both are 1 foot. The lab test results show that the prototype system can measure the thermoplastic type thickness with error in 5mil at laser scanning rate up to 50Hz.

Calculation of watermarking capacities of private Laplacian watermarking systems with the magnitude-error distortion measure under fixed attacks is addressed. First, in the case of an additive Laplacian attack, a nice closed-form formula for the watermarking capacities is derived, which involves only the distortion level and the parameter of the Laplacian attack. Second, in the case of an arbitrary additive attack, a general, but slightly more complicated formula for the watermarking capacities is given. Finally, calculation of the joint compression and private watermarking rate region of Laplacian watermarking systems with an additive Laplacian attack is considered.

This paper considers the problem of interference suppression in GPS coarse/acquisition (C/A) signals. Particularly, an anti-jam receiver is developed utilizing the unique repetitive feature of the GPS C/A-signal. The proposed receiver does not require the knowledge of the transmitted GPS symbols or the satellite positions. It utilizes the repetition of the Gold code within each navigation symbol to simultaneously suppress the interference of all satellites.