Constellation design has been previously addressed by assuming that there is a linear equalizer at the receiver side. However, the widely linear equalizer is well known to outperform the linear one with no significant complexity increase; we derive optimum and suboptimum techniques for constellation design in presence of such an equalizer. The proposed techniques adapt the circularity properties of the transmitted signals to the specific channel to be equalized; their performance analysis shows that also the simplest suboptimum procedure provides significant improvements over a fixed-constellation scheme.

This thesis deals with the multiple-input multiple-output (MIMO) equalization with widely linear (WL) filtering. The WL filtering outperforms the linear one in presence of rotationally variant signals, with no significant complexity increase. It has been used in order to compensate the transmitter and receiver in-phase and quadrature (IQ) imbalance and to design the optimum constellation transmitted over a channel known at the receiver, according to minimum mean square error (MMSE) criterion. Moreover, with reference to OFDM-OQAM systems, it has been proposed an ad hoc MMSE WL-DF equalizer which performs a selective MIMO equalization according to the characteristics of the MIMO channel describing the system. The synthesis of the equalizer has been carried out by an algorithm which exhibits minimum storage requirements. The equalizer has proved efficient for OFDM-OQAM systems, achieving the best trade-off between performance and complexity.

The wide spread of low-cost fabrication technologies gives rise to unpredictable imperfections associated with the analog stages which perform the frequency conversion. More specifically, it is well known that such stages suffer from the in-phase (I) and quadrature (Q) imbalance. In this paper, with reference to a single-carrier time-dispersive noisy channel, we address the receiver design when both the transmitter and the receiver are affected by the IQ imbalance, and we propose to resort to the widely linear filtering in order to improve the performances of the conventional minimum mean square error (MMSE) linear equalizer. The results show that the adoption of WL filters allows one to achieve considerable gains both in terms of MMSE and symbol error rate, with a limited increase in the computational complexity of the equalization stage.