A novel equalization and precoding joint optimization algorithm is proposed in dual hop multiple input multiple output (MIMO) relay communication systems. This solution combines the equalization for the first phase and precoding for the second phase in relay system, therefore the joint optimization can be equaled with the unique optimization for the product matrix of detection and precoding, which avoid the non convergence problems in iterating process, compared with iterative optimization, the complexity burden of this algorithm is greatly reduced. And from simulation results the performance is only a little degraded. Therefore, when complexity and performance are considered, this algorithm reached a better compromise.