Hybrid Analog-Digital Filter Designs for mmWave Multipair Two-Way Relaying Systems

In this correspondence, we propose an efficient hybrid beamforming algorithm for multipair two-way relaying systems with a large number of antenna elements. We first investigate an asymptotic behavior of the fully digital (FD) case where the number of radio frequency (RF) chains equals that of antenna elements. Our analysis reveals that the optimal RF filters can be determined by the array response vectors in the asymptotic region without consideration of the baseband (BB) filters. From this observation, we provide a new RF filter design based on the Gram-Schmidt process. Also, in order to manage interpair interference, we design BB filters based on the weighted sum minimum mean square error for given RF filters. Our simulation results demonstrate that as the number of antenna elements grows, our method achieves the optimal FD performance with much reduced computational complexity.