On joint modulation design in two-user non-orthogonal multiple access channels

In this paper, we consider the joint signal optimization in two-user non-orthogonal multiple access (NOMA) channels with finite-alphabet constellations. In the proposed design, the constellation of the second user is optimally rotated with the assumption that joint maximum-likelihood detection is employed at the receiver. With M antennas at the receiver, closed-form expressions for the optimal angles as a function of M, the channel conditions and the users' power levels are derived for 4-QAM constellation. Using rotated constellation, the performance of NOMA in terms of bit-error-rate is shown to be improved. The proposed approach is applicable in the uplink NOMA where users share the same time/frequency channel or in the downlink NOMA where users are multiplexed in the power domain.