Simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) can reconstruct indirect links when the direct link is blocked for millimeter wave (mmWave) communications. Moreover, the spectral efficiency of non-orthogonal multiple access (NOMA) can benefit from the highly directional transmission of mmWave. To capitalize on the complementarity of STAR-RIS, mmWave, and NOMA technologies, in STAR-RIS-assisted mmWave-NOMA downlink communication systems we propose the spectral efficiency optimization problem, considering the differences of the paired users’ channels. The problem is subject to maximum available transmission power, the constraints of the amplitude and phase-shift coefficients of each STAR-RIS element’s transmission and reflection, and so on. To solve the non-convex problem, the alternating iterative optimization algorithms (AIOAs) based on the successive convex approximation (SCA) and the semidefinite programming (SDP) are proposed, respectively. For the SCA-based AIOA algorithm, the effective channel gain is written as the sum of the square of the variable’ real part and the square of the variable’ imaginary part, which is further transformed into a convex expression after proving that the constraint on the sum of the amplitude coefficients of each STAR-RIS element’s transmission and reflection can be relaxed. For the SDP-based AIOA algorithm, the original problem is transformed into a SDP problem by introducing the auxiliary variables. Moreover, simulation results demonstrate that the two proposed algorithms improve spectral efficiency compared to existing algorithms, and NOMA is superior to orthogonal multiple access (OMA) in terms of spectral efficiency.