The Low Frequency Non-Separable Transform (LFNST) in VVC Draft v5 applies two different sizes of non-separable secondary transform kernel, the one with size 16x16 is applied for blocks with either width or height smaller than 8, the other one with size 16x48 is applied for blocks with both width or height greater than or equal to 8. To unify the LFNST design, in this contribution, it is proposed to use only the 16x48 kernels as in VTM-5.0 for LFNST. The reported benefits include: 1) the memory for storing the 16x16 LFNST kernels, i.e., 2KB memory, can be saved, 2) the calculation of LFNST is simplified because the switching of transform sizes is not needed, and 3) both VTM-5.0 and Spec text can be simplified. Experimental results reportedly show that, compared to VTM-5.0, the proposed method gives 0.05%, 0.01% luma BD-rate changes for All Intra (AI) and Random Access (RA) configurations, respectively, with 3% overall encoding time saving for AI. Compared to CE6-2.1a, the proposed method gives 0.00%, -0.01% luma BD-rate changes for AI and RA configurations, respectively.