Transition metal compounds with two-dimensional triangular lattice, such as LiVO2, are of particular interest, because they exhibit peculiar structural and magnetic behavior involving the frustration present in these oxides. An orbital ordering transition occurs near 500 K, which leads to a suppression of magnetic moment below the phase transition temperature Tt. We synthesized a series of Mg-doped LiVO2 single-phase samples and reported their structural and magnetic properties. The samples were characterized by x-ray diffraction, scanning electron microscope, differential scanning calorimetry, electrical resistivity, magnetic susceptibility, and specific heat measurement. For Mg-doped samples Li1-xMgxVO2 (x=0, 0.05, 0.10, and 0.15), the structural analysis show that, with increasing x, the lattice constants change monotonously; in the a-b plane, the lattice expands; while in the c-axis direction, the lattice is compressed. Substitution of Li with Mg ions results in the degeneration of two-dimensional characteristics and the distortion of the VO6 block, which significantly influence magnetic properties. The magnetic phase transition temperature falls with increasing x. The Mg-dopants play an important role on breaking the original moment equilibrium and suppressing the magnetic frustration.