This paper deals with novel formulas for calculating the mutual inductance and the magnetic force between two coaxial coils in air comprising a circular coil of the rectangular cross section and a thin disk coil (pancake). The mutual inductance and the magnetic force are expressed over the complete elliptical integrals of the first and second kinds, Heuman's lambda function, and three terms that must be solved numerically. All singular cases have been resolved analytically. We also give another approach as comparative method that is based on the filament method where all conductors are approximated by the set of Maxwell's coils. The expressions are obtained over the complete elliptical integrals of the first and second kinds. These new consistent expressions are accurate and simple for useful applications. All results obtained by different approaches are in excellent agreement.