On a Phase Transition in General Order Spline Regression

In the Gaussian sequence model
$Y= \theta _{0} + \varepsilon $
in
$\mathbb {R}^{n}$
, we study the fundamental limit of statistical estimation when the signal
$\theta _{0}$
belongs to a class
$\Theta _{n}(d,d_{0},k)$
of (generalized) splines with free knots located at equally spaced design points. Here
$d$
is the degree of the spline,
$d_{0}$
is the order of differentiability at each inner knot, and
$k$
is the maximal number of pieces. We show that, given any integer
$d\geq 0$
and
$d_{0}\in \{-1,0,\ldots,d-1\}$
, the minimax rate of estimation over
$\Theta _{n}(d,d_{0},k)$
exhibits the following phase transition:
$\begin{aligned} \inf _{ \widetilde {\theta }}\sup _{\theta \in \Theta _{n}(d,d_{0}, k)} \mathbb {E}_\theta \lVert \widetilde {\theta } - \theta \rVert _{}^{2} \asymp _{d} \begin{cases} k\log \log (16n/k), & 2\leq k\leq k_{0},\\ k\log (en/k), & k \geq k_{0}+1. \end{cases} \end{aligned}$
The transition boundary
$k_{0}$
, which takes the form
$\left \lfloor{ (d+1)/(d-d_{0}) }\right \rfloor + 1$
, demonstrates the critical role of the regularity parameter
$d_{0}$
in the separation between a faster
$\log \log (16n)$
and a slower
$\log (en)$
rate. We further show that, once encouraging an additional ‘
$d$
-monotonicity’ shape constraint (including monotonicity for
$d = 0$
and convexity for
$d=1$
), the above phase transition is removed and the faster
$k\log \log (16n/k)$
rate can be achieved for all
$k$
. These results provide theoretical support for developing
$\ell _{0}$
-penalized (shape-constrained) spline regression procedures as useful alternatives to
$\ell _{1}$
- and
$\ell _{2}$
-penalized ones.