On the convergence of linear stochastic approximation procedures

Many stochastic approximation procedures result in a stochastic algorithm of the form h/sub k+1/=h/sub k/+1/k(b/sub k/-A/sub k/h/sub k/), for all k=1,2,3,... . Here, {b/sub k/,k=1,2,3...} is a R/sup d/-valued process, {A/sub k/,k=1,2,3,...} is a symmetric, positive semidefinite Re/sup d/spl times/d/-valued process, and {h/sub k/,k=1,2,3,...} is a sequence of stochastic estimates which hopefully converges to h/sup /spl Delta//=[lim/sub N/spl rarr//spl infin//1/N/spl Sigma//sub k=1//sup N/EA/sub k/]/sup -1/ {lim/sub N/spl rarr//spl infin//1/N/spl Sigma//sub k=1//sup N/Eb/sub k/} (assuming everything here is well defined). We give an elementary proof which relates the almost sure convergence of {h/sub k/,k=1,2,3,...} to strong laws of large numbers for {b/sub k/,k=1,2,3,...} and {A/sub k/,k=1,2,3,...}.