A biological paradigm of versatile locomotion and effective motion control is provided by the polychaete annelid worms, whose motion adapts to a large variety of unstructured environmental conditions (sand, mud, sediment, water, etc.), and could thus be of interest to replicate by robotic analogs. Their locomotion is characterized by the combination of a unique form of tail-to-head body undulations (opposite to snakes and eels), with the rowing-like action of numerous lateral appendages distributed along their long segmented body. Focusing on the former aspect of polychaete locomotion, computational models of crawling and swimming by such tail-to-head body undulations have been developed in this paper. These are based on the Lagrangian dynamics of the system and on resistive models of its interaction with the environment, and are used for simulation studies demonstrating the generation of undulatory gaits. Several biomimetic robotic prototypes have been developed, whose undulatory actuation achieves propulsion on sand and other granular unstructured environments. Extensive experimental studies demonstrate the feasibility of robot propulsion by tail-to-head body undulations in such environments, as well as the agreement of its qualitative and quantitative characteristics to the predictions of the corresponding computational models.