Getting back on the road: Reformatting Flash memory on-board the Mars Exploration Rover Opportunity

In early 2011, NASA's Mars Exploration Rover (MER) Opportunity began experiencing intermittent errors when writing data products to Flash memory. Similar errors occurred with increasing frequency in early 2013 and many were accompanied by unexpected resets of the rover's flight computer (warm reboots). Depending on timing, warm reboots occasionally caused communication faults. Although these errors subsided in mid-2013, they returned at an increasing rate in mid-2014. The MER team developed techniques to mitigate the effects of infrequent warm reboots but these became ineffective as the rate and severity of the errors increased. Faced with these more frequent errors, the MER team decided to reformat the Flash memory. Using lessons learned from reformatting the Flash file system on the Spirit rover and implementing some new techniques, the MER team was able to reformat Flash and resume science operations much more quickly than anticipated. This paper will discuss the history of Flash related anomalies, suspect cause, initial operational mitigations, and the ultimate Flash reformat strategy and rover recovery activities.