Succinct data: Extreme compression for ODK forms making digital field assessment practical during disasters by minimizing data transmission size and costs

Field communications play a vital role in disaster relief and other humanitarian responses. However, the acute phase of these events is frequently characterized by scarcity of communications capacity, confounding the efforts of responders to develop the situational awareness that would enable them to maximize the effectiveness of their activities. Access to high-bandwidth global connectivity is often limited to a few sites where existing infrastructure is still functional, or where temporary access has been rapidly deployed. The need to gather data, e.g., standardized field assessment reports, is not limited to these locations. A related problem is that any available communications links are likely to be overloaded. Therefore it is desirable to create a system that can make use of hand-held and low-cost global communications options. One such family of devices is those that provide access to the Iridium constellation's Short Burst Data capability, effectively providing a satellite based SMS service. However making use of such devices is problematic because of the extremely limited bandwidth they offer, and high cost-per-byte. New Zealand Red Cross and the Serval Project [1 - 6] at Flinders University have responded to this need by creating the Succinct Data concept, wherein the salient information from a field report is automatically extracted, compressed and transmitted via any available means, with the full report following when the collection device, or another collection device it comes into contact with, eventually reaches a high-bandwidth communications link. In this way rich field assessment reports can be collected, with the highest priority information transmitted immediately, such as numbers of persons requiring assistance and basic assessment of dwellings, and with the additional rich information following when possible, such as detailed commentary, images and other media where appropriate. In this paper we describe the work to date on implementing Succinct Data, and the creation of a new compression scheme that can compress ODK form instances to less than 1/12th of their size - less than half the size achievable using gzip.