Research and Design of Sensor Data Management System Based on Distributed Storage

Sensor data is one of the basic source of big data in the Internet of Things(IoT). It is the data that is sensed, measured, and transmitted by sensors with real-time and dynamic characteristics. With the development of the IoT and big data technologies, new industries such as smart transportation, smart cities, and smart agriculture have gradually formed, resulting in an increasing demand for large-scale data analysis and calculations. However, the storage architecture based on relational databases has been unable to meet the current needs in terms of data storage and data read and write speed. Therefore, this paper mainly studies the sensor data management system based on distributed storage, builds a Hadoop cluster, proposes a storage structure that combines the relational database MySQL with the distributed database HBase, designs a data migration module to satisfy real-time sensor data monitoring and massive data storage requirements. In order to meet the needs of enterprises in actual production, SSM framework technology is used to build a sensor data management platform, covering the functions of system management, sensor data management, data exception management, sensor data real-time monitoring, sensor node status monitoring, data statistical analysis, etc. The platform has good security and reliability, and can effectively improve the management efficiency of the enterprise. Finally, the experiment provides that the system has obvious advantages in data upload and query ability compared with the traditional storage architecture. It provides a good data environment for data analysis and data mining in the future.