On Developing and Deploying Large-File Upload Services of Personal Cloud Storage

Personal cloud storage is rapidly gaining popularity. A number of Internet service providers, such as Google and Baidu, entered this emerging market and developed a variety of cloud storage services. These ubiquitous services allow people to access personal files all over the world at anytime. With the prevalence of mobile Internet and rich media on web, more and more people use cloud storage for storing working documents, music, private photos and movies. Nevertheless, the size of the media files is often beyond the upper limit that a normal form-based file upload allows hence dedicated large-file upload services are required to be developed. Although various cloud vendors offer versatile cloud storage services, very little is known about the detailed development and deployment of the large-file upload services. This paper proposes a complete solution of large-file upload service, with the contributions in manyfold: Firstly, we do not limit the maximum size of a large file that can be uploaded. This is extremely practical to store huge database files from ERP tools. Secondly, we developed large-file upload service APIs that have very strict verification of correctness, to reduce the risk of data inconsistency. Thirdly, we extends the service developed recently for team collaboration with the capability of handling large files. Fourthly, this paper is arguably the first one that formalizes the testing and deployment procedures of large-file upload services with the help of Docker. In general, most large-file upload services are exposed to the public, facing security and performance issues, which brings much concern. With the proposed Docker-based deployment strategy, we can replicate the large-file upload service agilely and locally, to satisfy massive private or local deployment of KDrive. Finally, we evaluate and analyze the proposed strategies and technologies in accordance to the experimental results.