Data Deduplication with Encrypted Big Data Management in Cloud Computing

Cloud storage service is an efficient solution for increasing storage needs of organizations and individuals. In order to ensure security and data privacy, users may encrypt their data before uploading to cloud. In such cases same or different users may up-load same data in encrypted form and it results in the existence of duplicated data. Data deduplication is a technique designed to identify and eliminate redundant data. When users are uploading data in encrypted form, then deduplication is a challenging problem. This project proposes a deduplication system in cloud storage that supports big data deduplication also. The sytem is based on symmetric encryption and re-encryption. The system aims at paragraph-level deduplication of text data files. A token will be generated for each paragraph in a file before uploading to cloud. When a new request for uploading is received, system checks the existence of duplicate files using these tokens. If the file is already exists in the cloud, then system rejects uploading, else system will perform deduplication using re-encryption algorithm.