Data Ingestion, Transformation, and Analytics

This chapter focuses on the following services: Amazon Web Services (AWS) Data Lake; AWS Glue; AWS Transfer Family; Kinesis Video Streams; Kinesis Data Streams; and Kinesis Data Firehose. A data lake is a centralized database that collects and stores massive amounts of structured and unstructured data from any number of places. AWS Transfer Family lets one to transfer data into and out of S3 and Elastic File System using three time‐tested, familiar protocols: File Transfer Protocol; Secure Shell File Transfer Protocol; and File Transfer Protocol over SSL. Amazon Kinesis is a collection of services that let one to collect, process, store, and deliver streaming data. Kinesis can perform real‐time ingestion of gigabytes per second from thousands of sources, making it perfect for things like audio and video feeds, application logs, and telemetry data.