AWS Batch Processing Solution Diagram (using AWS Glue)
2022-01-11 awsaws-batch-processingdata-engineeringsolution-diagram
This diagram shows a typical batch processing solution on AWS with Amazon S3, AWS Lambda, Amazon Glue and Amazon Redshift:
- Amazon S3 is used to store staging data extracted from source systems on-premises or on-cloud.
- AWS Lambda is used to register data arrival in S3 buckets into ETL frameworks and trigger batch process process.
- Amazon Glueis then used to integrate data like merging, sorting, filtering, aggregations, transformations and load the data.
- Amazon Redshift is then used to store the transformed data.
This diagram is forked from AWS Batch Processing Solution Diagram