https://github.com/aws-samples/aws-blog-firehose-custom-prefixes-for-s3-objects Destination configuration default Intermediate S3 prefix (Optional) To use the default prefix for Amazon S3 objects, leave this option
Category: Kinesis
AWS Stream from DDB to Datalake
Streaming Amazon DynamoDB data into a centralized data lake We use an AWS Key Management Service (AWS KMS) key for server-side encryption
Twilio Event Stream
Trust that your events will be queued if your system goes down and delivered as soon as they’re back up
AWS Kinesis to Eventbridge Pipe to Kinesis
Tutorial: Set up from Console https://www.eliasbrange.dev/posts/debug-dynamodb-with-eventbridge-pipes-and-cloudwatch/ Sample in pipe: { “kinesisSchemaVersion”: “1.0”, “partitionKey”: “1”, “sequenceNumber”: “49590338271490256608559692538361571095921575989136588898”, “data”: “SGVsbG8sIHRoaXMgaXMgYSB0ZXN0Lg==”, “approximateArrivalTimestamp”: 1545084650.987,
AWS Firehose to Elastic Search
How many shards are needed in a new elastic search domain Kinesis firehose destination to elastic search Firehose delivers to
AWS Kinesis Notes
Basic Examples Simple cloudformation example Don’t use EventSourceMapping. Just add event trigger with lambda function definition. Simple read Simple write
AWS Stream Batch Size and Batch Window
DynamicDB and Kinesis stream serverless settings Blog on parallel processing
AWS Kinesis
AWS research workshops What is Kinesis Amazon Kinesis is a managed, scalable, cloud-based service that allows real-time processing of streaming large amount