AWS Serverless Streaming vs. Messaging for Data Processing
Streaming for Data Processing
For asynchronous data processing, message services such as Amazon SQS or SNS might be used.
It depends on the type of processing and the type of data you're collecting.
The primary element in messaging systems is a single message.
Once messages have been used, they are erased from messaging services.
To bypass a record in streaming, you must provide error handling in your function.
Streaming vs. Messaging For Data Processing Video
W3schools.com collaborates with Amazon Web Services to deliver digital training content to our students.
Messaging for Data Processing
Streams are data buffers that don't care what their users do with them.
Regardless of user's action, data on the stream remains for a certain duration before disappearing.
You need to provide error handling.
The table below shows the differences between streaming and messaging.
| Messaging | Streaming |
|---|---|
| Individual messages are the core unit, and message rates vary | Stream of messages is typically continuous |
| Once a message has been read, it is removed | Data is kept on the stream for a certain amount of time. |
| For failures, you have to set up retries and dead-letter queues | The message is retried until it succeeds or runs out of time |
