Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.


Easy Message Service

Recipes for using reliable messaging for integration and performance. Amazon SQS and SNS are the backbone; for more information, see: http://developer.berkeley.edu/apidocs/ems

George Atala

on 29 October 2013

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Easy Message Service

Reliable Messaging
Boost Response Time
Typical Scenarios
Slow response times are common in two cases:
Traffic exceeds capacity
The service is a long-running process by design
Examples of long-running processes:
online order processing and airline bookings involve many steps and distributed service providers
Shorten the response time perceived by the user
How to Do It — High Level:

When a request is received, add it to a message queue, and return a response immediately to the user
Process the requests from the queue, return responses to a channel that's established with the client
The client (web or mobile) can check for responses asynchronously (pull message), and optionally update its display
Alternatively, the client can be notified when a response is ready (push message)
Shortening Response Time — More Details

For each "job queue" there has to be a corresponding "response queue"
Use a unique correlation key for each request/response pair to ensure responses are matched to the correct requests
In "pull" mode, clients query the response queue on the correlation key
In "push" mode, notification is sent to the client, which could be an email message or SMS in case of a user, or a webhook if the client is a system.
Send and respond
to Events

Scale to an Order
of Magnitude

(without adding hardware)
A Typical App Bottleneck
An application's database connection pool may hold 30 connections, but occasionally the app gets hundreds of concurrent users.
As a result, users can experience many types of problems:
Failed requests
Refused connections
Loss of data

Ultimately, the application can crash altogether.
The application can be modified to accept far more requests than it can process synchronously
How to Do It — More Details
Boosting Capacity without New Hardware
Add Resilience
& Fault Tolerance

Exchange data Securely
Decouple Systems
& Layers

Automate Business Processes
Closer Look at SQS & SNS
Click a Topic
& Use Arrow Buttons

in Full Screen

Recipes for Integration & Performance
A Business Process Manager (BPM) orchestrates the work:
Assigns tasks to process workers in the right order — sequentially or in parallel
Enforces process logic and handles errors
Decides whether a process instance was successful, and either 'commits' the result, or 'rolls back' the process
Process Management — More details
Use a task queue for each worker, which guarantees asynchronous delivery of tasks
Tasks in each process need to carry a common identifier so they can be grouped across workers
The process manager can send tasks sequentially, or broadcast them to multiple workers simultaneously
Use a response queue for each worker. Success can be based on responses from all workers being received, this is equivalent to all-or-nothing transactions
Systems often need to act when events occur, for instance, to perform a task or to notify users.
This is the essence of "event-driven architecture"
Messaging is the ideal tool to push event notifications reliably
Event producers publish event information on a topic
Event consumers subscribe to the topic to be notified of events.
Consumers can subscribe and receive notifications over a variety of protocols; in particular, EMS supports the following subscriptions:
HTTP, using a callback URL (webhook)
SQS queues

For more information: http://amzn.to/1546H4j
The asynchronous sharing of information has traditionally relied on FTP or secure FTP
Messaging provides an overheadless mechanism to share information, publicly or privately, coupled with a notification capability
Sharing Data — High Level
Create a queue for each consumer; queue names are globally unique, which makes queues private
For added security, use the encryption service to encrypt the payload
To send the same data to many consumers, use a topic & queue combination to 'multicast'
Note: message size is limited
Best Practice for SQS Queues
Use an exclusive queue per consumer
Always set a visibility time out that works for the consumer
The consumer should delete a message after consuming it
When done with a queue,
delete it
Set message attributes as needed: message size and retention period
Encrypt the payload as needed
If a consumer retrieves a message but fails to delete it, the message will be retrieved again: consumers should be able to handle processing a message more than once.
Not all SQS features are supported in EMS
Best Practice for SNS Topic
Use a topic to multicast the same message to many consumers, or to achieve a 'push notification' effect
Notifications to HTTP and SQS transports are supported in EMS
delete topics when done using them
Encrypt the payload as needed
Not all SNS features are supported in EMS
This is achieved by queuing the tasks in a reliable channel. The implications are:
The app can accept as much as an order of magnitude more requests
The app is much more resilient to DoS conditions
Once a request is received it won't be lost, even if the app crashes
This will achieve scalability, not necessarily response time. See recipe on performance.
Since requests will be processed asynchronously,
there are two options for handling client responses.
Client waits for response
Client receives confirmation immediately, disconnects.
Appropriate if wait time is short enough
Clients keep their positions in a reliable queue
The app is more scalable & resilient
Better option if wait time is long
Client gets confirmation of request received, then disconnects
Client can check back for responses,
or can be notified when response is ready
Huge scalability gains
Keep in Mind
SQS is distributed, here's what that means: http://amzn.to/1dwq8Iz
Messages in a queue are not ordered
Polling multiple times is necessary to retrieve all messages from a distributed queue
A quick response to the client is good for the client, but also critical for freeing up application resources
Typical Scenarios
A web or mobile application may act as a gateway to a complex process.
Example: amazon.com or expedia.com
An order placed on Amazon is processed by several discrete service providers: inventory management, packaging, shipping, payment processing, restocking... etc. It could take days to complete.
The challenge is to transfer the appropriate sub-task to each service provider, and to coordinate all sub-tasks as one unit of work
Process Management — High Level
Messaging is the communication medium between the BPM and the process workers:
Reliable: requests and responses are never lost
Asynchronous: systems are not impacted by delays elsewhere
Standard protocol: systems using hybrid technologies can exchange messages
Typical Scenario
Systems exchange information either synchronously (request / response), or asynchronously.
Managing SFTP and the relevant authorizations is non-trivial
Typical Scenario
students sign up for the waiting list of a class. When a seat becomes available, the event is published, and an enrollment process can react to the event and enroll students from the list
Publishing Events — High Level
Best in full screen
Decoupling Systems
Decoupling such systems is a powerful paradigm
Diverse technologies can still inter-operate
Changes in one system don't affect others, hence systems can evolve independently
Failure in one link doesn't impact all links
Decoupling can be asynchronous, which automatically enables throttling, load balancing, and fail-over
In an enterprise, many systems have to inter-operate
Decoupling via Messaging
Systems can communicate via synchronous or asynchronous interfaces, with varying degrees of decoupling.
An asynchronous API can be implemented as an exchange of asynchronous messages via SQS
Load Balancing, Failover, & Throttling with Queues
Each thread consumes messages as fast as its capacity allows, which achieves the
Load Balancing
behavior across all threads.
Queues offer an excellent mechanism for adding robustness to a distributed architecture.
If one thread fails, the remaining threads will continue processing messages, which achieves the
Queues 'absorb' all messages, no matter how fast they arrive, and allow consumers to process them at their own speed. This achieves the
To leverage this: a message consumer uses multiple threads (or servers) to retrieve messages from the same queue.
First: What's EMS?
Easy Message Service (EMS) is a RESTful messaging API that provides frictionless access to a robust messaging infrastructure.
EMS Features
Self-serve: users get instant access at: https://developer.berkeley.edu/apidocs/ems
HTTP only: no need for special messaging clients, EMS is pure HTTP
Amazon infrastructure: all messages are delivered over Amazon SQS and SNS
Developer friendly: use the the 'playground' to try the API before writing any code (see link above)
EMS is only available on the UC Berkeley network

In our first example, we'll see a request handler and backend processor operating synchronously.
Throughput and response time are entirely dependent on the processing speed of the backend.
The result is a large delay between when a client makes a connection and when the request is completed (about 25 seconds, in this case).
Shortening Response Time — Demo

In this next example, the request handler puts the information to be processed on a queue and immediately disconnects with the client.
The backend processor reads messages from the queue as fast as it can and processes the message contents.
Throughput increases by a factor of 6 as a result. The request handler is no longer bound by the processing speed of the backend.
Extra backend processors can be started to handle large loads without impacting the request handler.
Shortening Response Time — Demo

This demo shows a request handler which puts requests on a queue which two backend request processors read from to process requests.
One backend processor is faster than the other, but reading requests from a queue gives a natural load balancing behavior. Each processor proceeds at its own pace.
When one of the processors is terminated, the request handler and the other processor proceed with no problem (failover).
Load Balancing, Failover, & Throttling — Demo
Getting Started
Send & Receive Messages on an Exclusive Queue
Note: message size is limited
The Easy Message Service (EMS)
is a new API for reliable messaging
Access EMS at: https://developer.berkeley.edu/apidocs/ems

EMS is free and self-serve for UC Berkeley users
EMS is all HTTP, no other clients are necessary.
All messages are delivered via Amazon SQS and SNS.
1. Create a new queue, a globally unique ID is generated for it:
curl -X POST “https://apis-internal.berkeley.edu/ems/queue”
2. Share the secret ID with the target message consumer
3. Producer sends messages to the queue:
curl -X POST -d “Hello World” “https://apis-internal.berkeley.edu/ems/queue/{queue_id}/message”
4. Consumer reads messages from the queue:
curl -X GET “https://apis-internal.berkeley.edu/ems/queue/{queue_id}/message”
5. Once a message is consumed, the consumer deletes it.
Encrypt & Decrypt Messages
1. Generate a strong encryption key:
curl -X GET “https://apis-internal.berkeley.edu/ems/crypto/random-key”
2. Share the secret key with the target message consumer; it can be shared as a message on an exclusive queue, then delete the queue.
3. Encrypt messages using the secret key:
curl -X POST -H “Crypto-Key: {secret key}” -d “Encrypt This” “https://apis-internal.berkeley.edu/ems/crypto/encryption”
4. The consumer decrypts received messages using the same key:
curl -X POST -H “Crypto-Key: {secret key}” -d “Encrypted Text” “https://apis-internal.berkeley.edu/ems/crypto/decryption”
Broadcast to Many Consumers with Topics
1. Create a topic.
3. Subscribe each of the queues to the topic
curl -X POST “https://apis-internal.berkeley.edu//ems/topic/{topic_id}/subscription?queue_id=myqueue_id”
4. Publish messages to the topic, they will get delivered to all subscribed queues
In many cases, it's necessary to broadcast the same message to many consumers. This is achievable by combining queues with topics.
2. Create a queue for each consumer.
Send Notifications with Topics
1. The consumer implements a webhook (a web resource) to receive the notification.
Queues deliver messages when the consumer requests them.
This is known as a 'message pull'.
2. Subscribe the URL of the webhook to the topic, similarly to subscribing a queue
Topics offer a 'message push' option, which notifies the consumer when a message is available. This option uses a 'webhook' to notify the consumer.
The following 4 slides are a quick start guide
Full transcript