Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

ActiveMQ at Hightail

No description
by

Lawrence Cheng

on 10 January 2014

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of ActiveMQ at Hightail

ActiveMQ at Hightail
Today’s Agenda
Motivation for transitioning to ActiveMQ
Why ActiveMQ versus other message brokers?
ActiveMQ configuration
Network of Brokers
Balancing on Failures/Additions
Transitioning: SonicMQ/ActiveMQ split config
Integration: Spring Integration
Performance Gains
Additional Information
Evaluation of Message Brokers
Besides evaluating the SonicMQ broker, these brokers were also reviewed:

ZeroMQ
JBoss HornetQ
SpringSource RabbitMQ
Apache ActiveMQ
Apache Qpid

Motivation
Existing SonicMQ installation:

an older version (upgrade path complex and costly)
not very perfomant (low througput, high latency)
not scalable (active master, passive slave setup)
limited options for APIs and transport
monitoring and management consoles not very informative and easy to use

ZeroMQ
designed for performance
raw
no durability by default
no AMQP or JMS interfacing

JBoss HornetQ
polished and well documented
but no longer supported since 2010
no AMQP support

SpringSource RabbitMQ
pretty good flexibility/configurability
mirrored queue provides for good HA
all meta data replicated to all mirrors
won't scale to as many brokers as ActiveMQ
fast, especially for transactional messages
only supported via AMPQ (language/OS agnostic) so no JMS or JMX support by default

Apache ActiveMQ
very flexible/configurable (transport, storage, etc)
AMQP support since version 5.8
JMS/JMX support built-in
Comparable speed (slightly slower) to RabbitMQ, but horizontally scales to a larger number of broker nodes
Allows for easiest transition for existing systems relying upon JMS for interfacing

Master-Slave vs Network-of-Brokers
Balancing on Failure
Balancing on Addition
Split Config Example
{
"AllSplitConfigurations" : {
"applicationConfigMap" : {
"Alcatraz" : {
"destinationsSplitConfigList" : [ {
"destinations" : [ "AlcatrazAnalyzeMailQueue", "AlcatrazUpdateScoreQueue" ],
"split" : 50
}, {
"destinations" : [ "AlcatrazBadMailQueue" ],
"split" : 20
}, {
"destinations" : [ "AlcatrazGoodMailQueue" ],
"split" : 30
} ]
}
}
}
}

Split Config Administration
(web service)
http://<host>:<port>/<container>/jms-admin/split-configuration
methods:


GET
= get current config from the host's local in-memory configuration

POST
= posts (creates/updates) the config (in POST body) into global memcached AND into the local host's in-memory configuration

PUT
= puts (updates) the current config from global memcached into the local host's in-memory configuration

header:

Content-Type
: application/json
Spring Integration
Spring Integration Details
Motivation:
clean decoupling of code and messaging layer
easily swap between split multiple broker configuration and single broker configuration
ability to use other protocols and wire formats (e.g. AMQP and OpenWire)

Features:
compact library (small memory and disk footprint)
Backed by SpringSource community

A GatewayProxyFactoryBean will be used as the interface point for code initiated flows.

A message driven channel adapter and a service activator are used for incoming flows.
Spring Integration Example
(incoming)
Spring Integration Example
(outgoing)
<integration:gateway id="messageSendProxy"
service-interface="com.yousendit.common.util.jms.MessageSendProxy"
default-request-channel="scoreQueueOutChannel" />

<integration:channel id="scoreQueueOutChannel" />

<integration:channel id="activeMQScoreQueueOutChannel" />
<integration:channel id="sonicMQScoreQueueOutChannel" />

<integration:filter input-channel="scoreQueueOutChannel"
ref="spamAnaylzerBrokerResolver" method="useActiveMQ"
output-channel="activeMQScoreQueueOutChannel"
discard-channel="sonicMQScoreQueueOutChannel"/>

<int-jms:outbound-channel-adapter id="activeMQScoreQueueJmsOutAdapter"
channel="activeMQScoreQueueOutChannel"
connection-factory="activemqConnectionFactory"
destination-name="${spamanalyzer.jms.update.score.queue}"/>

<int-jms:outbound-channel-adapter id="sonicMQScoreQueueJmsOutAdapter"
channel="sonicMQScoreQueueOutChannel"
connection-factory="sonicmqConnectionFactory"
destination-name="${spamanalyzer.jms.update.score.queue}"/>
<int-jms:message-driven-channel-adapter id="activeMQAnalyzeQueueJmsInAdapter"
connection-factory="activemqConnectionFactory"
destination-name="${spamanalyzer.jms.analyze.mail.queue}"
concurrent-consumers="${spamanalyzer.jms.concurrent.consumers}"
max-concurrent-consumers="${spamanalyzer.jms.max.concurrent.consumers}"
max-messages-per-task="${spamanalyzer.jms.max.messages.per.task}"
send-timeout="${spamanalyzer.jms.receive.timeout}"
channel="analyzeQueueInChannel" />

<int-jms:message-driven-channel-adapter id="sonicMQAnalyzeQueueJmsInAdapter"
connection-factory="sonicmqConnectionFactory"
destination-name="${spamanalyzer.jms.analyze.mail.queue}"
concurrent-consumers="${spamanalyzer.jms.concurrent.consumers}"
max-concurrent-consumers="${spamanalyzer.jms.max.concurrent.consumers}"
max-messages-per-task="${spamanalyzer.jms.max.messages.per.task}"
send-timeout="${spamanalyzer.jms.receive.timeout}"
channel="analyzeQueueInChannel" />

<integration:channel id="analyzeQueueInChannel" />

<integration:service-activator
input-channel="analyzeQueueInChannel" ref="spamanalyzer" method="processMessage" />
Split Config Architecture
Additional Information
ActiveMQ broker and software configuration information can be found here:
http://confluence.yousendit.com/display/platform/ActiveMQ+at+Hightail

Server Configuration:
http://confluence.yousendit.com/display/platform/ActiveMQ+at+Hightail#ActiveMQatHightail-ServerConfiguration

ActiveMQ broker settings:
http://confluence.yousendit.com/display/platform/ActiveMQ+at+Hightail#ActiveMQatHightail-ActiveMQbrokersettings

Memcache storing split config:
http://confluence.yousendit.com/display/platform/ActiveMQ+at+Hightail#ActiveMQatHightail-Memcachestoringsplitconfig

Spring Integration:
http://confluence.yousendit.com/display/platform/ActiveMQ+at+Hightail#ActiveMQatHightail-SpringIntegration

Test parameters:
50 producer threads
20 consumers threads
1000 messages
200 byte per messages
Results
Full transcript