Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.


NoSQL - Maslow's Hierarchy of Reads (and Writes)

There are too many ways to read and write data, and polyglot persistence can get confusing, here is a guide to help... focus on memcached, SimpleDB and Cassandra

Adrian Cockcroft

on 30 April 2011

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of NoSQL - Maslow's Hierarchy of Reads (and Writes)

NoSQL - Maslow's Hierarchy of Reads (and Writes) by @adrianco
Netflix Cloud Architect
(with added discussion of Cassandra monitoring) Self - In Memory
Hold for the duration of a request
Cache repeated use of the same data
Selfish - no sharing, no copies
Reads and writes in microseconds
No network or disk access Esteem - Shared Memory
Share between requests
Maintain session state
Share across instances
Reads and writes in milliseconds
One network access, no disk access Belonging - Partitioned
Shard across shared memories
Improved scalability and availability
Reads and writes in milliseconds
One or two network accesss, no disk access Safe - Persisted
Write to disk, read from disk
Write copies to several disks
Lots of options - CAP Theorem
What happens when it breaks?
Most NoSQL support a range of options
Would you feel safer with MySQL? Durable
Data copied to multiple availability zones
Example S3, SimpleDB
Wide Area Multi-Region Support
Cassandra's Rack and Datacenter Aware Topology
Business Continuity Planning
Data on tape or copied to a different cloud NoSQL Needs Human Needs Self - In Memory Esteem - Shared Memory Belonging - Partitioned Safe - Persisted Durable - Backed-up Java
Concurrent Hashmap single memcached sharded memcached
hash routed at source http://www.java-tips.org/java-se-tips/java.util/how-to-use-of-hashmap.html http://memcached.org/ Consistent Writes (CP)
Guarantee that data is always in sync
Refuses writes when it's failed
May refuse reads if it can't be sure
Membase default behavior Available Writes (AP)
Guarantee that writes always succeed
Guarantee that reads always succeed
Data may not be latest or in sync
Inconsistencies are repaired eventually
SimpleDB, Cassandra, Riak normal case Apache Cassandra Mark Atwood - My-NoSQL Redux SimpleDB sharded membase read requests
vbucket routed at source memcached sharded membase write requests
vbucket routed at source, sideways copied and persisted SimpleDB Persists in the Cloud
behind the scenes it replicates sideways and persists Cassandra read/write requests (quorum)
connection to ring routes to location of data and replicates/reads desired number of copies Cassandra RF=2 Cross Region Cassandra read/write requests (local quorum)
connection to ring routes to location of data
async read-repair checks that all copies are good
async writes to remote non-quorum copies Local Cassandra Remote Cassandra wide area
link Cassandra Cassandra Backup and Restore via S3
snapshot, copy compressed SSTables to S3
Restore starts with S3, restores to new Cassandra instances. Full dump and incremental Disclaimers
This is a snapshot viewed through a lens
April 2010 - expect rapid change...
Stuff I have worked with (not all NoSQL)
My imperfect understanding of things
Stuff I care about - running NoSQL on AWS
Comments/corrections welcome
@adrianco Monitoring Cassandra
Underlying Data Sources
Java JMX and Garbage Collection
Disk space and performance
CPU, Network and Memory
Datastax.com OpsCenter
Cassandra Specific GUI
Application, Java and System Metrics http://prezi.com/veagqhsz38u8/nosql-maslows-hierarchy-of-reads-and-writes/ New in 3.2, Cassandra Support
(Screenshots by AppDynamics) AppDynamics Dashboard (old version) Single Region Deployment Patterns at Netflix
Tomcat based side process
Integrates with naming/discovery service
Manages token assignment
Manages instance replacement
Implements full and incremental backups
Configuration pattern
Replication Factor 3, Quorum writes
RackAware Topology mapped to AWS AZ Multi Region Deployment Patterns at Netflix
Tomcat based side process
Manages mapping to EIP
Cross region public/private IP issues
Configuration pattern
Replication Factor 3, Local Quorum writes
Secure inter-node transport
Datacenter Topology mapped to AWS Region
Full transcript