Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.



Real-Time & Tornado

Kiril Savino

on 21 May 2010

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of MongoNYC

Big Fat JSON Pipeline {_type: "stream",
_id: ObjectId("xxxx"),
game: {$ref: "game", $id: ObjectId("zzzz")},
scorer: {$ref: "user", $id: ObjectId("uuuu")},
events: [{$ref: "event", $id: ObjectId("yyyy")}]} GET /stream/xxxx
POST /stream/xxxx/save
DELETE /stream/xxxx REST(ish) API JSON Python Dictionaries Django Models Stupid SQL Relational Model The Original Stack 16,000 Queries. ...optimization... "Only" 500 Queries. WTF? play play_participant play_types play_properties play_participant_role play_property_type sport AHA! {_type: "event"
_id: ObjectId("aaaaaaaa"),
participants: [{player: {$ref: "player", $id: ObjectId("bbbbbb"), roles: ["pitcher"]}],
code: "SB",
properties: {...}} Performance != Scaling - marshalling
- calculations
- "business logic" Cache Everything! Denormalization Rules obvious choice: memcached non-obvious option: MongoDB? Now This is Caching {_id: "/stream/xxxxxxxx",
json: "{...}",
expires: "Fri May 21st...",
dependencies: ["/team/yyyyyyy", "/player/zzzzz"]} db.cache.ensure_index(("dependencies", 1))

db.cache.remove({"dependecies": {"$in": ["team/yyyyyyy"]}) yay! R W oh... Original GameStream <meta http-equiv="refresh" content="60" />
Entire page processed through Django templates. Optimization Strategies Baseline - 9 seconds for 9 inning game.
Optimizing data processing - 3 seconds.
Cache on read - 3 seconds on cache miss, 50 ms on cache hit. High enough volume, lots of people will experience a cache miss. Long Polling Using AJAX to retrieve data.
If there is data, return it immediately.
No data, leave the connection open and pend.
When data appears, send it and close the connection.
Client processes and then immediately opens a new connection to wait again.
url: '/push/stream/1234?index=10,
dataType: 'json',
type: 'GET',
error: ajaxError,
timeout: 0,
success: processEvents
}); Wait New data comes in Return JQuery + AJAX Tornado "push" Cheap reads $.ajax({
url: '/push/stream/1234?index=0,
dataType: 'json',
type: 'GET',
error: ajaxError,
timeout: 0,
success: processEvents
Returns immediately Tornado and Nginx 1000s of Open Connections

NGINX epoll > Apache New GameStream Cache on write Where does MongoDB fit in here? - Denormalization
- Caching
- Write Speed Caveats Mobile Browsers
Internet Explorer Learning MongoDB {_id: ObjectId("xxxx"),
event_block: "...lots and lots of json...",
save_date: "2010-05-21 12:08:00",
stream_id: ObjectId("yyyy")} ensure_index([("stream_id", 1)]) find(
{'stream_id': self.stream_id,
).sort('save_date', 1) ensure_index([("stream_id", 1), ("save_date", 1)]) $.ajax({
url: '/push/stream/1234?index=11,
dataType: 'json',
type: 'GET',
error: ajaxError,
timeout: 0,
success: processEvents
}); Synchronous Polling Still Fast Kiril Savino, @kirilnyc, kiril@gamechanger.io
Jerry Hsu, jerry@gamechanger.io
Full transcript