MongoEngine + Relational + Privileges = Goodness!

• not at all scalable without hard-coded SQL magic and lowest-level adding to Django and modding of Tastypie
• no object level privileges
• no offline working/syncing
• no `private clusters` for corporates through partitioning
• forced to optimize the wrong stuff in the wrong domain
• started to `feel wrong` (aka `got smelly`)

• manages changes to relations (1:1, 1:n, m:n)
• automatically updates the other side (atomic saves/deletes).
• memoizes document fields to monitor differences
• creates hooks for custom code when relational fields change
• correctly handles and compares mixed types and states(!) of mongoengine relations
• only 1 tiny extension to mongoengine's declarative syntax: just using the `related_name` argument sets everything up.

• transparently caches documents in a thread-safe, self-attaching DocumentCache on the request object
• setup couldn't be easier: DocumentCache( request ) suffices
• interact with the cache through `add`, `get`, `remove`, or let the overridden internals take care of it: accessing a related field with <document>.<fieldname> uses the cache

MongoEngine

+ Relational

+ Privileges

+ TastyMongo

= Goodness!

Progressive Planning

Marcel van den Elst

Why `Pyramid`?

Pyramid

• 'cause `they done things right`!
• better Request/Response objects (Webob) compared to Django
• nice traversal API
• we actually use 90% of Pyramid, and used only about 20% of Django

Why `TastyMongo`?

TastyMongo features

TastyMongo

(except for its gross inefficiency)

• we had a RESTful API (TastyPie) that worked like a charm with our backbone-relational framework
• we LOVED that functionality and separation of concerns!
• but Tastypie is Django-Tastypie and we ditched Django
• so we took all the good parts and created `TastyMongo`!

because we had no install base and legacy to support

again: no joins in NoSQL!

• RESTful API for mongoengine (HTTP conform)
• very similar to the great Django-Tastypie, but more consistent and simpler API
• declarative syntax (like Django's or mongoengine's)
• Django-style cross-relational filtering (`books__author__name`) that constructs the required separate NoSQL queries
• error handling (without forcing HTML down AJAX throat...)
• small functions that can easily be overridden where required
• managed relations between resources (mapping to Documents)
• Document field introspection and mapping to Resource fields
• custom URI schemas
• pagination
• throttling
• authentication
• (de)serialization (JSON, XML, CSV, whatever you write)

Why `mongoengine-privileges`?

mongoengine-privileges features

mongoengine-privileges

• mongoengine-privileges leverages mongoengine-relational
• mixin with straightforward API

• we need document-level and field-level permissions
• we couldn't find an existing mixin that correctly and transparently handled object level privileges across relations

may_foo(...)

Why `mongoengine-relational`?

mongoengine-relational features

mongoengine-relational

no joins!

• we have 2-way relations (1:n, n:m)
• we need denormalization
• we need caching
• we need triggers when stuff changes
• we don't want headaches

on_change(...)

on_change_foo()

hard-coded brittleness, inconsistency, boiler-plate

ObjectId strings, un-dereferenced ObjectIds, dereferenced objects, cached objects and DBRefs

bonus: DocumentCache

Why MongoEngine?

for our inherently highly related data

• NoSQL = NoSchema = No Deal
• small, transparent, active, mature, well-documented, readable, responsive authors
• Django is not all bad: it seemed easier to port our existing code to something with similar syntax

turned out to be true after we created TastyMongo, mongoengine-relational and mongoengine-privileges

Why adopt MongoDB?

Why abandon Django?

As it turned out we hardly used any of Django's features and most legacy stuff (non-AJAX tailored) actually got in the way

• started out as Django + PostgreSQL app (2009)
• Progressive Planning use case doesn't fit
• RESTful API through Django-Tastypie
• Wonderful. but. very. inefficient.

Django-Tastypie is lovely until you need performance

• everything is JSON (BSON)!
• deployment is a breeze
• web-scalable and flexible out of the box (Replica Sets)
• very active development + growing community
• `feels` much better and more human(e)

Why Progressive Planning?

Progressive Planning

(not about `resources` with statistical deviations that need to be measured and controlled)

• computers can do more than digitize paper-era-thinking
• `real` projects are about creative, excited, communicative, opinionated, multi-skilled human beings trying to achieve stuff together.
• `real` projects are volatile, highly interdependent, multi-constrained, interactive, networked, constantly changing, uncertain, stressful
• Progressive Planning supports REAL PROJECTS, i.e. YOU.