Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.


Going beyond Magento speed limits without HHVM and Varnish Mage Titans

No description

Ivan Chepurnyi

on 25 January 2016

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Going beyond Magento speed limits without HHVM and Varnish Mage Titans

Going Beyond
Magento speed limits*
* - without HHVM and Varnish
Why without HHVM and Varnish?
Compiles your code
Caches its output
Both does not solve a source of the issue
The issue is always in the code
Just take a look at this simple example...
Layout caches its structure based on current page handles
So almost every page has a cache footprint
Most of the developers treat Magento Core as a Holy Grail, that must never be touched...
But it MUST be touched for a good reason!
But what if layout is constructed
from pre-assembled parts?
So I've touched that Magento Core part!
... and covered that with tests
Layout handles are compiled into PHP files
Process of parsing XML is separated from building a page
You can create now your own layout file nodes
Lets see how it compares in numbers...
Gatling tool
Hypernode Vagrant box from @ByteInternet
Official Magento Sample Data
Redis Cache Backend
3 runs each for 60 seconds
Each 1st run is a warm up
First Run (Standard Layout)
First Run (Compiler Layout)
Mean time: 599ms
Standard Deviation: 302ms
Mean time: 488ms
Standard Deviation: 244ms
Second Run (Standard Layout)
Second Run (Compiler Layout)
Mean time: 566ms
Standard Deviation: 304ms
Mean time: 395ms
Standard Deviation: 177ms
So I decided to experiment more with Magento core!
... for every filter there is a cache footprint for every option combination?
Did you know that on category page...
... every filter loads data separately, even if data comes from the same table?
... terrible SQL queries are executed when you click on filters?
... everything is so tightly coupled, that you cannot separate facet from attribute?
But what if...
... facets would be separated from attributes
... all options would be retrieved at once and only for available combinations
... product collection would not be used for those calculations and loaded at the latest stage
So I decided to experiment...
... and cheated a little bit ...
... by using Sphinx for as storage
Lets see how it affects the numbers...
All Catalog Pages (Sphinx)
All Catalog Pages (Magento)
Mean time: 232ms
Standard Deviation: 86ms
Mean time: 395ms
Standard Deviation: 177ms
Category Pages (Sphinx)
Category Pages (Magento)
Mean time: 223ms
Standard Deviation: 51ms
Mean time: 463ms
Standard Deviation: 150ms
But Magento still has plenty of bottlenecks to solve...
Layout Compiler Module
Would be happy to provide you a quote for Sphinx integration
Load Test Results
Vagrant Box from ByteInternet (Sphinx pre-installed)
But that is not real life situation...
Sample data is too small...
So I created such database:
22,000 total SKU
~ 1000 items in each category
3 Store Views
And loaded it with 150 visitors.
With such a behavior:
Home Page
Category Page
Category Page with applied filter
Search Page
Filtered Category Page (Magento) Run #1
Category Page (Magento) Run #1
Mean time: 524ms
Standard Deviation: 219ms
Mean time: 529ms
Standard Deviation: 222ms
Search Page (Magento) Run #1
Mean time: 461ms
Standard Deviation: 211ms
Filtered Category Page (Magento) Run #2
Category Page (Magento) Run #2
Mean time: 493ms
Standard Deviation: 221ms
Mean time: 473ms
Standard Deviation: 219ms
Search Page (Magento) Run #2
Mean time: 441ms
Standard Deviation: 221ms
Filtered Category Page (Sphinx) Run #1
Category Page (Sphinx) Run #1
Mean time: 245ms
Standard Deviation: 41ms
Mean time: 236ms
Standard Deviation: 47ms
Search Page (Sphinx) Run #1
Mean time: 212ms
Standard Deviation: 49ms
Filtered Category Page (Sphinx) Run #2
Category Page (Sphinx) Run #2
Mean time: 233ms
Standard Deviation: 36ms
Mean time: 224ms
Standard Deviation: 36ms
Search Page (Sphinx) Run #2
Mean time: 209ms
Standard Deviation: 51ms
Who is that guy?
The person who you can blame for all terrible Magento 1.x architectural decisions :)
Why Sphinx?
No Java required (single binary)
Chosen by AirBnB and Craiglist
SphinxQL (MySQL client protocol)
Can be executed as your web app user
Configuration Merge
Product View
Downtime increases with the size your
catalog during full data re-index
Common bottlenecks
Indexation speed is not the best one you can achieve
Copy table, fill in new data and rename tables
on full re-index for zero downtime
What can be done
Write your own optimized indexes for your data structure
The more store views you have, the less you can scale
Common bottlenecks
Not all data you need for every request
Load configuration only for current store view
What can be done
Share only required data in configuration for another store view, lazy loading for admin operations
Redundant options are loaded
Common bottlenecks
Redundant EAV data is loaded
Flat collections are not used
Load only options you need for attributes
What can be done
Load partial product model
Use flat collections for relations, grouped and configurable products
In-efficient price retrieval from database
Common bottlenecks
Payment method API calls, when they are not needed
Shipping method API calls, when they are not needed
Use collection load events, to pre-load data at once, before it gets lazy loaded by Magento
What can be done
Optimize PSP and Shipping Provider code, to reduce API calls, cache them when possible
... and much more!
How about Magento 2?
My roadmap performance OS projects:
Layout Compiler (M2, Q1 2016)
Indexing Framework (M1, M2, Q2 2016)
Layered Navigation Framework (M1, M2, Q2-Q3 2016)
Full transcript