Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Scaling the Earth System Grid to 100 Gbps

SC11 Lawrence Berkeley National Laboratory Demonstration
by

Lauren Rotman

on 25 August 2013

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Scaling the Earth System Grid to 100 Gbps

Scaling the Earth System Grid to 100 Gbps
Climate Simulation Data is Critical
Climate Research is Growing in Data Intensity
ESG Data Snapshot
How else will we move
multiple Petabytes of
climate data??
Scaling to 100G
“Replica Core Archive” – The Coupled Model Intercomparison Project, Phase 5 (CMIP-5) used for the Intergovernmental Panel on Climate Change (IPCC) Fifth Assessment Report (AR5) is estimated to reach 1.5 - 2 Petabytes of data

Climate model data is projected to exceed hundreds of Exabytes by 2020 (BES Science Network Requirements Workshop, 2007)

The Earth System Grid (ESG) will need to scale to accommodate this massive growth
IPCC AR4 CMIP-3 LLNL ESG portal
35 TB of data at one location
83,337 files, model data from 13 countries
Generated by a modeling campaign coordinated by the Intergovernmental Panel on Climate Change (IPCC)
Over 600 scientific peer-review publications
CCSM NCAR ESG portal
237 TB of data at four locations (NCAR, LBNL, ORNL, LANL), 965,551 files
Includes the past 8 years of joint DOE/NSF climate modeling experiments
Broad investments in climate change research
Development of climate models
Climate change simulation
Model intercomparisons
Observational programs

Climate change research is increasingly data-intensive
Analysis and intercomparison of simulation and observations from many sources
Data used by model developers, impacts analysts, policymakers
Earth System Grid:
ESG is virtual collaborative environment linking distributed centers, users, models, and data.
The ESG research and development program was designed to develop and deploy the technologies required to provide scientists with virtual proximity to the distributed data and resources that they need to perform their research.


Presented by:
Mehmet Balman, Alex Sim
Scientific Data Management Research Group
Computational Research Division
Lawrence Berkeley National Laboratory
The Testbed
LBNL/NERSC: 20 nodes
ANL/ALCF: 15 nodes
ORNL/OLCF: 15 nodes
Each connection with a 10 Gbps interface
Climate demo moving datasets of
IPCC AR4 CMIP3
From NERSC to ANL over 100Gbps
Disk to memory
From NERSC to ORNL over 100Gbps
Disk to disk
jkbk
The Demo
Data preparation
35 Terabytes climate data
Select files from dataset
Climate Analysis
Retrieve files, extract data & perform analysis
Climate Analysis
Retrieve files, extract data & perform analysis
Full transcript