Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Copy of Copy of Machine 1

No description
by

on 15 July 2014

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Copy of Copy of Machine 1

input
data
input
data
input
data
input
data
Distributing Artificial Neural Networks Across a Cluster


Student: Andrew Trask
Advisor: Dr. William Hooper
Output Data
Input Data
A
B
C
D
E
F
G
H
O
P
Q
R
S
T
U
V
W
X
Y
Z
Input Layer
Output Layer
A Neural Network
One Computer
Limitations
All input/output data must fit on local hard drives
All Nodes/Edges must fit in Ram
Processing is limited to the number of threads that can fit on a single machine (large networks then take a long time to train)

State of the Art Solutions:
Replicate Entire Network Across Many Clusters, combine their answers (periodically or at the end)
Incrase the number of threads by utilizing GPUs
Partition Nodes across Cluster of computers, increasing both available RAM/Hard Drive space and concurrent computation

A
B
C
D
E
F
G
H
O
P
Q
R
S
T
U
V
W
X
Y
Z
NETWORK

P * Q
Messages
Computer 1
Computer 2
Computer 3
Computer 4
A
B
C
D
E
F
G
H
O
P
Q
R
S
T
U
V
W
X
Y
Z
Computer 1
Computer 2
Computer 3
Computer 4
input
data
input
data
input
data
input
data
Input Layer Node (count = P)
Output Layer Node (count = Q)
approaches
P*Q Network
Messages
A
B
C
D
E
F
G
H
O
P
Q
R
S
T
U
V
W
X
Y
Z
Follow The Data
One Data Sample
(One Row):
1
3
2
4
1
6
2
3
A + B + C + D + E + F + G + H = T
1 + 3 + 2 + 4 + 1 + 6 + 2 + 3 = 22
A
B
C
D
E
F
G
H
THE ORDER OF THE SUM
DOES NOT MATTER!
Input Layer Node (count = P)
Output Layer Node (count = Q)
computer 1
computer 2
computer 3
computer 4
count = P = 8
count = Q = 12
Full transcript