i486 neural networks

Bayesian Impact on Neural Networks

Bayes' Theorem

Bayes' Theorem shows the relation between two conditional probabilities.

The key idea:

The probability of an event A given B depends not only on the relationship between the two, but also the marginal probability of each event.

Bayesian Learning

Conventional Neural Networks

• Predictions found via maximization of posterior probability.

• For supervised networks, weights are trained based upon the difference between the observed value and the actual value, given a set of training data.

• Ultimately provides a single set of weights that can be used to make predictions.

vs.

Bayesian Neural Networks

• Predictions found by integration, which considers all possible parameters.

• Bayesian training results in a posterior distribution over network weights.

• This gives rise a distribution over the outputs of the network, which serves as the predictive distribution for the next case.

therefore...

Bayesian learning methods provide solutions to fundamental problems, such as:

• How to judge the uncertainty of predictions. The full predictive distribution (the output of each trial) provides the user with an uncertainty value, which can be important to some calculations.

• How to choose a suitable network architecture (the number hidden layers, the number of hidden units in each layer).

Internet Traffic Classification

• Implements the use of Bayesian Neural Networks to predict future internet traffic demands.

• Uses training data based upon past and current traffic.

• Once trained, the Bayesian network can achieve a high degree of accuracy in predicting traffic that a particular network will receive in the future.

• This helps companies financially plan for spikes or lulls in traffic, as well as assisting security personell in determining anamolous behavior.

Vehicular Traffic Prediction

• A study conducted at Texas A&M compared multiple Artificial Neural Network (ANN) models in terms of their accuracy at predicting vehicular traffic and crashes on highways.

• Bayesian Neural Networks (BNN) proved more effective than both Back-propogation Neural Networks (BPNN) and Negative Binomial (NB) regression models because it effectively alleviated the over-fitting problem commonly seen with the other two systems.

• BNN had better generalization abilities due to its superior scope of sampling potential values and predictions.

• The BNN approach will assist in future development of accident modification factors as well as improved prediction capabilities for highway design alternatives.

Adverse Drug Reaction Database Inference

• A large database of Adverse Drug Reactions (ADRs) held by the Uppsala Monitoring Centre (which holds data for the World Health Organization) contains nearly two million reports with nearly 35,000 new reports added quarterly.

• Due to its immense amount of data, the mining technique of choice for discovering potential new ADRs is a Bayesian confidence propogation neural network (BCPNN).

• Using this technique, a recent quarterly update was tested and resulted in the prediction of 307 potentially serious ADRs, 53 of which were related to new drugs.

• With this information, physicians can then perform tests to determine the validity of the ADR predictions.

• In essence...

IT SAVES LIVES.

Applications

P(B|A)P(A)

P(A|B) =

|

P(B)

Conventional Neural Networks

Nicholas Colvin and Kyle Nisswandt

Sources

http://en.academic.ru/pictures/enwiki/84/Thomas_Bayes.gif

http://www.isye.gatech.edu/~brani/isyebayes/bank/newbayes.jpg

http://www.google.com/url?sa=t&source=web&cd=4&sqi=2&ved=0CDQQFjAD&url=https%3A%2F%2Fceprofs.civil.tamu.edu%2Fdlord%2FPapers%2FXie_et_al_Bayesian_Neural_Networks.pdf&rct=j&q=bayesian%20neural%20network%20approach%20estimating%20energy%20equivalent%20speed&ei=NRprTbzEIYet8AbFmq2cCw&usg=AFQjCNFc6lJk_zxGCct4heep83yGOCZmmg&cad=rja

Bayes Image:

Sheen Image:

Motor Vehicle Collisions Info:

Internet Traffic Classification Info:

Drug Reactions Database Info:

BNN General Info:

What is Bayesian Learning?:

http://www.google.com/url?sa=t&source=web&cd=1&sqi=2&ved=0CB0QFjAA&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D4049810&rct=j&q=bayesian%20neural%20networks%20internet%20traffic%20classification&ei=-xprTc6oIcP78AbM56XoCw&usg=AFQjCNGuZxR7LkCX07RPueTS_dYPLldYOQ&cad=rja

http://dml.cs.byu.edu/~cgc/docs/atdm/BCPNN-ADR.pdf

http://www.google.com/url?sa=t&source=web&cd=1&sqi=2&ved=0CBkQFjAA&url=http%3A%2F%2Fwww.public.asu.edu%2F~huanliu%2Fdmml_presentation%2FP05-06%2FBayesian-Neural.ppt&rct=j&q=%22bayesian-neural-1.ppt%22&ei=vyNrTZeBH4O8lQf-5dyAAg&usg=AFQjCNEeqE5hOt2RVDY1DmaNbuQIDGvasw&cad=rja

http://dml.cs.byu.edu/~cgc/docs/atdm/BCPNN-ADR.pdf