Loading presentation...
Prezi is an interactive zooming presentation

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Naive-Bayes Inspired Effective Pre-Conditioner for Speeding-

No description
by

nayyar zaidi

on 17 December 2014

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Naive-Bayes Inspired Effective Pre-Conditioner for Speeding-

WANBIA-C and LR
Two stage learning.
1) learning generative parameters.
2) learning discriminative (weight) parameters.
Convex objective function.
Optimized through standard gradient descent.
Near-identical generalization performance to LR.
Introduction
WANBIA-C: An alternative Parameterization for LR
Naive Bayes and Logistic Regression
NB and LR are two equivalent models with different parameterization.
An Alternative Parameterization
Nayyar A. Zaidi, Mark J. Carman, Jesus Cerquides, Geoffrey I. Webb
Naive-Bayes Inspired Effective Pre-Conditioner for Speeding-up Logistic Regression
The equivalence becomes more clear if we take the exponential and log of NB:
Introduce a weight parameter per-attribute-value-per-class-value in Naive Bayes:
WANBIA-C vs. LR
WANBIA-C vs. LR
WANBIA-C vs. LR
(Learning Curves using SGD)
WANBIA-C vs. LR
WANBIA-C vs. LR
WANBIA-C vs. LR
Conclusion
Better convergence for both quasi-Newton and online stochastic gradient descent.
Information from generatively learned parameters serves as an effective pre-conditioner.
Adding a regularization term to the WANBIA-C objective function can smoothly interpolate between NB and LR.
Future work:
Theoretical analysis of WANBIA-C convergence rates.
Similar trick can be applied with other Bayesian Network structures.
Experiments
Optimizing conditional log likelihood is convex and hence must be minimized with weights such that
Full transcript