gErl

gErl

Architecture

Results

Rule Notation

• Simple non-recursive programs without background knowledge:
• ifthenelse, not, or, and, enjoysport, playtennis, cup, lens.

• Simple recursive programs without background knowledge:
• append, last, member

• Other programs with background knowledge:
• Web Classification [V. Estruch, 2005]
• Multiple substitutions on lists [F. Martínez, 2013]
• Simple English Grammar Problem
• Bunches of Keys
• Animal Kingdom
• ...

Last element of a list

Last element of a list

Defining Operators

Last element of a list

Meta-level facilities called meta-operators:

gErl provides the following meta-operators:

• meta_replace(Pos, f) : creates an operator that replaces in a rule a term located in the position Pos by the the term retuned by f.
• meta_insert(Pos, f) : creates an operator that inserts a term returned by the function f in the position Pos of a rule.
• delete(Pos) : creates an operator that deletes a term located in the position Pos of a rule.

IQ tests

Computational understanding of how intelligence tests can be solved:

• Through the use of a general learning tool, we can better understand the computational difficulty of them.

Raven's Progresive

Matrices

Odd One Out

Reusing Abstract Policies

Thurstone Letter

Series

RL-based heuristics

• Reuse of knowledge acquired in previous tasks in order to improve or accelerate the learning of future (possibly different) tasks.

• We used an hybrid approach between value-function and model-based methods:
• Generalise the state-value function Q(s, a)
• gErl uses a linear regression by default for generating Q , which is retrained periodically from Q .
• Q is used to obtain the best action a for the state s as follows:

M

.

t

Last element of a list

• Once the system has started, at each step, Q is updated using the following formula:

Last element of a list

Transfer Knowledge

s

• The abstract representation of states and actions (the and feature space) allows the system does not start from the scratch and reuse the optimal information
• The table Q can be viewed as knowledge acquired during the learning process that can be transferred to a new situation.
• When gErl learns the new task, Q is used to train a new model Q .
• Q is used from the first learning step and it is afterwords updated with the new information acquired using the model Q .

s

T

M

T

M

Conclusions

Flexible Architecture

• We have proposed that more general systems can be constructed by not only giving power to data and background knowledge representation but also:
• to a flexible operator redefinition and,
• the reuse of heuristics across problems and systems
• This flexibility also carries a computational cost.

Strengths

Closing...

Limitations

• Definition of customised operators (depending on the data structures and problem at hand)
• Genereralised heuristics (the choice of the right pair of operator and rule has been reframed as a decision process, as a reinforcement learning problem).
• Reuse of knowledge acquired in previous tasks in order to improve or accelerate the learning of future (possibly different) tasks.
• The abstract representation of states and actions (the and feature space) allows the system not to start from scratch and reuse the optimal information.

Further Work

gErl is only a prototype and, due its flexibility, it entails some risks:

• Significant amount of human effort and expertise required to design or select effective operators.
• Although we have developed an apparently good approach, it is required a further examination of the RL approach in order to obtain a better understand of how the system works.
• The system becomes very inefficient when the number of examples become large.

• Customised generalisation/specialization operators.
• Population of rules and programs evolved as in an evolutionary programming setting.
• Model-based Reinforcement Learning heuristic.
• Transfer learning between tasks in order to accelerate learning.
• Erlang functional programming language.

• The system can be used to investigate which operators are more suitable to solve a complex problem and allow us to better understand the role of operators and heuristics in machine learning.
• Using a subset of the examples as base examples and the rest for calculating coverage in order to make the system more efficient.
• To include features for the operators.
• The potential re-usability of the abstract knowledge learned.

Introduction

Why ?

Erlang/OTP is a functional programming language developed by Ericsson and "was designed from the ground up for writing scalable, fault-tolerant, distributed, non-stop and softrealtime applications".

• Free and open-source language with a large community of developers behind.
• Reflection and higher order.
• Unique representation language: operators, examples, models and background knowledge are represented in the same language.

a General Learning System

Reports and Papers inside the project

Thanks!

Introduction

- Fernando Martínez-Plumed, Cèsar Ferri, José Hernández-Orallo, M. José Ramírez-Quintana, On the definition of a general learning system with user-defined operators, 2013, CoRR abs/1311.4235

- Fernando Martínez-Plumed, Cèsar Ferri, José Hernández-Orallo, M. José Ramírez-Quintana, Policy Reuse in a General Learning Framework , Fifteenth Spanish Conference for Artificial Intelligence (CAEPIA 2013), Madrid, Spain, 2013, ISBN: 978-84-695-8348-7, pages 119-128.

- Fernando Martínez-Plumed, Cèsar Ferri, José Hernández-Orallo, M. José Ramírez-Quintana, Incremental learning with state features in gErl , In Alicia Troncoso, Jose C. Riquelme (eds) VII Symposium on Theory and Applications of Data Mining, TAMIDA 2013 (CEDI 2013), Madrid, Spain, 2013, ISBN: 978-84-695-8348-7, pages 1383-1392.

- Fernando Martínez-Plumed, Cèsar Ferri, José Hernández-Orallo, M. José Ramírez-Quintana, Learning with cofigurable operators and RL-based heuristics , In A. Appice et al.,editor, New Frontiers in Mining Complex Patterns , volume 7765 of Lecture Notes in Computer Science (LNCS), pages1-16. Springer-Verlag, 2013.

- Fernando Martínez-Plumed, Cèsar Ferri, José Hernández-Orallo, M. José Ramírez-Quintana, Towards the definition of learning systems with configurable operators and heuristics , in procedings of the 1st Workshop on New Frontiers in Mining Complex Patterns (NFMCP Workshop held at ECML-PKDD 2012) ,Bristol, UK, September 2012.

was born as an advocacy for general frameworks in inductive inference:

A general rule-based learning system where operators can be defined and customised according to the problem, data representation and the way the information should be navigated:

• Does not need to be modified when the data or problem changes.
• Preserves the original data representation.

http://www.dsic.upv.es/~fmartinez/gerl.html

Fernando Martínez Plumed

{fmartinez@dsic.upv.es}

DSIC, Universitat Politècnica de València

December 2013