Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.


Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Loop Mining in the Encyclopedia of World Problems

Prepared for the International Futures Conference in Turku (11-12. June, 2015)

Tomáš Fülöpp

on 11 November 2015

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Loop Mining in the Encyclopedia of World Problems

Problems are sociable; cannot exist in isolation. Solutions have to take into account the complex network of interrelations.

To understand more, we need to study the nature of the relationships involved.

In this work we examine one particular relationship type: "aggravation".

Problems chained
Loop Mining
in the Encyclopedia of World Problems
"Aggravation" is a systemic notion of one problem worsening, or being made worse by, another.

For example, the problem of cultivation of illegal drugs is aggravated by substance abuse.

Interesting because it forms long, pervasive chains that often intersect one another, and sometimes loops.

Relationships worsening
Most of the aggravating problem chains form straight strands.

A small subset of the chains loop back to their own beginnings or to intermediate nodes. In such positive feedback loops, problems aggravate each other in a closed circuit, reinforcing each other ad infinitum, constituting structures familiarly known as vicious cycles.

Vicious cycles emerging
Vicious cycles are hard to find even with most of the common computers. Hence the term: "loop mining".

Due to the computation speed barrier and complexity involved, we so far struggle to uncover the complete landscape of loops in the database.

Already able to shed some light on these peculiar emergent patterns and outline some pertinent questions worth further research.
Computation barrier
This work examines the intricate web of aggravating relations in the Encyclopedia of World Problems and Human Potential.

Resource attempting to represent the world problems as perceived by humans.

Refined profiles of almost 57,000 problems occurring around the world. Many of the problems fit the definition wicked problems.

The Encyclopedia is a long-term and open-ended project. The information it offers is always evolving and there is no notion of stable releases.

It prides itself on being fiercely unbiased and free of censorship.

The data source
Better understanding of the nature of vicious cycles emerging from a network of aggravating relationships between problems is promising to be a useful approach for systemic understanding of any problem's context.

It may have the potential to equip researchers and decision-makers with powerful analytical tools and strategies to minimize or completely solve problems they set out to deal with.
Encyclopedia of World Problems and Human Potential
Detection of all loops that exist in the data set is still an elusive goal because of the sharply increasing time it takes to process wider portion of the source data.

For example, computation of database-wide loops up to level 6 (still very shallow) found almost 2200 loops, but the process took 92 hours and 16 minutes.

It is evident that we have to improve the algorithm drastically in order to be able to examine the full depth of the loop mine in any reasonable time.

Mining time
This presentation and the full article is to be available at:

Comments? Questions? Complementary research or projects in mind?

We are actively seeking collaborators; please get in touch!

Learn more, get in touch
This is a presentation about mining. Mining of patterns. Patterns emerging from a complex network of problems.

Knowledge of such patterns may empower researchers and decision-makers to devise more efficient, context-aware strategies.

Work based on the dataset developed by a project called the Encyclopedia of World Problems and Human Potential.
A mining project
Aggravation is a directed relationship. Swapping the related end points changes the meaning of the situation. In other words: if problem A aggravates problem B, it does not mean that problem B aggravates problem A.

If problem A aggravates problem B then problem B indicates that it is aggravated by problem A, and vice versa. As these two notions mirror each other (A › B means both), it is enough to follow one of them to get a full picture of all involved problems in the database.

If A › B and B › C, then these form an aggravation path A › B › C. Thanks to the chaining, we see that while problem A directly aggravates problem B, it also indirectly aggravates problem C.
Basic structures
The Encyclopedia of World Problems and Human Potential is an outcome of a long-term research project initiated by the Union of International Associations (UIA) and Mankind 2000 in Brussels in 1972.

Both of the organizations are international non-profit associations.

Whilst Mankind 2000 has been dormant since the end of the 1980s, further work has been carried forward under the auspices of the UIA.
The origins
The Encyclopedia is a set of databases. Most notable are World Problems, Global Strategies, Human Values, and Human Development.

The databases are heavily interlinked both within and between them.

The Encyclopedia is to some degree also interlinked with other major databases maintained by the UIA, notably the Yearbook of International Organizations and the International Congress Calendar.

A group of databases
For the greater part of its long history, the Encyclopedia was published in book form. Initially as a single tome under the title Yearbook of World Problems and Human Potential in 1976, then under the surviving title in 1986, 1991, and 1994.

As early as 1999 it successfully moved to the web, with major redesign in 2003, and another one started in 2012 and still on-going.
Publication history
Information and activities reported by international organizations monitored by the UIA, complemented by information stemming from other sources, such as newspaper reports, scientific studies, books, etc.

Focus on variety of perspectives rather than on the perceived importance of particular problems. Entries were edited by trained UIA editors.

Expected increase of the variety of information sources thanks to crowdsourcing techniques,

Automatic augmentation from compatible databases, e.g. Wikipedia, WorldCat, Google Trends, etc. — secondary(e.g. illustrative images) or subject to frequent changes or evolution (e.g. statistics, trends).
Data collection
The database of World Problems contains hand-picked and well-refined profiles of almost 57,000 problems occurring around the world: starting with notorious global issues and ending with very specific and rare ones — from famine and child labour all the way to nail biting and abduction by aliens.

The Encyclopedia tries hard not to make judgement about what is or is not a problem. If an issue is perceived and reported as a problem by individuals or groups around the world, then it is likely to be included in the database.

Aggravation is just one type of relationship identified in the database. Others are the notion of reduction (one problem diminishes or is diminished by another), hierarchy (one problem being broader or narrower than another one), simple, undefined relatedness, and relations based on problem-specific typology, Encyclopedia-wide subject categorization, flat tagging, etc.

The World Problems database currently contains about 34,000 unique aggravating relationships.

Earliest studies analyzing feedback loops in the database of World Problems date back to 1995 (Anthony Judge & Nadia McLaren), using DOS and early Windows computer systems for days on end.
Earliest network analysis using the Encyclopedia dates back to the 1970s (even a silent movie made).
A web of relationships
Most of the aggravating relation chains form straight strands. Some are short, other amazingly long.

A small subset of the aggravating chains loop back to their own beginnings or to intermediate nodes. An example could be A › B › C (› A). In such positive feedback loops, problems aggravate each other in a closed circuit, reinforcing each other ad infinitum — constituting structures familiarly known as
vicious cycles

Let's define "loop level" as the number of nodes involved in a loop. For example, A › B would be a level 2 loop.
Loops emerging
Many of these chains are known to interlock. That happens whenever a problem aggravates more than one other problem. A simple example could be loop C › E › F (› C) which intersects with loop A › B › C (› A) in node "C".

There are several interlock possibilities — between straight chains, looping chains, or both. Loops can interlock in more than one node.
A simple example of a vicious cycle detected in the database:

Obesity > Abnormal blood fats > Diseases of metabolism (> Obesity)
Feedback loop examples
A real example of a 4-level loop:

Substance abuse > Cultivation of illegal drugs > Economic dependence of countries on the drug trade > Inadequate drug control (> Substance abuse)

In reality this loop shares 4 nodes and 3 edges with a larger loop involving “Illegal drug trafficking”; the result can be represented visually like this.

Level 4 loop
An example of a 5-node loop and how it interlocks with related loops:

Statelessness > Migrant labour > Discrimination against immigrants and aliens > Expulsion of immigrants and aliens > Refugees > Statelessness

Level 5 loop
Mining algorithm
Provisional results
The number of loops sharply increases with their length.

There are 179 loops of level 3, 263 loops of level 4, 502 loops of level 5, and finally 1212 of level 6. We see that the number of loops roughly doubles with every loop level.

This seems to be natural: while considering longer and longer chains the chances of some of them looping back increase as well.

Lengths increasing
A table of a complete excavation of several partial sets of increasing size and with a shallow but complete dataset examination in the last column.
Loop length frequency
In the following table we see a distribution of loops of various lengths detected in the same datasets. The whole database is bound to contain many thousands of loops.
Acknowledgements: This work would have been incomplete and impossible without giving credit to a number of Encyclopedia gurus and other present and former colleagues, including Tony Judge, James Wellesley-Wesley, Nadia McLaren, Jacques de Mévius, Sinead Mowlds, Clara Fernández López, Rachele Dahle, Ryan Brubaker, Tim Casswell, and many others.
Some feedback cycles, especially the shorter ones, may be obvious. Others, having been mined using intensive algorithms that were previously not available, may come as a surprise even to experts in the respective fields. Provision of the latter is one of the main inspirations powering the mining and research of loops in the network of world problem relationships.

It appears that meaningfulness of the detected loops varies with their level (i.e. length of chain). Let us have a closer look at the various groups of levels.

Meaning waning
We know that single-level loops are definitely meaningless (a problem aggravating itself). They are only useful for signalling a linking error to the editors.

Loops of level 2 are most unlikely to be valid, although we cannot rule them out. A loop of level 2 would need to mean that those two problems exist in a strangely small universe of their own.
Levels 1 - 2
Loops of levels 3-5 are much more interesting.

They are short enough to be either immediately known to researchers, or they can be relatively easily verified using thought experiments.
Levels 3 - 5
Loops of level 6 and more (approx.) tend to be too long to grasp immediately. In general, the longer loops feel less plausible.

Long loops are easier to disqualify because of a chance wrong link. But that does not mean an absence of absolutely valid vicious cycles of level 10, 20, 50, or more.

Or, are they intrinsically weak due to the yet unknown aggravation force distribution and dynamics?

Levels 6+
In a straight chain of aggravating relationships each problem is aggravated with a “force” related to all problems that aggravate it.

If A › B › C, to what degree does A aggravate C? Is it a combined force of A and B? Or does the force from A diminish with its distance from C? Do some nodes act as resistors — or conductors?

How is that situation different in a simple loop A › B › C (› A)? Does the force on C grow infinite because the aggravation has no ending point and therefore runs in a closed circuit, or does it run down due to a type of unknown friction?

The situation is even more complex in a network with interlocking straight and looping chains.

In visual spring/tension maps, nodes with more interlocks tend to become rigid, while loose strands and non-interlocking loop nodes are far easier to move around without disturbing the network. How do interlocks change complex tension balance and geometry and what does it mean for the nodes involved?
Force distributions and dynamics
Apart from two transient chain types we have been able to retrieve straight terminal paths and pure loops. We have observed that interlocking loops and chains are by far more common than isolated ones.
Whilst there are much less loops than straight chains, only the short ones are really rare. We have seen that the number of loops increases sharply with the loop lengths.
It appears that meaning or usefulness of detected loops depends on their length.

Loops of level 1 and 2 tend to be meaningless, then there is a short range of typically clear and useful looping structures, followed by a long tail of ever longer loops with decreasing perceived meaning. It remains a question whether that is an inherent property of long loops, or simply a result of lower comprehensibility and higher chance of systematic error.

It is possible that the longer loops, if properly validated, can surprise us with unexpected connections and dynamics which are otherwise hidden from us in too much complexity.
We have looked into force distribution and dynamics of aggravating paths. Problem nodes are subjected to exertions related to their position in the network. The distribution and dynamics of such forces is far from clear.
Thanks to an implementation of the BFS algorithms we have successfully mined all low-level feedback loops from the World Problems dataset of the Encyclopedia. We have also succeeded with several full-depth excavations, but — due to the computational complexity / speed barrier — so far only of narrower segments of the database.
More than once we have employed the metaphor of mining to the retrieval of vicious cycles. That metaphor is fitting in many ways — just like mining minerals there is need for powerful machines, good maps, and a lot of patience. The algorithmic approach taken (BFS) is akin to surface mining, excavating layer by layer of waste to get to the prized rare ore.

Vicious loops are intrinsically negative: our only chance at solving the problems they weave together is to break them free from the loops. This is where the metaphor stumbles a bit; we search for a treasure in order to be able to shatter it.
Examples of level 2 loops
Examples of level 3 loops
Example of a level 6 loop
Tomáš Fülöpp
17th International Futures Conference
"Futures Studies Tackling Wicked Problems"
Turku, Finland, 11-12. June 2015
A little window on a mesh created by aggravating relationships amongst world problems.
Trying to unravel...
Full transcript