Send the link below via email or IMCopy
Present to your audienceStart remote presentation
- Invited audience members will follow you as you navigate and present
- People invited to a presentation do not need a Prezi account
- This link expires 10 minutes after you close the presentation
- A maximum of 30 users can follow your presentation
- Learn more about this feature in our knowledge base article
Transcript of WWW 2013
modeling the evolution of user expertise
through online reviews
Julian McAuley, Jure Leskovec
Special effects that were good in 2003
may not be good in 2013
User and product features change over time
A child who likes Harry Potter in 2003
may have outgrown it in 2013
Even though today's children like Harry Potter,
the children of 2023 may not
Age of the product
Age ('development') of the user
Age ('zeitgeist') of the community
Movies may be preferred once they are considered 'classics' (Koren, 2010); new products may cause users to change their focus (Kolter & Malouf, 2007)
Users may influence each other (Ma et al., 2011; Moe & Schweidel, 2012), and communities may shift (Xiong et al., 2010)
How do users 'age'?
We don't model physical age, but rather development, or 'expertise'
A user should watch many movies before they will enjoy Citizen Kane
A user should drink many wines before they will enjoy a Romanee Conti
A user won't enjoy a bitter IPA, a smokey whiskey, or a strong cheese, until they have
acquired a taste
The very act of
will cause users' tastes to change
Each model makes different assumptions about which users are 'similar'
Users should be compared to others reviewing the product at the same time
Users should be compared to members of their community or social network
Users should be compared to
thers with the same
Our goal is to compare and model these effects
Models of temporal evolution
Start with the 'standard' latent factor model
Replace it by a series of models, indexed by a user's 'life stage'
Products/communities evolve over time:
Users evolve over time:
Users gradually become expert over time
All 'beginners' use the same model, regardless of when they arrive in the community
Not all users should evolve at the same rate!
the rate at which each user evolves:
Some users start out experienced
Others never become experienced
Each user evolves at their own rate
What are we really learning?
The constraint that users 'progress' between successive models means that all users evolve in the same 'direction'
Really, we're learning some property of evolution that's common to all users, regardless of when they arrive in the community
'Expertise' is the name we give to this property
We need to fit the parameters of each recommender system
We also need to fit users' experience progression
If we know each user's experience level, fitting the parameters is easy
Since users gain experience monotonically, we can fit experience using Dynamic Programming
The two steps repeat
for train/test splits, regularization etc., see our paper
Data are available on http://snap.stanford.edu/data/
How do experts differ from beginners?
Beloved products are most beloved by experts
Hated products are most hated by experts
Experts agree with each other
1) Experts give similar ratings when reviewing the same products
2) They also are individually more predictable (see paper)
(1) and (2) are arguably necessary conditions for users to be considered 'experts' (Einhorn, 1974)
Users who fail to become
experts are likely to quit
see also Danescu Niculescu-Mizil et al. (Thursday session 5)
Temporal evolution occurs on the level of products, users, and communities
We studied how users evolve as they review more products - or as they become more
Modeling expertise leads to more accurate performance, and leads to novel insights about how beginners and experts behave
J. Kolter and M. Maloof. Dynamic weighted majority: An ensemble method for drifting concepts. JMLR, 2007.
Y. Koren. Collaborative filtering with temporal dynamics. Commun. ACM, 2010.
H. Ma, D. Zhou, C. Liu, M. Lyu, and I. King. Recommender systems with social regularization. In WSDM, 2011.
W. Moe and D. Schweidel. Online product opinions: Incidence, evaluation, and evolution. Marketing Science, 2012.
H. Einhorn. Expert judgment: Some necessary conditions and an example. Journal of Applied Psychology, 1974.
Recommending products to users
We need to understand the
of the user, and the
of the product
e.g. How much will Julian like Harry Potter?
Users need to be
for the products they consume
(mean squared error of rating predictions on test set
users with at least 50 ratings)
'Global' versus 'Personal' clocks
Should users/products/communities be modeled in terms of their
or in terms of the
Recommending products to users
Goal is to predict users' ratings of products
Are 13 year old wines good, or are wines from 2000 good?
Did I start to enjoy Games of Thrones because it became popular, or because I became interested in fantasy?
Is Harry Potter popular in 2013, or amongst 13 year olds?
never became an expert
started out (somewhat) expert