Introducing
Your new presentation assistant.
Refine, enhance, and tailor your content, source relevant images, and edit visuals quicker than ever before.
Trending searches
voice search
5.9% Word Error Rate (Oct 2016, Microsoft)
Deep Neural Networks
Lawrence Cavedon
RealThing, Ltd.
Mark Sanderson
Bruce Croft
:-)
me
Johanne Trippas
Ameer Albahem
Australian Research Council, Linkage Project (LP130100563)
knowledge graph
context
ranking
diversification
recommendation
query suggestion
Screen readers are ineffectual [Sahib et al. 2012]
June 16, 2017
UNED
Madrid, Spain
User has a broader set of possible actions, that can be performed at any stage of the interaction:
Spoken SERP
Re-ranking
3
Short Summary
Longer Summary
Title
3
2
4
1
~
Ranking
1
3
https://sites.google.com/view/cair-ws/
- Iterative feedback for re-ranking
- Negative effect on user satisfaction described by Lee, Teevan and De la Chica [SIGIR'14] disappears due to the linearity of the channel.
query confirmation
query reformulation
SERP presentation/synthesis
...
Trippas, J.R., Cavedon, L., Spina, D., Sanderson M., How Do People Interact in Conversational Speech-Only Search Tasks: A Preliminary Analysis. CHIIR'17 (short paper). 2017.
Trippas, J.R., Cavedon, L., Spina, D., Sanderson, M., A Conversational Search Transcription Protocol and Analysis. CAIR SIGIR'17 workshop. 2017
Johanne R. Trippas
PhD Candidate
[Azzopardi, SIGIR'14] Azzopardi, L. Modeling Interaction with Economic Models of Search, SIGIR'14, 2014.
[Kiseleva et al., SIGIR'16] Kiseleva, J., Williams, K., Awadallah, A.H., Crook, A.C., Zitouni, I., Anastasakos, T., Predicting User Satisfaction with Intelligent Assistants. SIGIR'16, 2016.
[Lee, Teevan and de la Chica, SIGIR'14] Lee, C-J., Teevan, J. and de la Chica, S. Characterizing Multi-Click Behavior and the Risks and Opportunities of Changing Results during Use, SIGIR'14, 2014.
[Sahib et al., JASIST'12] Sahib, N.G., Tombros, A. and Stockman, T., A comparative analysis of the information seeking behavior of visually impaired and sighted searchers. JASIST, 2012.
[Spina et al., JASIST (to appear)] Spina, D., Trippas, J.R., Cavedon, L. and Sanderson, M.. Extracting Audio Summaries to Support Effective Spoken Document Search. JASIST, To appear.
[Trippas et al., SIGIR'15] Trippas, J.R., Spina, D., Sanderson, M. and Cavedon, L. Towards understanding the impact of length in web search result summaries over a speech-only communication channel. SIGIR'15. 2015
[Trippas et al. CHIIR'17] Trippas, J.R., Cavedon, L., Spina, D., Sanderson, M., How Do People Interact in Conversational Speech-Only Search Tasks: A Preliminary Analysis. CHIIR'17. 2017
[Trippas et al. CAIR'17a] Trippas, J.R., Cavedon, L., Spina, D., Sanderson, M., Crowdsourcing User Preferences and Query Judgments for Speech-Only Search. CAIR SIGIR'17 workshop. 2017
[Trippas et al. CAIR'17b] Trippas, J.R., Cavedon, L., Spina, D., Sanderson, M., A Conversational Search Transcription Protocol and Analysis. CAIR SIGIR'17 workshop. 2017
[Walker et al, EACL'97] Walker, M.A., Litman, D.J., Kamm, C.A. and Abella, A. PARADISE: A framework for evaluating spoken dialogue agents. EACL'97. 1997.
What are the properties of a "good" audio summary?
[Trippas et al., CAIR'17a]
Minimize interaction cost,
[Trippas et al., SIGIR'15]
[Spina et al., JASIST (to appear)]
maximize user satisfaction
@CAIRWorkshop
PARADISE framework [Walker et al., 2000]
- small set of subjective judgments
- 8-9 questionnaire judgments
- prediction target
- several parameters determined from interaction logs
- linear combination, ML
Q queries
V # results
S # summaries
A # assessed documents
Ts # time spent to listen summaries
...
C # clusters/sub-topics/aspects
L # labels
H # layers in the clustering hierarchy
[Azzopardi, SIGIR'14]
knowledge graph
context
ranking
diversification
recommendation
audio: linear channel
query suggestion
damiano.spina@rmit.edu.au
@damiano10
Recent research in IR evaluation methodologies enabled the study of more complex properties of IR user-system interaction
time-biased gain
summaries
[Smucker & Clarke, SIGIR'12 (best paper)]
conversational
search
search result browsing
cost models
[Azzopardi, SIGIR'14 (honourable mention)]
sessions
card playing model
[Zhang & Zhai, SIGIR'15; Zhai, SIGIR'15 keynote]