Towards Preschoolers' Automatic Satisfaction Assessment. An experience report

by Alexandra Turian·41 views·Updated May 22, 2020

Technology has penetrated almost every aspect of our life. Nowadays children grow and live surrounded by devices that provide them multimedia content to engage them and even teach them new things. It is considered that education should embrace the technology progress and include technology in the learning-assessment loop. The use of digital products in the learning process is effective if it provides appropriate content, the user spends a limited amount of time, and it brings satisfaction to the users. In this presentation we present our approach in assessing user satisfaction when a special kind of stakeholders are considered, namely kindergarten children. We present a case study we have performed, challenges and lessons learned while trying to automatically evaluate children satisfaction.

Audio TranscriptAuto-generated

Hello. My name is Alexander three and I want to
begin by thanking you for taking the time towards this
presentation. I'm grateful that the conference has not been canceled
in spite of these difficult pandemic situation today, I want
us to go together through our experience report towards preschoolers
Automatic satisfaction assessment.
In the first part, I'm going to say if you
were there in our approach, then in second part we
will dive in your case study and after that you
will learn about our results.
Well, at the end, I want to point out how
we're planning to improve and extend our idea conducted with
them. Babbage Boy, University of Krishna Polka.
Together with my teachers Adrianna Grande Angry good Attackers, Your
car. We performed a case study where we endeavoured into
integrating automatic emotion detection.
In the usability assessment process, we come together as researchers
who strive to integrate new technology into traditional way of
doing things.
On top of that, as professionals, we tackle user centered
design, whereas in Rohloff, parents were sick to raise tax
a V and tax eight Children.
What motivates us is the desire to enable education to
embrace technology by finding ways to build up to the
D off Children first design.
Now let's move on to the next topic.
In this case, study went on to compare results of
traditional usability testing methods such as observation post interview questions
and smile you meters with the automatic assessment off Children
emotions while they are exposed to an interactive applications the
application have chosen for our case.
Study is a storytelling game.
The system includes a digital storytelling part and a consolidation
part in the shape of four games.
We developed the system in the past by applying participatory
design and user centered design.
The system is there steam for Children aged 5 to
6 years attending the public formal preschool education system.
Its purpose is to provide a new directive way of
understanding in your story.
The application introduces Children into the story world, providing support
for consolidating knowledge from different domains, mathematics, natural sciences, language
and communication.
So let's look at one game as an example.
The mathematics came.
Children are prompted with a few questions, each question having
three possible answers.
The tasks are to compute the sum or difference off
the number of the animals on the screen.
Each task has explained the Children using audio off course.
The complexity is chosen.
Taking into consideration Children abilities, they just need to come
up to maximum 10.
The usability evaluation was conducted with five kindergarden Children aged
56 years.
All participants have previously interacted with similar systems.
Also be by students at the university.
What is important for us and I want to point
out, is that preschoolers in Romanian education system cannot read
or write and read, possess the skills to communicate their
opinion on product.
This makes preschoolers an age group that require extensive attention
when adopting traditional usability evaluation methods because of their short
attention span and their tendency to please adults taking into
consideration researchers.
Guidelines into organizing testing sessions with Children are valuation consisted
of Six Steps introduction, Ah, warm obsession, the actual play
session, the administration Off Smile You Meters, a post test
interview and the evaluation of emotions.
The session has been performed in the kindergarden in a
quiet room where there were no other Children.
To distract the participants, the computer game was accessible, using
one laptop and the mouths.
The minimal devices necessary to use the game for the
play testing session after listening for the story.
It's Child shows one or two games out of the
four available While they were interacted with the system, we
automatically captured pictures with Children's faces using the integrated laptop
camera. These are the pictures that were further analysed for
emotion detection.
Now that we have an understanding on how the case
study was performed, I want to share with you insights
off our results.
In order to achieve a clean comparison, we will go
over each of elevation method.
Asking Children additional questions after the session can be a
challenge. They might not want to spend extra time after
playing. Therefore, we kept the questions easy and short.
Also, we avoided the personalized or in the right questions
such as, I would like to know if you like
playing this game and chose questions such as Would you
like to play this game again with your best friend?
Childrens feedback was positive.
All of them said they will tell their parents about
the game, and all of them were curious about other
gains as well.
The other traditional method of evaluation was the use of
smiling meters value meters are a common way to allow
Children to express their attitude toward the product and measure
satisfaction and fun as it is.
Is it a complete and requires no writing for the
Children? All our participants chose happy faces.
Okay, I've explained how we apply traditional methods adapted to
preschoolers. Now here comes our idea in play, using face
detection and emotion recognition as an evaluation.
Math. During the play testing session, 140 pictures were automatically
taken using the laptop camera while Children were interacting with
the story telling system I described earlier after the session,
we analyzed all those pictures using Google Club Visionary for
this analysis, we built a small tool to make our
job easier in obtaining and visualizing the data.
We use Division Ah AP eyes label and face detection
features. Along with detecting labels and facial attributes, the A
P I also provides emotional likelihood for the four basic
emotions joy, sorrow, anger and surprise.
Looking at results after applying global detection, a surprisingly large
number of pictures have been labeled with smile 91 pictures
representing 65% off the total number of pictures from all
of this, though only three pictures have been labeled with
joy with height accuracy.
Still, we consider the results being encouraging and convergent toe
our findings from applying other assessment approaches.
In the interpretation of these results, there are three main
things to consider faced.
The number off pictures labelled with joy.
Second, the lack of negative emotions and, third, the algorithm
accuracy as the Children were engaged in a learning process
mediated by technology.
It is normal to be focused on listening to the
task statements and on performing the interaction required to solve
these tasks so their faces might not look joyful.
This is why the lack of negative emotions throughout all
the pictures was extremely relevant for us.
It means that the user did not feel frustrated or
angry at any time while interacting with the application tackling
the algorithm accuracy, I want to mention a major limitation
of the Google vision, a p I.
That is that if no specific emotion from this for
available can be detected with high confidence, data can combine
list if it cannot categorize the emotion, the A P
I returns very unlikely.
The limited emotional range does not make the comparison completely
fair. However, deciding whether faces happier, surprised, angry or confused,
said, or calm can be a difficult job even for
adults. To quickly recap.
In this presentation, I described our attempt of using an
automatic tool to identify Children's emotion while interacting with digital
story application.
After applying the tools over the set of pictures, we
discovered that most of the time Children had a great
time interacting with our application.
I would like to summarize by saying that face detection
brings great potential in usability testing with preschoolers, as long
as the products tested are suitable for the age and
their abilities.
Indeed, there are some limitations when working with Children.
Since facial expressions production is influenced both by age and
by gender, though the use of an automatic tool to
complete the image on user satisfaction is promising in the
future, we intend to include other information in our evaluation,
like hand gestures, voice recognition or posture, use other tools
to compare the results, include more participants and develop a
chrome extension that can be attached to anywhere application.
The track user inputs, such as drag and drop clicks,
browser actions along with real time emotion detection.
That concludes my talk for today.
If you have any questions, do not hesitate to contact
us. I sincerely appreciate that.
I've had the opportunity to present this.
You and I want to finish by thanking you for
your attention.