Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.



No description

Mark Begale

on 6 September 2018

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of MASTER SET

How many of you have built an eHealth intervention?
How many of you are interested in building your first?
Online is not Offline....
Think beyond existing evidence based care to leverage the benefits and advantages of technology platform--some examples:
Provider-to-patient direct access
Accessing the human network
Single user education
Demystifying the Technology
Desktop Computers
Mobile Devices
Touchscreen Smartphone
Keypad Smartphone
“Dumb” phone
Tablets (iPad)
Servers (Application / Database)
Connectivity (Bluetooth, Wifi, Ethernet)
Sensory devices
Biometric devices
Human Interaction Devices
A mouse
MEMS cap
Browser (Chrome, Firefox, Safari, and Internet Explorer)
Desktop Operating System (Windows, Mac, Linux)
Programming Languages
Desktop Applications
Microsoft Office
Desktop Publishing
Your Application?
Browser Applications
Server side: Google
Client side: Gmail
Your Application?
Development Platforms/Frameworks
mid level technologies that simplify programming tasks, (or "batches of code")
Content Management Systems
tools to allow non-programmers to author media content, such as text, movies, images, audio...
(features that can be modularly connected to platforms, like graphing applications)
The technology menu:
What would you like?

User centered Development
Field Testing and the Prototype Cycle
Alpha (controlled lab testing)
Beta (somewhat controlled field testing)
Release Candidate (uncontrolled field testing)
Final High Functioning Perpetual Beta
Going Digital:
Jennifer Duffecy, Ph.D., Mark Begale, B.A., William Riley, Ph.D., David C. Mohr, Ph.D.
History: How this got started....
Why should you use technology?
Access to different populations
Allows for better intervention adaptability
(changing it doesn’t “break it”)
Access to larger populations
Provides access to new sources
of study data
Standardizes the delivery of care
Provides easier access to narrative media
Games, movies, music, mainstream culture
Can provide just-in-time interventions
Because you’ll be left behind if you don’t
Developing a Data Management Plan
Key Questions
Where will data be stored?
Who will manage the data?
What data do you need?
Plan early—identify all needed
data before the project begins
Plan often—make sure that
technologists are notified if changes are needed
What types of data will you need?
Site usage
User authored content (what users filled in)
HOW are the data stored? Is it easy for your statisticians to utilize?
You have an idea of how it should work…
How do you ensure that it does?
Can we build an interface?
Should we build it?
Making an interface
Try it out conceptually
Verification: does it work?
Provides actual data about the usability of a project (task times, completion rates, satisfaction scores, etc)
User centered Development: Usability Testing

Focus groups
Think alouds (Brinck 2002)
Cognitive Walkthroughs (Wharton 1993)
Survey questionnaires (Lewis 1991, Lund 2001)
Card sorting

Click testing
Comparing user report data with objectively-collected site data
Visualizing user data into “narratives”
User centered Development:
REMEMBER the Field Testing and the Prototype Cycle
Alpha (controlled lab testing)
Beta (somewhat controlled field testing)
Release Candidate (uncontrolled field testing)
Final High Functioning Perpetual Beta
Training the best guard dog
Deferring to provided guard dogs (like your IT manager)
Compare your solution with others in your institution (and if you’re the first, plan to be challenged)
Know the policies:
Governmental (HIPAA / FISMA)
Structuring your data
Never store user information with study data
Never allow access to more than one type of data at any given time
Run analyses in protected computational environments (VPNs/Encrypted Laptops)
Audience Poll
A quick show of hands....
eHealth, mHealth,

Demystifying the Technology:
The Obligatory Buzzwords Page
Social Media:
Synchronous: Instant messaging, chat rooms, videoconferencing
Comprehensive: Facebook/MySpace/Google Plus
Constrained: Twitter
Specialized: YouTube/Blogs/Wikipedia
Web 2.0 (Sharing data):
Mash Ups: Using content from one site directly in another site
RSS/APIs: Making content easily available for reformatting
Semantic Web 3.0 (Categorizing and leveraging what is shared)
Ontologies (RDF): Defining the connections between data
Querying (SPARQL): Delivering data based on their ontology
Graphing: Using queries to deliver information in new ways
Demystifying the Technology
The limitations of technology, some examples:
Hardware Device Limitations (Computational Limitations, Platform/Browser Issues, Battery Issues, Failure)
Delivery Methodology Limitations (Mobile/tablet/desktop, Screen size, interface issues)
Connectivity Limitations (Speed, uptime, latency)
Development Platform Limitations (The tools don’t exist to build what you need)
Resource Limitations (Funds, Time, Experience)
Changing Climate of Technology
Issues of Feasibility
Your Technologist and You
(or, how I learned to stop worrying
and trust the techie)
Navigating the culture and language of technologists

The technology subculture (and the “pants” syndrome)
Differences in communication styles
Differences in language across disciplines
Understanding Technical Jargon (for Researchers), examples:
Client side / server side
Language about “equipment”
Understanding Research Jargon (for Programmers), examples:
What is a “study arm”?
What is “data analysis”?
Words that falsely translate:
What is a “schema”?
What is “data”?
What is “memory”?
What does it mean for something to be “coded”?
Choosing a technologist:
What should you look for?
Understanding the importance of deadlines
Examples of past work
Value continuum:
Experienced and expensive
Inexperienced and inexpensive
Vetting with existing trusted technologists
code reusability (will the work last for at least a year, two years?)
development methodologies (Agile, Waterfall)
Creating the Technology
(Building your BIT project)
BIT Development Process
How to optimize the building process: OVERVIEW
Develop a design document
Develop a “wishlist”
Develop a timeline
Develop a data management plan
Assign clear roles
Developing a wishlist
Make your “wishlist” of work priorities / defining scope of work
“Must have”
“Nice to have”
“If I had twice the money”
“If I had a blank check” (and if you do… tell us how you did it)
Decide early what components
you can afford to lose
Reprioritize development
tasks based on this wishlist
as changes inevitably arise
Developing a Timeline
Key Roles
Behavioral Scientists
Content Authors: build the actual substance of the BIT or intervention
Textual content
Graphical Content
Technology Support
Other (Medical/Anthropology/Etc)
User Centered Development
The user can be… wrong.
give them what YOU want
User centered Development: The user is always right
Who is the user? How to define your audience impacts everything.
How old are they? Are they children?
What is the target population?
What do they use technology for? [Ex: are they shoppers? Are they film lovers? Are they e-daters?]
Are they sophisticated users or basic users?
Will they have Internet connectivity?
Will they have motor skill, comprehension, or cognitive load issues?
More (and more) [and more…]
Dimensions of Usability
Effectiveness/Performance - how much time did it take, how many steps to complete a task?

Efficiency/Accuracy - how many mistakes were made?

Recall - how much does a person remember?

Satisfaction - Did they like it?
User centered Development:
The Role of Iterative Design
Nothing is EVER perfect
The disruptive nature of technological obsolescence
The disruptive nature of social trends
Building from your wish list
Building for the future
Security and Privacy
Be prepared and knowledgeable
Know the field and institutional repercussions (scare yourself into compliance)
Be ready for an explosion of data
Research Implementation: Issues of e/mHealth research
Assessing the participants expertise:
Different users require different levels of support
Managing studies at a distance
Handling emergencies
Resources for participants
Ditch the paper
Consider combining existing surveys and assessments into the technology
Data collection at a distance
Managing large datasets in a conscientious and efficient way must be deliberate—not an afterthought
Choosing the right technology: Summary
Examine your actual study needs critically
Learn as much as you can about the technologies you are thinking of using
Use the right tool for the job
Recognize the strengths and weaknesses of different approaches
Partner with the right people…
A Relationship Built on Trust
You will probably never know as much about technology as your technologist, but you should want to
Your technologist will probably never know as much about your specialty or research as you do—AND THEY SHOULD WANT TO
Choosing a technologist:
What are the roles?
Write code
Make things look pretty
Ux / HCI Specialists
User experience / Human Computer Interaction

Jack of all trades (at times masters of a few)
Choosing a technologist: Summary
Build a relationship based on trust
Learn to manage and navigate the cultural differences between researchers and technologists
Understand the different technologist roles
Understand the need for project management…
Research Study VS Product Development
Technology projects are PRODUCT driven (a functional thing must be provided at the end)
Products are often designed as static tools
Products are often expensive to iterate
Research projects have different goals and aims than technology products—
Evaluation: The data needed to optimize and validate a BIT is often much more extensive than data that is typically collected by commercial products
User management: Users aren’t just users, they are PATIENTS
Flexibility centers around understanding and increasing knowledge, not profit
What is a Design Document?

A communication tool where the project specifications are written down when they are agreed upon
A location to use for reference when misunderstandings occur
They contain a clear diagram of purpose, participant information, intervention components, and data needs
They contain a basic flowchart/diagrams of the graphical user interfaces (GUI) of your technology
They should be versioned and can be used as a historical document at the study’s conclusion
Key Questions-
How do you want your technology to be delivered?
What makes your behavioral intervention special?
What makes the use of technology in your BIT special?
How is your intervention staged?
Is your intervention tailored for a specific user, population or study arm?
Do you have the infrastructure in place to manage participants?
What kinds of reports and data do you need?
Resources and Limitations
Research VS Industry
Limited resource level requires proper allocation
User Centered Design: Summary
User focused (them)
Author focused (you)
Iterative—expect the need to constantly evolve
Incremental—grow as you go
Formative & Summative
Trust in field testing
Security and Privacy
Security and Privacy Failures
Research Implementation: The Role of Participants
Recruitment of participants
Online recruitment options

Enrollment of participants
Explaining privacy/security issues

Retention of participants
Types of outreach – adherence and connection to the trial
Ongoing tech needs
Research Implementation: Summary
Technology adds many options for participant recruitment and retention
Technology adds new complexities– be prepared!
Grant Applications
eHealth, mHealth, BITs...
Your Technologist and You
Building your BIT project
User Centered Development
Security and Privacy
Research Implementation
Grant Funding
Final Thoughts

Today's Agenda
Does the majority of the logical complexity occur on a user's machine?
Does it require an offsite server to perform the heavy lifting?
Always Online
(most web apps)
Sometimes Offline
(most phone apps)
Always Offline
(such as MEMS Caps / Actigraph)
Internet App
(the internet is a “place”)
Web App
(a manner in which the Internet is utilized, requires a browser)
Native App
(runs on a device WITHOUT requiring a web browser--Phone Apps or Microsoft Office)
shared vocabularies and taxonomies that model a domain
Technology failure
User assistance
N size
Integrating other information systems (like EMR, GIS)
“Setting it and forgetting it”

give them what THEY want
The user is right
The user may be wrong
Type of Breach
500+ Breaches
Location of Breach
500+ Breaches
We took existing offline interventions
Humble and Farley 2010
Project Onward
Embeds a web-based intervention in a peer network to facilitate management of distress among cancer survivors.
Duffecy J, Begale M, Mohr DC. Et al., Project onward: an innovative e-health intervention for cancer survivors. Psychooncology. Mar 21 2012.
Inaugurated in September of 2011, CBITs is supported by Northwestern University, Feinberg School of Medicine
Department of Preventive Medicine
Division of General Internal Medicine
Department of Psychiatry
Department of Medical Social Sciences.
What is a Behavioral Intervention Technology (BIT)?
eHealth: A broad term that can encompass web-based intervention, mHealth, use of EHRs for physician-patient and physician-physician communication, health informatics…
No good term for patient-facing technology-supported behavioral and psychological interventions
BITs integrate information, communications and media technologies with behavioral theory and science to create interventions that promote physical and mental health through behavior change.
A multi-media web-based intervention aimed at reducing depression among primary care patients
Monitoring of symptoms & behaviors
Graphic feedback
Mohr DC, Duffecy J, Begale., et al. J Med Internet Res. 2010;12(5):e48.
Mobile intervention aimed at reducing depression.
Develops context sensing. Embedded sensor data harnessed to interpret user
Social Context
Sensed states used for out reach to encourage positive behavior.
Burns, Duffecy, Begale, Mohr. J Med Internet Res. 2011;13(3):e55.
Purple Development Environment
BITs Development infrastructure that facilitates:
Lesson and tool authoring and editing
Social network management
Patient management and recruitment
Sensor data acquisition
Data mining
Reporting, graphing, &
Intervention tailoring
Storage of patient
use and outcome data
Deployment to web, phones,

PDE serves as a living repository of
information, tools, and expertise
from collaborators
Medication Adherence

Social Cognitive Theory
Cognitive Behavioral Theory
Theory of Planned Behavior

The study of the interaction between people and technologies
Technology Acceptance Model (TAM), a simple model of use derived from Theory of Planned Behavior (eg. Sumak, B. Comp Hum Beh 2011:27;2067-2077)
A BIT Ontology
Control Systems Engineering is one methodology that examines how to influence dynamical systems to achieve a desired outcome.
Mature computational framework for modeling feedback control systems, simulation and systematic decision making over time.
Riley, W. et al. Translational Beh Med 2011;1:53-71
Rivera, DE. Drug Alc Dep, 2007:88S;31-40
Relationship between behavioral scientist and technologist.
Ensuring understanding
Documenting processes
Process of development
Iterative and incremental
Speed of Change
Behavioral scientists come from an environment that evolves slowly
Technology evolves quickly and disruptively (obsolescence and innovation)
User behaviors are often different than expected
Unknowns in technology development (e.g. bugs)
BITs Development to Date
Much of the development of BITs to date has relied on existing models and theories of behavior change
Many of BIT design decisions are driven only loosely by behavioral and psychological theory, and mostly by educated guesses.
There is little theory available to guide us in the basic decisions that confront us in designing and developing BITs.
Granular Definition of Terms and Relationships
Ontology engineering studies methods of creating formal representations of concepts and relationships among the concepts.
For example: a granular definition of the BIT features, user interactions, and psychological and environmental factors that impact use of BIT features.
Development of these ontologies would
Provide a formal, shared, agreed-upon definition of terms and relationships
Facilitate the development of open architectures, interoperability, and sharing of knowledge, features, and technologies
Support more systematic investigation of methods of optimizing the efficacy of BITs
Ontologies must be optimally structured against the knowledge base they represent, and must be iterated and incremented
A BIT Ontology
“The most profound technologies are those that disappear.
They weave themselves into the fabric of everyday life
until they are indistiguishable from it.”
(Weiser, M., Scientific American. 1991, Sept, 94-100)
C, C++
Build Your Own
International Standard, ISO 9241-11
And some examples...
Browser Adoption Rates
Browser Performance Comparison
(ZDNet Performance Battery 2/12)
Higher is better
Building eHealth and mHealth Interventions
Realm of Human Computer Interaction Research
Realm of Standard Psychological Theories
Centered Development
And put them
Full transcript