A great way to make complex decisions in healthcare: Multiple criteria decision analysis

RB
by Richard Body1 view·Updated Sep 1, 2023
Create your own

Chapters

In this video I go through the basics of multiple criteria decision analysis: a great technique to use to guide decisions when there are multiple options and mutliple criteria to judge them against.

Audio TranscriptAuto-generated

  1. Hi. So you may know that I've got
  2. an interest in prehospital point of care testing
  3. and
  4. together with a team in Bristol, uh, led by Jonathan Beng.
  5. We were lucky enough to get an N.
  6. I HR grant to work up an application for a platform trial to evaluate
  7. prehospital point of care testing.
  8. And as part of this, we had to decide which tests
  9. might go into the platform first.
  10. So which ones were the highest priority to evaluate as part of a clinical trial?
  11. We needed a system to do that.
  12. Now.
  13. Many years ago, I did a course on multiple criteria decision analysis,
  14. and I've always wanted to use this technique.
  15. Well, now we've finally got to use it, and we've been doing this for a few months now.
  16. But today we just had a workshop where we put some of the
  17. finishing touches to our first multiple criteria Decision analysis or M CD A.
  18. And I want to tell you about it because this is
  19. a really powerful method that you could use to help inform your
  20. research or actually to make any complex decision where there are
  21. a number of different options and a number of different things,
  22. criteria that you've got to think about to weigh up against each other.
  23. So let me tell you about how you do multiple criteria, decision analysis.
  24. So I'm gonna just take you through a little tour, basically of the method.
  25. So,
  26. um, this method is used when you've got multiple options.
  27. So in this case, you know,
  28. we've got numerous different options for the platform trial.
  29. We could look at troponin testing for the diagnosing myocardial infarction,
  30. lactate for sepsis, lactate for trauma.
  31. We could look at B to HCG for pregnancy testing.
  32. We could look at ketones for diagnosing diabetic ketoacidosis.
  33. There's loads of different options and we can't choose them all.
  34. We've got to prioritise the top ones.
  35. So we've got to have a system to do that.
  36. But we also have multiple criteria. So there's not just one
  37. sort of key metric that we need to look at to decide that, Whether you know,
  38. this is the top priority to be evaluated in a trial,
  39. there are multiple different criteria that we might want to look at.
  40. So when those circumstances are present,
  41. that's when
  42. we might decide to use multiple criteria. Decision analysis.
  43. So the first thing we've got to think of in this case in our case
  44. study is what are we trying to achieve through prehospital point of care testing?
  45. What's the objective of it?
  46. And it may be
  47. that there's not just one objective, but there's multiple objectives.
  48. We're trying to do all sorts of different things. So, for example,
  49. we may be
  50. trying to improve patient safety.
  51. So it may be that we're trying to, uh, well,
  52. reduce mortality or morbidity for patients.
  53. Uh, it may be that we're trying to improve efficiency,
  54. So we're actually trying to make better use of healthcare, um, resources.
  55. Or it may be that we're trying to do more with the funds that are available to us.
  56. So we're essentially trying to find a lower cost option.
  57. And all of these things might be important
  58. for the thing that we're trying to achieve.
  59. And indeed, in a platform trial,
  60. when we think about prehospital point of care testing,
  61. all of those things are important.
  62. We've got to thinking about having an efficient health care system
  63. that,
  64. um
  65. has the lowest cost possible to get the best patient outcomes.
  66. But it's also going to have the best patient safety,
  67. and we can measure that in all sorts of different ways.
  68. So we have multiple different objectives.
  69. So how do we do this? Well, the first step is to identify
  70. the well, the options.
  71. First of all, I should say, the options, the tests, that might go into the, um,
  72. multiple criteria decision analysis.
  73. So we might do that.
  74. For example, by reviewing the literature,
  75. we might do a literature search to see what's out there.
  76. Um, what? We might look at manufacturers so we might go to industry bodies.
  77. We might go to companies that we know of, Um
  78. uh, or we might do searches to see where we see companies that, uh,
  79. that have an interest in this area.
  80. And then we might see what's available, what's on the market.
  81. Those might be the options, and then we can look for evidence around those,
  82. and then once we've got the options we're going to see Well,
  83. what a criteria are we going to measure them by?
  84. So these these These are the things that we're going to use
  85. to decide whether these tests should go into our platform trial,
  86. and it might be helpful to think of it instead of criteria.
  87. Think of it as a set of advantages and disadvantages of each option.
  88. OK, so we do that
  89. and then we've got to think right, So we've got a list of criteria
  90. and we're gonna do that with the options.
  91. We're gonna do that by looking at you know what, companies manufacturing.
  92. Looking at the published literature with the criteria.
  93. We've got to work with stakeholders.
  94. So we got to find out who the stakeholders are, invite them to take part
  95. and what we did in our programme. We, um
  96. we got the stakeholders to do a survey,
  97. so they took part in a survey to sort of brainstorm ideas.
  98. And then we held a workshop at which we discussed all
  99. of the ideas that have come out of the survey.
  100. Uh, and we refined them.
  101. We thought of more ideas at the workshop,
  102. and then we had a second round of survey where we
  103. put forward the ideas that have come together so far,
  104. and we tried to get consensus on some of those ideas.
  105. We also then use that survey on the workshop to try
  106. and help us understand which of those criteria are vital.
  107. So are there any deal breakers that mean
  108. that one of the options that we're considering
  109. is actually got to be thrown out?
  110. So is it that you know, for a point of care test in the
  111. hospital environment, for example, if it's massive
  112. and it's not gonna fit in an ambulance,
  113. then it's a bit of a deal breaker because we can't use it.
  114. If it doesn't fit in the ambulance, it hasn't got a power supply
  115. can use in an ambulance, and it's a deal breaker. It's gonna be out that sort of thing.
  116. We we wanted the stakeholders to try and identify
  117. what are the deal breakers,
  118. and that's gonna help us to trim down our long list
  119. of options down to a much more manageable shortlist for consideration.
  120. And then the last thing we're going to do
  121. is weight. The criteria,
  122. so really importantly, we've gonna be weighing out different outcomes,
  123. so we're gonna be looking at, For example,
  124. one pathway might have an advantage in that
  125. the mortality rate is reduced over standard care.
  126. But then that might cost a lot more than another pathway.
  127. Another car pathway
  128. might help us to,
  129. um, free up ambulances,
  130. but at the same time,
  131. uh, the accuracy might be slightly lower,
  132. so we we we can't wait all of those things equally.
  133. We're gonna have to have some kind of system
  134. to determine. Well, which one is most important? So we have to weight the criteria.
  135. We have to assign a weight.
  136. So it may be, for example, that we say, Well, you know,
  137. it's really important that the test has good accuracy, diagnostic accuracy,
  138. So we'll assign a really high weight to that.
  139. Uh, whereas cost, it's important, but it's not quite as important as accuracy.
  140. So we give a lower weight,
  141. and we're going to use that later when we get a scoring system
  142. to decide on which option is best.
  143. So what we did for that is we we set we we had to have classification systems.
  144. We proposed classification systems for each of the criteria,
  145. and we asked, How important do you think this is? And we did that from 0 to 10,
  146. and the stakeholders respond, and then we get the average of those responses,
  147. and that's what we used to assign the weight to determine how important.
  148. Each criterion is
  149. and then we have a workshop with all of the stakeholders.
  150. We present them with all of the information they have pre
  151. reading material and then we go through it at the workshop.
  152. They have a chance to ask questions
  153. and then we do the live scoring and that's where it gets really exciting.
  154. It's a bit like the Eurovision song contest,
  155. because we we score all all of these options as
  156. we go through and we get live results as well.
  157. So that's when it's really exciting.
  158. It's a great method. I really enjoy using it.
  159. We're going to write it up so we'll hear more about it.
  160. But I wanted to say a little bit about it now because hopefully you might
  161. be able to use it and it may not just be research you may want,
  162. Let's say, want to buy a new point of care test in your emergency department.
  163. You're not sure which one of several options to choose
  164. this could give you an easy method that you could apply
  165. to decide which one is for your emergency department.
  166. I hope that helps