Evidence-Based Management_Certified Course

Description

The basic premise behind evidence-based management (EBM) is that good decisions require both critical thinking and use of “best available evidence.” This “evidence” may come from scientific research, but good internal business information and mindful professional experience also constitute “evidence.” All practitioners use evidence in their decisions, but most pay little attention to the quality of the evidence and base their decisions on too few sources of evidence. The result is business decisions based on fads, so-called “cutting edge” solutions, and the pop concepts promoted by consulting firms and management gurus. The bottom line is bad decisions, poor outcomes, and no understanding of why things go wrong. EBM evolved in response to this problem with its goal of improving the quality of decision making by using critically evaluated evidence from multiple sources – organizational data, professional expertise, stakeholder values, and the scientific literature.

Although EBM sounds straightforward, gathering, understanding and applying evidence requires a particular set of skills and practical knowledge. These certified course modules will develop your evidence-based skills and enhance your understanding of how an evidence-based approach can support your organization’s decision-making.

CEBMa members

The first way to get access is by signing up as a member of The Center for Evidence Based Management (CEBMa). Associate members get access to four modules: module 1 to 4. Professional members get access to six modules: module 1 to 4 and module 6 & 7. After successfully completing the modules you will receive a CEBMa certificate of completion. You can find more information about becoming a CEBMa member here. (link: https://www.cebma.org/membership/)

CMU’s Heinz College

Another way to get access is through the executive education programs of Carnegie Mellon University’s Heinz College. After successfully completing additional modules you will receive a CMU certificate of completion. If you successfully complete all course modules, you will receive an official micro credential from Carnegie Mellon University. If you are interested in this option, please contact CEBMa for more information. (link: https://www.cebma.org/contact/)

Australian National University

The third way to get access is through the Australian National University’s Research School of Management. After successfully completing additional modules you will receive an ANU certificate of completion. If you successfully complete all course modules, you will receive an official micro credential from the Australian National University. If you are interested in this option, please contact CEBMa for more information.

The modules are developed by CEBMa and are part of a larger certified master course. If you are a teacher/lecturer and would like to use these (or additional) modules in your class, please contact CEBMa.

What students will learn

Module 1: The Basic Principles

Summarize the basic principles of evidence-based management

Explain why we need evidence-based management

Explain what counts as evidence

Determine which sources of evidence were consulted

Assess (coarsely and in general terms) the quality of evidence

Determine whether the ‘best available’ evidence was used in a decision-making process

Correct common misconceptions about evidence-based management.

Module 2: ASK

Identify (hidden) claims/assumptions regarding a practical issue

Determine whether an (assumed) problem is sufficiently clear

Determining whether there is sufficient evidence to support the (assumed) problem

Determine whether the preferred solution is sufficiently clear

Determine whether there is sufficient evidence (from multiple sources) to support the preferred solution.

Module 3: ACQUIRE – evidence from practitioners

Determine what evidence to acquire from practitioners

Determine how to prevent selection bias when acquiring evidence from practitioners

Determine the best method(s) to acquire evidence from practitioners

Determine whether bias could have affected evidence from practitioners

Formulate clear, unambiguous, and unbiased questions.

Module 4: APPRAISE – evidence from practitioners

Assess whether professional experience is valid and reliable

Grade the trustworthiness of professional experience

Recognize how system 1 thinking influences valid and reliable professional expertise

Determine whether a decision is based on system 1 or system 2 thinking

Recognize common cognitive biases

Identify ways cognitive biases can be overcome

Critically appraise evidence from practitioners.

Module 5: ACQUIRE – scientific evidence

Determine the most relevant online research database(s) given the question

Determine whether a journal is peer reviewed

Determine the most important PICOC terms

Search the Internet for relevant alternative and/or related terms

Search Google Scholar for related or broader academic terms

Test search terms to identify terms that yield the most relevant results

Apply Boolean operators to specify a search query

Use the history function to combine search queries

Apply methodological filters to identify meta-analyses and/or longitudinal/controlled studies

Narrowing search results by adding additional PICOC terms

Limit a search result by limiting the date range.

Module 6: A short introduction to science

Assess whether a study was conducted according to the scientific method

Recognize pseudo-science

Assess whether a statistically significant finding is of practical relevance

Assess whether methodological bias may have affected the results

Determine whether confounders may have affected the results

Assess whether a placebo effect may have affected the results

Identify moderators or mediators that may have affected the results

Distinguish quantitative research methods from qualitative research methods

Determine a study’s research design

Efficiently read a research paper.

Module 7: APPRAISE – scientific evidence

Assess the impact of an effect size

Assess whether a statistically significant finding is of practical relevance

Assess whether a confidence interval is sufficiently narrow

Assess whether an outcome was measured in a reliable way

Distinguish cause-and-effect questions from non-effect questions

Determine a study’s research design

Assessing whether a study’s research design is appropriate given the research question (methodological appropriateness)

Assessing a study’s methodological quality

Grading a study’s trustworthiness on the basis of it’s methodological appropriateness and quality

Summarizing a study’s main findings, weaknesses, and overall trustworthiness.

Module 8: ACQUIRE – organizational evidence

Module coming soon.

Module 9: APPRAISE – organizational evidence

Determine whether a logic model was used to collect and analyze evidence from the organization

Assess whether organizational data are relevant

Identify steps in the collection and processing of data that could introduce risk of inaccurate data

Determine whether contextual information is missing

Determine whether there could be measurement error

Assess whether there could be a small number problem

Determine whether a metric is a good representation of the data

Interpret a metric’s standard deviation

Assess whether a graph represents the data in a valid and reliable way

Interpret a correlation or regression coefficient

Determine whether a correlation- or regression coefficient is practically relevant

Assess whether there are outliers that may distort the evidence

Assessing whether range restriction may have affected the evidence

Assess whether a confidence interval is sufficiently narrow.

Module 10 & 11: ACQUIRE/ APPRAISE – stakeholder evidence

Module coming soon.

Module 12: AGGREGATE

Explaining what proof, evidence, chance, and ‘conditional’ probability means

Assessing the impact of a prior probability

Estimating the likelihood of the evidence: P(E|Htrue) and P(E|Hfalse)

Updating the probability of a claim/assumption/hypothesis when new evidence comes available

Aggregating evidence from multiple sources by applying Bayes Rule.

Module 13: APPLY

Use the PICOC method to determine whether the evidence applies to the organizational context

Determine whether a decision/intervention gives you the biggest bang for your buck

Assess the level of risk inherent in a decision/intervention

Identify ethical issues that need to be considered

Determine whether (and if so, how) the evidence is actionable

Determine whether there are moderators that need to be taken into account

Determine, given the type of decision at hand, how and in what form the evidence can be applied.

Module 14: ASSESS

Identify the type of decision (to be) made (routine, non-routine, or novel/hyper complex)

Determine whether a decision was executed as planned

Assess an outcome using the gold standard method

Assessing an outcome using quasi- or non-experimental methods

Suggest ways to improve the validity and reliability of an outcome assessment

Assess whether an outcome was measured in a reliable way

Assess whether indirect and intangible costs were taken into account

Assess the (unintended) consequences of a decision on stakeholders.