Introduce Research

• Study design/methods
• Appraisal
• Interpretation
• Application/utilization

Understand

• Evidence Informed Decision Making Model
• Sources of Evidence
• Types of research that inform practice (quantitative, qualitative, mixed methods)
• Why research studies should be critically appraised

1.2 Research, EBP & EIDM

• There are different ways of knowing; Empirical, Personal, Aesthetics, Ethical

1. Empirical (focus of class)

• Theories, models, facts
• Validation, confirmation
• Scientific competence

2. Personal knowledge

• Person stories, self
• Reflection, response
• Therapeutic use of self

3. Aesthetics knowledge

• Experience of nursing, health, illness
• Appreciation, grasp meaning
• Transformative act/acts

4. Ethical Knowledge

• Principles, codes
• Justification, dialogue

Evidence-Based Practice (EBP)

• A paradigm (model) and life-long problem solving approach to clinical decision-making that involves the conscientious use of the best available evidence (including a systematic search for and critical appraisal of the most relevant evidence to answer a clinical question) with one's own clinical expertise and patient values and preferences to improve outcomes for individuals, groups, communities, and systems
• Research utilization is different from EBP- utilization is just focused on using research and looking at the findings of research
• In the past, EBP was often criticized for being restrictive

Why must we use Evidence-Based Practice (EBP)?

• There are billions of dollars spent on health-related research
• It takes ~17 years to get research into recommended policy or practice (this is outrageous)
• 30-50% of people receive recommended care but 30-40% of patients do not get evidence-based care, they instead get care that is known to be ineffective or even harmful
• EBP improves outcomes, assesses risk vs harm for patients, we need to get into EBP to decrease the amount of time to get research into recommended policy

EIDM in a Clinical Setting (Clinical Expertise) - Model

1. Clinical state, setting and circumstances
2. Patient preferences and actions
3. Research evidence
4. Health care sources

• When making decisions, you must factor all of these bubbles and these bubbles may scale differently

• Think of context, preferences, evidence, and resources

7 Steps of EIDM

1. Define: Define the clinical issues & formulate a focused, structured research question (could be PICO or PS)
2. Search: conduct an efficient literature search to find evidence (search databases)
3. Appraise (focus of our class): critically appraise the evidence
4. Synthesize: combine & summarize evidence
5. Apply: Apply evidence to clinical issue and make decision using clinical expertise, patient's preferences and consideration of resources
6. Implement: implement intervention/treatment
7. Evaluate: outcome

1.3 The Research Process

1. Question formulation

• Identify a clinical problem or issue
• Review & critical appraisal of the research literature (for pre-context)
• Determine the need for & purpose of research

2. Study design & planning
3. Data collection
4. Data analysis
5. Conclusions

• Implications for practice & future research

6. Dissemination of study findings

• Spreading and sharing study findings

7. Utilization

• Apply research findings to clinical practice

• Here, you can see how research process ties into EBP

Study Designs & Methods

• Methods section needs to be very detailed; the designs and research methods should match the purpose of research and research question
• They must describe in detail the methods and design to know what was done
• They must have established criteria for appraising the appropriateness of the methods and assessing the validity of the results
• The design and methods should always follow from the research question (the question determines the method)
• Study designs and research methods should match the purpose of research and the type of question to be answered

Types of Research Methods

1. Quantitative Research (objective)

• Generates numerical data and results (raw data)
• Data collected using reliable & valid measurement tools
• Involves statistical anaylsis of data, usually 1/3 types; descriptive (prevalence of events/conditions, means and standard deviations), comparative (ex, difference in mean depression scores between intervention and control group), relationship (factors shaping quality of life)
• Quantitative study designs can be experimental or non-experimental
• Experimental designs are interventions such as randomized controlled trials or quasi-experimental
• Non-experimental designs are observational studies such as cohort design, case-control study, or cross-sectional study

2. Qualitative Research (subjective)

• Used to explore beliefs, experiences, attitudes, behaviors & interactions
• Generates non-numerical data
• Data collected through focus groups, individual interviews, document review, participant observation, etc
• Involves anaylsis of narrative data
• The big three of qualitative research designs includes grounded theory (sociology) which focuses on developing theory from data, phenomenology (philosophy) which focuses on experiences and ethnography (anthropology) which focuses on people and their cultures
• Other designs that are commonly used are qualitative descriptive (just describes), interpretive description (trying to extend and interpret raw data), focused ethnography, case studies

3. Mixed Methods Research

• Methodology that involves collecting, analyzing and integrating quantitative (ex, experiments, surveys) and qualitative (ex, focus groups, interviews) research

• Here, qualitative data can be collected and analyzed to support the development of the intervention, understand contextual factors related to the implementation of the intervention, and/or explain results

Research Questions (formulating the research question)

1. Quantitative (PICO) format for foreground questions

• P: Population
• I: Intervention/Exposure
• C: Comparison/Counter exposure
• O: Outcome
• Ex, In adults with untreated hypertension, what is the effect of polyphenol-rich dark chocolate compared to white chocolate in lowering blood pressure?
• Make sure you don't give accidental bias or direction

2. Qualitative- PS format for foreground question

• P: Population
• S: Situation
• Ex, What are the caregiving experiences of first generation immigrant parents of children with cancer?

1.6: Focused, Structured & Answerable Research Questions

Q: Why Write a Focused Research Question?

• PICO(T) & PS questions-> are searchable, answerable questions created from clinical issues
• The question determines the research method-> the best evidence comes from studies that use the study design most appropriate for the question -> seek pre-appraised evidence first (as high up the pyramid), and if there isn't any, then critically appraise studies yourself

Developing a Well-Built Clinical Q

• The anatomy consists of either PICO or PS
• The Question needs to identify 1) the key problem of the patient 2) what treatment or tests you are considering for the patient 3) what alternative treatment or tests are being considered (if any) and, 4) what is the desired outcome to promote or avoid
• Two additional elements to consider in building a clinical question is the 1) the type of question and 2) the type of study
• Remember the question determines the research method/study design

• If there is harm to a patient or in situations where it is dangerous to not receive a medication, we do not use a RCT
• Remember cohort, case controls and cross-sectional studies are quantitative data (non-experimental)

Acquiring the Evidence to Answer Well-Built Clinical Questions

• Constructing a well-built clinical question can lead directly to a well-built search strategy
• You may not use all the information in the clinical question for your search strategy
• Example:

1.7 The 5S & 6S Pyramids

Critical Appraisal

• Addresses two broad questions:

1. Internal validity: are the research design and methods likely to produce results that are true or valid? (think small first)
2. External validity: If the answer to the first question is 'yes', can the results be applied to the clinical issue or problem?

• There are several sources of pre-appraised and summarized research to simplify critical appraisal (5S/6S pyramid… see later)
• Where pre-appraised research is not available, current critical appraisal tools specific to each study design exist to help you conduct your own critical appraisal

Putting it all together

• Empirical knowledge= research evidence
• EBP & EIDM: Use of research evidence in clinical decision-making (combined with patient situation & preference , resources, expertise)
• Critical appraisal: finding high quality evidence for EBP & EIDM
• Critical appraisal recognizes that different clinical questions require different study designs as the question determines the method
• Final evidence- start at the top of the pyramid!

• In the 6S pyramid, guidelines are summaries, systematic reviews are syntheses, and individual studies are at the bottom

Hills Criteria of Causation (background)

• Hill was a British medical statistician (1897-1991) who developed 9 criteria for determining the causal link between a specific factor and a disease

Uses of Hill's criteria:

• Basis of epidemiological research, which attempts to establish scientifically-valid causal links between potential disease agents and many diseases
• Identification of study designs offering strongest evidence
• Appraising evidence obtained from multiple studies

1. Temporal Relationship

• Exposure always precedes the outcome
• "For example; if factor "A" is believed to cause the disease, then factor "A" must always precede the occurrence of the disease

2. Strength of Association

• The stronger the association, the more likely the relationship between "a" and "b" is causal
• For example; the higher the correlation between sodium consumption and hypertension, the stronger the relationship between sodium and hypertension
• The higher the odds of 30-day hospital re-admission in post-surgical patients, the stronger the relationship between surgery and 30-day hospital re-admission

3. Dose-response Relationship

• An increasing amount of exposure (dose), increases the amount of response (outcome)
• If a dose-relationship relationship exists-> stronger evidence for causal relationship (compared to absence of relationship)
• Similarly, if a certain factor is the cause of a disease, the incidence of a disease should decline when exposure to the factor is reduced or eliminated

4. Consistency of Association

• Finding a consistent relationship between a factor and outcome across different studies with different populations-> stronger evidence of causal relationship (compared to inconsistent findings)

5. Biological Plausibility

• Association agrees with accepted understanding of pathological processes (there needs to be a theoretical basis aka it needs to make sense) from physiological/pathophysiological perspective

6. Experimental Evidence

• The association between exposure and outcome can be supported with experimental evidence -> is there research to substantiate this causal relationship

7. Alternate Explanations

• Determine the extent to which researchers have taken other possible explanations into account and ruled out these alternate explanations (leaves this explanation to standalone)

8. Specificity

• Established when a single cause produces a specific effect (single cause causes a specific effect)
• Weakest of criteria for causation
• Absence of specificity does not negate a causal relationship

9. Coherence

• The association should be compatible with existing theory and knowledge
• Can be substantiated through existing theory or prior knowledge

Features, Advantages & Disadvantages of Different Quantitative Designs

Q: What is a study design?

• Refers to the way a study is organized/constructed & methods used
• Quantitative Study designs are best for questions about:

1. Cause of disease (etiology)
2. Prognosis
3. Diagnosis
4. Prevention
5. Treatment
6. Economics of a health problem

Q: Quantitative Study Designs: Hierarchy

1. Experimental Designs (best)

• RCTs ; patients are randomly assigned to treatment groups
• In order for randomization to occur, there should be a equal chance for individuals to be in intervention or treatment group
• You can do this by generating a randomizer sequence via computer and allocation of sequence is concealed (no one can manipulate sequence)
• Point of randomization is to ensure each of groups are balanced in terms of confounders
• We want to ensure they're balanced because they can influence the outcome
• RCT is gold standard because we can be confident that this intervention caused this outcome
• Participants are followed forward in time (prospectively) from exposure to outcome
• Two types outcomes can be measured; continuous and discrete ; continuous outcomes are numeric values that can continually be evaluated such as BP and measured, meanwhile discrete outcomes are more concrete such as outcomes such as yes or no

Q: What are some strengths and weaknesses of RCT?

• Cost is expensive in the sense of time; time is needed to ensure good randomization
• We want to see 80% of patients retained in study so anything greater than 20% dropout is a red flag
• Generalizability= selection bias
• Ethics: it may not always be appropriate; especially if you are withholding a particular treatment that you know has benefits; is it fair to deny the intervention to the control group if for instance you deny cancer patients access to a drug that offers hope with no known risks
• Or is it feasible to deliver the intervention to only some members of the eligible population such as studying the effects of a new management system for hospital nursing staff (usually these are implemented hospital wide so you would need to implement for one hospital while not implementing for another)

2. Quasi-Experimental Designs

• Cohort analytic: only difference from RCT is that it's not random
• Participants can choose to in treatment or control group
• Selection bias; no equal chance so raises confounding variable and decreases our confidence in the outcomes

Q: What are some strengths and weaknesses of cohort analytic?

• Cost can be expensive depending on intervention and how long
   

3. Non-experimental designs (worst)

• Cohort
• Case-control
• Cross-sectional
• These studies are usually associated with PECO questions
• Looks at predictive factors such as a sugar as a predictor for obesity

1. Cohort

• Interested in the likelihood that people will experience or develop an outcome given their exposure to a disease, condition or situation (prognosis question)
• Ex, how likely are patients with ulcerative colitis to develop bowel cancer?
• Where is the comparator though?

Q: What are the strengths and weaknesses of cohort designs?

• Best evidence for evaluating risk factors
• Big sample size
• No one to compare to
• Because of time factor, confounding variables can creep into play as well

2. Case-control study

• Participants with and without the outcome are identified
• Investigators look back in time (retrospectively) from outcome to exposure
• Investigators often try to match cases & controls; this helps to ensure groups are as similar as possible regarding important variables that may influence the outcome (ex, age, sex)

• From an ethics perspective; you don't want to cause the exposure

Q: What are the strengths and weaknesses of the case-control design?

• Control group: it doesn't mean it's difficult to find control group, but it is difficult to find control group that is comparable for confounding factors

3. Descriptive/cross-sectional survey

• A group of people are interviewed or asked to complete a survey to determine whether they have experienced an exposure of interest and an outcome of interest
• Exposure and outcome are measured simultaneously
• Ex, a group of women interviewed to determine if use of technology & whether they had a miscarriage

• Now here, it might be difficult to find control group at all

Questions & Designs

1. Typically questions about:

• Effectiveness of prevention and treatment interventions-> RCTs are the best
• Cause of health problem (causation )-> RCTs are the best again; but if random allocation is unethical, then cohort analytic design is used and if outcome is rare or takes too long to develop, then case-control design
• For the course of a health state or condition (prognosis): a cohort study is used

Sources of Bias in Quantitative Research Studies

1. Publication Bias: bias against negative findings (not published)* studies that were not published since they didn't achieve significant results that they wanted
2. Selection Bias: related to sampling; those who get into either group should not be different from each other or the population they represent
3. Exposure & Recall Bias: Studies can struggle to confirm temporality of exposure and/exposure characteristics (ex, case control studies)
4. Measurement bias: Instruments should be validated & reliable for study populations

• Validated means that it can measure what it needs to measure and reliable means that it can repeatedly do that
• Interviewer: interviewers aware of the status of participants may probe ore or less deeply

5. Confounders: alternative explanation of findings; can occur when a factor is related to BOTH exposure and outcome and is not recognized/controlled

• Can make effects inflated or deflated
• Example; strong relationship between coffee consumption & lung cancer is confounded by smoking (likely cause of both)

Ways to Minimize Bias

• Search for unpublished results (ex, find abstracts of conferences, call researcher)
• Blinding/masking (participants, interventionists, data collectors, outcome assessors, data analysts) but this can't always happen
• Strict adherence to research protocol; establish and publish study protocol, ensure rigorous study design, develop prospective hypotheses and analytical plans, applies to intervention procedures and measuring outcomes
• Strict follow-up of participants; compare groups; compare completers to dropouts
• Careful matching/control groups

Dependent vs Independent Variables

• Independent variable is the presumed cause ; variable that is varied or manipulated by researcher
• Dependent variable is the presumed effect ; response that is measured

Data Types (levels of measurement)

1. Nominal (attributes are only named); weakest
2. Ordinal (attributes can be ordered) such as a pain scale but don't know the magnitude of differences
3. Interval (distance is meaningful), such as temperature and distance
4. Ratio scale (absolute zero)

• These are important as it helps to interpret the data-> if a variable is just nominal (Categorical), then you know the numerical values are just codes for name
• If variable is nominal, you would not report means but instead report the number and proportion of sample having each value
• If a variable is ratio, you would report means

Reliability vs Validity

• A measurement must be reliable to be valid but reliability is not a guarantee of validity

Ethics in Choosing a Study Design (TCPS 2)- Canada

Respect for human dignity through 3 core principles;

1. Respect for persons

• Participants autonomy (free, informed, ongoing consent)
• Protecting vulnerable people (incapable of autonomy)

2. Concern for welfare

• Impacts to health or circumstances (physical, economic, social)

3. Justice

• Treating people fairly, equitably
• Recruitment process important

Sampling Concepts

1. Sample

• The group of people in a study; a subset of the population

2. Population

• All individuals to whom the study results should be applicable to; target group for study

3. Sampling

• Selecting a proportion of the population to study

4. Generalizability

• A portion of the population of interest is studied in the hope that the findings can be generalized beyond the sample to the population

5. External Validity

• Degree to which the findings of a study can be generalized beyond the study sample
• Can we apply our findings from our sample to the larger population?

Major Categories of Sampling

1. Probability Sampling

• Random selection of subjects where each subject has an equal chance of being selected
• Most likely representative of population (not perfect but close)
• Can be very expensive and intensive

2. Non-probability Sampling

• Selection of subjects by a non-random method
• Rarely representative of the target population
• May have limited generalizability due to sampling bias (interpret with caution)
• Convenient and economical

*: non-probability sampling may be used in quantitative or qualitative studies, although non-probability sampling is more likely to be used in qualitative studies

Probability Sampling: Examples

1. Simple Random Sampling

• Sampling frame (or a list of all population elements) is obtained

• A sample (of appropriate size) is selected at random- usually via a table of random numbers, a computer program, or an organization with expertise/services for supporting RCTs

2. Stratified Random Sampling

• Sample is divided into strata (subsets) and subjects are randomly selected from each stratum
• Enhances the sample's representatives

3. Cluster Sampling

• Sample is selected based on successive random sampling of units (ex, selection of a sample of schools, from which a sample of students are then randomly selected)
• Often used for national surveys

4. Systematic Sampling

• Sample size is determined; first subject is randomly selected and then every (k)th individual is selected (k= sampling interval)

Issues with Sampling

• Sampling Error: the gaps between the sample's representativeness and the population's known and unknown characteristics
• Generally decreases as sample size increases

1. Sampling Bias 1: Referral Filter Bias

• Type of healthcare access bias ; patients referred to tertiary care centres are typically more ill or have more rare disorders than patients who are well enough to be followed in community hospitals
• Therefore, selecting samples from tertiary care centres may mean your sample doesn't represent a typical patient with a particular disease (hard to generalize)

2. Sampling Bias 2: Volunteer Bias

• Individuals who volunteer to participate in studies may be different than those who tend not to volunteer (ex, may have different exposures or outcomes)

Assessing Sampling Bias

To assess whether sampling bias exists, ask:

• Who was included in the sample?
• What was the source of recruitment into the study?
• How were the subjects recruited?
• Which people were approached to be in the study? (Ie, consecutive sample, volunteers, other?)
• What were the inclusion and exclusion criteria? What were the demographics of the sample? Did they have medical conditions or other factor that could affect the results of the study?

Critical Appraisal of Intervention Studies

You need to ask yourself:

1. Are the results valid; how serious was the risk of bias? Are the methods done in a way where I am not confident in the study?

• Randomization of patients to control and intervention group: ensures groups are the same for known & unknown factors that might influence outcome; randomization is dependent on 1) generation of randomization sequence and 2) allocation concealment
• Allocation Concealment: ensures randomization by preventing investigators from knowing upcoming assignment (and trying to change them)
• If they do not talk about allocation concealment; say they didn't talk about it for RCTs; they should have at least mentioned it
• Length and completeness of follow up: ensures sufficient time has elapsed to see the outcome; and ensure, sufficient number complete the intervention
• Intention to treat: ensures randomization persist by analyzing patients in groups to which assigned (once randomized, always analyzed) and handling missing data
• Blinding: applies to each group involved in the execution, monitoring & reporting phases (patients, clinicians, data collectors, outcome assessors, data analysts)
• Baseline characteristics: similarity of groups (links to randomization)
• Variation between groups: limit variation to the intervention (intervention should be only difference)
• Measurement bias: instruments should be reliable and valid; assessors should be blinded

2. What are the results?
3. How can I apply the results to patient care?

Random Allocation (Randomization)

Q: Were participants randomized to the treatment and control groups?

• "Who/What" can be randomized?
• Individuals, families, hospitals, wards or health units (cluster) or towns
• Best methods for randomization in order include computer-generated methods (which does not allow for manipulation of randomization), an agency that has no involvement in patient recruitment (ex, pharmacy department), an external trials office, random number table, or coin toss (cumbersome)
• Non-random methods of allocation which are less than ideal are birth dates (odd/even), chart numbers, list of next up, day of week in clinic, convenience

Randomization: The Benefits

Q: Is the sample representative and large enough?

• Randomization aims to ensure the sample is representative of the population from which it originated
• Sample size depends on factors like the desired sampling error, power and effect size
• Larger samples provide more accurate results and enables you to generalize to the population
• There are also algorithms to calculate a prior sample size based n percentage of sampling error

Q: Are there baseline differences?

• Randomization reduces the likelihood of these differences

Allocation Concealment

Q: Was the process of allocation concealed?

• To ensure the clinician recruiting patients are unaware of which group the next patient will be allocated to (and thus cannot influence which group an individual is assigned to)
• Method of allocation concealment: use of sequentially numbered, opaque, sealed envelopes but even with this approach, people still peek
• The best approach is independent randomization service

Completeness and Length of Follow Up

Q: What was the follow-up? (i.e, how long? How complete?)

• Length: judgement as to whether the time period was long enough for individuals to experience the outcome of interest ; use clinical experts and common sense to make this judgement
• Completeness (or loss to follow up): percent of patients that complete the study
• Follow-up of greater than or equal to 80% is considered most ideal and compare intervention and control group
• If you have a large percentage dropping out, is there an explanation?
• Theoretically, everyone should be accounted for in a study

Intention to Treat Analysis

Q: Was there a ITT analysis?

• Were patients analyzed in the groups to which they were initially randomized?
• Once randomized, always analyzed; regardless of whether patients completed intervention, got the wrong treatment, got partial treatment, died partway through study
• Imputation may be done to achieve ITT where they take average values of those who completed study and use value for that missing individual
• ITT preserves randomization; if you start pulling people out, there are imbalances of prognostic factors so you undo randomization

What is ITT linked to?

• Linked to methods of handling missing data
• Problem is that depending on the assumption made about missing data, some methods of analysis that include all randomized individuals may be less valid than methods that do not include all randomized individuals
• The recommendation is that: employ ITT analysis strategy, comprising of a design that attempts to follow up all randomized individuals, a main analysis that is valid under a stated plausible assumption about the missing data, and sensitivity analyses that include all randomized individuals in order to explore the impact of departures from the assumption underlying the main analysis

Study Blinding

Q: Was the Study blinded?

• You need to consider if all individuals involved in the study (ex, patients, investigators, research staff) unaware of who is assigned to treatment or control groups?
• Patients, study personnel, data collectors, outcome assessors, data analysts, report writers, sponsors, healthcare providers can all be blinded
• To aid in blinding, some trial participants may get something that is not really a treatment or for real such as a placebo but less used today
• It's considered unethical though because you are tricking people

Contamination & Co-Intervention

Q: Were both groups of participants (intervention and control) treated equally except for the intervention?

• Co-intervention: extra care or treatment given to one group and not the other ; typically an add on t the intervention
• Contamination: participants in the control group accidentally get the intervention and vice versa

Baseline Differences

Q: Were the two groups similar at the start of the study?

• If not, was this difference taken into consideration during the analysis?
• Consider baseline or entry characteristics of participants in each group: emphasis on characteristics that are known to, or believed to, have an influence on the outcome of interest

Measurement Bias

• Are the measure reliable and valid?
• Is the measurement subject to bias? -> researcher can introduce bias through form of interview bias
• There may be bias in the tools you use if same tools are not consistently being used

Hypothesis Testing

• Quantitative research uses statistical measures to understand relationships between exposures/interventions and outcomes
• Answering research questions uses a process called hypothesis testing- process begins with null hypothesis:
• "Null hypothesis" / H(o) states that there is no difference between the intervention(s) and control group(s)
• The null hypothesis is assumed to be true until anaylsis of data from study suggests otherwise:
• If the difference between the groups is large enough, we may reject the null hypothesis-> accepting the alternative hypothesis stating that there is a difference between the groups

Hypothesis Testing: Two Errors

• Type I error: "false positive"- claiming that there is a difference when there isn't one (rejecting the null hypothesis when it's true)
• Type II error: "false negative"- claiming that there isn't a difference when there is one (accepting the null hypothesis when it's false)
• Alpha= error risk for type I error while B= error risk for type II error
• They are related in that when alpha gets smaller, Beta gets bigger
• Power= 1-B

Type I error

• Claiming that there is a difference between the groups when there isn't one
• To limit this error, researchers declare a level of significance (a) for the hypothesis test
• a= probability researchers accept of making a type I error
• For hypothesis test with a 95% confidence level, a= 5%

Type II error

• Claiming that there isn't a difference between groups when there is one
• Researchers would like to limit this error too, but:

1. Type I & II errors are related, decreasing one increases the otter
2. Type I error receives priority in hypothesis testing
3. Type I error (a) = type II error (b)= 20%
4. Most common reason for type II error: small sample size (smaller effects require large samples to detect)

Continuous Outcome Data

Example: Study testing a new weight loss drug compared to a currently prescribed drug (weight loss is measured in lbs)

• To analyze our results, we would look at the two groups as a whole by calculating the mean and standard deviation (SD) for each group:
• New drug (intervention) group of 25 people: mean weight loss of 14.4 lbs (SD= 8.5)
• Existing drug (control) Group of 25 people: mean weight loss of 7.3 lbs (SD= 4.5)
• There seems a difference between the means but is it a true difference?

The T-test (&ANOVA)

• We need a statistical test and appropriate confidence level to compare weight loss in the two groups
• Confidence level: 95% CI, alpha= 5%
• Statistical tests for comparing groups on a continuous outcome:

1. T-test is appropriate for comparing 2 groups
2. ANOVA is appropriate for comparing 3 + groups
3. Both tests determine if the difference between mean values of the groups are statistically significant

P-values

• p-value is the probability of obtaining a test statistic equal to or higher than the one observed if H(o) is true
• Therefore, large p-values indicate that H(o) is quite plausible given the data
• Small p-values indicate that H0 is implausible; small p-values esp <0.05 mean that it is likely implausible and statistically significant
• A statistically significant p-value tells us what the difference is unlikely to be (zero, the null hypothesis), but it doesn't tell us what the difference is likely to be (we need confidence intervals here)

Confidence Intervals (another way to tell statistical significance)

• Confidence intervals help us focus on the magnitude of the difference (among an outcome)
• This helps us to understand study precision & clinical significance
• The CI asks: "what is the range of difference between the two groups within which we would find the true difference"
• We can control the confidence intervals of intervention group + control group to each other and if the upper limit of control fails to exceed lower limit of drug group= no overlap, we can conclude that there is a difference between the two groups -> this is why we don't want confidence intervals where the mean group difference cross zero= statistically insignificant
• Narrow confidence intervals= precise study with large sample size
• The middle line is typically a 0 for mean difference in continuous variables

Categorical Outcome Data: Dichotomous Outcomes Analyzed using 2x2 Contingency Tables

Dichotomous Outcome Data

• Clinical Research is often concerned with the practical issues of preventing disease, treating illness, prolonging life, and reducing acute care admissions
• The object of measurement associated with dichotomous outcome data= determining the presence of absence of risk factors, assessing the presence or absence of particular diseases or conditions (ex, asthma, no asthma), estimating survival (dead or alive), estimating a 30-day hospital re-admission (yes or no )

Dichotomous Data

• These measures are all nominal categories (just names) and counts of people in each category
• Non-parametric methods are used to analyze them-> non-parametric methods are methods that do not rely on a normal distribution- why- because categorical variables are typically not normally distributed
• Chi-square: common non-parametric test that measures the statistical significance of an association between nominal (categorical) variables

More Association Tests for Dichotomous Data

• Health research often concerned with presence or absence of outcomes due to an intervention or exposure
• A number of measures are used with these data:

1. Relative Risk (RR)
2. Relative Risk Reduction (RRR)
3. Absolute Risk Reduction (ARR)
4. Odds Ratio (OR)
5. Number Needed to Treat (NNT)/Number Needed to Harm (NNH)

The 2x2 Table

• To compute these measures of association, which indicate discrete outcomes, we "count bodies or events" that belong to different categories
• We look at these variables in terms of proportions and we use a 2x2 table

Q: Let's look at an example: people with severe asthma who have been admitted to hospitals at least once in the past year

• Split these asthma sufferers into 2 groups:

1. Group that receives special care from a nurse specialist (intervention)
2. Group that controls their asthma as usual (control)

• Observe these asthma suffers over the next year for the outcome (readmission to hospital)

First Variable of Interest: Relative Risk

• Relative risk is the ratio of the probability of an event occurring in the exposed group vs the probability of the event occurring in the non-exposed group  aka; what is the ratio of the outcome occurring in the intervention group vs the control group?
• Formula [a/(a+b)] / [c/(c+d)] -> 12/84=0.1429 and 26/81=0.321 -> 0.1429/0.321=0.4452
• RR of readmission is 44.5% in the intervention group compared to the control group OR risk in intervention group is 0.445 times the risk of control group

Second variable of interest: Relative Risk Reduction (RRR)

• Estimates the proportion of risk of "bad outcome" reduced by intervention
• RRR= 1-RR= 1-0.445=0.555 which means the specialist nurse (intervention) decreased the risk of hospital readmission by 55.5% compared with regular care

Third variable of interest: Absolute Risk Reduction (ARR) or risk Difference (RD)

• The percent of patients that will be spared the adverse outcome if they receive the intervention
• Subtract the intervention from the control [c/(c+d)]-[a/(a+b)]
• 26/81=0.321, 0.321-0.143=0.178 so 17.8%
• This means that 17.8% of asthma patients will not be readmitted to the hospital if they receive the intervention 

Good Outcomes

• We use the same measure (RR), but reverse our interpretation:

1. RR < 1.0 for the intervention favours the control
2. RR >1.0 for the intervention favours the intervention

Bad Outcomes

• For bad outcomes, the interpretations are opposite

1. RR <1.0 for the intervention favours the intervention
2. RR >1.0 for the intervention favours the control

Number Needed to Treat (NNT)

• The number of people that we need to treat to have an impact on one person ; the number of people we would need to treat to prevent one "bad outcome"= 1/ARR
• In other words, how many asthmatics would have to see the specialist nurse in order to prevent one of them from being readmitted
• Here, we want to number to be lower

Number Needed to Harm (NNH)

• The number of patients who if they receive the intervention, would result in one additional patient being harmed
• Can result with good or bad outcomes; if intervention reduces the chance for a "good" outcome compared to the control OR if intervention increases the chance for a "bad outcome" compared to control
• Can be used to express harmful effect of intervention
• Same calculation 1/ARR
• If NNH is substantially lower than NNT, the risk-benefit ration would argue against drug/intervention

Confidence Intervals

• The concept of confidence intervals applies equally to dichotomous data, except the "no difference" marker is 1 (RR/OR) instead of 0
• The same three questions of interpretation are the same; does it cross 1? How wide? Where is the upper and lower limit?

Using CIs to Interpret Trial Results

Ex, a trial to determine the effect of calcium supplementation (taken to reduce fractures) on myocardial infarctions (MI) in healthy menopausal women

• RR: 3.5; means that the risk of MI in the Ca2+ group is 3.5 the risk in the control group but the true RR may be as low as 2.1 or as high as 4.47
• The CI doesn't include an RR of 1.0 which means the result is statistically significant
• Then we ask is it clinically significant (check at both ends of the spectrum); many clinicians would regard differences at either end of the CI as an important clinical difference and we would want to inform our patients about this risk)

Ex,2: a trial in a neonate intensive care unit to compare mortality rates of infants cared for by NPs vs pediatric residents

• Results for NP intervention group; 4.6% of NP patients died while 5.9% of residents patient's died
• RR= 0.78 and the 95% CI is 0.43-1.4
• This means that the risk of mortality is reduced in the NP group by 22%, but the true RR could be consistent with a 57% reduction or a 40% increase in neonatal mortality in the NP group
• The CI includes an RR of 1 so the difference between groups is not statistically significant
• The trial failed to show differences between the 2 groups, but it also failed to exclude the possibility of importance differences in neonatal mortality (because CI was so large)
• Overall, not significant

Odds Ratio

• An alternative to RR that is similar in interpretation
• OR will be approximately equal to RR when the prevalence of the outcome is rare (<5-10%), but will be higher if not rare
• RRs are calculated in RCTs and cohort studies (ORs can be calculated, but RRs preferred as measure of risk)
• ORs are calculated in case-control studies (RRs cannot be calculated)
• Odds ratio= (a/b) / (c/d)

1. Disease OR: odds of having outcome/condition for exposed group compared to odds of having outcome for non-exposed group
2. Exposure OR: odds of having been exposed in outcome/condition group vs odds of having been exposed in non-outcome group

• They are equal numerically, even though they are interpreted differently

Disease Odds Ratio

Ex. 1: Using asthma hospital example, calculate OR for the asthma hospital readmission study ("disease" is an outcome of hospital readmission)

• Intervention: (a/b)= 12/72=0.1667
• Control: (c/d)= 26/55=0.4727
• Divide the odds of the intervention by the control; OR= 0.1667/0.4727=0.3527
• This means that the odds or risk of readmission for the intervention group are 0.353 times that of the control group (or 35.3% of the risk of control group)
• This was disease OR

Exposure Odds Ratio

Ex: 2: Calculate OR for the asthma hospital readmission study where exposure is having the specialist nurse

• Same math
• This means that the odds of having a specialist nurse for the readmitted group are 0.353 times that of the not admitted group

RRs: Interpretation, Pitfalls

Ex, 1: RR>1, for ex, RR= 4.2

1. Correct:

• Intervention subjects had 4.2 times the risk of the outcome compared to control subjects
• Intervention subjects had a 420% more risk of getting the outcome compared to control

2. Incorrect:

• Intervention subjects had 4.2 times more risk of outcome compared to control 
• ORs: Use the same wording as RRs, since they're interpreted as if they were RRs

Ex. 2: RR<1, for example RR: 0.57

1. Correct:

• Intervention subjects had 0.57 times the risk of the outcome compared to control
• Intervention subjects had a 43% less risk of getting the outcome compared to control subjects

2. Incorrect:

• Intervention subjects had 57% less risk of outcome compared to control

Categorical Outcome Data: Dichotomous Outcomes Analyzed Using a Modelling Approach

• Modelling approaches use a statistical procedure to estimate ORs and RRs
• Common statistical procedures include regression methods suitable for dichotomous outcomes, such as: logistic regression for estimating ORs or log-binomial or modified Poisson regression for estimating RRs
• Modelling approaches increasingly used to estimate ORs and RRs in RCTs and observational studies why: this allows for the use of categorical and/or continuous risk factors (exposures) (note: a 2x2 table only applies to categorical factors)
• Modelling also reduces risk of potential bias in RR and OR estimates by adjusting for confounding variables
• ORs and RRs generated by modelling methods interpreted the same way as those derived from 2x2 contingency tables

Ex. 1: Framingham Heart Study

• Large Prospective cohort study (24 years of follow-up)
• Research question: does cholesterol level predict development of angina over 24 year follow-up?
• Outcome (dichotomous): developing angina during 24 year follow up (yes/no)
• Primary predictor of interest (continuous): serum cholesterol level (mg/dL)
• Potential confounding variables (categorical and continuous): sex, smoking status, diabetes diagnosis, age, BMI, heart rate
• Anaylsis (OR): logistic regression [note a 2x2 contingency table to estimate the OR cannot be used because 1: primary predictor is continuous and 2) significant confounding may bias the OR estimate so should adjust

• We see that for both adjusted and unadjusted 95% CIs, the exclude 1 meaning that they are statistically significant (higher cholesterol levels are associated with higher odds of developing angina)
• Not a large difference between unadjusted and adjusted OR meaning that there isn't much confounding effect of cholesterol on angina as a result of other predictors included in the adjusted model

4.5- Hazard Ratios (HRs)

• HR (Hazard Ratio): ratio of the rates of an event (hazard) in two groups, takes timing into account, and tell us if an intervention/risk factor changes the rate at which an event occurs
• RRs + ORs focus on the total number of events over entire study period but does not take timing into account, tells us if an intervention/risk factor changes the risk of the event or the odds of an event
• HR + RR + OR are all measures of risk, each suited to different study designs and research questions
• All are ratios, all have CI, if it excludes the 1, they are statistically significant
• HRs used with modelling approach called survival anaylsis, yields survivorship curves that show: occurrence of some event over time within a group, vertical axis corresponds to proportion experiencing the event of interest within each group and horizontal axis= time

Interpreting HR:

• The curve shows the rates that event occurs within a group, can compare rates for different groups
• HR=1 indicates that the event occurs at the same rate in each group at any given time interval
• HR not equal to 1 indicates that the event occurs at a different rate in each group at any given time interval

HR= RR?

• Some sources state that the two measures are more or less equal, hazard ratios are useful when the risk is not constant over time, it uses information collected at different times
• However, with HR, we need to state in which time interval, the probabilities are calculated, thus the hazard can differ depending on the time interval chosen
• HR for entire study period= weighted RR during entire study period
• For short time intervals, HR and RR are similar but differ over time

Ex. Survival Rate in Cirrhosis Patients with less than/equal 90 day hospital readmission vs greater than 90 day readmission

• Large retrospective cohort study
• Research q: Is early hospital readmission associated with survival rate in patients with cirrhosis?
• Outcome (dichotomous): died (yes/no)
• Primary predictor of interest (categorical): early (less than 90 days) vs late (greater than 90 day0 hospital readmission
• Potential confounding variables (Categorical and continuous): age, sex, ethnicity, income, Charlson score
• Analysis (HR); survival analysis

• 95% CIs both exclude 1 so statistically significant, but the difference between adjusted and non-adjusted HRs indicate confounding effect due to other predictors included in adjusted model