click below
click below
Normal Size Small Size show me how
WEEK 8:
Making sense of the evidence - systematic reviews and meta analysis:
| Question | Answer |
|---|---|
| systematic review | overview of all primary studies that contain explicit statements of objects, materials and methods and has been conducted according to explicit reproducible methodology |
| stages in conducting a systematic review (5) | Frame the question Identify relevant literature Assess the quality of literature Summarise the evidence Interpret the findings |
| framing the question includes | PICO/PICOS patient or problem (general population - no age or sex restrictions) intervention (eg given 0.2g/day or more of vitamin C) comparison (placebo) outcome (incidence, duration and severity of common cold) study (type of study) |
| identifying relevant literature includes | performing comprehensive search strategy to identify literature (including unpublished studies) and even the references of those studies |
| assess quality of literature includes | was study randomised? was there allocation concealment? was there evidence of blinding? how many patients lost to follow up (attrition)? were the thing being tested and placebo indistinguishable? only using good quality studies |
| how to summarise and interpret evidence qualitatively | via narrative or qualitative summary eg what proportion of studies found significant results? what were the key findings? |
| how to summarise and interpret evidence quantitatively | meta analysis (not all systematic reviews will contain a meta analysis), statistical method used to combine data (RR/OR) from 2 or more studies (usually same design) to produce new single 'summary estimate' of effect of treatment w measure of uncertainty) |
| how to summarise and interpret evidence quantitatively is not | a simple average of results |
| how to summarise and interpret evidence quantitatively essentially weights results based on | size of study (sometimes other factors too) |
| bigger studies = | more believable results = given more weight in meta analysis |
| when is qualitative/narrative synthesis suitable | when studies cant be statistically combined eg because they are very heterogenous (you dont want to compare apples with oranges) |
| when is meta analysis (quantitative) suitable | when studies are very similar ie largely match across the PICOS acronym and have similar design |
| ratio 1 | exactly consistent with our null hypothesis (i.e. no effect) |
| ratio below 1 | means outcome is less likely in exposed group |
| ratio above 1 | means outcome is more likely in exposed group. |
| CI that crosses 1 from either direction means results are | are not statistically significant (i.e. the null hypothesis is a fairly likely explanation and we cannot reject it). |
| Statistical heterogeneity (or simply heterogeneity) | Variability in the intervention effects being evaluated Observed intervention effects are more different from each other than one would expect due to random error (chance) alone |
| Statistical heterogeneity may be the result of: | clinical heterogeneity and methodological diversity |
| clinical heterogeneity | no two studies are identical and results of included studies may vary eg differences in study population, differences in interventions or exposures, and differences in outcome definition |
| methodological diversity | Variability in study design and quality |
| ways to address heterogeneity via modelling the variation (2) | fixed effect model and random effect model |
| fixed effect model | assumes a single underlying effect size (i.e. that all included studies are reporting on a single ‘true effect’ size of Lariam that is the same across all studies) |
| random effect model | assumes studies are estimating similar but not identical effect sizes (eg A conducted in slightly different population group compared to B) |
| heterogeneity being addressed allows CI to | be wider to allow for extra variation between studies |
| ways to address heterogeneity via exploring variation | random effect model and sub group analysis |
| random effect model can | account for variation but not explain it |
| sub group analysis (by study characteristics or participant characteristics) may explore | treatment effects further eg in the Vitamin C studies, splitting the studies into two categories helped show what was happening. |
| problems with meta analysis | publication bias |
| publication bias | bias occurring when research that is published is not representative of all completed studies and applies particularly to smaller studies |
| effect of publication bias | distort results and conclusions drawn from a meta analysis |
| potential solution for publication bias | require drug manufacturers and other researchers to register clinical trials BEFORE they start |
| why use grey literature | finding relevant research that may not have been published |
| funnel plot | plot of measure of study size or study precision against a measure of effect (OR) where smaller/less precise studies expected to vary further from central effect size (more random variation) |
| what do you expect a funnel plot to look like (in absence of bias) | balanced funnel- would expect approximately equal numbers of studies to fall on each side |
| grey literature can include (3) | Conference proceedings Dissertations Reports by non-academic organisations (e.g. government departments, charities) |
| what is grey literature | studies that are not published |
| when plot goes over 1 it is | not significant |
| when plot does not go over 1 it is | highly significant |