Help
Options
Didn't know it?
click below
click below
Knew it?
click below
click below
Don't Know
Remaining cards (0)
Know
retry
shuffle
restart
0:00
Embed Code - If you would like this activity on your web page, copy the script below and paste it into your web page.
Normal Size Small Size show me how
Normal Size Small Size show me how
RD Final
Research Design Final
| Question | Answer |
|---|---|
| Case Study | a form of qualitative or mixed methods research involving intense examination of an individual, group, or organization. Some CS referred to as ethnography, field study, and participant observation. |
| What do case studies examine? | The relationship of and interaction of all variables in order to provide as complete an understanding of an event or situation as possible. |
| Advantages of case studies | -May provide an abundant/fertile source of information -Possible to study a characteristic/attribute quite intensively -can focus on rare, unique, and extreme cases |
| Disadvantages of case studies | -this method is not scientific- nonrandom sample, non-generalizable conclusion -observer may be biased -subject is usually aware they are being studied (Hawthorne effect) -Many threats to internal validity may occur such as maturation, testing, history |
| The value of case studies | -source of ideas and hypothesis -source for developing therapy techniques -Study of rare phenomena -Persuasive and motivational value-provides evidence for grants application that would fund a larger empirical study. |
| Case Study Approaches | Approach or type of case study will likely be determined by the research objective/question. -3 main types 1) explanatory, 2) exploratory, and 3) descriptive |
| Designing the case study | -what questions to study-must have clear research question -what data is relevant -what data to collect -How to analyze that data |
| Yin's 5 basic components of case study design | 1.The study's question(3) 2.The study's propositions (if any) 3.The study's units of analysis(es) 4.The logic linking of the data to the propositions 5.the criteria for interpreting the findings |
| Six types of data that can be collected in case studies | 1.Documents 2.Archival records 3.Interviews 4.Direct Observation (may include videos) 5.Participant observation 6.Artifacts (in ethnographies) |
| Evaluation of data | -Researchers interpret data either holistically or through coding. Some designs allow use of statistical techniques to assess results. -Evaluation mainly depends on the type of case study one is conducting |
| Single-Case Design 4 features | 1. continuous assessment 2. baseline assessment 3. stability of performance 4. use of different treatment phases -methodology unique to single-case designs involves assessing an individual's behavior using repeated observations over time. |
| Continuous assessment | refers to the use of repeated observations of an action/behavior over time. Observations may take place several times a day or multiple times per week or month. |
| Baseline assessment | generally refers to the observation of an action/behavior for several days before an intervention is applied -must be stable/little variation, 7 or 8 data points/sessions |
| Stability of performance | need data (measuring action/behavior) to be consistent/steady over time. Variability or fluctuation of measurements would make it difficult to draw any conclusions about the intervention |
| Different phases | different conditions individual or case unit is exposed to during the study. A phase may be a baseline condition w/ no intervention or treatment condition. Another phase may be time period where individual case is receiving treatment/intervention |
| A-B design | simplest of single case design options. A=baseline phase-no treatment or intervention B=treatment or intervention phase -Only need repeated measurements of problems/condition before treatment (A) and during treatment (B). |
| A-B limitations | -does not provide enough evidence for a cause and effect between the treatment and the behavior or actions being measured. |
| A-B advantage | does not require the collection of numerous data points per phase. Statistical method (time series analysis) can be used to detect a trend using as few as 8 data points per phase. |
| A-B-A design | -allows for two transitions (B-A & A-B)can demonstrate the effectiveness of Tx -Strengthens the ability to draw cause-and-effect conclusion; preferred design for single case research -considered unethical to return to baseline to determine causality |
| Multiple-baseline design | -effects of intervention assessed across several partics., behav. &/or settings -helps control for confounds by introducing tx at diff times for diff p, b, &/or s. -important baseline data are consistent for second p,b,or s before introducing tx |
| Advantages of multiple-baseline designs | -do not require any reversal, end in an intervention condition, and don't require a return to baseline condition to show experimental control |
| Limitations of multiple-baseline designs | -measurements/behaviors must be monitored continuously & concurrently across all conditions. time consuming and expensive -if data is unstable during any phase, tx to others may be delayed. -prolonged baseline=adverse experience for p- unethical |
| nonstatistical evaluation | examination of data (charting, graphing, eyeing) and determining whether the intervention had an effect by VISUAL INSPECTION (descriptive statistics) |
| Effect Size (ES) | -measure the magnitude of a tx or relationship effect. -Important because significance tests are a function of sample size. -ES MEASURES ARE NOT A FUNCTION OF SAMPLE SIZE IN SINGLE STUDIES -summarize findings from spec. area of research in meta-a |
| History of ES | -Early 1990's suggested to include ES in all APA articles but didn't catch on. -1999 Task force on Statistical Inference (TFSI)created to solve significance testing debate and promote use of ES -APA manual required all publications to include ES |
| Cohen's d | -effect size measure to use in the context of a t-test or one-way ANOVA -d=difference between two means divided by pooled Sd for those means -means further apart= greater effect size -tends to be used most often |
| Glass's Delta | -estimator of effect size that uses only Sd of the second group (control) -when several tx groups were compared to control, better to use just Sd for control, effect sizes wouldn't differ under equal means & different variances |
| Hedges' g | -suggested by Larry Hedges in 1981, like other measures based on Sd, but pooled Sd is computed slightly different than Cohen's d -usually lower/more conservative than Cohen's d, but will Cohen's d approach as sample size increase |
| Analysis of Variance Families (ANOVA, ANCOVA, MANOVA, MANCOVA) | -Eta-squared and partial eta-squared: estimates of the degree of association for the sample SSB/SST= proportion of variance in Y explained by X; How much effect can be explained by tx |
| Effect size for Chi-squared tests | -when there is 1 df for chi-squared test, ES measure is PHI -when more than 1 df, ES measure should be Cramer's V. |
| Caution when interpreting effect size | -whether an ES should be interpreted as small, medium, or large depends on its substantial context and its operational definition -Just report the ES and don't include size classifiers let the reader decide that |
| Nonprobability sampling | -type of sampling that doesn't involve random selection -purposive sampling, convenience sampling, and quota sampling |
| Purposive sampling | sampling for only a specific group, purposely seeking individuals that meet the criteria (grad students, individuals with certain medical condition) -snowball sampling is part of this method |
| Snowballing | -may be used to study individuals who are hard to identify or find -Identify one individual with the desired attribute and then use that individual to find others (exotic dancers) |
| Convenience sampling (aka accidental) | -Usually undergrad psych students are the participants in most research from universities -Another example is going to the mall to gather participants who happen to be there |
| Quota Sampling (similar to stratified sampling without random selection) | -Stratify your participants on some characteristic to mirror that of the general population or the study population of interest -Ex. 100 white and 20 black participants to meet quota |
| Disadvantages of nonprobability sampling | -potential for selection bias (participants choose to participate due to interest) -Likely to NOT be able to generalize findings to the general population (major problem) -Sampling error cannot be known |
| Sampling Error | the difference between the sample value (e.g. percentage of individuals diagnosed with PTSD) and the true value in the population -It can't be known if the true percentage of prevalence of something is generalizable to the population |
| Observational research methods | -observational studies often performed in clinical settings and seek to determine either prevalence, incidence, cause, or prognosis |
| When might observational studies be used? (2) | 1. Used to study events that might be considered unethical to study in an experiment or quasi-experiment design 2. rare events or conditions |
| Incidence vs Prevalence | -Incidence = rate- specific time -Prevalence is not a rate, it's the total number of people with an illness/disorder at any given time. -Cohort studies are best used to determine incidence of condition/disease/syndrome |
| Cohort Study (aka longitudinal study) | -Group of individuals you will follow over time -Data will be collected either prospectively or retrospectively |
| Prospective cohort study (single group) | -individuals selected do not have condition of interest (e.g. substance abuse). -They are followed over a period of time and variety of variables may be collected at baseline and at multiple time periods depending on the study. |
| Prospective cohort study (single group) (cont) | -All observed, see who will develop the condition. Those individuals who do not develop the condition are considered as INTERNAL CONTROLS. -EX: Chicago HS students from graduation through life to see how many develop certain disorders, cause/effect |
| Prospective cohort study (two groups) | -two groups (cohorts) are followed. One has been exposed to certain condition & the other has not. Non-exposed group know as an EXTERNAL CONTROL EX: war vets active duty combat or active duty not combat, measure PTSD -same length, more participants |
| Retrospective cohort study | -data are already collected, researcher simply records what is known or has been collected on a group/cohort. -Takes less time b/c data are already collected & may even be computerized for easy accessibility. |
| Advantages of cohort studies 1 | Allows a researcher to study something that would be unethical in a randomized control study (substance use, smoking, exposing someone to something potentially harmful to them) |
| Advantages of cohort studies 2 | -Since these studies allow the researcher to demonstrate that the cause(s) occurred before the event of interest, it enables him/her to determine "cause" instead of not knowing which is the cause and which is the effect. Can id cause before effect happens |
| Advantages of cohort studies 3 | -Also, a single study can measure several variables that may be both risk factors or outcomes (e.g., depression, anxiety, PTSD). |
| Disadvantages of cohort studies 1 | Cohort studies are not as reliable as randomized controlled studies, since the two groups may differ in ways other than the variable under study. |
| Disadvantages of cohort studies 2 | They require a large sample size, are inefficient for rare outcomes, and can take long periods of time if retrospective -If studying something uncommon it will be difficult to acquire a sample large enough |
| Disadvantages of cohort studies 3 | Participant loss (i.e., attrition) is often a problem as time increases. -morality of studies- people don't want to come in and you have to provide incentive. Participants could die, be arrested, or otherwise unavailable. -START W/LARGE SAMPLE |
| Cross-sectional study | -Used to determine the prevalence of condition, disease, or syndrome. Also used to infer causation. -assesses participants at one point in time. Data collected on whether individual was exposed to variable, & whether they have outcome of interest. |
| Advantages of cross-sectional studies 1 | -only one group is used and data are collected at only one time- may be conducted relatively quickly w/o a lot of expense. The use of questionnaires is generally the cheapest method of data collection, but can result in a low response rate. |
| Advantages of cross-sectional studies 2 | -Best way to determine prevalence & may be used to identify associations that can then be studied with a cohort study to provide more evidence |
| Disadvantages of cross-sectional studies 1 | The use of questionnaires is generally the cheapest method of data collection, but can result in a low response rate and may result in biased data due to potential differences in responders and nonresponders |
| Disadvantages of cross-sectional studies 2 | Data may be collected via interviews (in person or by phone); however, this method is more time consuming and expensive if several people must be hired to do interview. -higher response rate but requires more time and money |
| Disadvantages of cross-sectional studies 3 & 4 | Can't differentiate between cause and effect therefore the researcher can't provide an explanation for the findings. -Can't study rare outcomes or conditions. Even a large samaple may miss detecting some one with the event. |
| Case-control study | -generally retrospective in that individuals with the condition are usually matched with a control who does not have the condition -May determine the importance of a predictor variable in relation to the presence or absence of the condition of interest |
| Case-control study (cont) | Case-control studies are generally used to calculate the odds of someone developing the event or condition given they have, or have been exposed to, the predictor variable/risk factor. |
| Advantages of case-control studies | -best way to study uncommon or rare conditions -does not require very many subjects relative to cross-sectional and cohort studies (easy in hospitals or clinics to gather participants) |
| Advantages of case-control studies (cont) | -can produce information that may be used for future, more elaborate studies, or provide evidence for grant application |
| Disadvantages of case-control studies | 1. can only look at one outcome/event/condition 2. Bias is a problem (not much of a prob with genetic causes, but w/environmental it's more of a prob -two types: sampling bias and observation and recall bias |
| Sampling bias | How were the participants with the condition sampled/obtained |
| Observation and recall bias | Since data are collected retrospectively, may be bias on the part of the subject and researcher to determine if predictor or risk variables occur -influencing recall, difficult to get good info b/c recall is possibly incorrect |
| Publication Bias (AKA file drawer problem) | -Publications, editors, journals ant to make money and they don't want to publish a negative/no finding article |
| Publication Bias (AKA file drawer problem) (cont) | -Any finding is a finding, if you don't publish "no association" research, someone else will waste the time and resources to research that. -can show bordering significant findings but a need for more research w/later sample size. |
| Content Analysis | -concept of CA began in 1930's, initially limited to studies that examined texts for frequency of occurrence of terms (word counts), -By 50's started focusing on concepts rather than words and semantic relationship rather than just presence |
| Use of Content Analysis | -Method of research in marketing & media studies, literature & rhetoric, ethnography & cultural studies, gender & age issues, sociology & political science, psychology & cognitive science, & is beginning to be used in the legal field (case law) |
| Types of content analysis | Two main types: Conceptual analysis and relational analysis |
| Conceptual analysis | establishes the existence and frequency of concepts most often represented by words or phrases in the text. |
| Relational analysis | goes one step further by examining the associations among concepts in a text. |
| Steps of Conducting Content Analysis (1-5) | 1)Decide the level of analysis 2)Choose how many concepts to code 3)Decide whether to code for existence or frequency of a concept 4)Decide on how you will distinguish among concepts 5)Develop rules for coding your texts |
| Steps of Conducting Content Analysis (6-8) | 6)Decide what to do with "irrelevant" information 7)Code the texts 8)Analyze your results |
| 1. Decide Level of Analysis/Coding Unit(s) (5) | Word, Theme, Item, Character, Time & Space |
| Word | Analyze for sex related words in different magazines |
| Theme | Analyze for occasions, in children's literature, on which boy/girl initiates and gets praised for aggressive behaviors |
| Item | Look for whole stores, e.g., Number of newspaper articles on I'm a Celebrity, Get Me Out of Here |
| Character | Analyze types of character occurring in TV cartoons |
| Time & Space | Measure the time or space (e.g., column inches) devoted to particular issues in the media |
| 2. Choose how many concepts to code | -code every single word or only certain ones deemed most relevant to research question? -Determine flexibility allowed when coding -Determining certain # and set of concepts allows researcher to examine text for very specific things, keeping on task |
| 2. (continued) | -introducing level of coding flexibility allows new, important material to be incorporated into the coding process that could have significant bearings on results. |
| 3. Decide whether to code for existence or frequency of a concept | -This question changes the coding process -Coding for existence, the "coding unit" would only be counted once, no matter how many times it appeared. Very basic coding process, would give the researcher a vet limited perspective of the text. |
| 4. Decide on how you will distinguish among concepts | -decide on level of generalization, i.e. whether concepts are to be coded exactly as they appear, or if they can be recorded as the same even when they appear in different forms (ex: coding unit like "expensive" may also appear as "expensiveness") |
| 5. Develop rules for coding your texts | Researcher should create a set of translation rules that will allow him/her to streamline and oragnize the coding process |
| 6. Decide what to do with "irrelevant" information | Researcher must decide whether irrelevant info should be ignored, or used to reexamine and/or alter coding scheme. -words like "and" and "the" as the appear by themselves would be ignored in previous example, add nothing to the qualification of words |
| 7. Code the texts | Done either by hand, i.e. reading through the text and manually writing down concept occurrences, or w/ computer programs -programs will code data for you but depend on researcher's preparation and category construction. Manual better to see errors. |
| 8. Analyze your results | Researcher examines data and attempts to draw whatever conclusion and generalizations possible. Conceptual analysis is limited by the qualitative nature of it's examination. Relational analysis required to fully explore concept relationship |
| Advantages of Content Analysis (1-3) | -looks directly at communication via texts or transcripts, gets at the central aspect of social interaction -can allow for both quantitative and qualitative methods -can provide valuable historical/cultural insights over time through analysis of texts |
| Advantages of Content Analysis (4-5) | -Is an unobtrusive means of analyzing interactions (no participants= no IRB needed) -provides insight into complex models of human thought and language use |
| Disadvantages of Content Analysis (1-2) | -can be extremely time consuming -is subject to increased error, particularly when relational analysis is used to attain a higher level of interpretation |
| Disadvantages of Content Analysis (3-4) | -is often devoid of theoretical base, attempts too liberally to draw meaningful inferences about the relationships and impacts implied int he study (but can use this to develop a theory) -tends too often to simply consist of word counts |
| Disadvantages of Content Analysis (5-6) | -often disregards the content that produce the text, as well as the state of things after the text is produced -can be difficult to automate or computerize |
| Literature review | -may be classified as "narrative" or "systematic" |
| Narrative review | -based on a subjective section of publications, reviewer qualitatively addresses a question summarizing the findings of previous studies and drawing a conclusion. -potential for author(s) bias b/c review doesn't follow clear methodology. |
| Narrative review (cont) | -lack of specific search strategy increases risk of failing to identify relevant or key studies on a given topic, allowing for questions to arise regarding conclusions made. -Narrative reviews considered more as opinion pieces than evidence-based. |
| Characteristics:Review Question | Systematic review:strictly formulated Narrative review:broadly formulated |
| Characteristics:Methodology | SR: Clearly defined NR: Not or insufficiently described |
| Characteristics:Search Strategy | SR: clearly defined NR: Not described |
| Characteristics:Selection of the studies | SR: clearly defined NR: not described |
| Characteristics:Ranking of the studies | SR: By levels of evidence NR: Not performed |
| Characteristics:Analysis of the studies | SR: Clearly described NR: Not described |
| The flaws and limitations of a narrative review can be restricted by _________? | conducting a systematic review -systematic reviews are considered to provide the highest level of evidence |
| the term "systematic" refers to _______? | a strict approach (clear set of rules) used to identify relevant studies |
| Systematic approach includes | -use of accurate search strategy to identify all studies addressing specific topic -establishing clear inclusion/exclusion criteria -well-defined methodological analysis of selected studies |
| What does a properly performed systematic review do? | -reduce potential bias in identifying the studies, limiting the possibility of the authors to select the studies arbitrarily considered the most "relevant" for supporting their own opinion or research hypothesis. |
| Meta-analysis | IMPORTANT:a systematic review always precedes a meta-analysis, or a meta-analysis must contain a systematic review |
| Meta-analysis history | -in 1976 Gene Glass proposed a method to integrate and summarize the findings from the scientific literature, called the method META-ANALYSIS and it's basically the statistical analysis of a compilation of individual studies -"the analysis of analyses" |
| -Meta-analysis is most often used to assess ____________. | -the CLINICAL EFFECTIVENESS OF HEALTHCARE INTERVENTIONS (ex: effectiveness of CBT) -It does this by combining data from two or more randomized control trials |
| Meta-analysis of trials provides _____________. | -a PRECISE ESTIMATE OF TREATMENT EFFECT giving due weight to the size of the different studies included |
| The validity of the meta-analysis depends on the _______________. | -QUALITY OF THE SYSTEMATIC REVIEW on which it is based -Systematic review is the most important part of a meta-analysis |
| Good meta-analyses aim for ________, look for ______, and explore the robustness of the main findings using __________. | -COMPLETE COVERAGE OF ALL RELEVANT STUDIES -the PRESENCE OF HETEROGENEITY -SENSITIVITY ANALYSIS |
| Requirements for meta-analysis | -WELL-EXECUTED SYSTEMATIC REVIEW -the main requirement of systematic review is a complete unbiased collection of all the original studies of acceptable quality that examine the same therapeutic question |
| Steps in conducting meta-analyses (4) | 1)location of studies 2)quality assessment 3)calculating effect sizes 4)checking for publication bias |
| Location of studies | -comprehensive search strategy interrogates several electronic databases -Hand-searching of key journals, checking of reference lists of papers obtained is also recommended -search strategy/key terms need to be developed with care |
| Location of studies (cont) | -the strategy is written in a sequence of requirements: include papers with specific terms, exclude papers that do not meet certain criteria (ex: age or diagnostic group), only include studies that follow certain research designs (randomized controlled) |
| Quality assessment-decide how each thing is defined | -decisions msut be taken about which studies are sufficiently well conducted to be worth including. -potential for bias so good meta-analysis will use EXPLICIT AND OBJECTIVE CRITERIA FOR INCLUSION OR REJECTION of studies on quality grounds |
| Calculating effect sizes | -Previously most common summary measure of ES was the ODDS RATIO, but now THE RISK RATIO (RELATIVE RISK) can be given -they are generally interpreted the same way -Risk ratio of 2 (odds ratio of .5) implies defined outcome happens twice as o |
| Checking for publication bias | -Key concern is pub. bias pertaining to negative findings (less likely to be published) -Examine a FUNNELL PLOT to assess the likely presence of pub. bias |
| Funnel Plot | -displays the studies included in the MA in a plot of ES against sample size -Smaller studies have more chance variability than larger studies, should look like symmetrical inverted funnel. Asymmetric= MA maybe missed trials- smaller, no effect studies |
| Funnel Plot (cont) | -measure of study size (i.e., standard error) on verticle axis as function of ES on horizontal axis. -Larger studies appear towards top, tend to cluster near mean ES -smaller studies towards bottom, tend to disperse across range of values |
| Funnel Plot (cont) | -absence of pub. bias= distribution would be symmetrical about the combined ES -Presence of bias= bottom of plot would show higher concentration of studies on one side of mean-reflects fact that smaller studies more likely published if have + results |
| Potential sources of asymmetry in funnel plots (3) | -Selection bias -true heterogeneity -Data irregularities |
| Selection biases | -pub bias -location biases-where are participants from -language bias-native language -citation bias- not including articles that didn't cite preferred journal(s) |
| True heterogeneity | -size of effect differs according to study size -intensity of intervention -differences in underlying risk |
| Data irregularities | -poor methodological design of small studies -inadequate analysis -fraud |
| Forest Plot | -usual way of displaying data from a meta-analysis is by a pictorial representation -displays the findings from each individual study as a blob or square |
| Forest Plot (cont) | -squares towards left= new tx better -those on right= new tx less effective -size of square/blob proportional to precision of study (sample size) -horizontal line drawn around each square to represent uncertainty of estimate of tx effect |
| Forest Plot (cont) | Diamond= aggregate ES obtained by combining all studies |
Created by:
klthomas0123
Popular Psychology sets