Help
Options
Didn't know it?
click below
click below
Knew it?
click below
click below
Don't Know
Remaining cards (0)
Know
retry
shuffle
restart
0:00
Embed Code - If you would like this activity on your web page, copy the script below and paste it into your web page.
Normal Size Small Size show me how
Normal Size Small Size show me how
RS Ch 9
RS Ch 9 Book and PPT
| Question | Answer |
|---|---|
| Internal validity | is the degree to which the relationship between the independent & dependent variables is free from the effects of extraneous factors. |
| External validity | is the degree to which results of a study can be generalized to persons or settings outside the experimental situation.Concerned with the usefulness of the information outside the experimental situation |
| THREATS TO DESIGN VALIDITY Four Categories of Threats: | 1. Statistical Conclusion validity 2. Internal validity 3. Construct validity 4. External Validity |
| Statistical conclusion Validity:(def) | Refers to the appropriate use of statistical procedures for analyzing data,Concerns the inappropriate use of statistical procedures for analyzing dataThis can lead to invalid conclusions about the relationship between IV and DV |
| Statistical conclusion Validity:(quote) | Is there a relationship bewtween the independent and dependent variables? |
| Internal validiy- (def from threat to validity) | refers to the potential for confounding factors to interfere with the relationship between the IV and DV variables, Extraneous factors interfere with cause-and-effect inferences, Randomization & control groups help avoid extraneous factors |
| Internal validity: (quote) | Is there evidence of a casual relationship between independent and dependent variables |
| Construct validity of causes and effects: | Refers to the theoretical conceptualization of hte independent and dependent variables: It concerns how the intervention and response variables are theoretical conceptualized by the researchers |
| Construct validity (quote0: | To what theoretical constructs can results be generalized? |
| External Validity (def from threat to validity): | Refers to extent to which results fo a study can be generalized outside the experimental situation. |
| External Validity (quote) | can the results be generated to other persons, setting or times? |
| Threats to Statistical Conclusion Validity- Potential threats: | Low statistical POWERViolated assumptions of statistical testsError rateReliability and validity These threats involve concepts of statistical inference |
| Low Statistical Power | Power of a statistical test concerns its ability to reject the null hypothesisThis allows a conclusion to be made that a real relationship exists between the IV and DVs |
| What happens if : If the sample size is too small or extraneous sources of variation are not controlled | significant effects may be missed(Low Statistical Power) |
| Violated Assumptions of Statistical Tests | Most statistical tests are based on a variety of assumptions about the experimental data, If theses assumptions are not met, statistical outcomes may lead to erroneous inferences |
| That all data points make up a normal distribution (bell curve) --- Is an example of what? | Violated assumptions of statistical Tests |
| Error Rate | With the use of certain statistical tests, the probability of making a wrong conclusion increases as the number of repeated tests increase. There are statistical procedures that can usually control for this |
| Having 3 DVs & running a separate t-test on each variable instead of using a multivariate test -------- Is an example of what? | Error Rate |
| Reliability and Variance | Statistical conclusions are threatened by any extraneous factors that increase variability within the data. These threats contribute to error varianceIn short, this is variance in the data that cannot be explained by treatment effects |
| Unreliable measurementFailure to standardize the protocolEnvironmental interferencesHeterogeneity of subjects These are examples of? | Reliability and Variance |
| Failure to Use Intention to Treat Analysis | This is a way to avoid bias in data analysis when the original random assignment cannot be maintained |
| Not using this analysis (Failure to Use Intention to Treat Analysis) technique will potentially result in | potentially result in the researcher overestimating a treatment effect |
| INTERNAL VALIDITY- Three components of causality: | Temporal precedence, Covariation of cause and effect, No plausible alternative explanations |
| Threats To Internal Validity - Single Group Threats: | History, Maturation, Attrition (aka experimental mortality), Testing, Instrumentation, Statistical regression |
| Threats To Internal Validity - Multiple Group Threats (Selection interactions): | Selection-history effects, Selection-maturation effects, Selection-attrition effects, Selection-testing effects, Selection-instrumentation interaction, Selection-regression |
| Threats To Internal Validity - Social Threats: | Ambiguity about the Direction of Causal Influence, Diffusion or Imitation of Treatments, Compensatory Equalization of Treatments, Compensatory Rivalry, Resentful Demoralization |
| Single Group Threats- History | Confounding effect of specific eventsThe event occurs after introduction of the IV or between the pretest and posttestNot as problematic if timeframe is short |
| Examples:Participation in similar situation outside of treatment setting, Conversation between subjects, Subject’s moving around, Personnel changes during course of study.... are examples of what? | Single group threat- History |
| Single Group Threats- Maturation | Passage of time independent of external eventsMay cause participants to respond differently on subsequent testsA concern if intervals between tests are long |
| Examples:Growing older, Becoming stronger, Becoming healthier, More experienced, Fatigue, Bored---- are examples of what? | Single Group Threats- Maturation |
| Single Group Threats- Attrition or Experimental Mortality | Pt dropout be4 study completed, Dropout occurs for specific reasons related to the study, Should determine if dropout occurs due to random or biased reasons, determine if 1 group effected more than another, RS should report reasons for attrition in writ |
| One example is:Exercise intervention study with people having anginaPurpose of study is to increase anginal threshold Subject or subjects drop out because of increased pain associate with beginning an exercise program--- is an example of? | Single Group Threats- Attrition or Experimental Mortality |
| INTERNAL VALIDITY- Three components of causality: | Temporal precedence, Covariation of cause and effect, No plausible alternative explanations |
| Threats To Internal Validity - Single Group Threats: | History, Maturation, Attrition (aka experimental mortality), Testing, Instrumentation, Statistical regression |
| Threats To Internal Validity - Multiple Group Threats (Selection interactions): | Selection-history effects, Selection-maturation effects, Selection-attrition effects, Selection-testing effects, Selection-instrumentation interaction, Selection-regression |
| Threats To Internal Validity - Social Threats: | Ambiguity about the Direction of Causal Influence, Diffusion or Imitation of Treatments, Compensatory Equalization of Treatments, Compensatory Rivalry, Resentful Demoralization |
| Single Group Threats- History | Confounding effect of specific eventsThe event occurs after introduction of the IV or between the pretest and posttestNot as problematic if timeframe is short |
| Examples:Participation in similar situation outside of treatment setting, Conversation between subjects, Subject’s moving around, Personnel changes during course of study.... are examples of what? | Single group threat- History |
| Single Group Threats- Maturation | Passage of time independent of external eventsMay cause participants to respond differently on subsequent testsA concern if intervals between tests are long |
| Examples:Growing older, Becoming stronger, Becoming healthier, More experienced, Fatigue, Bored---- are examples of what? | Single Group Threats- Maturation |
| Single Group Threats- Attrition or Experimental Mortality | Pt dropout be4 study completed, Dropout occurs for specific reasons related to the study, Should determine if dropout occurs due to random or biased reasons, determine if 1 group effected more than another, RS should report reasons for attrition in writ |
| One example is:Exercise intervention study with people having anginaPurpose of study is to increase anginal threshold Subject or subjects drop out because of increased pain associate with beginning an exercise program--- is an example of? | Single Group Threats- Attrition or Experimental Mortality |
| Single Group Threats- Testing | Potential effects of pretesting or repeated testing on the DVJust the act of collecting data changes the response that is being measuredReactive measurements: |
| What are Testing Reactive Measurements? | Tests that can potentially change the response they are measuring |
| Repeated measures of ROM may increase the range without the intervention ever being introduced--- This is an example of what | Single Group Threats- Testing Reactive Measurements |
| Single Group Threats- Instrumentation | Concerned with the reliability of the measurement or the ratersRaters can become more skilled between the pretest and the posttest. Can be addressed by:Consistent calibrationDetermining reliability |
| Lack of poor calibration of measuresChanges in criteria followed by tester..... are examples of what? | Single Group Threats- Instrumentation |
| Single Group Threats- Statistical Regression | Rgrss twrd the mean can occur even without the introduction of the interventionThe amt of stat rgrss is directly related to the degree of measurement error in the DV |
| Statistical Regression is also associated with __________ | reliability of a test |
| When does statistical regression become a problem? | It becomes a real problem if groups chosen on the basis of extreme scores |
| Poor reliability can cause extreme scores on a pretest to regress toward the mean on the posttest is an example of what? | Single Group Threats- Statistical Regression |
| Multiple Group Threats- Selection Interaction | Refers to factors other than the experimental intervention that can influence posttest differences between groupsCan be a problem if groups differ prior to data collection due to differential selection |
| Multiple Group Threats- Selection Interaction, can be problematic if: | Can be problematic if intact groups usedCan be problematic if IV is an attribute variable |
| Multiple Group Threats- Selection Interaction can be controlled by | Can be controlled by random assignmentCan also decrease threats if use matched pairs or ANCOVA |
| Multiple Group Threats- Selection-history effects | Experimental groups have different experiences between pretest & posttest |
| Multiple Group Threats- Selection-maturation effects | Experimental groups experience maturational change at difference speeds |
| Multiple Group Threats- Selection-testing effects | The pretest affects groups differently |
| Multiple Group Threats- Selection-instrumentation effects | The test is not consistent across groups Often due to variances in reliability |
| Multiple Group Threats-Selection-regression effects | A concern if groups are specifically divided based on higher and lower pretest scores |
| Social Threats | These are threats to internal validity that occur because of the interaction of subjects and investigatorsThese refer to pressures that can occur in research situations that may lead to difference between groups |
| Why do most Social Threats occur? | Most occur because those involved are aware of the other groups’ circumstances or are in contact with one another |
| Ambiguity About Direction Of Causal Influence | Did X cause Y or Y cause X? |
| Diffusion Or Imitation Of Treatments | Experimental information about the IV is not always intended for all groupsA threat potentially occurs if groups have the opportunity to communicate with one another and share information about treatments being received |
| If the control group becomes aware of the experimental situation members may change their behavior accordingly.... this is an example of : | Diffusion Or Imitation Of Treatments....This communication or knowledge will potentially diffuse the treatment effect making it impossible to distinguish the behaviors of the two group |
| Compensatory Equalization of Treatments | The experimental treatment is considered a desirable service or condition by the person administering the conditionConsequently, those administering the service or condition may try to even out experiences for those NOT receiving the condition |
| This will potentially obscure the treatment effects and make the groups look more alike.... is an example of what? | Compensatory Equalization of Treatments |
| Compensatory Rivalry & Resentful Demoralization | These two effects represent opposite reactions to the same situationThose not receiving what is perceived to be the “desirable” treatment may try to compensate by working extra hard to achieve similar results |
| the subject is responsible for equalizing the effects, is an example of what? | Compensatory Rivalry & Resentful Demoralization |
| Concerns the researcher’s goals & how well experimental results can be generalized... is an example of which threat to validity? | CONSTRUCT VALIDITY OF CAUSES AND EFFECTS |
| Importance lies in operational definitionsand A study can be internally sound but have no practical application... is an example of which threat to validity? | CONSTRUCT VALIDITY OF CAUSES AND EFFECTS |
| Potential biases can be introduced into a study by the researchers or the subjects... is an example of which threat to validity? | CONSTRUCT VALIDITY OF CAUSES AND EFFECTS |
| Importance lies in operational definitions... is an example of which threat to validity? | CONSTRUCT VALIDITY OF CAUSES AND EFFECTS |
| What are Operational Definitions | The belief that most constructs require use of multiple treatment methods & multiple measurement methodsIf these are only one dimensional the results will only apply to a limited aspect of the construct |
| Multiple-treatment interaction can create | carryover or combined effects |
| Order effects can occur if treatments or measurements are | are consistently given in same order |
| Multiple-Treatment Interaction | Participants receive more than one treatment or measurement, Carryover or combined effects can occur making it difficult to interpret the results out of the context of several responses |
| Order effects can also occur if treatments or measurements are given | given in the same order each time administered |
| In Multiple-Treatment Interaction does order matter? and can findings be generalized? | The order can potentially influence subsequent responsesThe findings cannot be generalized to situations where a single treatment or measurement is administered |
| Length of Follow-Up | Conclusions and interpretations cannot be made beyond the scope of the studyThe timeframe used for data collection must be considered when looking at the construct validity of the IV |
| Length of Follow-Up becomes a problem when: | Becomes a potential problem when a study addresses responses over timeIf the time is too short, cannot make generalizations about long-term effects of treatment |
| Experimental Bias | Biases introduced because of participant or researcher expectations: Hawthorne Effect, Rosenthal Effect, Avis Effect |
| Avis Effect: | Participants in control group try harder simply because they are in the control group |
| Rosenthal Effect: | Researcher’s behaviors or appearance impact how the participant responds |
| Hawthorne Effect: | Participants experience change as a result of simply being in a research project |
| The generalizability of a study is primarily related to the specific patient context and conditions under investigation is which threat to validity? | External validity |
| What are the 4 threats to external validity? | Interaction of Treatment and Selection, Interaction of Treatment and Setting, Interaction of Treatment and History, Reactive or Interactive Effects of Testing |
| Threat to External Validity- Interaction Of Treatment And Selection | Becomes a problem if samples are confined to certain types of participants, Cannot generalize to those who don’t have these characteristics, Adherence when participants don’t comply with experimental protocol |
| Threat to External Validity- Interaction Of Treatment And Setting | Results obtained from pts in one setting cannot be generalized to another settingTreatments that are effective under very constrained conditions (laboratory) may not effective under less constrained conditions (clinic)This is the “real world” issue |
| Threat to External Validity- Interaction Of Treatment And Setting AKA: | aka Reactive Effects of Experimental Arrangements |
| One specific example is:Study done in a hospital can’t generalize to an outpatient setting is an example of what? | Threat to External Validity- Interaction Of Treatment And Setting |
| Threat to External Validity- Interaction Of Treatment And History | Cannot generalize results to different periods of time |
| Elderly in the 1956 are not the same as elderly in 2010 is an example of what? | Threat to External Validity- Interaction Of Treatment And History |
| Threat to External Validity- Reactive Or Interactive Effects Of Testing | A pretest makes a participant more aware or sensitive to upcoming treatmentAs a result, the treatment may not be as effective without the pretest |
| CONTROLLING THREATS To Internal Validity- Random assignment and use of a control groups help guard against: | History up to the start of the study, Maturation, Selection bias, Statistical regression, Testing, Instrumentation & selection interactions |
| CONTROLLING THREATS To Internal Validity- Other techniques: | Matched-pairs, Within subjects designs, Placebo, Blind setup, Double-blind setup |
| Uncontrolled Threats To Internal Validity- Random assignment and use of a control group cannot rule out effects of: | Attrition, Imitating treatments, Compensatory reactions, Double-blinding will help control theseReactive or interactive effect, Instrumentation, Attrition |
| CONTROLLING THREATS to External Validity - Random Selection | External validity has to do with generalizability so you want the sample to be representative of the target populationRandom assignment is the best way to potentially obtain a representative sample of the target population |
| Levels of treatment can be randomly selected from the possible levels, Experimental situations from all possible situations, The dependent variable or variables can be randomly selected from a pool of potential dependent variables: can all be -______ | randomly selected |
| What is usually the best way to control most the threats to the external validity of a study? | Random Selection |
| The one threat to external validity that is more difficult to control is | the “reactive effects of experimental arrangement” which is problematic when conducting research under very controlled situations |
| Is this a true statement? When trying to increase on type of validity the other type is usually compromised | YES!!!! The researcher should make the decision up front, “a priori”, whether internal or external validity is most important OR whether a balance between the two is more important |
Created by:
NicoleB