click below

click below

Normal Size Small Size show me how

# Psychometric Methods

### Midterm

Question | Answer |
---|---|

Measurement | rules for assigning numbers to objects and properties of objects |

psychological assessment | collecting information for making behavioral predictions |

nominal scale | naming scale, each number takes on meaning of verbal label, not treated as numbers ex. numbers on sports teams |

ordinal scale | tells us order in which people stand, but not how much separates them, 1st, 2nd, 3rd (difference between 1st and 2nd isn't necessarily same as difference between 2nd and 3rd) |

interval scale | equal numerical differences represent equal differences in the trait ex (temperature, score on a test) |

ratio scale | when scale is constructed so a score of 0 means none of a trait, allow us to make proportional statements about a scale ex( taking 10 minutes on a test is 2x as long as taking 5 minutes on a test)f |

z-score | the distance between his raw score and mean/standard deviation (X-M)/(SD) |

Mode | the score that occurs most often |

Median | the middle score when all scores are listed smallest to largest |

Mean | the average score |

norm referenced test | having peers provide a standard, quality of performance is determined by comparison with others scores, designed to compare/rank individuals in relations to others |

Criterion referenced test | using a predetermined standard to determine the level of mastery on a specific skill, deliberately constructed to yield measurements that are directly interpretable in terms of specified performance standards, |

content validity | assessment of whether a test contains appropriate content and requires that appropriate processes be applied to that content, the extent to which a measure represents all facets of a given social construct. Consists of face and logical validity |

face validity | the appearance of reasonableness of a test (does it seem like it measures what it should, does a math test measure your knowledge of mathematics or reading) |

Logical validity | does the measure adequately sample what is needed to measure the thing? Depends on expert consensus , want to ensure its comprehensively mapped |

criterion-related validity | consists of two facets: predictive and concurrent validity is said to have this when the test has demonstrated its effectiveness in predicting criterion or indicators of a construct. |

predictive validity | if the test info is used to forecast future criterion performance (delay between collection and use as in SAT to predict GPA) extent to which score on assessment predicts future performance |

concurrent validity | scores on test and criterion are obtained at about the same time |

construct validity | extent to which assessment accurately assesses the construct "Are we measuring what we thing we are?" 3 parts: group contrasts, correlational studies, experimental studies |

construct underrepresentation | a threat to construct validation, occurs when the scope of test is too narrow and fails to include impt aspects of the construct |

convergent validity | the degree to which two measures of constructs that theoretically should be related, are in fact related ex: test the CV of a measure of self-esteem, researcher show that measures of similar constructs, such as self-worth, confidence, are also related |

discriminant validity | tests whether concepts or measurements that are supposed to be unrelated are, in fact, unrelated ex: intelligence should not correlate to height |

construct-irrelevant test variance | a threat to construct validation, refers to presence of reliable variance that is extraneous to the construct being quantified (c-it difficult-something makes test harder than it should be like anxiety c-it easy- something makes test easier) |

nomological net | a representation of the constructs of interest, their observable manifestations, and the interrelationships among and between these. a framework to organize construct validity |

what are the 3 primary sources of a nomological net? | group contrasts correlational studies experimental studies |

What are the 2 main components of a nomological net? | construct representation nomothethic span |

construct representation | underlying theoretical construct that explains the empirical |

nomothetic span | network of correlates and empirical findings |

Compare and contrast the construct representation and the nomothetic span. | both are parts of the nomological net, the construct representation is closely tied to the theoretical understanding of a construct while the nomothetic span encompasses the correlates and empirical findings |

the "twisted pear" | a heteroscedastic plotting of data points where the data closer to the origin have a higher correlation and predictability than those further from the origin which are spread out further than the regression line |

homoscedasticity | can predict uniformly along the regression line, if range is restricted, the correlation DECREASES |

heteroscedasticity | observed y scores around the regression line are not uniform, a restricted range INCREASES the correlation |

standard error of estimate | SD of observed scores around regression line |

multiple regression | uses several explanatory variables to predict the outcome of a criterion variable. (College GPA predicted by SAT and HS GPA) |

suppressor variable | has 4 traits: independent of criterion variable, (-)Beta weight in regression equation, significantly correlated with predictor variable, increases multiple R2 significantly (accounts for some noise in an equation and makes the other variable better) |

coefficient of determination R^2 | portion of the criterion variable that can be accounted for by the predictor variables (part of circle that overlaps and explains it) |

coefficient of non determination 1-R^2 | portion of the criterion variable that cannot be accounted for by the predictor variables (part where circles don't over lap) |

coefficient of alienation | the sqrt of the coefficient of nondetermination (sqrt 1-R^2) z-scores->SEE how much can you reduce the confidence intervals around y predicted below chance (1) 0=perfect prediction 1=no validity |

Incremental validity (via multiple regression) | ? |

Incremental validity (via Taylor-Russell Tables) | Hit Rate-Base Rate |

T-R: Base rate | how successful the original screening measures are at effectively choosing |

T-R: Selection ratio | what percentage of the total "applicant pool" how many you must select |

T-R: False Positive | accepting an applicant who fails |

T-R: True Positive | accepting an applicant who succeeds |

formative evaluation | assessment to guide future instruction |

summative evaluation | provides a summary of student achievement |

mental measurements yearbook | publication from Nebraska U begun by Oscar Buros to critically review tests/questionnaires to help determine their quality |

rival hypothesis | hypothesis that compete with traditional views, alternative explanation for a phenomenon |

meta-analysis | combines information from many studies to make the confidence intervals smaller and more precise |

validity generalization | capacity of an instrument to generalize to other settings (cross cultural, to different schools, different regions) the study determines the bounds of the generalizability |

Compare and contrast the two types of criterion related validity. | Predictive validity and concurrent validity, both are important parts of criterion related validity and both are related to time. Predictive validity involves a temporal gap between when collected and compared while concurrent |

compare and contrast convergent and discriminant validity | both are necessary aspects for a nomological net to be accurate convergent-the things which should be related are Discriminant-unrelated variables do not have strong correlation |

Can range truncation increase a validity coefficient? If so how? | Yes! In the case of heterosckedastic distributions, a smaller range has a higher correlation |

Why does the construct validation process never end? | you can always come up with additional relevant studies that could contribute to the construct and further support or reduce it |

What is a nomological net and what is it good for? | A frame work to organize construct validity, consists of both the theoretical and empirical information related to a construct, has 3 primary sources (group contrasts, correlational studies, experimental studies) never ending |

Discuss utility of Taylor-Russell tables. What kinds of info do you need to use them? | Precisely estimate proportion of true +/- and false +/- that assessment procedure gives you over what is currently in place. Need BR, SR, VC to use |

What is a suppressor variable, what do they do, how do you know you have one? | A supressor variable, when controlled for, strengthens the relationship between the other variables, it can detract from the actual relationships (verbal skills in test taking to determine pilot success) |

How do you know when you have a suppressor variable? | significant correlation to predictor variable independent of criterion variable -Beta weight in regression equation increases multiple R2 significantly |