Question | Answer |
Measurement | rules for assigning numbers to objects and properties of objects |
psychological assessment | collecting information for making behavioral predictions |
nominal scale | naming scale, each number takes on meaning of verbal label, not treated as numbers
ex. numbers on sports teams |
ordinal scale | tells us order in which people stand, but not how much separates them, 1st, 2nd, 3rd (difference between 1st and 2nd isn't necessarily same as difference between 2nd and 3rd) |
interval scale | equal numerical differences represent equal differences in the trait
ex (temperature, score on a test) |
ratio scale | when scale is constructed so a score of 0 means none of a trait, allow us to make proportional statements about a scale
ex( taking 10 minutes on a test is 2x as long as taking 5 minutes on a test)f |
z-score | the distance between his raw score and mean/standard deviation
(X-M)/(SD) |
Mode | the score that occurs most often |
Median | the middle score when all scores are listed smallest to largest |
Mean | the average score |
norm referenced test | having peers provide a standard, quality of performance is determined by comparison with others scores, designed to compare/rank individuals in relations to others |
Criterion referenced test | using a predetermined standard to determine the level of mastery on a specific skill, deliberately constructed to yield measurements that are directly interpretable in terms of specified performance standards, |
content validity | assessment of whether a test contains appropriate content and requires that appropriate processes be applied to that content, the extent to which a measure represents all facets of a given social construct.
Consists of face and logical validity |
face validity | the appearance of reasonableness of a test (does it seem like it measures what it should, does a math test measure your knowledge of mathematics or reading) |
Logical validity | does the measure adequately sample what is needed to measure the thing? Depends on expert consensus , want to ensure its comprehensively mapped |
criterion-related validity | consists of two facets:
predictive and concurrent validity
is said to have this when the test has demonstrated its effectiveness in predicting criterion or indicators of a construct. |
predictive validity | if the test info is used to forecast future criterion performance (delay between collection and use as in SAT to predict GPA)
extent to which score on assessment predicts future performance |
concurrent validity | scores on test and criterion are obtained at about the same time |
construct validity | extent to which assessment accurately assesses the construct
"Are we measuring what we thing we are?"
3 parts: group contrasts, correlational studies, experimental studies |
construct underrepresentation | a threat to construct validation, occurs when the scope of test is too narrow and fails to include impt aspects of the construct |
convergent validity | the degree to which two measures of constructs that theoretically should be related, are in fact related
ex: test the CV of a measure of self-esteem, researcher show that measures of similar constructs, such as self-worth, confidence, are also related |
discriminant validity | tests whether concepts or measurements that are supposed to be unrelated are, in fact, unrelated
ex: intelligence should not correlate to height |
construct-irrelevant test variance | a threat to construct validation, refers to presence of reliable variance that is extraneous to the construct being quantified (c-it difficult-something makes test harder than it should be like anxiety
c-it easy- something makes test easier) |
nomological net | a representation of the constructs of interest, their observable manifestations, and the interrelationships among and between these. a framework to organize construct validity |
what are the 3 primary sources of a nomological net? | group contrasts
correlational studies
experimental studies |
What are the 2 main components of a nomological net? | construct representation
nomothethic span |
construct representation | underlying theoretical construct that explains the empirical |
nomothetic span | network of correlates and empirical findings |
Compare and contrast the construct representation and the nomothetic span. | both are parts of the nomological net, the construct representation is closely tied to the theoretical understanding of a construct while the nomothetic span encompasses the correlates and empirical findings |
the "twisted pear" | a heteroscedastic plotting of data points where the data closer to the origin have a higher correlation and predictability than those further from the origin which are spread out further than the regression line |
homoscedasticity | can predict uniformly along the regression line, if range is restricted, the correlation DECREASES |
heteroscedasticity | observed y scores around the regression line are not uniform, a restricted range INCREASES the correlation |
standard error of estimate | SD of observed scores around regression line |
multiple regression | uses several explanatory variables to predict the outcome of a criterion variable.
(College GPA predicted by SAT and HS GPA) |
suppressor variable | has 4 traits: independent of criterion variable, (-)Beta weight in regression equation, significantly correlated with predictor variable, increases multiple R2 significantly
(accounts for some noise in an equation and makes the other variable better) |
coefficient of determination R^2 | portion of the criterion variable that can be accounted for by the predictor variables (part of circle that overlaps and explains it) |
coefficient of non determination 1-R^2 | portion of the criterion variable that cannot be accounted for by the predictor variables (part where circles don't over lap) |
coefficient of alienation | the sqrt of the coefficient of nondetermination (sqrt 1-R^2)
z-scores->SEE
how much can you reduce the confidence intervals around y predicted below chance (1)
0=perfect prediction 1=no validity |
Incremental validity (via multiple regression) | ? |
Incremental validity (via Taylor-Russell Tables) | Hit Rate-Base Rate |
T-R: Base rate | how successful the original screening measures are at effectively choosing |
T-R: Selection ratio | what percentage of the total "applicant pool" how many you must select |
T-R: False Positive | accepting an applicant who fails |
T-R: True Positive | accepting an applicant who succeeds |
formative evaluation | assessment to guide future instruction |
summative evaluation | provides a summary of student achievement |
mental measurements yearbook | publication from Nebraska U begun by Oscar Buros to critically review tests/questionnaires to help determine their quality |
rival hypothesis | hypothesis that compete with traditional views, alternative explanation for a phenomenon |
meta-analysis | combines information from many studies to make the confidence intervals smaller and more precise |
validity generalization | capacity of an instrument to generalize to other settings (cross cultural, to different schools, different regions) the study determines the bounds of the generalizability |
Compare and contrast the two types of criterion related validity. | Predictive validity and concurrent validity, both are important parts of criterion related validity and both are related to time. Predictive validity involves a temporal gap between when collected and compared while concurrent |
compare and contrast convergent and discriminant validity | both are necessary aspects for a nomological net to be accurate
convergent-the things which should be related are
Discriminant-unrelated variables do not have strong correlation |
Can range truncation increase a validity coefficient? If so how? | Yes! In the case of heterosckedastic distributions, a smaller range has a higher correlation |
Why does the construct validation process never end? | you can always come up with additional relevant studies that could contribute to the construct and further support or reduce it |
What is a nomological net and what is it good for? | A frame work to organize construct validity, consists of both the theoretical and empirical information related to a construct, has 3 primary sources (group contrasts, correlational studies, experimental studies) never ending |
Discuss utility of Taylor-Russell tables. What kinds of info do you need to use them? | Precisely estimate proportion of true +/- and false +/- that assessment procedure gives you over what is currently in place. Need BR, SR, VC to use |
What is a suppressor variable, what do they do, how do you know you have one? | A supressor variable, when controlled for, strengthens the relationship between the other variables, it can detract from the actual relationships (verbal skills in test taking to determine pilot success) |
How do you know when you have a suppressor variable? | significant correlation to predictor variable
independent of criterion variable
-Beta weight in regression equation
increases multiple R2 significantly |