Psychology owes the advanced development of reliability assessment to the early work of
Charles Spearman
Abraham introduced the notion of
sampling error
The term error refers to discrepancies that will always exist between....... ability and the ...... of ability
true, measurement
the difference between a persons true score and observed score results from
measurement error
the equation for true/observed score
X = T + E
Spearman published the earliest work on reliability theory in his 1904 article
"The proof and measurement of Association between two things"
The distribution of random errors is what shape
bell shaped
Dispersion around the observed scored tells us
how much error there is in the measurement
Classical teat tehory uses the standard deviation of this distribution as a basic measure of error called
the standard error of measurement
standard error of measurement tells us
on average, how much an observed score deviates from a true score
if three random samples of items are selected from the entire domain of all possible items, each of the three samples should yield an unbiased estimate of the ........ related to the domain
true score
Reliability coeffiecients are
ratio coefficients that can also be expressed as the ratio of the variance of the true score on a test to the variance of the observed score on a test
reliability coefficients cab be interpreted as the proportion of variation in observed scores that can be attributed to
actual differences among people
three methods of estimating test reliability
test-retest, parallel form, and internal consistency
test-retest
consideres the consistecy of the test results when the test is admistered on differen occasions
parallel forms
evaluate the test across differen froms of the test
internal consistency
we examine how people perform on similar subsets of items selected from the same form of the measure
Test-Retest is appropritate when
it is assumed a trait being measured does not change over time... E.G intelligence
Carryover effect
occur when something about taking the test the firsttime influences the second time the test is taken
carryover effect can lead to an.......... of the true reliability of a test
overestimate
Parallel forms
the correlation between form a and from b of a test
split half method
a test given and divided into halves that are scored separately
Split half method can measure
internal consistency
Problem with split half
how its split; one half may have more difficult problems than the other half
to correct for half lenght you can use the.... formula
spearman-brown formula
spearman brown formula allows you
to estimate what the correlatoin between the two halves would have been if each half had been the length of the who test
spearman brown formula
R= 2r/(1+r) where r is the correlation between the two halves
the spearman brown formula increases the
reliability
Coefficient alpha
estimates teh internal consistency of tests in which the items are not scored 0 to 1 (right or wrong) in doing so he developed a more general reliability estimate
under what cirumstance is coefficient alpha appropriate measure of internal consisteny reliability of a test
when items are not dichotomosly scored
all measures of internal consistency (split half, coefficient alpha) evaluate the extent to which the different items on a test
measure the same ability of trait
if a test is designed to measure several different traits or characteristics, .................., is a good way to examine whether items within subgroups or components are internally consistent
factor analysis
behavirol psychologist prefer to use
observation of behavior opposed to psychological test
To asses the behavioral observations we must asses the reliability of the observer: two ways
interrater reliability or kappa
interrater reliability
the consistency among different judges who are evaluating the same behavior
three ways to an observer can evaluate behavior
record the precentage of times that two or more observers agree, kappa statistics
kappa statistics was introduced by
J. Cohen in 1960
Kappa statistics is
an agreement between two judges who each rate a set of objects using nominal scales.
the standard error of measurement is used to form .........around observed scores. Th
confidence intervals
Confidence intervals allow us to estimate the probability that the ........... falss within certain interval
true score
the interval around the observed score is the
confidence interval
if we have an 82 observed score and a 95% CI this means that
we are 95% that the true score relies around 78-84 interval
when importnat decisions are to be made on the basis of test socre, reliability coefficients should be ...... or higher
.95
basic research should have a reliability coefficient of
.70 to .80
allows test developers to estimate what the correlation between two measures would have been if the they had not been measured with error
correction for attenuation
a reliability of a difference score is expected to be
lowere that the reliability of either test on which it is based
administering two supposedly equivealent forms of test (form a and form b) to the same group of individuals yilelds a correlation coefficient indicating
alternative forms reliability
what did the suprem court rule in the case of Griggs vs. Duke Power?
Employees must provide evidence that a test used for selection of promotion of employees has a specific meaning.
Validity is
the agreement between test score or measure and the quality it is believed to mesure. "Does the test measure what is supposed to?"
Three primary types of evidence that can suppor a tests validity
face validity, content, and criterion related validity,
Face validity
mere appearance that a mesure has validity. If the items seem to be reasonably related to the perceived purpose of the test.
Face validity is not really validity because
it does not offer evidence to support conclusions drawn from test scores
consideres the adequacy of representation of the coneptual domain the test is designed to cover
content-related validity
has the test been constructed adequately would be what type of validity
content
contruct under representation is
the failure to capture important components of a construct
If a math test on included algebra not geometry this would be an example of
consrtuct under representation
Construct irrelevant variance is
when scores are influenced by irralevent factors
Test Anxiety, illiness, sleepy, are all factors that might influence a test this is temred
construct irrelevant variance
Criterion Related Evidence for Validity
tells us how well a test corresponds with a particulare criterion. Provided by high correlations between a rest and a well-defined criterion measure
predicitve validity
the forecasting function of test
Concurrent Relate evidence for validity is
assessments of thei simultaneous relationships between the test ad the crieterion-- such as between a learning disability and school performance
Validity coefficient
relationship between a test and a criterion is usually expressed a correlation called Validity coefficient
Are scale correlates well with other scales is an example of what type of evidence
Concurrent evidince (under criterion)
Construct validity evidence
is established through a series of activities in the which a reseracher simultaneously defines some construct and develops the instrumentation to measure it.
A construct is
something built by mental synthesis. Ex. Intelligence, love, curiosity, or mental health
convergent evidence
- When a measure correlates well with other tests believed to measure the same construct
- Convergent evidence is obtained in one of two ways
- 1) we show that a test measures the same things as other tests used for the same purpose.
MedTemCP5 Word+Pics
