Test and Measurement
Quiz yourself by thinking what should be in
each of the black spaces below before clicking
on it to display the answer.
Help!
|
|
||||
|---|---|---|---|---|---|
| show | The consistency or stability of scores.
🗑
|
||||
| Classical Test Theory | show 🗑
|
||||
| Random Measurement Error | show 🗑
|
||||
| show | Unsystematic Error
🗑
|
||||
| How do we increase our confidence in scores? | show 🗑
|
||||
| Content Sampling Error | show 🗑
|
||||
| show | Random and transient. Situation-Centered influences (e.g., lighting & noise) and Person-Centered influences (e.g., fatigue, illness)
🗑
|
||||
| show | Administration errors (e.g., incorrect instructions, inaccurate timing) and Scoring errors (e.g., subjective scoring, clerical errors).
🗑
|
||||
| Classical Test Theory | show 🗑
|
||||
| X | show 🗑
|
||||
| show | True score (reflects stable characteristics)
🗑
|
||||
| E | show 🗑
|
||||
| show | reflects the temporal stability of a measure. Most applicable with tests administered more than once and/or with constructs that are viewed as stable. It is important to consider length of interval.
🗑
|
||||
| show | “carry-over effects.” Appropriate for tests that are not appreciably impacted by carry-over effects.
🗑
|
||||
| Alternate Form Reliability | show 🗑
|
||||
| show | reflects error due to temporal stability and content sampling.
🗑
|
||||
| Simultaneous Administration for Alternate Form Reliability | show 🗑
|
||||
| Alternate Form Reliability limitations | show 🗑
|
||||
| show | Estimates of reliability that are based on the relationship between items within a test and are derived from a single administration of a test.
Split-Half Reliability
🗑
|
||||
| Split-Half Reliability reflects what type of error? | show 🗑
|
||||
| Coefficient Alpha & Kuder-Richardson (KR 20) | show 🗑
|
||||
| show | For speed tests, reliability estimates derived from a single administration of a test are inappropriate. For speed tests, test-retest and alternate-form reliability are appropriate, but split-half, Coefficient Alpha and KR 20 should be avoided.
🗑
|
||||
| Inter-Rater Reliability | show 🗑
|
||||
| show | Difference scores are calculated when comparing performance on two tests.
Why is the reliability of the difference between two test scores generally lower than the reliabilities of the two tests?
🗑
|
||||
| show | When there are multiple scores available for an individual one can calculate composite scores (e.g., assigning grades in class).
What is the important issue with reliability of a composite score?
🗑
|
||||
| show | If a test score is used to make important decisions that will significantly impact individuals, the reliability should be very high
🗑
|
||||
| show | If a test is interpreted independently but as part of a larger assessment process (e.g., personality test), most set the standard as .80 or greater.
🗑
|
||||
| Standards for Reliability in research or composite score | show 🗑
|
||||
| show | Increase the number of items (i.e, better domain sampling). Use multiple measurements (i.e., composite scores). Use “Item Analysis” procedures to select the best items.Increase standardization of the test (e.g., administration and scoring).
🗑
|
||||
| Standard Error of Measurement (SEM) | show 🗑
|
||||
| show | the average amount of error in test scores. Technically, the SEM is the standard deviation of error scores around the true score.
🗑
|
||||
| How is the SEM calculated? | show 🗑
|
||||
| What is the relationship between rxx and SEM? | show 🗑
|
||||
| show | reflects a range of scores that will contain the test taker’s true score with a prescribed probability.
What is used to calculate confidence intervals?
🗑
|
||||
| show | is that they remind us that measurement error is present in all scores and we should interpret scores cautiously.
🗑
|
||||
| show | “The range within which a person’s true score is expected to fall -- % of the time.”
🗑
|
||||
| What is the relationship between SEM and confidence intervals? | show 🗑
|
||||
| show | An extension of Classical Test Theory. Classical Theory - all error is random. Generalizability Theory - recognizes sources of systematic error.
🗑
|
||||
| show | If you have a situation in which there is no opportunity for systematic error to enter the model, Classical and Generalizability are mathematically identical.
🗑
|
||||
| show | objective tests are administered under standardized conditions (e.g., SAT or GRE).
🗑
|
||||
| When is Generalizability Theory useful | show 🗑
|
Review the information in the table. When you are ready to quiz yourself you can hide individual columns or the entire table. Then you can click on the empty cells to reveal the answer. Try to recall what will be displayed before clicking the empty cell.
To hide a column, click on the column name.
To hide the entire table, click on the "Hide All" button.
You may also shuffle the rows of the table by clicking on the "Shuffle" button.
Or sort by any of the columns using the down arrow next to any column heading.
If you know all the data on any row, you can temporarily remove it by tapping the trash can to the right of the row.
To hide a column, click on the column name.
To hide the entire table, click on the "Hide All" button.
You may also shuffle the rows of the table by clicking on the "Shuffle" button.
Or sort by any of the columns using the down arrow next to any column heading.
If you know all the data on any row, you can temporarily remove it by tapping the trash can to the right of the row.
Embed Code - If you would like this activity on your web page, copy the script below and paste it into your web page.
Normal Size Small Size show me how
Normal Size Small Size show me how
Created by:
kxiong