Help
Options
Didn't know it?
click below
click below
Knew it?
click below
click below
Don't Know
Remaining cards (0)
Know
retry
shuffle
restart
0:00
Embed Code - If you would like this activity on your web page, copy the script below and paste it into your web page.
Normal Size Small Size show me how
Normal Size Small Size show me how
Item Analysis
| Question | Answer |
|---|---|
| Item Analysis | “The better the items, the better the test.” |
| Item Analysis | The reliability of test scores and the validity of the interpretation of test scores are dependent on the quality of the items in the test. |
| p | # of Subjects with Correct Answer/ Number of Subjects |
| Item Difficulty Index | Ranges from 0.0 to 1.0. Large p indicates an easy item; a small p indicates a difficult item. |
| Items that are correctly answered by every subject (p = 1.0) and items that are missed by every subject (p = 0.0) provide what? | no information about individual differences and are of no value from a measurement perspective. |
| What are desirable p values? | For maximizing variability and reliability, the optimal item difficulty value is 0.5. |
| Too many difficult items and few subjects get them correct results in what? | in reduced variability. |
| Too many easy questions and most subjects get them correct results in what? | reducing variability. |
| Due to the effects of guessing | for selected response items the optimal p is > 0.50. |
| In practice, a general recommendation is to use items with p values with a range of what? | approximately 0.20 around the optimal values. For example, if your optimal p value is 0.50, you would select items ranging from 0.40 to 0.60, with a mean of 0.50. |
| p Values & Mastery Tests | On mastery tests in schools the students either pass or fail and there is the expectation that most students will eventually be successful. As a result, on many mastery tests it is common for items to have p values that are relatively high. |
| In mastery testing situations where you want to select what? | a specific proportion of subjects (e.g., employment screening), the optimal item difficulty value will equal the proportion of subjects you expect to pass the test. If you want to identify the top 50% of a sample, the optimal p value is? |
| If you want to identify the top 20%, the optimal p value is? | 0.20. |
| Other Considerations with p | If you need to distinguish among the most talented individuals, it may be desirable to include some items with low p values to challenge the most gifted individuals. |
| Item Difficulty Index | Requires correct/incorrect scoring. If there is partial credit awarded, you should dichotomize the scoring. (anything below 5 = 0, above 5 =1) |
| Why is Item difficulty is specific to the sample? | For example, an item could be difficult for a young sample (1st grade) and easy for an older sample (6th grade). |
| Item discrimination refers to what? | how well an item can discriminate or differentiate among test takers who differ on the construct being measured by the test. On a test of reading comprehension, item discrimination reflects an item’s ability to distinguish between individuals with good a |
| D | pT - pB |
| pT | the proportion of people in the top group who are correct on the item |
| pB | the proportion of people in the bottom group who are correct on the item |
| What is one common approach is to Item Discrimination Index? | compare the top 27% versus bottom 27% and exclude the middle 46%. |
| What is a general rule suggested for items with D values? | over 0.30 are acceptable and items with D values below 0.30 should be reviewed and possibly revised or deleted. |
| Items with D values close to 0.0 | do not contribute to the ability of the test of discriminate. |
| Items with negative values | do not measure the same characteristic as the overall test and actually detract from the test. |
| very difficult & very easy items tend to have | smaller D values. |
| Item-Total Correlations | A procedure where performance on individual items is correlated with the total test score (typically using the point-biserial correlation). |
| What is unadjusted test score? | The total test score is usually the total number of items answered correctly (unadjusted) |
| What is adjusted test score? | the total number of answers correctly omitting the item being examined (adjusted). |
| A large item-total correlation is taken as evidence of what? | that an item is measuring the same construct as the overall test and discriminates between individuals high on the construct and those low on the construct. |
| The square of the item-total correlation indicates what? | the proportion of variance on the test predictable from performance on the item. |
| Item Discrimination & Mastery Tests | Since it common for items on Mastery Tests to have high p values, standard procedures for determining item discrimination need to be modified.Special procedures have been developed for these tests (e.g., D based on subjects who received instruction versu |
| On speed tests, measures of item difficulty and discrimination will largely reflect what? | the location of the item in the test rather than the item’s actual difficulty level and ability to discriminate. |
| Distracter Analysis | With multiple-choice items, an incorrect alternative is referred to as a distracter. |
| If the item has a marginal p or D and you want to do what? | revise it, Distracter Analysis may help. |
| Distracter Analysis allows you to do what? | evaluate the percentage of people selecting each distracter to determine which ones are useful. |
| What is positive discrimination? | The discrimination index tells you if more people in the top group than the bottom group selected the correct answer |
| What is negative discrimination? | With distracter analysis you examine the individual distracters to see if they have negative discrimination (i.e., selected more by people in the bottom group). |
| Distracter Power Analysis | Distracters should demonstrate negative discrimination. (don’t want top group to be selecting the wrong answers). If they have positive discrimination, there is likely some ambiguity present. |
| If a distracter is rarely or never selected, what does that mean? | it is probably so obviously wrong that no one selects it - it is ineffective and should be revised. |
| Items with intermediate p values have the greatest potential for what? | for being the best at discriminating. |
| The selection of distracters strongly influences what? | the difficulty level of the item. |
| Qualitative Item Analysis | Set the test aside and after a break carefully proof the test. Have a colleague familiar with the content area review the test for errors. After administering the test, solicit feedback from examinees about ambiguous items. |
| Item Characteristic Curves | ICC is a graph on which ability is plotted on the horizontal axis and the probability of a correct response is plotted on the vertical axis. |
Created by:
kxiong