Chapter 12: Evaluating the Methods Section—Instruments

Test Bank

Multiple Choice

1. ______ scores are based on a predetermined level of performance to a comparison study.

A. Criterion-referenced

B. Norm-referenced

C. Analytically referenced

D. Reliant-referenced

Cognitive Domain: Knowledge

Answer Location: Standardization of Instruments

Difficulty Level: Easy

2. In ______ approach, scores are interpreted based on the comparison of one group’s performance with other groups’, all of whom represent a clearly defined population.

A. criterion-referenced

B. norm-referenced

C. analytically referenced

D. reliant-referenced

Cognitive Domain: Knowledge

Answer Location: Standardization of Instruments

Difficulty Level: Easy

3. Thorndike and Thorndike-Christ noted that many tests and measures are designed to “reflect the values and experiences of typical ______.”

A. middle class

B. lower class

C. upper class

D. poor

Cognitive Domain: Knowledge

Answer Location: Culture, Gender, and Diversity Bias

Difficulty Level: Easy

4. Reliability is reported as a correlation expressed as the degree of the relationship between two sets of scores. Possible scores in these correlations can range from ______.

A. –2.0 to + 2.0

B. 0 to +1.0

C. –3.0 to +3.0

D. –1.0 to +1.0

Cognitive Domain: Knowledge

Answer Location: Reliability of Instruments

Difficulty Level: Easy

5. Strong indicators of reliability in a particular test or measure have a score of at least ______.

A. .75

B. .60

C. .80

D. .50

Cognitive Domain: Knowledge

Answer Location: Reliability of Instruments

Difficulty Level: Easy

6. ______ concerns the consistency of a measure based on content, for instance, when two forms of the measure are administered to a sample and reliability coefficients are obtained between them.

A. Test–retest reliability

B. Scorer reliability

C. Alternate-form reliability

D. Interrater reliability

Cognitive Domain: Knowledge

Answer Location: Reliability of Instruments

Difficulty Level: Easy

7. Which type of reliability involves the measurement of the consistency of scores over a period of time?

A. test–retest reliability

B. scorer reliability

C. alternate-form reliability

D. interrater reliability

Cognitive Domain: Knowledge

Answer Location: Reliability of Instruments

Difficulty Level: Easy

8. A basic type of validity that concerns the extent to which responses on test items represent a particular content is known as ______.

A. construct validity

B. criterion-related validity

C. content validity

D. convergence and divergence evidence

Cognitive Domain: Knowledge

Answer Location: Validity of Instruments

Difficulty Level: Easy

9. Which type of validity involves obtaining evidence of prediction of a particular construct, or accuracy in identifying a construct in practice?

A. construct validity

B. criterion-related validity

C. content validity

D. internal validity

Cognitive Domain: Knowledge

Answer Location: Validity of Instruments

Difficulty Level: Easy

10. All the potential effects that alter the true score, such as the limitations of the test, time of day in responding to the measure, and lack of consistent administration procedures, are referred to as ______.

A. truer score

B. error

C. formal structure

D. criterion structure

Cognitive Domain: Knowledge

Answer Location: Reliability of Instruments

Difficulty Level: Easy

11. Factor analysis is a source for ______.

A. criterion-related validity

B. convergence and divergence evidence

C. content-related validity

D. construct validity

Cognitive Domain: Knowledge

Answer Location: Validity of Instruments

Difficulty Level: Easy

12. An approach to measuring dependent variables where scores are used to evaluate behavior is known as ______.

A. self-report

B. ratings

C. interviews

D. direct observation

Cognitive Domain: Knowledge

Answer Location: Approaches to Measuring the Dependent Variable

Difficulty Level: Easy

13. The strength of this approach to measuring dependent variables is ease and time of administration and does not require extensive training is ______.

A. ratings

B. interviews

C. self-report inventories

D. physiological method

Cognitive Domain: Knowledge

Answer Location: Approaches to Measuring the Dependent Variable

Difficulty Level: Easy

14. One of the cornerstone tools for the assessment of behavioral, social, and emotional problems is ______.

A. direct observation

B. indirect observation

C. interviews

D. self-report

Cognitive Domain: Knowledge

Answer Location: Quantitative Research Examples

Difficulty Level: Easy

15. Assessments that measure heart rate, ECG, blood pressure, and galvanic skin response are known as ______.

A. direct observation

B. indirect observation

C. physiological methods

D. self-report

Cognitive Domain: Knowledge

Answer Location: Approaches to Measuring the Dependent Variable

Difficulty Level: Easy

True/False

1. Instrumentation is that area of the methods section where the researchers systematically describe the dependent measures used in the study.

Cognitive Domain: Knowledge

Answer Location: Evaluating the Methods Section—Instruments

Difficulty Level: Easy

2. Standardization in administering the dependent measure theoretically provides more objective data.

Cognitive Domain: Knowledge

Answer Location: Standardization of Instruments

Difficulty Level: Easy

3. Researchers and practitioners have noted the importance of using measures of the dependent variable that are culturally, gender, and diversity-issue appropriate.

Cognitive Domain: Knowledge

Answer Location: Culture, Gender, and Diversity Bias

Difficulty Level: Easy

4. Use of an inappropriate test or dependent measure can result in test bias and problems with interpreting outcomes.

Cognitive Domain: Knowledge

Answer Location: Culture, Gender, and Diversity Bias

Difficulty Level: Easy

5. A higher positive correlation indicates a weak relationship between the two sets of scores.

Cognitive Domain: Knowledge

Answer Location: Reliability of Instruments

Difficulty Level: Easy

6. Scorer reliability is the most frequently reported type of reliability.

Cognitive Domain: Knowledge

Answer Location: Reliability of Instruments

Difficulty Level: Easy

7. The most general type of validity is construct validity because it supposedly concerns the degree to which a test measures an intended theoretical concept or construct.

Cognitive Domain: Knowledge

Answer Location: Validity of Instruments

Difficulty Level: Easy

8. Without information on the reliability and validity of the instruments used in a quantitative study, it is difficult to conclude whether the study results obtained are accurate.

Cognitive Domain: Knowledge

Answer Location: Summary

Difficulty Level: Easy

9. Psychometricians have long acknowledged that validity is the least important characteristic of a test.

Cognitive Domain: Knowledge

Answer Location: Validity of Instruments

Difficulty Level: Easy

10. Self-report and inventories are not the most frequently used methods of measuring the dependent variable in psychology, counseling, and education.

Cognitive Domain: Knowledge

Answer Location: Approaches to Measuring the Dependent Variable

Difficulty Level: Easy

Short Answer

1. Provide an example of a criterion-referenced score.

Cognitive Domain: Application

Answer Location: Evaluating Single-Subject Research

Difficulty Level: Medium

2. Provide an example of a norm-referenced score.

Cognitive Domain: Application

Answer Location: Evaluating Single-Subject Research

Difficulty Level: Medium

3. List a few of the ways to determine reliability.

Cognitive Domain: Comprehension

Answer Location: Reliability of Instruments

Difficulty Level: Easy

4. Give an example of the test–retest method of determining reliability.

Cognitive Domain: Application

Answer Location: Reliability of Instruments

Difficulty Level: Medium

5. Define validity.

Cognitive Domain: Comprehension

Answer Location: Validity of Instruments

Difficulty Level: Easy

6. What are the three components of construct validity Urbina (2004) identified?

Cognitive Domain: Comprehension

Answer Location: Validity of Instruments

Difficulty Level: Easy

7. Provide an example of content validity.

Cognitive Domain: Application

Answer Location: Validity of Instruments

Difficulty Level: Medium

8. Provide an example of a self-report inventory.

Cognitive Domain: Comprehension

Answer Location: Approaches to Measuring the Dependent Variable

Difficulty Level: Easy

9. Define reliability.

Cognitive Domain: Comprehension

Answer Location: Reliability of Instruments

Difficulty Level: Medium

10. What is the calculation of Kappa and when would you use this?

Cognitive Domain: Comprehension

Answer Location: Evaluating Single-Subject Research

Difficulty Level: Easy

Essay

1. An important consideration in assessing the objectivity and quality of a dependent measure is the extent to which the procedures and scoring methods are standardized. Discuss the standardization methods delineated in the text.

Cognitive Domain: Analysis

Answer Location: Standardization of Instruments

Difficulty Level: Medium

2. Discuss the APA Standards for Educational and Psychological Tests and the American Counseling Association Code of Ethics regarding tests and diversity.

Cognitive Domain: Analysis

Answer Location: Culture, Gender, and Diversity Bias

Difficulty Level: Medium

3. Researchers have noted that one of the most important considerations in determining the quality of the dependent measures is the reliability of the instruments and dependent variables. Define and discuss reliability of the instruments and the various forms of determining reliability.

Cognitive Domain: Analysis

Answer Location: Reliability of Instruments

Difficulty Level: Medium

4. Discuss the advantages and disadvantages of using self-report inventories.

Cognitive Domain: Application

Answer Location: Approaches to Measuring the Dependent Variable

Difficulty Level: Hard

5. Discuss alternate-form reliability. What is a benefit of using this approach?

Cognitive Domain: Application

Answer Location: Reliability of Instruments Methods

Difficulty Level: Hard