Ch4 Validity And Reliability: Threats To Research Test Bank

CHAPTER 4

TRUE/FALSE

1. Validity is closely related to the principle of operationalization, which addresses how variables are measured.
2. Oftentimes a single measure may not provide a true measure of a variable. In this case, researchers can combine several different measures together.
3. Higher values of Cronbach’s alpha represent higher reliability of a certain scale or index.
4. Many researchers make a useful analogy to help understand validity and reliability when developing a case by comparing them to a target and marksmanship. The tightness of the bullet holes reflects validity.
5. Reliability can be tested in several methods, including test–retest reliability, intraobserver reliability, interitem reliability, and alternate-forms reliability.
6. Natural disaster or climate change is the only example of history threatening internal validity.
7. History is the most difficult threat to internal validity because there is not much to mitigate its effect.
8. Testing effect is a problem when change in the measuring instrument is made between pretest and posttest.
9. Because of statistical regression, it is important to analyze the data for outliers and consider their effects.
10. The best way to overcome selection bias is to use randomization when assigning research participants to different groups.
11. Diffusion of treatment in regards to validity and reliability means that the stimulus in an experiment was administered on time.
12. More than one threat to internal validity can occur simultaneously, which can be named confounding interactions.
13. The Hawthorne effect and placebo effect are examples of reactivity.
14. Criterion validity determines how well a measure covers the meanings included in the concept.
15. To reify something means to treat something that is not real as if it were real.

MULTIPLE CHOICE

1. Which of the following is closely related to the term validity?
2. Approximation to the truth
3. Reflection of a characteristics or property in the real world
4. Accuracy
5. All of the above
6. Which of the following is closely related to the term reliability?
7. Consistency
8. Repeatability
9. Precision
10. All of the above
11. To improve the reliability of an instrument, researchers may use the same exam or questionnaire more than one time, which is known as __________.
12. test–retest reliability
13. intraobserver reliability
14. interim reliability
15. alternate-forms reliability
16. Qualitative researchers make observations more than one time, and the same result is obtained, which is called ___________ reliability.
17. test–retest
18. intraobserver
19. interim
20. alternate-forms
21. To determine reliability, researchers can take multiple questions when measuring the factor of interest, which is known as __________ reliability.
22. test–retest
23. intraobserver
24. interim
25. alternate-forms
26. Which of the following is false regarding internal validity?
27. Internal validity exists when a relationship between two variables is causal.
28. In other words, each of the variables must vary together.
29. However, the direction of the change between variables does matter.
30. The covariance between variables may result from some unknown variable(s) called exogenous variables.
31. __________ validity is concerned with how well the study findings are generalizable to other settings.
32. Internal
33. External
34. Statistical conclusion
35. Content
36. Among various threats to internal validity, __________ refers to any events that happen during the course of a study that may affect the study outcomes.
37. history
38. maturation
39. testing effect
40. instrumentation
41. Which of the following can be an example of history endangering internal validity?
42. Natural disaster such as Hurricane Katrina
43. Terrorist attack
44. Economic crisis
45. All of the above
46. Among threats to internal validity, __________ refers to any physical or psychological changes taking place within participants that occur with the passing of time regardless of the experimental manipulation.
47. history
48. maturation
49. testing effect
50. instrumentation
51. Which of the following is false regarding maturation?
52. Maturation threatens internal validity mainly because research participants get older and experienced.
53. It is true that research participants’ emotional make-up can influence a study’s outcome.
54. There is no solution to maturation effects to interval validity.
55. None of the above
56. __________ occurs when research participants become so familiar with the survey instruments that their answers are influenced.
57. History
58. Maturation
59. Testing effect
60. Instrumentation
61. Which of the following can be a solution to testing effect?
62. Random assignment
63. Altering the original questions with wording
64. Providing post–test
65. All of the above
66. __________lowers the level of internal validity when change in the measuring instrument is made between the applications.
67. History
68. Maturation
69. Testing effect
70. Instrumentation
71. Which of the following is false regarding intercoder reliability?
72. Intercoder reliability occurs when a multiple number of researches interview the same participant, but get different responses.
73. It is possible that researchers interpret the same participant’s responses differently.
74. Intercoder reliability threat can occur only in qualitative research.
75. Intercoder reliability threat can be reduced with proper training and by utilizing research assistants who are motivated and conscience.
76. __________ occurs when outliers move closer to the mean on subsequent testing.
77. Normal distribution
78. Statistical regression
79. Validity
80. Selection bias
81. What is the simplest solution to statistical regression?
82. To remove outliers from the study or convert outliers to the mean value.
83. To allow research participants to retake the exam until they are satisfied with their result
84. To retrieve another sample
85. There is no solution to statistical regression.
86. __________ occurs when the research participants in the experimental group are not similar to those in the control group.
87. Statistical regression
88. Selection bias
89. Mortality
90. Diffusion of treatment
91. __________ occurs when research participants drop out of the study before it concludes.
92. Mortality
93. Diffusion of treatment
94. Compensatory rivalry
95. Experimenter expectancy
96. The research participants in the control group may get the information on the treatment that the participants in the experimental group receive, which can contaminate the study outcome. This threat to internal validity refers to
97. mortality.
98. diffusion of treatment.
99. compensatory rivalry.
100. experimenter expectancy.
101. __________ occurs when the research participants in the control group learn of special treatment given to the experimental group and alter their behavior.
102. Mortality
103. Diffusion of treatment
104. Compensatory rivalry
105. Experimenter expectancy
106. Researchers can subconsciously and unintentionally make their research participants aware of the research hypotheses, which ultimately influences their behavior. This threat to internal validity refers to
107. compensatory rivalry.
108. experimenter expectancy.
109. halo effect.
110. Hawthorne effect.
111. Which of the following is true regarding external validity?
112. External validity means the ability to generalize the current findings across types of persons, settings, and times.
113. Reactivity is a threat to external validity known as Hawthorne effect.
114. Another form of reactivity is the placebo effect.
115. All of the above
116. The Graduate Record Examination (GRE) is argued to help predict students’ potential for success in graduate programs. Similarly, the LSAT is used to predict law school applicants’ potential for success in legal studies. GRE and LSAT are thought to have strong __________ validity.
117. internal
118. external
119. criterion
120. construct
121. One’s hair color is not a predictor of future athletic success because it lacks __________ validity.
122. external
123. criterion
124. construct
125. content

FILL-IN-THE-BLANK

1. __________ is the ability to measure what one is intending to measure.
2. __________ is the ability to consistently produce the same results over time.
3. __________ is a collection of questions to measure a research concept.
4. __________ is a statistical procedure to measure scale reliability. It produces a numerical value indicating how good the scale is actually measuring what it intended.
5. __________ occurs when two identical measures of the topic are taken on two separate occasions and the results are the same or when two separate researchers administer a test and obtain the same results.
6. __________ is measured when research participants are given slightly different questions of the same concept.
7. __________ addresses if a relationship between two variables is causal.
8. __________ is the attempt to statistically account for all of the variables that could influence the variable of interest.
9. __________ is the ability of a specific study outcome to be able to generalize to a larger population or setting.
10. __________ in a research study is the external events that occur with potential to influence the findings.
11. __________ is any changes in the respondent due to maturity, socialization, age, or education, which can influence the findings.
12. __________ occurs when respondents become so familiar with the survey instrument that it influences the findings.
13. __________ is a threat to the quality of a study when changes are made to the testing instrument.
14. __________ is a threat to research when a respondent’s scores move closer to mean values overtime.
15. __________ are the extreme values that jeopardize a normal distribution of data.
16. __________ is a threat to internal validity that occurs in sampling process when random assignment to research groups is not properly conducted.
17. __________ is a threat to internal validity when original participants in a study drop out, die, or are otherwise excluded from the research project before the study concludes.
18. __________ is a threat to internal validity when research participants learn too much information about the study.
19. __________ is a threat to internal validity when control group participants learn of potential benefits of the experimental group’s treatment and then alter their own behavior based o the information.
20. __________ is a threat when a researcher’s judgment, opinions, or own personal bias about participants influence the study outcome.
21. __________ is a label given when respondents acknowledge that they are being studied and somehow alter their normal behavior or opinion.
22. __________ is when a respondent is given a “make believe” treatment but believes the effects are real.
23. __________ exclude participants and researchers from knowing which groups of respondents belong to the experimental group.
24. __________ utilizes a well-established measure to compare to an alternative measure that together provides more predictability.
25. __________ is when multiple measures of the constructs of a theory are related to one another in the expected direction.