Ch11 Research Validity All Methods Test Questions & Answers - Educational Research 6e Answer Key + Test Bank by Robert Burke Johnson. DOCX document preview.
Chapter 11: Validity of Research Results in Quantitative, Qualitative, and Mixed Research
Learning Objectives
- Explain the meaning of confounding variables.
- Explain the meaning of statistical conclusion validity, construct validity, internal validity, and external validity and their importance in the research process.
- Identify and explain the types of evidence that are needed to reach a causal conclusion.
- Explain the threats to internal validity and be able to identify when they might exist in a research study.
- Explain the threats to external validity and when they might exist in a research study.
- Explain the role of operationalization of constructs in research.
- Identify and explain the types of validity or trustworthiness used in qualitative research.
- Identify and explain the types of validity or legitimation used in mixed research.
Multiple Choice
1. An extraneous variable is:
a. Any variable other than the independent variable that might influence the dependent variable.
b. A variable that was left out of the study because it was irrelevant.
c. A dependent variable that is redundant with the other dependent variables.
d. Any variable other than the dependent variable that might influence the independent variable.
Learning Objective: 1
Cognitive Domain: Knowledge
Answer Location: Validity Issues in the Design of Quantitative Research
Difficulty Level: Easy
2. When an extraneous variable systematically varies with the independent variable and influences the dependent variable, it is called:
a. Another dependent variable
b. A confounding variable
c. A moderating variable
d. An unreliable variable
Learning Objective: 1
Cognitive Domain: Knowledge
Answer Location: Validity Issues in the Design of Quantitative Research
Difficulty Level:
3. A researcher conducts an experiment on memory strategies where the research assistant unknowingly runs control participants only in the afternoon and the experimental group only in the morning. Time of day in this case very well might be a(n):
a. Independent variable
b. Dependent variable
c. Mediating variable
d. Confounding variable
Learning Objective: 1
Cognitive Domain: Application
Answer Location: Validity Issues in the Design of Quantitative Research
Difficulty Level: Medium
4. When the researcher can draw clear causal conclusions from what went on in a study, this is called:
a. Internal validity
b. Internal reliability
c. Internal consistency reliability
d. External validity
Learning Objective: 2
Cognitive Domain: Comprehension
Answer Location: Internal Validity (i.e., Causal Validity)
Difficulty Level: Easy
5. To infer causality, three conditions must be met. One of them is that:
a. X and Y occur at the same time.
b. X is not related to Y.
c. Confounding variables can be ruled out as causes.
d. Manipulating one variable does not influence another variable.
Learning Objective: 3
Cognitive Domain: Comprehension
Answer Location: Criteria for Inferring Causation
Difficulty Level: Medium
6. Which of the following statements is true?
a. A statistical relationship is sufficient evidence to infer causality.
b. Temporal order of the cause and effect is not important in inferring causality.
c. A statistical relation of X and Y is insufficient evidence for inferring causality.
d. Temporal order of the cause and effect variables and a statistical relationship are all that is needed to infer causality.
Learning Objective: 3
Cognitive Domain: Analysis
Answer Location: Criteria for Inferring Causation
Difficulty Level: Hard
7. When an unplanned event occurs during the course of the study that might influence the dependent measure, this is a called:
a. History effect
b. Maturation effect
c. Testing effect
d. Instrumentation effect
Learning Objective: 4
Cognitive Domain: Comprehension
Answer Location: History (threat to one-group designs)
Difficulty Level: Medium
8. A state passes a law requiring mandatory seatbelt use and claims it has reduced traffic fatalities. At the same time that the seatbelt law was changed, speed limits were also reduced from 70 MPH to 55 MPH by the federal government. Traffic fatalities decreased during the first year of the seatbelt law. The change in the speed limits represents what kind of threat to internal validity?
a. Maturation effect
b. Instrumentation effect
c. Testing effect
d. History effect
Learning Objective: 4
Cognitive Domain: Application
Answer Location: History (threat to one-group designs)
Difficulty Level: Medium
9. When physical or mental changes occur in individuals, that influences their scores on the dependent variable(s). This is called:
a. History effect
b. Maturation effect
c. Testing effect
d. Instrumentation effect
Learning Objective: 4
Cognitive Domain: Comprehension
Answer Location: Maturation (threat to one-group designs)
Difficulty Level: Medium
10. A researcher does a one-group pretest-posttest study where she examines the effects of a reading program on children’s reading comprehension level. She finds that the students’ scores at the end of the school year are higher than at the beginning of the school year. She attributes the changes to the reading program. Her colleague argues you would expect that the children would improve anyway from the beginning of the year to the end of the year. Her colleague is arguing that the results are potentially due to:
a. History effect
b. Multiple-treatment interference
c. Testing effect
d. Maturation effect
Learning Objective: 4
Cognitive Domain: Application
Answer Location: Maturation (threat to one-group designs)
Difficulty Level: Medium
11. When scores improve simply due to having taken a pretest, this threat to validity is called:
a. History effect
b. Multiple-treatment interference
c. Testing effect
d. Maturation effect
Learning Objective: 4
Cognitive Domain: Comprehension
Answer Location: Testing (threat to one-group designs)
Difficulty Level: Medium
12. A researcher does a study of a mathematics program in a one-group pretest-posttest design. She gives the group of second graders a multiple-choice pretest. Unknown to her, the children had never taken a multiple-choice test. The children scored higher on the same test given at posttest. She argues that the children improved because of the mathematics program. That the children had never taken a multiple-choice test before and showed improvement at posttest suggests that one possible threat to her conclusion is a:
a. History effect
b. Multiple-treatment interference
c. Testing effect
d. Selection effect
Learning Objective: 4
Cognitive Domain: Application
Answer Location: Testing (threat to one-group designs)
Difficulty Level: Medium
13. If the way a variable is measured changes during the course of a study, this threat to validity is called:
a. Testing effect
b. Instrumentation effect
c. Maturation effect
d. Multiple-treatment interference
Learning Objective: 4
Cognitive Domain: Comprehension
Answer Location: Instrumentation (threat to one-group designs)
Difficulty Level: Easy
14. A researcher is carrying out a study of the effects of teaching different approaches to argumentation on writing persuasive essays. He pretests the participants by having them write three persuasive essays prior to the study, and three essays after the study. He has two individuals grade all the essays. The two individuals were given no training prior to the study. The essays from the students in the study actually looked worse rather than better after the study. Based on the information presented above, what threat to internal validity could explain the contradictory finding?
a. Selection
b. Maturation effect
c. Instrumentation
d. History effect
Learning Objective: 4
Cognitive Domain: Analysis
Answer Location: Instrumentation (threat to one-group designs)
Difficulty Level: Hard
15. The tendency of scores that are well below the mean to move up toward the mean on retesting, and the tendency of scores that are well above the mean to move down toward the mean on retesting is called:
a. Selection effect
b. Maturation effect
c. Instrumentation effect
d. Regression artifact effect
Learning Objective: 4
Cognitive Domain: Comprehension
Answer Location: Regression artifacts (threat to one-group designs)
Difficulty Level: Medium
16. A school district examines a program that uses mentors to help very poor readers improve their reading performance. The children in the program are at the 4th percentile at pretest. At posttest they are around the 20th percentile. While it is possible that the program made the difference, another reason for the change in scores could be:
a. History
b. Regression artifact
c. Multiple-treatment interference
d. Differential selection
Learning Objective: 4
Cognitive Domain: Application
Answer Location: Regression Artifacts (threat to one-group designs)
Difficulty Level: Medium
17. When the participants in the different treatment conditions have different characteristics that vary along with the independent variable, this is called:
a. Maturation
b. Regression artifacts
c. History
d. Differential selection
Learning Objective: 4
Cognitive Domain: Comprehension
Answer Location: Differential Selection (threat to multi-group designs)
Difficulty Level: Medium
18. When the different threats to internal validity act differently in different comparison groups, this is known as:
a. Additive and/or interactive effects
b. A testing effect
c. A maturation effect
d. A regression artifacts effect
Learning Objective: 4
Cognitive Domain: Comprehension
Answer Location: Additive and Interactive Effects (threats to multi-group designs)
Difficulty Level: Easy
19. A group of researchers conduct a study where children from particular classrooms are assigned to treatment or control conditions. After the study, the researcher finds out that the students in the control group are higher achievers than those in the experimental group. He found no treatment effect. The failure to find a treatment effect might be due to:
a. A treatment effect
b. A testing effect
c. A differential selection effect
d. A maturation effect
Learning Objective: 4
Cognitive Domain: Application
Answer Location: Differential Selection (threat to multi-group designs)
Difficulty Level: medium
20. A researcher examines a program looking at the effects of mentoring on poor readers’ reading achievement. He looks at two different schools. One serves as the control and the other the experimental group. Both schools had reading achievement that was around the 50th percentile. During the time that the mentoring program is in place in the experimental group, a statewide reading initiative is started in randomly selected schools. The experimental, but not the control school is involved in the statewide reading initiative. At the end of the year, the experimental group does better than the control. From the information presented above, a likely threat to the internal validity of the study is:
a. Selection by mortality interaction
b. Mortality
c. Selection-history effect
d. Selection-maturation effect
Learning Objective: 4
Cognitive Domain: Analysis
Answer Location: Additive and Interactive Effects (threats to multi-group designs)
Difficulty Level: Hard
21. When there is differential maturation in two groups, this is also called:
a. Mortality effect
b. Selection-history effect
c. Selection-regression artifacts effect
d. Selection-maturation effect
Learning Objective: 4
Cognitive Domain: Comprehension
Answer Location: Additive and Interactive Effects (threats to multi-group designs)
Difficulty Level: Medium
22. When there is differential drop out between groups in a study, this problem is called:
a. Maturation effect
b. Differential attrition
c. History effect
d. Testing effect
Learning Objective: 4
Cognitive Domain: Comprehension
Answer Location: Additive and Interactive Effects (threats to multi-group designs)
Difficulty Level: Easy
23. A personnel committee is evaluating professors. Professor X has the highest student ratings in the department. One of the committee members suggests that Professor X should receive the highest merit raise in the department. Another committee member raises an objection. She states, “Professor X scores are high, compared to other professors, because over half of the students drop out of Professor X’s courses, not because Professor X is an excellent professor.” What threat to the validity of the conclusion that Professor X is an excellent professor is the committee member describing?
a. Maturation effect
b. History effect
c. Testing effect
d. Differential attrition effect
Learning Objective: 4
Cognitive Domain: Analysis
Answer Location: Additive and Interactive Effects (threats to multi-group designs)
Difficulty Level: Medium
24. The ability to generalize the results of a study to individuals not in the study is called:
a. Internal validity
b. Temporal validity
c. Ecological validity
d. Population validity
Learning Objective: 5
Cognitive Domain: Comprehension
Answer Location: Population Validity
Difficulty Level: Easy
25. The ability to generalize the results of a study to a population is greatest when:
a. Random assignment is used
b. Random selection is used
c. Convenience sampling is used
d. Purposive sampling is used
Learning Objective: 5
Cognitive Domain: Comprehension
Answer Location: Population Validity
Difficulty Level: Medium
26. It is sometimes forgotten that the ability to generalize to the target population requires that:
a. The characteristics of the accessible population match the target population
b. The accessible population be 100 or greater
c. Random assignment took place
d. The study sample be a convenience sample
Learning Objective: 5
Cognitive Domain: Comprehension
Answer Location: Population Validity
Difficulty Level: Medium
27. When the target population has several subpopulations within, it is possible that:
a. The overall results might not hold for subpopulations
b. The overall results will always hold for all subpopulations
c. Generalizing across the subpopulations is easily accomplished
d. The overall results will apply to all subpopulations
Learning Objective: 5
Cognitive Domain: Analysis
Answer Location: Population Validity
Difficulty Level: Hard
28. The ability to generalize results across settings is called:
a. Multiple treatment interference
b. Population validity
c. Ecological validity
d. Internal validity
Learning Objective: 5
Cognitive Domain: Comprehension
Answer Location: Ecological validity
Difficulty Level: Easy
29. The ability to generalize the results across variations of the treatment is called:
a. Temporal validity
b. Outcome validity
c. Treatment variation validity
d. Ecological validity
Learning Objective: 5
Cognitive Domain: Comprehension
Answer Location: Treatment Variation Validity
Difficulty Level: Easy
30. The ability to generalize the results across different but related dependent variables is called:
a. Temporal validity
b. Outcome validity
c. Treatment variation validity
d. Ecological validity
Learning Objective: 5
Cognitive Domain: Comprehension
Answer Location: Outcome validity
Difficulty Level: Easy
31. When the researcher is unable to provide clear evidence that the independent variable preceded the effect, we have the threat to internal validity known as:
a. Ambiguous temporal precedence
b. Proper time order
c. Improper causal precedence
d. Linear temporal precedence
Learning Objective: 3
Cognitive Domain: Application
Answer Location: Criteria for Inferring Causation
Difficulty Level: Medium
32. Reactivity can influence:
a. The ecological validity of a study
b. The internal validity of a study if it varies across comparison groups
c. Both ecological and internal validity of a study
d. Neither ecological nor internal validity
Learning Objective: 5
Cognitive Domain: Comprehension
Answer Location: Ecological Validity
Difficulty Level: Medium
33. An alteration of performance that occurs when an individual is aware that he or she is participating in a study is called:
a. Multiple treatment interference
b. Reactivity
c. Camera effect
d. Experimenter effects
Learning Objective: 5
Cognitive Domain: Comprehension
Answer Location: Ecological Validity
Difficulty Level: Medium
34. A researcher places a video camera in a classroom to tape lessons that are part of one of the treatment conditions. The video camera is likely to increase the probability of:
a. Multiple treatment interference
b. Selection effects
c. Reactivity
d. Maturation
Learning Objective: 5
Cognitive Domain: Application
Answer Location: Ecological Validity
Difficulty Level: Medium
35. The extent to which a study can be generalized across time is called:
a. Incremental validity
b. Ecological validity
c. Population validity
d. Temporal validity
Learning Objective: 5
Cognitive Domain: Comprehension
Answer Location: Temporal Validity
Difficulty Level: Easy
36. Roger is concerned with generalizing the results of his study to and across different groups of people, settings, times, and treatments. His concerns revolve around:
a. Internal validity
b. External validity
c. Causation
d. Temporal precedence
Learning Objective: 5
Cognitive Domain: Analysis
Answer Location: External Validity (i.e., Generalizing Validity)
Difficulty Level: Medium
37. To most accurately measure a construct, it is recommended that:
a. Researchers use only one measure
b. Researchers use effective items to measure the construct
c. Researchers not study constructs over time
d. Researchers avoid multiple operationalism
Learning Objective: 6
Cognitive Domain: Comprehension
Answer Location: Construct validity
Difficulty Level: Easy
38. A major problem for research validity in qualitative research is:
a. The researcher does not include quantitative measures
b. There is no way to test the internal and external validity of qualitative studies
c. There is no way to validate descriptions of settings
d. Researcher bias can go unrecognized and can invalidate conclusions
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: Research Validity (or Trustworthiness) in Qualitative Research
Difficulty Level: Easy
39. Reflexivity in qualitative research refers to:
a. Using multiple sources of data
b. Reflection after data collection about hypotheses
c. Self-reflection by the researcher on his or her biases and predispositions
d. Only self-reflection by the researcher about what literature to include in the literature review
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: Research Validity (or Trustworthiness) in Qualitative Research
Difficulty Level: Medium
AACSB Standard:
40. One way that a qualitative researcher can help reduce researcher bias is to use:
a. Negative case sampling
b. Random assignment to groups
c. Statistical regression
d. Snowball sampling
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: Research Validity (or Trustworthiness) in Qualitative Research
Difficulty Level: Easy
41. In qualitative research, descriptive validity refers to:
a. The mean, standard deviation, and the range
b. The validity of theory that the researcher uses
c. The validity of tests used in the study
d. Factual accuracy of accounts reported by researcher
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: Descriptive Validity
Difficulty Level: Easy
42. One common way to ensure descriptive validity in qualitative research is to:
a. Use multiple theoretical perspectives
b. Compute means and standard deviations
c. Use multiple investigators
d. Using pattern matching
Learning Objective: 7
Cognitive Domain: Applications
Answer Location: Descriptive Validity
Difficulty Level: Medium
43. Interpretive validity relates to:
a. Accuracy of researcher in portraying the research participants’ thoughts and meanings
b. Factual accuracy of the account reported by a researcher
c. Validity of the theory applied to the data
d. Degree to which the results can be generalized to other populations
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: Interpretive or Participant Validity
Difficulty Level: Medium
44. In his study, Patrick obtained feedback from his participants and used many of their descriptive words in writing up the study for publication. These strategies provide evidence of:
a. Interpretive validity
b. Descriptive validity
c. Reflexive validity
d. Theoretical validity
Learning Objective: 7
Cognitive Domain: Analysis
Answer Location: Interpretive or Participant Validity
Difficulty Level: Hard
AACSB Standard:
45. Anna is writing up her qualitative research study. She took in depth notes on what took place with the participants as well as what she saw and heard when working with the participants. She did this to improve the study’s:
a. Interpretive validity
b. Participant validity
c. Theoretical validity
d. Descriptive validity
Learning Objective: 7
Cognitive Domain: Analysis
Answer Location: Descriptive Validity
Difficulty Level: Medium
46. Sarah is conducting an ethnographic study of fifth grade classes using an interdisciplinary curriculum. After she collects her data, she goes back and asks each teacher to help her out by checking the accuracy of the descriptions she has written of the classrooms. This technique is called:
a. Participant observation
b. Participant feedback
c. Participant validity
d. Theoretical validity
Learning Objective: 7
Cognitive Domain: Application
Answer Location: Interpretive or Participant Validity
Difficulty Level: Medium
47. Theoretical validity refers to:
a. Accuracy and defensibility of the theoretical explanations given
b. Accuracy of the descriptions of the data
c. Accuracy of the researcher’s description of the meanings ascribed to participants
d. Accuracy of the low inference descriptors used
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: Theoretical Validity
Difficulty Level: Easy
48. To increase the theoretical validity of his study, a qualitative researcher should use:
a. Low inference descriptors
b. Peer review
c. Data from observations only
d. Data from interviews only
Learning Objective: 7
Cognitive Domain: Application
Answer Location: Theoretical Validity
Difficulty Level: Medium
AACSB Standard:
49. Pattern matching refers to:
a. Looking at the match between the actual data and a theoretically predicted set of results
b. Looking at the match between the descriptions of researchers and the descriptions of participants in a qualitative study
c. The researcher matching what participants do by imitating it
d. The researcher finding the match for his or her results in another study
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: Theoretical Validity
Difficulty Level: Medium
50. One way to promote theoretical validity in qualitative research is to:
a. Use low inference descriptors
b. Avoid participant observation
c. Use extended fieldwork
d. Use data from interviews only
Learning Objective: 7
Cognitive Domain: Application
Answer Location: Theoretical Validity
Difficulty Level: Easy
51. Richard has made predictions about the data he will collect for his research study. After his data are collected he compares them to his predictions. He does this to assess which type of validity?
a. External validity
b. Internal validity
c. Population validity
d. Theoretical validity
Learning Objective: 7
Cognitive Domain: Analysis
Answer Location: Theoretical validity
Difficulty Level: Hard
52. Multiple theoretical perspectives refer to:
a. Using multiple investigators to determine the accuracy of a researcher’s account of an event
b. Using multiple data sources to confirm a theory
c. Using multiple theories or perspectives to interpret and explain the data
d. Using extended fieldwork to test whether one theory fits the data
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: Theoretical Validity
Difficulty Level: Medium
53. Internal validity in qualitative research refers to:
a. The validity of cause and effect conclusions
b. The validity of the research results for other settings
c. The validity of the research results for other populations
d. The validity of the theories that guided the research
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: Internal Validity (in Research Validity (or “Trustworthiness” in Qualitative Research section)
Difficulty Level: Medium
54. William has conducted a qualitative study. He has internally validated his findings by ruling out rival explanations for the results. This is referred to as the:
a. Researcher-as-laborer metaphor
b. Researcher-as-teacher metaphor
c. Researcher-as-legislator metaphor
d. Researcher-as-detective metaphor
Learning Objective: 7
Cognitive Domain: Application
Answer Location: Internal Validity (in Research Validity (or “Trustworthiness” in Qualitative Research Section), Table 11.2
Difficulty Level: Medium
55. Mary wanted to demonstrate that her qualitative research results held in other settings and with other populations. In order to do this she should:
a. Use low inference descriptors
b. Use multiple sources of data
c. Use multiple theoretical perspectives
d. Use naturalistic generalization
Learning Objective: 7
Cognitive Domain: Application
Answer Location: External Validity (in Research Validity (or “Trustworthiness” in Qualitative Research section)
Difficulty Level: Medium
56. Qualitative researchers can determine whether their results would hold in other settings and with other populations by using:
a. Replication logic
b. Low inference descriptors
c. Data multiple sources of data
d. Multiple theoretical perspectives
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: External Validity (in Research Validity (or “Trustworthiness” in Qualitative Research Section)
Difficulty Level: Easy
57. Dr. Martin’s research team was able to generalize the results of their study because they saw the same results in several other studies. These researchers were using:
a. Theoretical transfer
b. Replication logic
c. Low inference descriptors
d. Participant feedback
Learning Objective: 7
Cognitive Domain: Analysis
Answer Location: External Validity (in Research Validity (or “Trustworthiness” in Qualitative Research Section)
Difficulty Level: Medium
58. External validity in qualitative research:
a. Is always irrelevant in qualitative research
b. Is confirmed through random sampling in qualitative research
c. Is confirmed through random assignment in qualitative research
d. Can be obtained by replication in qualitative research, but it is often of little interest to the qualitative researcher
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: External Validity (in Research Validity (or “Trustworthiness” in Qualitative Research Section)
Difficulty Level: Medium
59. Dr. Johnson conducted a research study in which he investigated the effect of three different methods of teaching mathematics to three different classes of fifth grade students. However, one of the classes had smarter students than the other two and as a result this class demonstrated the most improvement in their mathematics scores. In spite of this problem, he concluded that the method used to teach mathematics to this class of students was the most effective. In this experiment, the intellectual ability of the students in the various classes was:
a. A confounding extraneous variable
b. An extraneous variable that did not confound the results
c. An independent variable
d. A dependent variable
Learning Objective: 1
Cognitive Domain: Analysis
Answer Location: Validity Issues in the Design of Quantitative Research
Difficulty Level: Hard
60. A qualitative researcher is often concerned with identifying the immediate, intentional, particular, complex, and local causes of attitudes, actions, and events. This is known as:
a. Nomothetic causation
b. Idiographic causation
c. Replication causation
d. Naturalistic causation
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: External Validity (in Research Validity (or “Trustworthiness” in Qualitative Research Section)
Difficulty Level: Medium
61. When we talk about inferring that the independent and dependent variables are related in the population of interest and the strength of the relationship is accurate, we are talking about:
a. Statistical conclusion validity
b. Internal validity
c. Construct validity
d. External validity
Learning Objective: 2
Cognitive Domain: Comprehension
Answer Location: Statistical Conclusion Validity
Difficulty Level: Medium
62. If a research study revealed that the independent and dependent variables covary and they really do covary, the study has:
a. Been demonstrated to be worthwhile
b. Revealed a causal relationship
c. Statistical conclusion validity
d. Revealed a spurious relationship
Learning Objective: 2
Cognitive Domain: Application
Answer Location: Statistical Conclusion Validity
Difficulty Level: Medium
63. Which of the following does not represent a threat to internal validity?
a. Random assignment
b. History
c. Differential attrition
d. Regression artifact
Learning Objective: 4
Cognitive Domain: Comprehension
Answer Location: Threats to Internal Validity in Single-Group Designs, Additive and Interactive Effects (threats to multi-group designs)
Difficulty Level: Medium
64. Dr. Kenji wants to determine whether his new phonics program helps children’s reading comprehension. He gives a fifth grade class the program and also studies a second grade class that does not go through the program. He finds that the fifth graders read with better comprehension. The confounding variable in this research design is:
a. The grade the students are in
b. The phonics program
c. The reading comprehension scores
d. Not giving one group the program
Learning Objective: 1
Cognitive Domain: Analysis
Answer Location: Validity Issues in the Design of Quantitative Research
Difficulty Level: Hard
65. The confounding effects referred to collectively as “maturation” include all of the following plausible rival hypotheses except:
a. Aging
b. Instruction
c. Fatigue
d. Boredom
Learning Objective: 4
Cognitive Domain: Application
Answer Location: Maturation (Threat to one-group designs)
Difficulty Level: Hard
66. Dr. Faust investigated the overall effectiveness of an experimental program to teach reading to children from cultures other than the U.S.. He found that the experimental program was much more effective than the standard reading program typically administered to these children. In this study, Dr. Faust investigated:
a. Causal explanation
b. Ambiguous temporal precedence
c. Statistical conclusion validity
d. Causal description
Learning Objective: 3
Cognitive Domain: Analysis
Answer Location: Two Major Types of Causal Relationships
Difficulty Level: Hard
67. Dr. Sanjay is studying whether his racism awareness program increases his research participants’ awareness of issues of race and ethnicity in his city. On Monday, he gives an attitude survey to his participants to gauge their awareness. On Tuesday through Thursday, he puts them through his program. On Wednesday, a local predominantly black church is firebombed and the KKK claims responsibility. On Friday, Dr. Sanjay gives his awareness survey again and finds awareness of racial issues has increased dramatically. This example illustrates the possibility of ______________ as a plausible rival explanation.
a. History
b. Maturation
c. Regression artifact
d. Differential selection
Learning Objective: 4
Cognitive Domain: Application
Answer Location: History (threat to one-group designs)
Difficulty Level: Medium
68. Fred took the SAT to meet an entrance requirement for a college. On his first attempt he scored far below average. He decided to study for the test and take it again. This time he scored higher and closer to the average of all people taking the test. This example illustrates the possibility of _______________ as a rival plausible explanation.
a. History
b. Maturation
c. Regression artifact
d. Differential selection
Learning Objective: 4
Cognitive Domain: Application
Answer Location: Regression Artifacts (threat to one-group designs)
Difficulty Level: Medium
69. A school counselor was concerned with the rising level of violence at the school in which she was employed so she decided to try an experimental program to reduce the level of violence. She identified two groups of violent children at her school and randomly assigned them to two groups. Both groups of children are taught effective methods of dealing with other children and with frustration, but one group is reinforced every time they use any of these techniques and isolated when they are violent. The other group is isolated and punished in the form of having to do additional duties such as cleaning the bathrooms whenever they act violently. Some of the participants in both groups fail to complete the study but most of the ones that drop out are in the punishment group. The study found that both groups reduced their violent behavior by equal amounts. Which is the best conclusion to draw from this study?
a. There is no difference in the effectiveness of reinforcement versus punishment
b. The study is confounded by differential attrition and this might explain the failure to find any group differences
c. The study is confounded by a history bias and this might explain the failure to find any group differences
d. Reinforcement would have been demonstrated to be more effective if the study had lasted longer
Learning Objective: 1, 4
Cognitive Domain: Analysis
Answer Location: Validity Issues in the Design of Quantitative Research, Additive and Interactive Effects (threats to multi-group designs)
Difficulty Level: Hard
70. If the results of your study can be generalized to other people, in other settings, using different treatment variations and outcome measures, and the results have held up over time you have the ultimate in:
a. Statistical conclusion validity
b. Internal validity
c. External validity
d. Construct validity
Learning Objective: 5
Cognitive Domain: Comprehension
Answer Location: External Validity (i.e., Generalizing Validity)
Difficulty Level: Medium
71. The extent to which a researcher understands, uses, and presents participants’ but maintains her objective outsider view is known as:
a. Paradigmatic legitimation.
b. Inside-outside legitimation.
c. Philosophical legitimation.
d. Sequential legitimation.
Learning Objective: 8
Cognitive Domain: Comprehension
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
72. Mr. Jones has demonstrated that his experimental reading program is very effective with 4th and 5th grade children with a learning disability. Ms. Thornton is using his experimental reading program but does not use it exactly like the instructions specify. Although Ms. Thornton has varied the way in which she is implementing the reading program she also finds it to be very effective with her students with learning disabilities. This is a demonstration that the reading program has:
a. Population validity
b. Temporal validity
c. Ecological validity
d. Treatment variation validity
Learning Objective: 5
Cognitive Domain: Application
Answer Location: Treatment Variation Validity
Difficulty Level: Medium
73. Dr. Smith conducted a mixed methods study on delay of gratification. In this particular study, delay of gratification was measured by the amount of time that would elapse between when a person saw a desired food and when he or she would eat it. In addition, participants were interviewed about their reactions to the delay of gratification they experienced in this study. What type of legitimation did Dr. Smith use?
a. Inside-outside legitimation.
b. Commensurability approximation legitimation.
c. Sequential legitimation.
d. Weakness minimization legitimation.
Learning Objective: 8
Cognitive Domain: Analysis
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
74. The results of Dr. Jarvis’ study were replicated by three other teams of researchers. As a result Dr. Jarvis’ study has:
a. Important results
b. Valid results
c. Research reliability
d. Research validity
Learning Objective: 2
Cognitive Domain: Analysis
Answer Location: Validity Issues in the Design of Quantitative Research
Difficulty Level: Medium
75. If the inference that is made from the results of a study are correct, then the study has:
a. Important results
b. Valid results
c. Research reliability
d. Research validity
Learning Objective: 2
Cognitive Domain: Comprehension
Answer Location: Validity Issues in the Design of Quantitative Research
Difficulty Level: Medium
76. A meta-inference is an inference:
a. From philosophy
b. From a qualitative study
c. Using both qualitative and quantitative findings
d. That has relevance to the total population
Learning Objective: 8
Cognitive Domain: Comprehension
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
77. Validity that makes use of both the emic and etic viewpoints is called ______.
a. inside-outside validity
b. paradigmatic/philosophical validity
c. commensurability approximation validity
d. weakness minimization validity
Learning Objective: 8
Cognitive Domain: Comprehension
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
78. If a researcher understands and explains his or her epistemological, ontological, axiological, methodological, and rhetorical beliefs, the study has:
a. Inside-outside validity
b. Paradigmatic/philosophical validity
c. Commensurability approximation validity
d. Weakness minimization validity
Learning Objective: 8
Cognitive Domain: Comprehension
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
79. Dr. Pitts has successfully integrated her qualitative and quantitative data, analysis, and conclusions in her study. As a result the study has:
a. Pragmatic legitimation
b. Integration legitimation
c. Multiple validities
d. Sociopolitical legitimation
Learning Objective: 8
Cognitive Domain: Application
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
80. If a researcher has addressed the interests, values, and standpoints of many stakeholders, the researcher has achieved:
a. Inside-outside legitimation
b. Sociopolitical legitimation
c. Commensurability approximation legitimation
d. Weakness minimization legitimation
Learning Objective: 8
Cognitive Domain: Comprehension
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
81. If a researcher is able to integrate the two views of a quantitative researcher and qualitative researcher into a broader viewpoint, he/she has achieved:
a. Inside-outside legitimation
b. Paradigmatic/philosophical legitimation
c. Commensurability approximation legitimation
d. Weakness minimization legitimation
Learning Objective: 8
Cognitive Domain: Comprehension
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
82. In a mixed methods study, if the quantitative part of the study appropriately followed from the previous phase of qualitative data collection, the study has achieved:
a. Sequential legitimation
b. Conversion legitimation
c. Sample integration legitimation
d. Socio-political legitimation
Learning Objective: 8
Cognitive Domain: Application
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
83. If a researcher has successfully quantitized qualitative data and qualitized quantitative data, he/she has achieved:
a. Sequential legitimation
b. Conversion legitimation
c. Sample integration legitimation
d. Socio-political legitimation
Learning Objective: 8
Cognitive Domain: Comprehension
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
84. If the qualitative and quantitative parts of a mixed study have used different sets and numbers of research participants and you have made appropriate generalizations when you combine the data from these sets of research participants, you have achieved:
a. Sequential legitimation
b. Conversion legitimation
c. Sample integration legitimation
d. Socio-political legitimation
Learning Objective: 8
Cognitive Domain: Application
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
85. If you have addressed the interests, values, and viewpoints of multiple stakeholders in your mixed research study, then you have attained:
a. Sequential legitimation
b. Conversion legitimation
c. Sample integration legitimation
d. Socio-political legitimation
Learning Objective: 8
Cognitive Domain: Application
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
86. If you have successfully addressed all the various validity issues pertaining to your mixed research study then you have achieved:
a. Pragmatic Legitimation
b. Multiple validities
c. Integration legitimation
d. Conversion legitimation
Learning Objective: 8
Cognitive Domain: Comprehension
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
87. A researcher tested a group of students first time in second grade and the second time in the middle of the third grade. The second test indicated significant improvement, but after reviewing the test scores the researcher became concerned that he could not be sure whether the scores were a result of the treatment or a result of the students’ having gained more life experience in a year-and-a-half. What threat to internal validity is this researcher worried about?
a. Maturation
b. Testing
c. Differential selection
d. Instrumentation
Learning Objective: 4
Cognitive Domain: Analysis
Answer Location: Maturation
Difficulty Level: Hard
88. The basic problems of history, maturation, instrumentation, and testing can all be eliminated by altering the study in what way?
a. Making the study qualitative
b. Adding a control group
c. Using a pretest and posttest
d. Checking for regression artifacts
Learning Objective: 4
Cognitive Domain: Analysis
Answer Location: Threats to Internal Validity (in Single-Group Designs)
Difficulty Level: Medium
89. If the results of a study can be generalized from the sample of research participants to individuals who were not included in the study, the study is said to have what?
a. Population validity
b. External reliability
c. Representative validity
d. Internal validity
Learning Objective: 4
Cognitive Domain: Comprehension
Answer Location: Population validity
Difficulty Level: Medium
90. If a study can be generalized over time, it has what?
a. Temporal validity
b. Treatment variation validity
c. Outcome validity
d. Operational validity
Learning Objective: 4
Cognitive Domain: Comprehension
Answer Location: Temporal validity
Difficulty Level: Medium
91. In his study, Joe used member checking as a strategy to ensure what type of validity?
a. Construct validity
b. Participant validity
c. Theoretical validity
d. Triangulation validity
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: Interpretive or Participant Validity
Difficulty Level: Medium
92. Dr. Halen did an extensive literature search. As he worked his way through the literature he searched for research that disconfirmed his hypotheses. This is an example of:
a. Peer review
b. Negative case sampling
c. Reflexivity
d. Theoretical validity
Learning Objective: 7
Cognitive Domain: Application
Answer Location: Research Validity (or “Trustworthiness”) in Qualitative Research
Difficulty Level: Hard
93. A critical friend is a type of peer review focused on:
a. Interpretive validity
b. Internal validity
c. Theoretical validity
d. Descriptive validity
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: Theoretical Validity
Difficulty Level: Medium
94. In Sally’s research study, a member of a control group was exposed to part of the treatment condition. This is known as:
a. Effect size
b. External validity
c. Replication
d. Treatment diffusion
Learning Objective: 2
Cognitive Domain: Analysis
Answer Location: Treatment Diffusion
Difficulty Level: Hard
AACSB Standard:
True-False
1. Internal validity refers to the degree to which a researcher can accurately claim that a causal relationship exists between two variables and that the observed strength of the relationship is accurate.
a. True
b. False
Learning Objective: 2
Cognitive Domain: Comprehension
Answer Location: Statistical Conclusion Validity
Difficulty Level: Easy
2. Internal validity refers to the degree to which a researcher can accurately claim that a causal relationship exists between two variables.
a. True
b. False
Learning Objective: 2
Cognitive Domain: Comprehension
Answer Location: Internal Validity (i.e., Causal Validity)
Difficulty Level: Easy
3. External validity refers to the degree to which the study results can be generalized to and across populations of persons, settings, times, outcomes, and treatment variations.
a. True
b. False
Learning Objective: 2
Cognitive Domain: Comprehension
Answer Location: External Validity (i.e., Generalizing Validity)
Difficulty Level: Easy
4. Ecological validity is present when a study results can be generalized to different settings.
a. True
b. False
Learning Objective: 2
Cognitive Domain: Comprehension
Answer Location: Ecological Validity
Difficulty Level: Easy
5. When a significant amount of time passes between the first and second treatment so that one or more things could be at least partially responsible for a change seen in the research participants, it could be an example of regression artifacts.
a. True
b. False
Learning Objective: 4
Cognitive Domain: Application
Answer Location: History, Regression Artifacts
Difficulty Level: Medium
6. When a significant amount of time passes between the first and second treatment so that participants may have experienced physical or mental changes that could be at least partially responsible for a change seen in the research participants, it could be an example of a history effect.
a. True
b. False
Learning Objective: 4
Cognitive Domain: Application
Answer Location: Maturation (threat to one-group designs)
Difficulty Level: Medium
7. Attrition refers to the loss of people who do not complete the study.
a. True
b. False
Learning Objective: 4
Cognitive Domain: Comprehension
Answer Location: Additive and Interactive Effects (threats to multi-group designs)
Difficulty Level: easy
8. When researchers exhibit bias they might have a tendency find exactly what they were looking for.
a. True
b. False
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: Research Validity (or “Trustworthiness”) in Qualitative Research
Difficulty Level: Easy
9. The use of multiple investigators is a way to provide evidence of descriptive validity in qualitative studies.
a. True
b. False
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: Research Validity (or “Trustworthiness”) in Qualitative Research
Difficulty Level: Easy
10. Qualitative researchers are often concerned with nomothetic causation because it helps them to understand causes in specific situations and some of these causes might be unique to the situation or context.
a. True
b. False
Learning Objective: 7
Cognitive Domain: Comprehension
Answer Location: External Validity (in Research Validity (or “Trustworthiness”) in Qualitative Research Section)
Difficulty Level: medium
11. Differential selection produces similar comparison groups.
a. True
b. False
Learning Objective: 4
Cognitive Domain: Comprehension
Answer Location: Differential Selection (threat to multi-group designs)
Difficulty Level: Easy
12. Treatment diffusion occurs when the researcher has not operationalized a concept completely.
a. True
b. False
Learning Objective: 2
Cognitive Domain: Comprehension
Answer Location: Treatment Diffusion
Difficulty Level: Medium
13. Jason’s study looked at the reactions of members of the two major political parties to the person who was elected as president. It is possible that his study had selection-maturation validity threats.
a. True
b. False
Learning Objective: 4
Cognitive Domain: Analysis
Answer Location: Additive and Interactive Effects (Threats to Multi-Group Designs)
Difficulty Level: Hard
14. Multiple validity is used to describe how well a researcher using mixed methods has tackled and resolved the applicable validity types.
a. True
b. False
Learning Objective: 8
Cognitive Domain: Comprehension
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
15. In conducting her literature review, Carol noticed that most of the articles she was finding were 25 years old and she used them to develop her study. As a result her study’s temporal validity may be affected.
a. True
b. False
Learning Objective: 5
Cognitive Domain: Analysis
Answer Location: Temporal Validity
Difficulty Level: Hard
16. In his study, Collin only included one outcome measure. This probably had a negative effect on the study’s external validity.
a. True
b. False
Learning Objective: 5
Cognitive Domain: Analysis
Answer Location: Outcome Validity
Difficulty Level: Hard
17. Multiple validity is used to describe how well a researcher using mixed methods has tackled and resolved the applicable validity types.
a. True
b. False
Learning Objective: 8
Cognitive Domain: Comprehension
Answer Location: Research Validity (or “Legitimation”) in Mixed Research
Difficulty Level: Medium
Essay Questions
1. What evidence is needed to conclude causality in research?
Learning Objective: 3
Cognitive Domain: Comprehension
Answer Location: Criteria for Inferring Causation
Difficulty Level: Medium
2. Name three threats to external validity and explain how they can be avoided.
Learning Objective: 5
Cognitive Domain: application
Answer Location: External Validity (i.e., Generalizing Validity)
Difficulty Level: hard
3. Name and define the four types of validity discussed in this chapter and describe their importance to research.
Learning Objective: 2
Cognitive Domain: Analysis, Application
Answer Location: Internal Validity (i.e., Causal Validity), External Validity (i.e., Generalizing Validity), Construct Validity, Statistical Conclusion Validity.
Difficulty Level: Hard
4. Describe three of the strategies a qualitative researcher can use to ensure the research validity of her study.
Learning Objective: 7
Cognitive Domain: Application
Answer Location: Research Validity (or "Trustworthiness) in Qualitative Research
Difficulty Level: Medium
5. Name and describe three of the types of research validity or legitimation for mixed research.
Learning Objective: 8
Cognitive Domain: Application
Answer Location: Research Validity (or “Legitimation”) in Mixed Research.
Difficulty Level: Medium
Document Information
Connected Book
Educational Research 6e Answer Key + Test Bank
By Robert Burke Johnson
Explore recommendations drawn directly from what you're reading
Chapter 9 Six Major Methods of Data Collection
DOCX Ch. 9
Chapter 10 Sampling in Quantitative, Qualitative, and Mixed Research
DOCX Ch. 10
Chapter 11 Research Validity – All Methods
DOCX Ch. 11 Current
Chapter 12 Experimental Research Weak and Strong Designs
DOCX Ch. 12
Chapter 13 Experimental Research Quasi and Single-Case Designs
DOCX Ch. 13