Test Bank

Chapter 10: Evaluation and Policy Analysis

Multiple Choice

1. Research with an impact on policy choices in the immediate future is known as ______ research.

a. statistical

b. applied

c. sampling

d. singular approach

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Knowledge

Answer Location: What Do We Have In Mind?

Difficulty Level: Easy

2. Regardless of the treatment or program being examined evaluations are systematic because they employ social research approaches to ______.

a. gathering valid and reliable data

b. finding reliable data

c. understanding why people act the way they act

d. looking at programs in a singular way

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: A Brief History of Evaluation Research

Difficulty Level: Easy

3. The process of service delivery is also known as the program ______.

a. evaluation

b. outputs

c. process

d. delivery

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Knowledge

Answer Location: Evaluation Basics

Difficulty Level: Medium

4. Variation in both outputs and outcomes in turn influences the inputs to the program through ______.

a. the recruitment process

b. an output system

c. a never-ending loop

d. a feedback process

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: Evaluation Basics

Difficulty Level: Easy

5. The evaluation process as a whole and the feedback process in particular can only be understood in relation to the interests and perspectives of ______.

a. stakeholders

b. the government

c. exploratory research

d. highest scientific standards

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: Evaluation Basics

Difficulty Level: Medium

6. The difference between evaluation research and traditional social science research is that evaluation research ______.

a. is designed to test the implications of a social theory

b. is not designed to test the implications of a social theory

c. uses only quantitative research methods

d. creates a broad theoretical explanation for what is discovered

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: Evaluation Basics

Difficulty Level: Medium

7. Stakeholders are those individuals and groups who ______.

a. are peripherally related to the program

b. are uninterested parties with relation to the program

c. have some basis of concern with the program

d. are, in reality, the researchers themselves

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Application

Answer Location: Evaluation Basics

Difficulty Level: Medium

8. After a program has begun, evaluators are often called on to determine ______.

a. if the theoretical aspects are sound.

b. what resources are being expended in the conduct of the program.

c. if there is congruence between expenditures and theoretical underpinnings.

d. a similar population might benefit from a program.

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Application

Answer Location: Is the Program Working as Planned?

Difficulty Level: Medium

9. A type of evaluation research that attempts to determine the needs of some population that might be met with a social program is a(n) ______.

a. needs assessment

b. evaluability assessment

c. process evaluation (monitoring)

d. program montage

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: Is the Program Working as Planned?

Difficulty Level: Medium

10. A type of evaluation research conducted to determine whether it is feasible to evaluate a program’s effects within the available time and resources is a(n) ______.

a. needs assessment

b. evaluability assessment

c. process evaluation (monitoring)

d. program montage

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: Is the Program Working as Planned?

Difficulty Level: Medium

11. Evaluation research that investigates the process of service delivery is a(n) ______.

a. needs assessment

b. evaluability assessment

c. process evaluation (monitoring)

d. program montage

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: Is the Program Working as Planned?

Difficulty Level: Medium

12. It was not until the end of the ______ that social research became immersed in the workings of government with the common goal of improving society.

a. 1950s

b. 1960s

c. 1970s

d. 1980s

Learning Objective: 10.2: Describe the history of evaluation research and its current status.

Cognitive Domain: Knowledge

Answer Location: A Brief History of Evaluation Research

Difficulty Level: Easy

13. A process evaluation can employ a wide range of ______.

a. variables

b. indicators

c. formats

d. modalities

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: Is the Program Working As Planned?

Difficulty Level: Medium

14. Did the treatment or program (independent variable) effect change in the dependent variable?” This part of the research is variously called ______.

a. impact evaluation or impact analysis

b. modalities

c. outcome evaluation

d. independent/dependent analysis

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: Did the Program Work?

Difficulty Level: Medium

15. D’Amico and Fromme (2002) studied the impact of the Risk Skills Training Program (RSTP) and how it compared to the Drug Abuse Resistance Education-Abbreviated (DARE-A) program, using which type of design?

a. computer assisted surveys

b. true experimental design

c. quasi-experimental design

d. snowball design

Learning Objective: 10.2: Describe the history of evaluation research and its current status.

Cognitive Domain: Comprehension

Answer Location: Case Study: The Risk Skills Training Program (RSTP) Compared With Drug Abuse Resistance Education-Abbreviated (DARE-A)

Difficulty Level: Hard

16. A type of evaluation research that compares program costs to program effects is ______.

a. cost-benefit analysis

b. efficiency analysis

c. cost-effectiveness analysis

d. an RSTP program

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: Is the Program Worth It?

Difficulty Level: Medium

17. Piza et al. (2016) conducted a ______ of a program in Newark, NJ pairing proactive CCTV monitoring with directed police patrol.

a. efficiency analysis

b. cost-effectiveness analysis

c. cost-benefit analysis

d. black box evaluation

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Application

Answer Location: Case Study: Cost-Benefit Analysis of CCTV Monitoring and Direct Police Patrol

Difficulty Level: Hard

18. The process evaluation component of the research into Project Safe Neighborhoods by McGarrell, et al., included surveys given to CAGI project coordinators to ______.

a. determine the proper population for the research

b. evaluate whether the population was correct, as chosen

c. evaluate whether the initiative was implemented as planned

d. determine the influence of CAGI as implemented

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Application

Answer Location: Case Study: Process Evaluation of an Antigang Initiative

Difficulty Level: Hard

19. A necessary component of an evaluation research project that compares program effects with costs is known as a(n) ______.

a. efficiency analysis

b. cost-benefit analysis

c. impact/efficiency plan

d. experimental design

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: Is the Program Worth It?

Difficulty Level: Medium

20. In black box evaluation, the focus of the researcher is on whether cases seem to have changed as a result of their ______.

a. entrance into some social science program

b. exit out of a social science program

c. entrance into a program to measure their outputs

d. exposure to the program between the time they entered the program as inputs and when they exited the program as outputs

Learning Objective: 10.4: Explain the concept of “black-box” evaluation and the value of opening the black box.

Cognitive Domain: Application

Answer Location: Black-Box Evaluation or Program Theory

Difficulty Level: Medium

21. If an investigation of program process is constructed, a(n) ______.

a. theory-driven outline may help define the researcher orientation

b. program theory may be developed

c. evaluation of results in the professional social science community is appropriate

d. methodology may have to be created

Learning Objective: 10.4: Explain the concept of “black-box” evaluation and the value of opening the black box.

Cognitive Domain: Comprehension

Answer Location: Black Box Evaluation or Program Theory

Difficulty Level: Hard

22. ______ approaches encourage researchers to be responsive to people who are involved with the program.

a. Social science

b. Integrative

c. Stakeholder

d. Entrepreneurial

Learning Objective: 10.4: Explain the concept of “black-box” evaluation and the value of opening the black box.

Cognitive Domain: Comprehension

Answer Location: Black-Box Evaluation or Program Theory

Difficulty Level: Medium

23. ______ approaches emphasize the importance of researcher expertise

a. Social science

b. Integrative

c. Stakeholder

d. Entrepreneurial

Learning Objective: 10.4: Explain the concept of “black-box” evaluation and the value of opening the black box.

Cognitive Domain: Comprehension

Answer Location: Black-Box Evaluation or Program Theory

Difficulty Level: Medium

24. An orientation to evaluation research that expects researchers to respond to concerns of people involved with stakeholders as well as to the standards and goals of the social scientific community are known as ______ approaches.

a. social science

b. integrated

c. stakeholder

d. entrepreneurial

Learning Objective: 10.4: Explain the concept of “black-box” evaluation and the value of opening the black box.

Cognitive Domain: Comprehension

Answer Location: Black-Box Evaluation or Program Theory

Difficulty Level: Medium

25. The clustering of violent crime at particular locations suggests that there are important features or dynamics at these locations that ______.

a. give rise to violent situations.

b. lead them to have lower criminogenic clusters

c. are increasingly identified as urbanogenic clusters

d. lead to community uprisings of placid behavior

Learning Objective: 10.6: Understand why a true experimental design is the strongest method for conducting impact evaluations.

Cognitive Domain: Application

Answer Location: Case Study: Problem-Oriented Policing in Violent Crime Areas—A Randomized Controlled Experiment

Difficulty Level: Hard

26. Data obtained from a true experimental design provide the best way to determine that ______.

a. the assessment of change in the independent variable is present

b. showing the time ordering was not necessary

c. all three criteria necessary for establishing a causal relationship has been shown

d. there is no need for random assignment because the population was not random

Learning Objective: 10.6: Understand why a true experimental design is the strongest method for conducting impact evaluations.

Cognitive Domain: Comprehension

Answer Location: Strengths of Randomized Experimental Designs in Impact Evaluations

Difficulty Level: Medium

27. The primary reason why randomization is likely not possible in evaluation research is that ______.

a. it can be difficult to obtain the cooperation of the research board

b. the program is usually outside the control of the evaluator

c. although there may be cooperation of the stakeholders there likely will not be from the other researchers

d. experimental designs are not very powerful

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Application

Answer Location: When Experiments Are Not Feasible

Difficulty Level: Hard

28. ______ designs are the most powerful alternatives to true randomized experimental designs.

a. Integrated

b. Quasi-experimental

c. Stakeholder

d. Program theory

Learning Objective: 10.6: Understand why a true experimental design is the strongest method for conducting impact evaluations.

Cognitive Domain: Application

Answer Location: When Experiments Are Not Feasible

Difficulty Level: Medium

29. Because Taylor and Woods (2010) could not randomly assign LEAs to experimental and control conditions, they used a(n) ______ design.

a. equivalent group

b. quasi-experimental

c. nonequivalent control group

d. true experimental

Learning Objective: 10.6: Understand why a true experimental design is the strongest method for conducting impact evaluations.

Cognitive Domain: Application

Answer Location: Case Study: Nonequivalent Control Group Design: Decreasing Injuries From Police Use of Force

Difficulty Level: Medium

30. The goal of policy research is to inform those who make policy about ______.

a. (only) the positive effects of the research

b. (only) the negative effects of the research

c. the possible alternative courses of action in response to some identified problem

d. taking a less rigorously design to preserve noneffectiveness

Learning Objective: 10.7: Explain the differences between applied and basic research.

Cognitive Domain: Comprehension

Answer Location: Increasing Demand for Evidence-Based Policy

Difficulty Level: Hard

31. One private organization that has become a leader in publicizing reviews related to criminal justice policy research is the ______.

a. Carter Collaboration

b. Smithson Collaborative

c. Rand Institute

d. Campbell Collaboration

Learning Objective: 10.2: Describe the history of evaluation research and its current status.

Cognitive Domain: Comprehension

Answer Location: Increasing Demand for Evidence-Based Policy

Difficulty Level: Hard

32. Educational and vocational training opportunities in prison, the availability of legal counsel, and treatment for substance abuse are all potentially important benefits, and an evaluation research project ______.

a. may impact the cost-benefit analysis of them

b. could increase the types of availability

c. can change both their type and their availability

d. may impact effectiveness

Learning Objective: 10.7: Explain the differences between applied and basic research.

Cognitive Domain: Comprehension

Answer Location: Ethics in Evaluation

Difficulty Level: Hard

33. Human subjects’ considerations multiply when the focus is ______.

a. needs assessment

b. cost-benefit analysis

c. cost-effectiveness analysis

d. program impact

Learning Objective: 10.7: Explain the differences between applied and basic research.

Cognitive Domain: Application

Answer Location: Ethics in Evaluation

Difficulty Level: Medium

34. It may not be ethical simply to bar some potential participants from the programs when ______.

a. resources are scarce

b. an ongoing entitlement program is being evaluated

c. there are less effective benefits available

d. successful programs are not quantifiable

Learning Objective: 10.7: Explain the differences between applied and basic research.

Cognitive Domain: Application

Answer Location: Ethics in Evaluation

Difficulty Level: Hard

35. By the mid-1970s, evaluators were called on not only to assess the overall effectiveness of programs but also to determine ______.

a. whether programs were being implemented as intended

b. if there were enough bureaucratic staff to follow rules & regulations

c. what other types of semi-rigorous experimentations needed to be done

d. if practitioners had made appropriate decisions about programs

Learning Objective: 10.2: Describe the history of evaluation research and its current status.

Cognitive Domain: Analysis

Answer Location: A Brief History of Evaluation Research

Difficulty Level: Medium

True/False

1. Evaluation research is a unique component of research designs.

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: A Brief History of Evaluation Research

Difficulty Level: Medium

2. The process of evaluation research is a simple systems model.

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Knowledge

Answer Location: Evaluation Basics

Difficulty Level: Easy

3. The difference between evaluation research and traditional social science research is that evaluation research is designed to test the implications of a social theory.

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: Evaluation Basics

Difficulty Level: Medium

4. The direct products of the program’s service delivery process are its outputs.

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Knowledge

Answer Location: Evaluation Basics

Difficulty Level: Easy

5. A quasi-experimental design is the preferred method for maximizing internal validity.

Learning Objective: 10.2: Describe the history of evaluation research and its current status.

Cognitive Domain: Application

Answer Location: Did the Program Work?

Difficulty Level: Medium

6. Program impact may be evaluated with quasi-experimental designs or survey or field research methods without a randomized experimental design.

Learning Objective: 10.2: Describe the history of evaluation research and its current status.

Cognitive Domain: Comprehension

Answer Location: Did the Program Work?

Difficulty Level: Easy

7. When an ongoing entitlement program is being evaluated and experimental subjects would normally be eligible for program participation, it may be ethical simply to bar some potential participants from the programs.

Learning Objective: 10.3: Diagram the evaluation research process as a feedback system.

Cognitive Domain: Comprehension

Answer Location: Ethics in Evaluation

Difficulty Level: Medium

8. A needs assessment attempts to answer whether a new program is needed.

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: Do We Need the Program?

Difficulty Level: Easy

9. Evaluability assessments often rely on quantitative methods.

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Application

Answer Location: Can the Program Be Evaluated?

Difficulty Level: Medium

10. Evaluators are often called on to document the extent to which implementation has taken place, whether the program is reaching the target individuals or groups, whether the program is actually operating as expected, and what resources are being expended in the conduct of the program.

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Application

Answer Location: Is the Program Working as Planned?

Difficulty Level: Medium

11. As part of its Project Safe Neighborhoods, the U.S. Department of Justice (DOJ) provided funding to 12 relatively large cities to develop and implement a comprehensive anti-gang initiative (CAGI) intended to prevent and reduce gang violence.

Learning Objective: 10.2: Describe the history of evaluation research and its current status.

Cognitive Domain: Application

Answer Location: Case Study: Process Evaluation of an Antigang Initiative

Difficulty Level: Medium

12. The analysis of the extent to which a treatment or other service has the intended effect is an outcome evaluation.

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Application

Answer Location: Did the Program Work?

Difficulty Level: Easy

13. A program theory describes what has been learned about how the program has its effect.

Learning Objective: 10.5: Discuss the role of program theory and its value in evaluation research.

Cognitive Domain: Comprehension

Answer Location: Black-Box Evaluation or Program Theory

Difficulty Level: Easy

14 When a program theory helps to guide the investigation of program process in the most productive directions, it is a theory-driven evaluation.

Learning Objective: 10.5: Discuss the role of program theory and its value in evaluation research.

Cognitive Domain: Application

Answer Location: Black-Box Evaluation or Program Theory

Difficulty Level: Easy

15. The first evaluator of the evaluation research is the funding agency, not the professional social science community.

Learning Objective: 10.5: Discuss the role of program theory and its value in evaluation research.

Cognitive Domain: Application

Answer Location: Researcher or Stakeholder Orientation

Difficulty Level: Medium

16. The program theory is developed by the researcher to clarify and develop the researchers’ theory of the program.

Learning Objective: 10.5: Discuss the role of program theory and its value in evaluation research.

Cognitive Domain: Comprehension

Answer Location: Researcher or Stakeholder Orientation

Difficulty Level: Medium

17. One of the only true experiments to examine the efficacy of problem-oriented policing was conducted by Anthony Braga and his colleagues in 1999, and it remains a model evaluation study today.

Learning Objective: 10.6: Understand why a true experimental design is the strongest method for conducting impact evaluations.

Cognitive Domain: Application

Answer Location: Case Study: Problem-Oriented Policing in Violent Crime Areas—A Randomized Controlled Experiment

Difficulty Level: Medium

18. Data from a quasi-experimental design provide the best way to determine that all criteria have been met in a study.

Learning Objective: 10.6: Understand why a true experimental design is the strongest method for conducting impact evaluations.

Cognitive Domain: Comprehension

Answer Location: Strengths of Randomized Experimental Designs in Impact Evaluations

Difficulty Level: Medium

19. Evaluation research questions are amenable to a true experimental design.

Learning Objective: 10.6: Understand why a true experimental design is the strongest method for conducting impact evaluations.

Cognitive Domain: Comprehension

Answer Location: When Experiments Are Not Feasible

Difficulty Level: Medium

20. Typically, evaluation research that attempts to identify the effects of a treatment, law, or program typically is quantitative.

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: Qualitative and Quantitative Methods

Difficulty Level: Medium

21. Because policy research often draws on the findings of evaluation research projects and involves working for a client as is the case in evaluation research, policy researchers confront many of the same challenges as do evaluation researchers.

Learning Objective: 10.5: Discuss the role of program theory and its value in evaluation research.

Cognitive Domain: Comprehension

Answer Location: Increasing Demand for Evidence-Based Policy

Difficulty Level: Medium

22. A quasi-experimental design in which there are experimental and comparison groups that are designated before the treatment occurs but are not created by random assignment is a nonequivalent control group design.

Learning Objective: 10.6: Understand why a true experimental design is the strongest method for conducting impact evaluations.

Cognitive Domain: Knowledge

Answer Location: Case Study: Nonequivalent Control Group Design: Decreasing Injuries from Police Use of Force

Difficulty Level: Medium

23. In quasi-experimental designs, there is a comparison group that is comparable to the experimental group in critical ways but the subjects are not randomly assigned to the comparison and experimental groups.

Learning Objective: 10.6: Understand why a true experimental design is the strongest method for conducting impact evaluations.

Cognitive Domain: Knowledge

Answer Location: Case Study: Nonequivalent Control Group Design: Decreasing Injuries from Police Use of Force

Difficulty Level: Medium

24. A private organization that has become a leader in publicizing reviews related to criminal justice policy research is the Carter Collaboration.

Learning Objective: 10.2: Describe the history of evaluation research and its current status.

Cognitive Domain: Application

Answer Location: Increasing Demand for Evidence-Based Policy

Difficulty Level: Medium

25. Assessing needs, determining evaluability, and examining the process of treatment delivery have many special ethical dimensions.

Learning Objective: 10.7: Explain the differences between applied and basic research.

Cognitive Domain: Analysis

Answer Location: Ethics in Evaluation

Difficulty Level: Medium

Short Answer/Essay

1. What is evaluation research? Is it a method of data collection or unique research design?

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: A Brief History of Evaluation Research

Difficulty Level: Medium

2. What does it mean to say that evaluation process as a whole can be understood in relation to stakeholders? As part of your answer, explain what stakeholders are and how this type of research is different than traditional social science research.

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Application

Answer Location: Evaluation Basics

Difficulty Level: Medium

3. Describe the process of evaluation research.

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Application

Answer Location: Evaluation Basics

Difficulty Level: Medium

4. Give an examples of the types of questions that evaluation projects may focus on that are related to the operation and impact of a program.

Learning Objective: 10.1: Define each of the types of evaluation research, and explain when each is appropriate.

Cognitive Domain: Comprehension

Answer Location: Questions for Evaluation Research

Difficulty Level: Medium

5. What is a Needs Assessment? How is it accomplished?

Learning Objective: 10.3: Diagram the evaluation research process as a feedback system.

Cognitive Domain: Application

Answer Location: Do We Need the Program?

Difficulty Level: Medium

6. What is an Evaluability Assessment? How is it accomplished?

Learning Objective: 10.3: Diagram the evaluation research process as a feedback system.

Cognitive Domain: Comprehension

Answer Location: Can the Program Be Evaluated?

Difficulty Level: Medium

7. What is Process Evaluation (Program Monitoring)?

Learning Objective: 10.3: Diagram the evaluation research process as a feedback system.

Cognitive Domain: Application

Answer Location: Is the Program Working as Planned?

Difficulty Level: Hard

8. Once we have decided on or identified the goal or focus for a program evaluation, there are still important decisions to be made about how to design the specific evaluation project. What are the most important decisions to be made?

Learning Objective: 10.4: Explain the concept of “black-box” evaluation and the value of opening the black box.

Cognitive Domain: Application

Answer Location: Design Decisions

Difficulty Level: Medium

9. What is a Black Box evaluation?

Learning Objective: 10.4: Explain the concept of “black-box” evaluation and the value of opening the black box.

Cognitive Domain: Application

Answer Location: Black-Box Evaluation or Program Theory

Difficulty Level: Medium

10. Most social science research assumes that the researcher specifies the research questions, the applicable theory or theories, and the outcomes to be investigated. In program evaluation, however, the research question is often set by the program sponsors or the government agency that is responsible for reviewing the program. How does that affect the research?

Learning Objective: 10.4: Explain the concept of “black-box” evaluation and the value of opening the black box.

Cognitive Domain: Comprehension

Answer Location: Researcher or Stakeholder Orientation

Difficulty Level: Medium

11. What is the social science approach? Stakeholder approach? How are they different?

Learning Objective: 10.4: Explain the concept of “black-box” evaluation and the value of opening the black box.

Cognitive Domain: Application

Answer Location: Research or Stakeholder Orientation

Difficulty Level: Hard

12. Describe the research done on Problem-Oriented Policing in Violent Crime Areas. What kind of design was used? What did they discover?

Learning Objective: 10.6: Understand why a true experimental design is the strongest method for conducting impact evaluations.

Cognitive Domain Analysis

Answer Location: Case Study: Problem-Oriented Policing in Violent Crime Areas—A Randomized Controlled Experiment

Difficulty Level: Hard

13. Did the research by Braga et al. (1999) meet all of the criteria for a true experimental design? Describe the research and what they found.

Learning Objective: 10.6: Understand why a true experimental design is the strongest method for conducting impact evaluations.

Cognitive Domain: Application

Answer Location: Strengths of Randomized Experimental Designs in Impact Evaluations

Difficulty Level: Hard

14. How has evaluation research changed over the years, particularly from the 1500s forward?

Learning Objective: 10.2: Describe the history of evaluation research and its current status.

Cognitive Domain: Comprehension

Answer Location: A Brief History of Evaluation Research

Difficulty Level: Medium

15. Describe the research done by the US Department of Justice as part of its Project Safe Neighborhoods. Was it deemed to be effective? Why?

Learning Objective: 10.2: Describe the history of evaluation research and its current status.

Cognitive Domain: Application

Answer Location: Case Study: Process Evaluation of an Antigang Initiative

Difficulty Level: Medium