Test Bank

Chapter 13: Evaluation and Policy Research

Multiple Choice

1. Information about service delivery system outputs, outcomes, or operations that can guide program input is called ______.

A. outputs

B. outcomes

C. feedback

D. program process

Difficulty Level: Easy

2. Policy researchers need to include policy makers and regulators in their planning and should consider who will be affected by a policy decision. What caution to policy research does this best represent?

A. credible

B. meaningful and engaging

C. responsible

D. creative

Difficulty Level: Hard

3. What is one of the greatest challenges for designing quality policy research.

A. making sure all negative consequences are considered

B. coming up with new solutions

C. credibility of evidence

D. timeframe difference between the research process and policy decision-making

Difficulty Level: Hard

4. Number of meals served or number of people sheltered over a year are examples of ______.

A. inputs

B. outputs

C. outcomes

D. program progress

Difficulty Level: Easy

5. Case managers and volunteers are examples of ______.

A. inputs

B. outputs

C. outcomes

D. program progress

Difficulty Level: Easy

6. Evaluation research is unlike traditional social science research because ______.

A. it is designed to test the implications of a social theory

B. program stakeholders have influence in how the study is designed

C. evaluation research cannot ethically use randomization

D. evaluation research rarely uses quantitative methods

Difficulty Level: Medium

7. Evaluation research began and developed in which time period?

A. 1910–1920s

B. 1930–1940s

C. 1950–1960s

D. 1970–1980s

Difficulty Level: Easy

8. The economic value of a social program when compared to the costs of that program is established in ______.

A. a cost–benefit analysis

B. a cost-effectiveness analysis

C. a process analysis

D. a summative assessment

Difficulty Level: Medium

9. Theory-driven evaluation does which of the following?

A. assesses the relative costs of a social program

B. determines the generalizability of a program

C. guides the investigation of a program process

D. suggests which outcomes should be measured

Difficulty Level: Hard

10. The Government Performance and Results Act of 1993 requires what of government programs?

A. IRB oversight

B. cost–benefit of analysis

C. needs assessment

D. some sort of evaluation

Difficulty Level: Easy

11. Which of the following is not a typical input in social programs?

A. clients

B. customers

C. feedback

D. participants

Difficulty Level: Medium

12. The services delivered or new products produced by a social program are ______.

A. program process

B. program outputs

C. program theory

D. program participation

Difficulty Level: Easy

13. To find out if a new program is needed in a community, conduct a/an ______.

A. process evaluation

B. evaluability assessment

C. impact analysis

D. needs assessment

Difficulty Level: Medium

14. One important difference between evaluation research and other social scientific research is ______.

A. evaluation research is often guided by politics

B. evaluation research rarely uses qualitative research

C. evaluation research rarely uses quantitative research

D. evaluation research is conducted by professional researchers

Difficulty Level: Hard

15. When evaluating their program, in which group exercise classes tended to increase several measures of participant health, researchers concluded that it didn't really matter how the group classes worked, just so long as it worked. This is an example of ______.

A. black box theory

B. program theory

C. utilization theory

D. theory-driven evaluation

Difficulty Level: Hard

16. If a program evaluation is written without regard to whether the program had the intended effect, but with very careful attention to how it will be received by other social science peers, it has what kind of orientation or approach?

A. process

B. stakeholder

C. social science

D. theoretical

Difficulty Level: Hard

17. Evaluability assessments tend to rely on ______.

A. experimental designs

B. qualitative methods

C. surveys

D. all of these

Difficulty Level: Medium

18. In Project New Hope, 677 low-income adults were offered jobs, childcare, and healthcare, while others were offered none of these things. In the end, only 27% of those who were offered jobs stuck with it long enough to lift themselves out of poverty. Levels of depression did not decrease, nor did self-esteem increase. However, the classroom performance of the participants' male children increased. This example demonstrates the importance of ______.

A. theory-driven evaluation

B. process evaluation

C. measuring multiple outcomes

D. efficiency analysis

Difficulty Level: Hard

19. A community hires a researcher to determine whether a community center would be a benefit, in terms of reducing delinquency, promoting community service, and developing networks. To provide an answer, the researcher would conduct what sort of evaluation on community services?

A. needs assessment

B. evaluability assessment

C. process evaluation

D. impact analysis

Difficulty Level: Hard

20. A study specifically designed to investigate whether a program can be evaluated is known as a(n) ______.

A. needs assessment

B. evaluability assessment

C. process evaluation

D. impact analysis

Difficulty Level: Medium

21. Which of the following is a typical reason why a program cannot be evaluated?

A. Management only wants its superior performance confirmed.

B. Staff doesn't trust management to check on their performance.

C. Personnel have no clear sense of what the program is trying to achieve.

D. All of these

Difficulty Level: Medium

22. Evaluation research that investigates the process of service delivery is known as ______.

A. needs assessment

B. evaluability assessment

C. process evaluation

D. impact analysis

Difficulty Level: Easy

23. The preferred method for maximizing internal validity in impact analysis is ______.

A. process theory

B. stakeholder orientation

C. black box orientation

D. experimental design

Difficulty Level: Medium

24. Impact analysis generally relies on what sort of research design?

A. experimental and quasi-experimental

B. open-ended surveys

C. closed-ended surveys

D. one-shot case studies

Difficulty Level: Medium

25. Cost–benefit analyses and cost-effectiveness analyses are both what kind of evaluation research?

A. needs assessment

B. evaluability assessment

C. efficiency analysis

D. impact analysis

Difficulty Level: Medium

26. To lessen the potential detrimental impact of social programs, researchers should do all of the following, EXCEPT ______.

A. vary treatments among settings rather than among individuals within a setting

B. use the minimum sample size required to adequately test the results

C. minimize the number in the untreated control group

D. compare the presence and absence of treatments, rather than those that vary in intensity

Difficulty Level: Hard

27. The historical growth of evaluation research by social scientists is linked to which of the following?

A. growth in private-sector social programs

B. growth in government-funded social programs

C. growth in faith-based social programs

D. decline in government-funded social programs

Difficulty Level: Easy

28. If an evaluation takes a black box approach, which is least important?

A. evaluability

B. social science orientation

C. peer evaluation

D. causal mechanism

Difficulty Level: Medium

29. In the DARE program, which of the following was NOT a stakeholder?

A. Parents worried about drug abuse.

B. Police officers trained as DARE officers.

C. Law enforcement in general.

D. All of these were stakeholders.

Difficulty Level: Hard

30. If a researcher wants to find out how a program actually produces its outcomes, she or he could develop a/an ______.

A. impact evaluation

B. process evaluation

C. program theory

D. efficiency analysis

Difficulty Level: Hard

31. Appreciative inquiry does NOT do which of the following?

A. consider participants' ideas about a program

B. evaluate a social program

C. have a professional social researcher

D. create a structured dialogue about needed changes

Difficulty Level: Medium

32. When the goal of evaluation is to compare alternative programs, it may be preferable to ______.

A. randomize whole groups, not individuals, to different groups

B. randomize individuals, not whole groups, to different groups

C. randomly sample small groups from a small population

D. make sure to have a small sample size

Difficulty Level: Hard

33. If a researcher takes a social science approach to conducting a program evaluation, the people who will assess the quality of that researcher's report will be ______.

A. other social scientists

B. the funding agency

C. the participants

D. the stakeholders

Difficulty Level: Medium

34. To evaluate a program designed to improve math scores for elementary school girls, girls were divided into treatment and control groups. Girls in the treatment group were required to attend an after-school program, while girls in the control group went home on time. Math exam scores were then compared between the two groups of girls. This type of program design is associated with ______.

A. needs assessment

B. evaluability assessment

C. process evaluation

D. impact analysis

Difficulty Level: Hard

35. Who is eliminated from the research process in appreciative inquiry?

A. stakeholders

B. sponsors

C. professional researchers

D. staff

Difficulty Level: Easy

True/False

1. Evaluation research developed in tandem with government expansion during the Great Depression and WWII.

Difficulty Level: Easy

2. Evaluation research is conducted to investigate social programs.

Difficulty Level: Easy

3. According to U.S. law, some sort of evaluation is required of all government programs.

Difficulty Level: Easy

4. The direct product that a program delivers is the outcome.

Difficulty Level: Easy

5. The type of evaluation research that determines if a new social program is needed or an old social program is still needed is called an impact assessment.

Difficulty Level: Easy

6. Stakeholders objectively define needs in a needs assessment.

Difficulty Level: Easy

7. Evaluability assessments generally rely on quantitative methods.

Difficulty Level: Medium

8. The investigation of how a social program works is called a mechanism evaluation.

Difficulty Level: Easy

9. If evaluation findings will be used to help shape and refine a social program, it is known as a formative evaluation.

Difficulty Level: Easy

10. Process evaluation investigates how a service is delivered.

Difficulty Level: Easy

11. Quasi-experimental designs are the preferred method for proving that a program created (or caused) a desired effect.

Difficulty Level: Medium

12. In impact analysis, an experimental design is preferred to maximize internal validity.

Difficulty Level: Easy

13. With a black box evaluation study, understanding how a program works is not important.

Difficulty Level: Easy

14. Researchers should always emphasize a strict social science orientation when conducting evaluation research.

Difficulty Level: Easy

15. Impact analyses typically use quantitative data.

Difficulty Level: Easy

Essay

1. How and why did evaluation research develop? How does it differ from other types of social science research?

Difficulty Level: Medium

2. Describe the potential problems for stakeholder orientation in evaluation research, in general and using the example of reducing welfare payments in favor of a workfare program.

Difficulty Level: Hard

3. In what circumstances should qualitative methods be used in evaluation research? In what circumstances should quantitative methods be used? Use examples to demonstrate your answer.

Difficulty Level: Hard

4. Based on the information in Chapter 13, how would you evaluate the success of the DARE program? Explain your answer fully.

Difficulty Level: Medium

5. Why is evaluation research more political than other forms of social research?

Difficulty Level: Medium

6. Outline the basics of evaluation research. How would a researcher use these basics in order to evaluate the DARE program detailed in the text?

Difficulty Level: Hard

7. What is the difference between an evaluation research project that assumes a black box and one that seeks to develop a program theory? What are the advantages and disadvantages to each?

Difficulty Level: Hard

8. What steps can be taken to lessen the potential detrimental impact of social programs on participants? What other ethical issues should evaluation researchers consider both in terms of social science ethics and federally mandated criteria?

Difficulty Level: Hard

9. What are the limitations of evaluation research when compared to other forms of social scientific research?

Difficulty Level: Hard

10. Describe the special ethical considerations associated with evaluation research. How might a researcher lessen the detrimental impacts to subjects?

Difficulty Level: Hard