Measurement Ch.5 Full Test Bank

CHAPTER 5

Measurement

Multiple Choice

1. A researcher interested in measuring religiosity first defines the meaning and identifies possible dimensions of this concept. Which step in the measurement process does this represent?
   1. conceptualization
   2. validity assessment
   3. operationalization
   4. index construction
2. Sociologist Robert Blauner identified four dimensions of alienation: powerlessness, meaninglessness, isolation, and self-estrangement. Blauner’s analysis represents one aspect of
   1. conceptualization.
   2. operationalization.
   3. reliability assessment.
   4. validation.
3. A researcher interested in measuring religiosity decides to ask respondents how often they pray and how often they listen to religious programs on the radio. These questions represent the researcher’s attempt to
   1. provide a conceptual definition of the concept.
   2. refine the meaning of the concept.
   3. specify empirical indicators.
   4. assess the quality of the researcher’s operational definition.
4. Which of the following is an empirical indicator of social capital?
   1. good will, fellowship, and sympathy among the individuals that make up a social unit
   2. the bonding and bridging of people with one another
   3. the number of people individuals report that they can rely on in a time of need
   4. the ability to accrue benefits through membership in social networks
5. Some sociologists define “formalization” as the extent of regulation in an organization. In one study this was indicated by the number of the following types of documents that organizations had created: written employment contracts, information booklets, an organization chart, written job descriptions, and a manual of procedures. The total count of documents is an example of
   1. conceptualization.
   2. operationalization.
   3. reliability assessment.
   4. validity assessment.
6. Which of the following is an empirical indicator of alienation?
   1. a feeling of estrangement from a situation, group, or culture
   2. the inability to control one’s immediate work processes
   3. lack of a sense of purpose that connects one’s job to the overall organization of production
   4. whether an individual agrees with the statement “I am often left out of things going on around me”
7. Which of the following is an example of an operational definition?
   1. fear of crime: a series of complex changes in bodily functioning that alerts an individual to potential danger
   2. life satisfaction: relatively stable assessment of how well people’s lives match their expectations
   3. mastery: people’s sense of control over their environment
   4. social isolation: the breakdown of social relations that provide meaning and social support
   5. criminal activity: the number of crimes reported to the police per 1,000 population in the city or town where a respondent resides
8. In social measurement, indicator is to index/scale as __________ is to __________.
   1. measured; manipulated
   2. one questionnaire item; two or more items
   3. observation; verbal reports
   4. reliable; unreliable
9. Manipulation operations are used most commonly in
   1. experiments.
   2. surveys.
   3. qualitative field research.
   4. the analysis of existing data.
10. In Tilcsik’s experimental study of job discrimination against gay men, what type of operational definition did he use to measure “employment discrimination”?
    1. manipulation operations
    2. verbal report
    3. observation
    4. archival records
11. In selecting or creating an operational definition of a concept, the textbook suggests all of the following guidelines except
    1. carefully consider the theory in which the concept is embedded.
    2. use more than one indicator to measure complex concepts.
    3. select an operational definition that captures the meaning of the concept.
    4. begin by identifying empirical indicators of the concept.
    5. use quantitative measures whenever possible.
12. A variable’s level of measurement indicates the
    1. number of questions used to measure the variable.
    2. number of variable categories.
    3. kinds of comparisons that can be made between cases in different variable categories.
    4. degree of correspondence between conceptual and operational definitions.
13. Which of the following is an example of nominal-level measurement?
    1. family size (number of children in a family)
    2. political participation (number of times voted in the last 10 civic elections)
    3. educational attainment (highest year or grade in school completed)
    4. political party affiliation (Republican/Democrat/Independent/Other/None)
    5. religiosity (whether one attends religious services frequently, occasionally, or never)
14. Suppose a researcher uses the following set of categories to measure employment status: working full-time; working part-time; in school. This set of categories is
    1. exhaustive but not mutually exclusive.
    2. mutually exclusive but not exhaustive.
    3. both exhaustive and mutually exclusive.
    4. neither exhaustive nor mutually exclusive.
15. A survey question measuring frequency of exercise asks respondents, “On how many of the past 7 days did you exercise or participate in sports activities?” The answer categories are 0, 1 – 3, 3 – 5, 5 or more. This set of categories is
    1. exhaustive but not mutually exclusive.
    2. mutually exclusive but not exhaustive.
    3. both exhaustive and mutually exclusive.
    4. neither exhaustive nor mutually exclusive.
16. As a measure of religiosity, a researcher asks, “How important would you say religion is in your own life? Is it very important, somewhat important, not very important, or not at all important?” This is an example of __________-level measurement.
    1. nominal
    2. ordinal
    3. interval
    4. ratio
17. A measure of attendance at religious services asks respondents to place themselves in one of the following categories: every week, two to three times a month, once a month, less than once a month. Which level of measurement is this?
    1. nominal
    2. ordinal
    3. interval
    4. ratio
18. Suppose we measured length of residence by asking people, “How many years have you lived at your present address?” This constitutes __________-level measurement.
    1. nominal
    2. ordinal
    3. interval
    4. ratio
19. The GSS measure of television viewing asks, “On the average day, about how many hours do you personally watch television?” Which level of measurement is this?
    1. nominal
    2. ordinal
    3. interval
    4. ratio
20. If you measured age by calculating the difference between the year a respondent was interviewed and the year the respondent was born, then your level of measurement would be __________; if you measured age by placing people in one of four categories (18 and under, 19–39, 40–59, 60 and over), then your level of measurement would be __________.
    1. ratio; ratio
    2. ratio; interval
    3. ratio; ordinal
    4. ordinal; ordinal
21. Which level of measurement has an absolute or “true” zero point?
    1. nominal
    2. ordinal
    3. interval
    4. ratio
22. Which of the following factors would not be a source of measurement error for scores on this exam?
    1. differences among students in the ability to take multiple choice exams
    2. differences in knowledge and understanding of research methods
    3. ambiguously worded questions
    4. students’ state of health
23. Systematic error would be least likely to occur with which of the following measures?
    1. a miscalibrated yardstick that is an inch short
    2. an IQ test developed for English-speaking U.S. citizens that is administered to French Canadians
    3. a question measuring child abuse that asks, “How often do you hit your child when she doesn’t obey you?”
    4. scores on the multiple choice portion of this exam
    5. reports of gambling winnings on IRS tax returns
24. The social desirability effect refers to
    1. class differences in happiness and mental health.
    2. respondents’ tendency to give answers that project a positive self-image.
    3. the tendency to agree with a statement regardless of its content.
    4. higher response rates obtained by pleasant than unpleasant interviewers.
25. Research indicates that in recent years respondents’ answers to survey questions measuring racial prejudice underestimate actual racial prejudice. This underestimate is an example of __________ measurement error, which affects measurement __________.
    1. random; reliability
    2. random; reliability and validity
    3. systematic; validity
    4. systematic; reliability and validity
26. Suppose that on the SAT (Scholastic Aptitude Test), some test-takers do better than others because they make more lucky than unlucky guesses on questions that they cannot solve. This constitutes __________ measurement error, which lowers the SAT’s __________.
    1. random; measurement level
    2. random; reliability and validity
    3. systematic; validity
    4. systematic; reliability and validity
27. All of the following combinations of reliability and validity are possible except
    1. low reliability; low validity
    2. low reliability; high validity
    3. high reliability; low validity
    4. high reliability; high validity
28. Which of the following combinations is possible?
    1. low random error; low reliability
    2. high random error; high reliability
    3. low random error; low validity
    4. high random error; high validity
29. Measures that are subject to very little random error but a major source of systematic error
    1. are reliable but not valid.
    2. are valid but not reliable.
    3. are neither reliable nor valid.
    4. may be both reliable and valid.
30. Which of the following questions most clearly concerns validity and not reliability?
    1. Do self-reports of alcohol consumption underestimate the incidence of binge drinking?
    2. Do respondents’ answers to the question change with repeated questioning?
    3. What is the average inter-item correlation among the scale items?
    4. Do different coders agree in their ratings of violence on Saturday morning cartoons?
31. Which of the following questions concerns reliability?
    1. What does the Scholastic Aptitude Test (SAT) really measure?
    2. Do attitude scales tap personal feelings or just what people think investigators want to hear?
    3. Is poverty best represented by measures of low absolute income (a minimum subsistence) or low relative income (e.g., 50 percent of national median income)?
    4. How closely does self-reported church attendance correspond to actual church attendance?
    5. Assuming that students did not learn the answers in the interim, would they score the same on this exam if they took it again tomorrow?
32. Survey estimates of church attendance based on self-reports are very stable, as respondents tend to give consistent answers. Observations of church attendance indicate, however, that it is about one-half the level reported in surveys. This suggests that, as a measure of church attendance, self-reported attendance is
    1. reliable but not valid.
    2. valid but not reliable.
    3. neither reliable nor valid.
    4. both reliable and valid.
33. Test-retest, split-half, and internal consistency are forms of
    1. measurement control.
    2. construct validation.
    3. reliability assessment.
    4. validity assessment.
34. Suppose you used a single questionnaire item to measure a particular concept. Which of the following methods could not be used to assess its reliability or validity?
    1. test-retest reliability
    2. internal consistency reliability
    3. face validity
    4. criterion-related validity
    5. construct validity
35. Suppose a researcher asks the exact same question at two different points in a lengthy interview, near the beginning and near the end. If respondents give the same answer each time the question is asked, this supports the question’s
    1. test-retest reliability.
    2. internal consistency.
    3. convergent validity.
    4. construct validity.
36. For various reasons, which form of reliability assessment is least likely to be used?
    1. intercoder reliability
    2. internal consistency
    3. test-retest reliability
37. Two observers rate individual leadership by recording the number of times that each member of a group initiates discussion. The level of agreement between the observers’ independent records provides evidence of
    1. reliability.
    2. validity.
    3. neither reliability nor validity.
    4. both reliability and validity.
38. Which of the following statements correctly applies to the reliability of this exam?
    1. If nearly everyone misses a particular question, this will lower the exam’s reliability.
    2. The best way to assess the exam’s reliability is to carefully judge how representative the questions are of the course content.
    3. Ambiguously worded questions that evoke guessing have little or no effect on reliability.
    4. Increasing the number of questions on the exam is likely to increase its reliability.
39. Reliability assessment is a matter of checking for __________, whereas validity assessment is a matter of determining the degree of __________.
    1. accuracy; consistency
    2. consistency; accuracy
    3. reactivity; similarity
    4. accuracy; similarity
40. Suppose a researcher constructs a six-item index for measuring risk-taking propensity. Evidence that eighth graders who score high on the index are more likely to engage in risky behaviors (e.g., initiate sexual activity, use drugs) supports the index’s
    1. test-retest reliability.
    2. internal consistency.
    3. intercoder reliability.
    4. convergent validity.
41. By reporting that scores on his Love Scale were correlated with the feeling of being “in love,” Zick Rubin provided evidence of the scale’s
    1. internal consistency.
    2. test-retest reliability.
    3. intercoder reliability.
    4. convergent validity.
42. Which of the following statements is true regarding the assessment of measurement validity?
    1. Subjective evaluation of operational definitions has no place in social science.
    2. Face validity is an adequate means of assessing the validity of most operational definitions.
    3. Criterion-related validity is the most common method of validity assessment in sociological research.
    4. Construct validity depends on an accumulation of research evidence rather than a single prediction.
43. A comprehensive exam purported to cover an entire semester’s work covers only material from the first eight weeks of the course. As a measure of how much students learned overall, this exam lacks
    1. reliability.
    2. construct validity.
    3. content validity.
    4. criterion-related validity.
44. Suppose that a large company administers a new employment test to fifty applicants for the position of clerk-typist. All fifty are hired, and then, after nine months on the job, are rated on their job performance by their supervisors. Correlating test scores with supervisor ratings is an attempt to assess the test’s
    1. test-retest reliability.
    2. internal consistency reliability.
    3. criterion-related validity.
    4. construct validity.
45. Evidence that scores on the Rosenberg Self-Esteem Scale are associated, as expected, with measures of depression and peer popularity supports the scale’s
    1. test-retest reliability.
    2. internal consistency.
    3. discriminant validity.
    4. construct validity.
46. A near-perfect correlation between a measure of romantic love and a measure of liking suggests that the measure of romantic love
    1. is reliable.
    2. is internally consistent.
    3. lacks discriminant validity.
    4. lacks convergent validity.
47. In contrast to criterion-related validity, construct validity is important to assess when
    1. the measure in question is intended for practical application rather than theoretical purposes.
    2. there is a single, agreed-upon criterion for assessing validity.
    3. the primary concern is the accuracy of prediction.
    4. the primary concern is with capturing the theoretical meaning of the concept being measured.
48. In assessing your newly developed scale for measuring anti-Semitism, you get highly similar results when you administer your scale twice to the same group. You also get a high correlation between your new anti-Semitism scale and another scale designed to measure the tendency of people to give socially desirable responses. According to this evidence, your scale is
    1. reliable but not valid.
    2. valid but reliable.
    3. both reliable and valid.
    4. neither reliable nor valid.
49. The logic of construct validation is based on all of the following except
    1. theoretical relations between the concept being measured and other concepts.
    2. the internal consistency of variables with which the concept is theoretically related.
    3. theoretical predictions of variables with which the concept should be associated.
    4. theoretical predictions of variables with which the concept should not be associated.
    5. an accumulation of evidence rather than a single association between the concept and a related variable.

True and False

T F 1. The measurement process moves from the concrete to the abstract.

T F 2. Operational definitions are intended to capture the meaning of conceptual definitions.

T F 3. Operational definitions may be manipulated or measured.

T F 4. The most common form of social measurement is direct observation.

T F 5. One empirical indicator is as good as another.

T F 6. Whenever possible, researchers should avoid the use of multiple indicators of a concept.

T F 7. Often, many different indicators may be chosen to measure a given concept.

T F 8. There is almost always an imperfect correspondence between conceptual and operational definitions.

T F 9. Measurement level is determined by the number of empirical indicators used to operationalize a concept.

T F 10. In nominal measurement, the numbers assigned to categories may be added together but not multiplied.

T F 11. The lowest level of measurement is nominal.

T F 12. Interval-level measures have an arbitrary zero point.

T F 13. Class rank in a graduation class is an example of ratio measurement.

T F 14. It is possible for a measure to be reliable but invalid.

T F 15. It is possible for a measure to be valid but unreliable.

T F 16. If all the variation in a measurement is due to true differences in the concept, then the measure is both reliable and valid.

T F 17. Systematic measurement error tends to lower reliability.

T F 18. The social desirability effect is an example of random measurement error.

T F 19. The cultural bias of IQ tests is an example of systematic measurement error.

T F 20. Reliability assessment is a matter of checking for consistency.

T F 21. The most common approach to reliability assessment is the test-retest method.

T F 22. Internal consistency is used to assess the reliability of single-item measures.

T F 23. Using additional indicators of a concept (e.g., additional questionnaire items) can increase reliability.

T F 24. All forms of measurement validation are subjective to a certain extent.

T F 25. Criterion-related validity generally is reserved for tests and measures that are intended for practical application.

T F 26. The social desirability effect lowers measurement validity.

T F 27. Construct validation depends on an accumulation of research evidence rather than a single prediction.

T F 28. Correlations among different measures of the same concept are evidence of the measures’ convergent validity.

T F 29. Correlations among different measures of the same concept are evidence of the measures’ discriminant validity.

T F 30. A near-perfect correlation between level of education and scores on a mental health scale suggests that the scale lacks discriminant validity.

Essay

1. Briefly explain how the process of measurement moves from the abstract to the concrete.
2. Explain how the text’s example of a recipe for carrot cake illustrates an operational definition in social research. What are the recipe ingredients analogous to in social measurement? What do the steps for combining ingredients and baking the cake illustrate about an operational definition?

1. Levels of measurement indicate the inferences that can be drawn when comparing one category or value of a variable with another: Do they differ? Is one ranked higher than the other? What is the numerical difference? What is the ratio of the values? Identify the four levels of measurement. Then, for each level describe the inferences that can be made when comparing units (e.g., people) who are in different variable categories.
2. Describe the relationship between reliability and validity. (NOTE: Your description should answer the following two questions: Is it possible to have a reliable but invalid measure? Is it possible to have an unreliable but valid measure?)
3. Several studies have shown that feelings of empathy motivate altruistic behavior. The 2002 GSS showed a correlation of .19 between the Davis Empathy Scale and a composite measure of altruistic behavior based on whether respondents had engaged in 15 different activities during the past year. Among these activities were giving directions, giving money to charity, helping carry a stranger’s belongings, and donating blood. This modest association may reflect difficulties in measuring altruistic behavior. Identify two possible sources of error in the GSS measure of altruistic behavior and carefully explain how each could introduce random or systematic measurement error.
4. Explain the difference between criterion-related validity and construct validity. What type of evidence does each method use to validate measures?