Chapter 10 Observation & Meta-analysis Exam Prep

Test questions

1. Which of the following is an advantage that observational methods have over tightly controlled experimental procedures?

1. There is greater construct validity in both naturalistic and structured observational methods than in tightly controlled experimental procedures.
2. Observational methods have minimal ethical concerns as compared to tightly controlled experimental methods.
3. There is greater generalizability in both naturalistic and structured observational methods than in tightly controlled experimental procedures.
4. Observational methods are vulnerable to participant reactivity, which is not the case in tightly controlled experimental methods.

Pg. 8, 10, 19, 20

2. Which of the following options is the best description of concealment?

1. It involves hiding the presence of the observer and any recording equipment to prevent individuals from changing their natural behavior.
2. It involves hiding the presence of recording equipment but not the presence of a human observer to prevent individuals changing their natural behavior.
3. It involves hiding the purpose of the study from individuals who are being observed to prevent them from changing their natural behavior.
4. It involves hiding the researcher’s records of observed behavior from individuals who are being observed to prevent them from changing their natural behavior.

Pg. 10

3. Julie is a developmental psychologist who is interested in studying young children’s helping behavior in a playroom setting. Her lab is set up to resemble a playroom containing toys and child-sized furniture. Adjacent to the “playroom” is a room for Julie and her research assistants to observe the behavior of children who participate in their study. There is a one-way mirror installed on the wall that separates the playroom from the observation room, as a strategy dealing with:

a. habituation.

b. reactivity.

c. ethical concerns.

d. construct validity.

Pg. 10

4. In ________________________________ research, the researcher manipulates the setting to a greater extent than in naturalistic observational studies.

1. archival
2. case study
3. meta-analysis
4. structured observational

Pg. 6, 26, 31, 33

5. Piaget's theory of cognitive development was partly based on the _________________

observation of his infant son.

A) experimental

B) naturalistic

C) concealed

D) cross-sequential

Pg. 6

6. Questions 6 – 8 refer to the abstract below.

The abstract below is taken from the 2013 issue of the journal Education and Treatment of Children. The authors of the paper examined whether children with autism can be conditioned and taught to use eye contact. Instructors are advised to use the reference below to retrieve the original abstract for their use.

Carbone, V. J., O'Brien, L., Sweeney-Kerwin, E. J., & Albert, K. M. (2013). Teaching eye contact to children with autism: A conceptual analysis and single case study. Education and Treatment of Children, 36(2), 139-159.

Which of the following is a reasonable assumption that the researchers are making about conclusions of past literature that points to the importance of this case study?

1. Children with autism also learn through social connections and visual input.
2. Children with autism do not necessarily need to learn through visual input.
3. The learning mechanisms in children with autism are fundamentally different from the learning mechanisms in typically developing children.
4. Children with autism do not learn from auditory input.

7. Many of the sessions in this case study were videotaped and analyzed. What would be the best procedure for coding the data?

a. The data should be coded by at least two observers together in discussion.

b. The data should be coded by at least two observers independently, but later discussed to resolve any inconsistencies.

c. The data should be coded by at least two observers independently, and analyzed without any discussion regardless of agreement.

d. The data can be coded by only one individual because this is a case study.

Pg. 24, 25

8. Which of the following is not a valid reason to employ the design that the authors chose?

a. Conditioning procedures may harm the child, and little is known about the consequences of this treatment.

b. The researchers were hoping to achieve a large amount of quantitative data to statistically test the outcomes of the treatment.

c. The chosen design might have been the more cost effective.

d. Withholding a potentially beneficial treatment comes with ethical considerations.

Pg. 29, 30

9. Which of the following reasons is not an argument that supports coding qualitative data?  

1. It can make summarizing the data easier. 
2. It can allow for the use of statistical tests more suitable for quantitative data.
3. It can lead to richer data with more details.  
4. It simplifies the data.

Pg. 23, 24

10. Archival research often provides researchers with __________________________ than conducting a single study.

1. a larger sample size
2. reduced statistical power
3. greater control over the questions asked
4. more ethical challenges

Pg. 31, 32

11. Meta-analyses often draw directly on which of the following statistics in order to form conclusions?

1. Sample size across studies
2. The level of significance in the findings across studies
3. Effect sizes across studies
4. Confidence intervals of the relevant findings across studies

Pg. 34, 35

12. Which of the following methods of sampling benefits the most from the use of some type of recording technology to track all types of behaviors?

A) Specimen record

B) Time sampling

C) Event sampling

D) Coding record

Pg. 21, 22, 24

13. In observational studies, researchers often use concealment to deal with the problem of reactivity. What is necessary on the researcher's part in order for concealment to be properly executed?

A) Deception

B) Double blind design

C) Manipulation checks

D) Control group manipulation

Pg. 10, 11

14. Which of the following is a valid criticism of observational methods in psychological research?

A) Because they are not real experiments, observational methods are not considered real science.

B) Because they do not involve a lab setting, observational methods are not publishable in peer-reviewed journals, and thus do not provide generalizable science.

C) Because they involve very little manipulation on the researcher's part, and do not generally employ random assignment, observational methods rarely yield data supporting causal relationships.

D) Because the data is generally qualitative in nature, observational methods do not yield data that can be analyzed with inferential statistics and do not meet the rigor that psychological scientists require.

Pg. 9, 10, 20

15. Which of the following describes a disadvantage of archival research?

a. Archival research usually contains a large amount of data from a relatively representative sample.

b. Archival research is exempted from review from most Institutional Review Boards.

c. Archival research contains data that is limited what has already been collected.

d. Archival research is often open for researchers to access.

Pg. 33

16. When conducting a meta-analysis, which of the following statements is true?

A) Meta-analyses should include studies of all types, including those of high and low quality to capture the entire scientific repertoire.

B) Meta-analyses should contain an equal number of studies reporting significant and non-significant results so as to avoid biases in the forthcoming analyses.

C) Meta-analyses should have strict criterions for inclusion, and only include high-quality studies.

D) Meta-analyses should only include statistically significant findings because researchers conducting these analyses are primarily interested in the size of effects that were found. This allows researchers to gauge whether or not the findings are reliable.

Pg. 33, 34

17. The file-drawer problem influences the results of a meta-analysis by:

a. reducing its statistical power.

b. reducing construct validity.

c. decreasing the quality of published studies.

d. decreasing the accuracy of the standardized effect size.

Pg. 39, 40

18. The textbook discussed several examples of case studies. In what way was the study of H.M. most different from other case studies such as the one about Nadia?

A) H.M. was an adult, whereas Nadia was a child, and therefore the study of H.M. came with fewer ethical concerns.

B) H.M. could not consolidate new memories, and therefore might not have been vulnerable to feeling bored and fatigued from repeated sessions with an experimenter or researcher.

C) H.M. had a rare tumor, which made the findings non-generalizable to non-clinical populations.

D) The data obtained from H.M. were mostly through naturalistic observations, whereas the data generated from studying Nadia were mostly through a microgenetic approach.

Pg. 26 - 28

19. Jessica is interested in studying how popular leaders behave towards members of their team, as compared to leaders who are unpopular. She is especially interested in examining this behavior in a business context and obtaining high external validity. Which of the following methods is best suited to her goals?

a. Case study

b. Meta-analysis

c. Experimental study run in a laboratory setting

d. Structured observation

Pg. 19, 20

20. In a study by Rosengren, Schein, and Gutiérrez (2010), scale errors committed by children between the ages of 18 and 29 months were recorded while they were at play in a laboratory preschool. The researchers placed miniature replica toys in the preschool classroom during specific time periods to increase the likelihood of observing scale errors. This study is an example of a:

a. naturalistic observational design.

b. case study design.

c. structured observational design.

d. single-subject experimental design.

Pg. 20, 21

21. Recorded video data of an observational study does not offer the advantage of:

a. more accurate coding.

b. greater reliability in coding.

c. less variability in coding.

d. fewer problems of participant reactivity.

Pg. 23

22. In a meta-analysis, articles that ________________ should be excluded.

a. have different sample sizes

b. report non-significant results

c. are low on internal validity

d. are not in peer-reviewed journals

Pg. 36

23. Janet is deeply interested in examining the phenomenon of self-imposed social isolation (i.e., hikikomori). She wants to investigate the possible causes for this phenomenon. After doing a brief literature search, she finds several case studies on individuals with hikikomori and numerous media articles reporting on these case studies. Which of the following is the best step for Janet to obtain a list of potential triggers of the behavior?

1. Read the media articles, since the journalists have already summarized and evaluated the case studies.
2. Contact a researcher who had conducted one of the case studies and interview the researcher about the subject.
3. Read all the case studies and identify common factors that the case subjects shared.
4. Read only the case studies that were reported on by the media, since these case studies must have been most impactful.

Pg. 27

24. The General Social Survey contains demographic and attitudinal data collected from respondents over a few decades. Researchers have used this data set to explore the relation between income inequality and happiness. This is an example of:

a. archival research.

b. meta-analysis.

c. case study.

d. naturalistic observational design.

Pg. 31

25. If a comprehensive record of all initiated psychological experiments and their results can be maintained, this would greatly reduce:

a. the need for case studies.

b. the file-drawer problem.

c. participant bias.

d. non-significant findings.

Pg. 40

15 true/false questions

1. Observational methods are research methods that do not have rules or structure. False

Pg. 5

1. Only structured observational methods can lead to any useful theory in psychology, because naturalistic observation is not systematic. False

Pg. 6

1. The most common way of conducting naturalistic observation is for the researcher to enter a setting of interest and wait for any interesting behavior to appear. False

Pg. 7

1. Structured observation is most useful after the researcher has done some preliminary work on the phenomenon of interest. True

Pg. 19

1. Structured observations offer researchers high external validity. True

Pg. 20

1. In the specimen record approach of coding, all of the behaviors in a given time period are recorded. True

Pg. 24

1. An advantage of the case study method is that it allows researchers to obtain rich data. True

Pg. 29

1. Most studies based on archival data do not require approval of the Institutional Review Board. True

Pg. 31

1. When researchers come up with research questions, they are sometimes able to use existing data to answer those questions. True

Pg. 32

1. Case study data allow researchers to use complex statistical techniques. False

Pg. 29, 30

1. A drawback of archival research is that the data available might not directly answer your research questions. True

Pg. 33

1. If only a single study has been done on your topic of interest, it is too early for a meta-analysis. True

Pg. 33

1. Effect size can only be measured when you have more than one study. False

Pg. 34

1. Meta-analysis allows researchers to better evaluate the existence or strength of a psychological phenomenon as compared to a single study. True

Pg. 39

1. The results of a meta-analysis are always unbiased because they take into account the findings of multiple studies. False

Pg. 39, 40

1. Explain what is meant by “reactivity” in observational methods and define two strategies to reduce the effect of reactivity.
2. Angela is interested in how children of different ages use tools in the environment to solve problems. She is deciding between using naturalistic observation and structured observation for her study. Describe the advantages and disadvantages of each method.
3. Describe the process of meta-analysis. What is a potential challenge to the validity of meta-analysis, and what steps can be taken in response?
4. What are the advantages of conducting a meta-analysis on existing research, as compared to designing and conducting an empirical study?
5. Briefly explain what is meant by archival research. Describe two advantages and a disadvantage of archival research.