Validity Evidence Based on Content | 138 ■ Validity Evidence Based on Internal Structure | 138 ■ Validity Evidence Based on Relations to Other Variables | 139
Using Reliability and Validity Information | 140
Sources of Information About Tests | 141
Sampling Methods | 141
Terminology Used in Sampling | 141
Random Sampling Techniques | 144
Simple Random Sampling | 145
Stratified Random Sampling | 146
Cluster Random Sampling | 149
Systematic Sampling | 149
Nonrandom Sampling Techniques | 150
Random Selection and Random Assignment | 151
Determining the Sample Size When Random Sampling Is Used | 152
Sampling in Qualitative Research | 153
Summary | 154
Key Terms and Concepts | 155
Related Internet Sites | 156
Practice Test | 156
Challenge Exercises | 157
Research Validity | 158
Introduction | 159
Overview of Four Major Types of Validity | 159
Statistical Conclusion Validity | 160
Construct Validity | 160
Threats to Construct Validity | 161
Participant Reactivity to the Experimental Situation | 161 ■ Experimenter Effects | 164
Numbers | 465 ■ Physical Measurements | 465 ■ Presentation of Statistical Results | 465 ■ Tables | 465 ■ Figures | 467 ■ Figure Legends and Caption | 467 ■ Figure Preparation | 467 ■ Reference Citations | 468 ■ Reference List | 469 ■ Preparation of the Manuscript for Submission | 471 ■ Ordering of Manuscript Pages | 471
Submission of the Research Report for Publication | 471
Acceptance of the Manuscript | 473
Presenting Research Results at Professional Conferences | 473
Oral Presentation | 474
Poster Presentation | 474
Summary | 476
Key Terms and Concepts | 477
Related Internet Sites | 477
Practice Test | 478
Challenge Exercises | 478
Appendix | 479
Glossary | 480
References | 495
Index | 507
Preface
Welcome to Research Methods, Design, and Analysis . You are embarking on a study that will help you to think critically and creatively in Psychology and other disciplines. We have three goals for this text. First, we have focused on writing a book that provides an understanding of the research methods used to investigate human thought and behavior. Research methods tend to change slowly, but they do change. This book provides coverage of the complete range of research methods available today. Psychology tends to favor experimental methods so we devote more time to experimental research methods. Because survey research also is used in many areas of psychology, we carefully cover this method, including how to write a proper questionnaire. Because of the rapid growth of qualitative and mixed methods in psychology, we carefully cover these methods to complement the more traditional methods and to add to each student’s repertoire of research skills. A second overarching goal that has been maintained throughout all editions of the textbook is to present information in a way that is understandable to students. We have attempted to meet this goal by presenting material in as simple and straightforward a manner as possible and by accompanying complex material with illustrations taken from the research literature. We believe that such illustrations not only assist in clarifying the presented material but also bring the material to life when it is placed in the context of actual research studies. This allows the student not only to learn the material but also to see how it is used in a research study.
Overview and Organization of the Textbook
Research Methods, Design, and Analysis is written at the undergraduate level and is intended for use in the undergraduate methods course. The book provides an introduction to all aspects of research methodology, and assumes no prior knowledge. The chapters are divided into seven major parts, as follows:
Part I. Introduction (Chapters 1 and 2)
This section begins with a discussion of knowledge and science in an effort to provide students with an understanding of the nature, goals, and outcomes of science. We believe that most students have an incomplete understanding of science and that they must understand its goals and limitations in order to appreciate and understand the nature of the research process. This is followed by a discussion of
the major types of research used to investigate mind and behavior in an attempt to make sure that the students connect the various research approaches with science. We also discuss the major methods of data collection to help students see how empirical data are obtained.
Part II. Planning the Research Study (Chapters 3 and 4)
In this section, the focus of the book moves to some general topics involved in all research studies. First, we explain how to come up with a research idea, conduct a literature review, and develop a research question and hypothesis. Second, we explain the key ethical issues that must be considered when planning and conducting a research study. We explain the ethical guidelines sanctioned by the American Psychological Association.
Part III. Foundations of Research (Chapters 5 and 6)
In Part III, we cover some concepts that the researcher must understand before critiquing or conducting a research study. We begin with a discussion of measurement. We define measurement, and explain how measurement reliability and validity are obtained. Next, we explain how researchers obtain samples of research participants from targeted and accessible populations. We explain the different methods of random and nonrandom sampling, and we show the important distinction between random selection and random assignment. We also briefly explain the sampling methods used in qualitative research. Next we explain how research validity (i.e., valid results) is obtained. This includes discussions of the major kinds of research validity (internal, external, statistical conclusion, and construct) that must be addressed and maximized in empirical research.
Part IV. Experimental Methods (Chapters 7–11)
Part IV is focused on, perhaps, the most prominent approach to research in psychology and related disciplines (i.e., experimental research). The section includes (a) a chapter explaining the control techniques required to obtain valid research results, (b) a chapter explaining how to select and/or construct a strong experimental research design, (c) a chapter explaining the procedure and details of carrying out an experimental study, (d) a chapter explaining how to select and/or construct a quasi-experimental research design when needed, and (e) a chapter explaining when single-case designs are needed and how to select and/or construct an appropriate single-case design.
Part V. Survey, Qualitative, and Mixed Methods Research (Chapters 12 and 13)
This section includes chapters on additional major research methods used in psychology and related disciplines. First, the student is introduced to the goals, design, and conduct of survey research. The student will also learn how to correctly construct a questionnaire and/or interview protocol to be used in survey research. Second, the book includes a full chapter on qualitative and mixed methods research. The relative
strengths and weaknesses of quantitative, qualitative, and mixed methods research are discussed, the different qualitative and mixed methods approaches and designs are explained, and information is provided about how to conduct a defensible and rigorous qualitative or mixed methods study.
Part VI. Analyzing and Interpreting Data (Chapters 14 and 15)
This section explains descriptive and inferential statistics in a way that is both rigorous and fully accessible to students with no prior background in statistics. The descriptive statistics chapter explains the graphic representation of data, measures of central tendency, measures of variability, measures of relationship between variables, and effect size indicators. Chapter 15, “Inferential Statistics,” explains how researchers obtain estimates of population characteristics based on sample data and how researchers conduct statistical hypothesis testing. In an effort to connect design and analysis, the appropriate statistical tests for the experimental and quasiexperimental research designs covered in earlier chapters are discussed. The student will also learn how to present the results of significance tests using APA style.
Part VII. Writing the Research Report (Chapter 16)
In Part VII we explain the basics of writing a professional, informative, and accurate research manuscript that can be submitted for publication. The guidelines from the latest edition of the Publication Manual of the American Psychological Association are explained in this chapter.
Pedagogical Features
The pedagogical features include concept maps and objectives at the beginning of each chapter. Each chapter highlights important terms and concepts and includes definitions of these in the chapter margins. These terms and concepts are highlighted not only to point out to students that they are important but also to increase the ease with which students can learn these terms and concepts. Study questions are spaced throughout each chapter to help students review the material after they have finished reading a section; this feedback system will assist students in learning the material and assessing whether they understand the material. Each chapter ends with several learning aids. First, a summary of the material, a list of the key terms, and a set of useful Internet sites are provided. Next, to help students access their knowledge of the chapter material, a Practice Test is provided at the end of each chapter. These tests include several multiple choice questions that students can use to assess their knowledge of the chapter material. The Practice Test is followed by a set of Challenge Exercises; these are designed to provide students with exposure to and experiences with activities required in the conduct of a research study.
In addition to the pedagogical aids included in the book, the twelfth edition includes a MySearchLab with eText (www.mysearchlab.com) integrated Web site. MySearchLab contains an eText that students can access anywhere they have an Internet connection, including tablet devices, making it easier for them to study on
the go. Interactive glossary flashcards and practice tests help them prepare for exams. MySearchLab also includes Simulations of classic experiments and research inventories, giving students firsthand experience with common research methodologies. The Simulations anonymously track participant data that can be downloaded by instructors and distributed to students for analysis.
One of the major challenges of a Research Methods course is engaging students in the subject matter and promoting critical thinking. MySearchLab also includes Operation ARA, a critical thinking game developed by Keith Millis, Art Graesser, and Diane Halpern. Operation ARA is a role play game that uses a “save the world” storyline to engage students as they learn scientific thinking and research methods. Students progress through three levels in the game: from Basic Training, where they learn the skills, to the Proving Ground, where they demonstrate their mastery of the skills, to Active Duty, where they must apply their skills to stop the world from certain destructions. A separate Instructor’s Guide is available to adopters. MySearchLab is available for purchase standalone, or it can be packaged at no additional cost with the textbook.
New to the Twelfth Edition
Many minor changes have been made to the twelfth edition to update references, clarify material, and improve the student learning process. The major changes are as follows:
1. Added a new comprehensive MySearchLab with eText so that this book can be used for online, blended, and regular classroom courses.
2. Added audio file for each chapter so students can hear the authors read the chapter at their convenience.
3. Added learning objectives to the beginning of each chapter.
4. In Chapter 4, updated ethical principles to match the new APA guidelines.
5. In Chapter 8, added material on mixed experimental research designs.
6. In Chapter 8, added internal validity tables modeled on the classic work by Campbell and Stanley, 1963 (and updated based on Shadish, Cook, and Campbell, 2002), specifically Table 8.1 Summary of Threats to Internal Validity for Weak Experimental Designs and Table 8.2 Summary of Threats to Internal Validity for Strong Experimental Designs.
7. In Chapter 10, added Table 10.2 Summary of Threats to Internal Validity for QuasiExperimental Designs.
8. In Chapter 13, added a new section on Research Validity in Mixed Methods Research.
Acknowledgments
As with all previous editions, we offer our sincere appreciation and gratitude to our editor Stephen Frail, his editorial assistant Caroline Beimford, the Pearson production team, our students, and all of our external reviewers of past editions of this book.
Cha P ter
Introduction to Scientific Research 1
Introduction to Scientific Research
Introduction to Scientific Research
Learning Objectives
• Explain what knowledge is and how it is obtained.
• Describe the current conception of science and describe its history.
• Understand the basic assumptions underlying scientific research.
• Describe the characteristics of scientific research and understand why each of these is necessary.
• Explain the difference between logic of discovery and logic of justification.
• Describe the characteristics that typify the person who is adept at pursuing scientific research.
• Describe the objectives of scientific research. • Differentiate pseudoscience from scientific research.
Introduction
In our daily lives, we continually encounter problems and questions relating to thoughts and behavior. For example, one person might have a tremendous fear of taking tests. Others might have problems with alcoholism or drug abuse or problems in their marriage. People who encounter such problems typically want to eliminate them, but often need help. Consequently, they seek out professionals, such as psychologists, for help. Likewise, business professionals might enlist the assistance of psychologists in understanding the thinking and behavior of others. For example, salespeople differ greatly in their ability to understand customers and sell merchandise. One car salesperson might be capable of selling twice as many cars as another salesperson. If the sales manager could discover why such differences in ability exist, he or she might be able to develop either better training programs or more effective criteria for selecting the sales force.
In an attempt to gain information about mental processes and behavior, people turn to the field of psychology. As you should know by now, a great deal of knowledge about information processing and the behavior of multiple types of organisms has been accumulated. We have knowledge that enables us to treat problems such as test anxiety and depression. Similarly, we have identified many of the variables influencing persuasion and aggression. Although we know a great deal about mental processes and behavior, there is still much to be learned. In order to learn more about such psychological phenomena, we must engage in scientific research.
The course in which you are now enrolled will provide you with information about conducting scientific research. Some students might feel that understanding research is important only for professional scientists. But, as Table 1.1 reveals, there are many reasons why students should take a research methods course. One reason identified in Table 1.1 is to help students become more informed and critical consumers of information. We are all bombarded by the results of scientific and pseudoscientific research, and we all need tools to interpret what is being reported. For example, saccharin has been demonstrated to cause cancer in laboratory animals, yet there are many people who consume saccharin and do not contract cancer. You as a consumer must be able to resolve these discrepancies in order to decide whether or not you are going to eat foods containing saccharin. Similarly, television commercials often make claims of “scientific proof” regarding the effectiveness of their products. First of all, science does not provide “proof” for general laws; instead, it provides evidence, often very strong evidence. Second, upon closer examination, almost all of the “scientific tests” reported in television commercials would likely be shown to be flawed.
Table 1.1
Reasons for Taking a Research Methods Course
• Learn how to conduct psychological research.
• Provides a foundation for topic-specific courses such as abnormal, social, cognitive, biopsychology, and developmental psychology.
• Can be a more informed and critical consumer of information.
• Helps develop critical and analytical thinking.
• Provides information needed to critically read a research article.
• Necessary for admission into most graduate programs in psychology.
Methods of acquiring Knowledge
There are many procedures by which we obtain information about a given phenomenon or situation. We acquire a great deal of information from the events we experience as we go through life. Experts also provide us with much information. In this chapter, we will briefly discuss four ways by which we acquire knowledge, and then we will discuss the scientific approach to acquiring knowledge. Each of the successive approaches is a more acceptable means of acquiring knowledge. You will also see that although the earlier approaches do not systematically contribute to the accumulation of scientific knowledge, they are used in the scientific process. The scientific approach is a very special hybrid approach to generating and justifying knowledge claims and to accumulating this knowledge over time.
Intuition
Intuition
Intuition occurs when one feels they have direct knowledge or insight but cannot state any observation or reason for the knowledge.
Intuition is the first approach to acquiring knowledge that we examine. Webster’s Third New International Dictionary defines intuition as “the act or process of coming to direct knowledge or certainty without reasoning or inferring.” Such psychics as Edgar Cayce seem to have derived their knowledge from intuition. The predictions and descriptions made by psychics are not based on any known reasoning or inferring process; therefore, such knowledge would appear to be intuitive. Intuition relies on justification such as “it feels true to me” or “I believe this point, although I can’t really tell you why.” The problem with the intuitive approach is that it does not provide a mechanism for separating accurate from inaccurate knowledge. The use of intuition is sometimes used in science (Polanyi & Sen, 2009), and it is probably seen most readily in the process of forming hypotheses. Although most scientific hypotheses are derived from prior research, some hypotheses arise from hunches and new ways of looking at the literature. You might, for example, think that women are better at assessing the quality of a relationship than are men. This belief might have been derived from things others told you, your own experience, or any of a variety of other factors. Somehow you put together prior experience and other sources of information to arrive at this belief. If someone asked you why you held this belief, you probably could not identify the relevant
Authority
A basis for acceptance of information, because it is acquired from a highly respected source
factors—you might instead say it was based on your intuition. From a scientific perspective, this intuition could be molded into a hypothesis and tested. A scientific research study could be designed to determine whether women are better at assessing the quality of a relationship than are men.
authority
Authority as an approach to acquiring knowledge refers to the acceptance of information or facts stated by another person because that person is a highly respected source. For example, on July 4, 1936, the government of the Soviet Union issued a “Decree Against Pedology” (Woodworth & Sheehan, 1964), which, among other things, outlawed the use of standardized tests in schools. Because no one had the right to question such a decree, the need to eliminate standardized tests had to be accepted as fact. The problem with the authority approach is that the information or facts stated by the authority might be inaccurate.
If the authority approach dictates that we accept whatever is decreed, how can this approach be used in science? In the beginning stages of the research process, when the problem is being identified and the hypothesis is being formed, a scientist might consult someone who is considered “the” authority in the area to assess the probability that the hypothesis is one that is testable and addresses an important research question. Virtually every area of endeavor has a leading proponent who is considered the authority or expert on a given topic.
Authority is also used in the design stage of a study. If you are unsure of how to design a study to test a specific variable, you might call someone who is considered an authority in the research area and get his or her input. Similarly, if you have collected data on a given topic and you are not sure how to interpret the data or how they fit with the other data in the field, you might consult with someone who is considered an authority in the area and obtain input. As you can see, the authority approach is used in research. However, an authority is an expert whose facts and information are subject to testing using the scientific process.
Rationalism
The acquisition of knowledge through reasoning
Rationalism
A third approach to gaining knowledge is rationalism This approach uses reasoning to arrive at knowledge and assumes that valid knowledge is acquired if the correct reasoning process is used. During the sixteenth century, rationalism was assumed to be the dominant mode by which one could arrive at truth. In fact, it was believed that knowledge derived from reason was just as valid as, and often superior to, knowledge gained from observation. Its leading advocate was the philosopher René Descartes (1596–1650). Descartes, who famously claimed, “I think, therefore I am,” argued that “clear and distinct ideas” must be true, and from those foundational ideas one should deduce all other beliefs. One danger of relying solely on rationalism for acquiring knowledge is that it is not unusual for two well-meaning and honest individuals to reach different conclusions. This does not mean that science does not use reasoning or rationalism. In fact, reasoning is a vital element in the scientific process. Scientists make use of
Empiricism
The acquisition of knowledge through experience
reasoning not only to derive some hypotheses but also to identify the outcomes that would indicate the truth or falsity of the hypotheses. Mathematics, which is a type of rationalism, is used extensively in many areas of science such as physics. There is also a well-developed line of research in mathematical psychology. In short, rationalism can be very important for science, but by itself it is insufficient.
empiricism
A fourth approach to gaining knowledge is through empiricism. In its naïve form, this approach would say, “If I have experienced something, then it is valid and true.” Therefore, facts that concur with experience are accepted, and those that do not are rejected. This approach was used by some individuals in the 1960s who stated that satanic messages were included on some records. These individuals had played the records backward and had heard messages such as “Oh Satan, move in our voices.” Because these individuals had actually listened to the records and heard the messages, this information seemed to be irrefutable. However, later research indicated that individual expectations influenced what people “heard” (Vokey & Read, 1985). Therefore, naïve empiricism can be problematic; however, empiricism in its more realistic form can be very useful, and, as you will see, it is an important part of the scientific approach.
Empiricism as a systematic and well-developed philosophy is traced to John Locke (1632–1704) and David Hume (1711–1776). These philosophers argued that virtually all knowledge is based on experience. Locke put it well when he claimed that each person is born a tabula rasa (i.e., individuals’ minds are blank slates or tablets upon which the environment writes). The origin of all knowledge is from our senses (sight, hearing, touch, smell, and taste). Our senses imprint ideas in our brains that then are further worked upon (combined, related) through cognitive processes. The early system of psychology known as associationism arose out of empiricist philosophy, and one might view it as the first “school of psychology” (Heidbreder, 1933). Although the empirical approach is very appealing and has much to recommend it, several dangers exist if it is used alone. Our perceptions are affected by a number of variables. Research has demonstrated that such variables as past experiences and our motivations at the time of perceiving can drastically alter what we see. Research has also revealed that our memory for events does not remain constant. Not only do we tend to forget things, but at times an actual distortion of memory might take place.
Empiricism is a vital element in science, but in science, empirical observations must be conducted under controlled conditions and systematic strategies must be used to minimize researcher bias and to maximize objectivity. The later chapters in this book will carefully explain how to carry out empirical research that is scientific and, therefore, reliable and trustworthy.
Science
Science
The most trustworthy way of acquiring reliable and valid knowledge about the natural world
The word science had its ancient origins in the Latin verb scire, meaning “to know.” However, the English word “science,” with its current meaning, was not coined until the nineteenth century by William Whewell (1794–1866). Before that time, scientists were called “natural philosophers” (Yeo, 2003). Science is a very important way of acquiring knowledge. Although it is a hybrid of the forms discussed earlier, it is superior in the sense that it is designed to systematically produce reliable and valid knowledge about the natural world. One might think that there is only one method by which scientific knowledge is acquired. While this is a logical thought, Proctor and Capaldi (2001) have pointed out that different scientific methods have been popular at different points in time. That’s because science continues to develop and improve all the time. We now take a brief historical tour of scientific methods.
Induction and deduction
Induction
A reasoning process that involves going from the specific to the general
As classically defined by Aristotle (384–322 BCE), induction is a reasoning process that involves going from the specific to the general.1 For example, if on a visit to a daycare center you see several children hitting and kicking other children, you might infer that many children in that center are aggressive or even infer that children in daycare centers across the country tend to be aggressive. This inference is an example of induction, because you moved from the particular observations to a much broader and general claim. Induction was the dominant scientific method used from the late seventeenth century to about the middle of the nineteenth century (Proctor & Capaldi, 2001). It was during this time that scientific advances were made by careful observation of phenomena with the intent to arrive at correct generalizations. Both Francis Bacon (1561–1626) and Isaac Newton (1642–1727) advocated this approach. Newton, for example, has stated that “principles deduced from phenomena and made general by induction, represent (italics ours) the highest evidence that a proposition can have .” (Thayer, 1953, p. 6).
Induction is still used very frequently in science. For example, Latané (1981) observed that people do not exert as much effort in a group as they do when working alone and inferred that this represented the construct of social loafing. When Latané made this generalization of social loafing from the specific observation that less effort was expended in a group, he was engaged in inductive reasoning. Inductive reasoning is also seen in the use of statistical analysis in psychological research. When researchers rely on samples and generalize to populations, they are using inductive reasoning. Inductive reasoning is, therefore, an integral part of science. It is not, however, the only reasoning process used in science. Deductive reasoning is also used.
1In the philosophy of logic, induction and deduction have slightly different meanings from what is presented here. In philosophy of logic, inductive reasoning refers to drawing of a conclusion that is probably true, and valid deductive reasoning refers to the drawing of a conclusion that is necessarily true if the premises are true (Copi & Cohen, 2005).
Deduction
A reasoning process that involves going from the general to the specific
Hypothesis testing
The process of testing a predicted relationship or hypothesis by making observations and then comparing the observed facts with the hypothesis or predicted relationship
Deduction, as classically defined by Aristotle, refers to going from the general to the specific. For example, Levine (2000) predicted that a person who views the group’s task as important and does not expect others to contribute adequately to the group’s performance will work harder. Here, Levine was logically moving from the general proposition of social loafing and deducing a specific set of events that would reduce social loafing. Specifically, Levine deduced that viewing the group’s task as important and not expecting others to contribute adequately would cause a person to work harder or counter the social loafing effect. Today, when researchers develop hypotheses, they routinely deduce the observable consequences that must occur if they are going to claim (after collecting data) that the hypothesis is supported or not supported.
In sum, science makes use of both inductive and deductive thinking. However, neither of these approaches is the only or primary approach to current science.
Hypothesis Testing
Hypothesis testing refers to a process by which an investigator formulates a hypothesis to explain some phenomenon that has been observed and then compares the hypothesis with the facts. Around 1850, induction was considered to be inadequate for the task of creating good scientific theories. Scientists and philosophers suggested that hypothesis testing should be formally added to the scientific method (Proctor & Capaldi, 2001). According to Whewell (1847/1967), “The process of scientific discovery is cautious and rigorous, not by abstaining from hypothesis, but by rigorously comparing hypothesis with facts, and by resolutely rejecting all which the comparison does not confirm” (p. 468). According to this approach, scientific activity involves the testing of hypotheses derived from theory or experience. Whewell suggested that science should focus on the confirmation of predictions derived from theory and experience.
Proctor and Capaldi (2001) argue that the era of hypothesis testing extended from approximately 1850 to about 1960. However, an examination of the psychological research literature shows that hypothesis testing has been, and still is, a very important part of scientific activity in psychology. For example, Fuller Luck, McMahon, and Gold (2005) investigated cognitive impairments in schizophrenic patients. They hypothesized that schizophrenics’ working memory representation would be abnormally fragile, making them prone to being disrupted by distracting stimuli. They then designed a study to collect data that would test the adequacy of this hypothesis.
Logical positivism
A philosophical approach that focused on verifying hypotheses as the key criterion of science
Hypothesis testing as a scientific methodology was associated with the logical positivist movement. Logical positivism was the outgrowth of a group of scholars at the University of Vienna with a scientific background and a philosophical bent. This group became known as the Vienna Circle and the group’s viewpoint was called logical positivism (Miller, 1999). One of the central views of the Vienna Circle was that a statement is meaningful only when it is verifiable by observation or experience. Logical positivists believed that the most important aspect of science was the verification of hypotheses by objective observation or experience. Logical positivist Moritz Schlick (1882–1936) said in 1934, “Science
Falsificationism
A deductive approach to science that focuses on falsifying hypotheses as the key criterion of science
makes prophecies that are tested by ‘experience’ ” (in Ayer, 1959, p. 221). The logical positivists ultimately hoped to show that the natural world followed universal scientific laws.
Although logical positivism had many supporters, it was also criticized. One of the most severe critics was the philosopher of science Karl Popper (1902–1994). Popper pointed out that the verification approach of the logical positivists was based on a logical fallacy (known as affirming the consequent). To fix this “error,” Popper argued that science should rest on a deductively valid form of reasoning (1968). One can claim conclusively using deductive reasoning that a general law is falsified if the data do not support the hypothesis, and this deductively valid approach is what Popper advocated. He argued that science should focus on stating bold hypotheses followed by attempts to falsify them. Popper’s approach is known as falsificationism
Duhem–Quine principle
States that a hypothesis cannot be tested in isolation from other assumptions
Naturalism
Position popular in behavioral science stating that science should justify its practices according to how well they work rather than according to philosophical arguments
Empirical adequacy
Present when theories and hypotheses closely fit empirical evidence
A major strength of Popper’s approach is that it helps eliminate false theories from science. However, Popper’s approach also was criticized because it focused only on falsification and completely rejected induction. Popper stated “There is no induction; we never argue from facts to theories, unless by way of refutation or ‘falsification’ ” (Popper, 1974, p. 68). Unfortunately for Popper, induction is required in order to claim what theories are best supported and to what degree, and, therefore, what theories we should believe. Popper’s approach was also criticized because even if the data appear to falsify a hypothesis, one still cannot conclude that the theory is necessarily false. That’s because you have to make many assumptions during the hypothesis testing process, and one of those assumptions, rather than the hypothesis, might have been false. This idea that a hypothesis cannot be tested in isolation (i.e., without making additional assumptions) is called the Duhem–Quine principle.
A key point is that psychologists today rely on a hybrid approach to hypothesis testing that includes probabilistic thinking, preponderance of evidence, and a mixture of the logical positivists’ verification approach and Popper’s falsification approach. It is important to remember that hypothesis testing produces evidence but does not provide proof of psychological principles.
naturalism
Since the 1960s we have entered a methodological era in science that has evolved from a movement in the philosophy of science called naturalism (Proctor & Capaldi, 2001). Naturalism rejects what is called foundational epistemology, which assumes that knowledge is a matter of deductive reasoning and that knowledge is fully certain, much like a mathematical or geometrical proof. Instead, naturalism takes the position that science should be studied and evaluated empirically, just like a science studies any other empirical phenomenon. Naturalism is a pragmatic philosophy of science that says scientists should believe what is shown to work. When it comes to judging scientific beliefs, naturalism says we should continually evaluate our theories based on their empirical adequacy. That is, do the empirical data support the theory, does the theory make accurate predictions, and does the theory provide a good causal explanation of the phenomenon that you are studying?
Normal science
The period in which scientific activity is governed and directed by a single paradigm
Paradigm
A framework of thought or beliefs by which reality is interpreted Revolutionary science
A period in which scientific activity is characterized by the replacement of one paradigm with another
If you look at the history of science, you can see that approaches to science can change over time. Science uses many approaches that have been shown to be helpful to the advancement of valid and reliable knowledge. Naturalism takes a practical approach to methods and strategies. Next we briefly mention some historical influences since about 1960 that were precursors to today’s scientific naturalism.
Kuhn and Paradigms Thomas Kuhn (1922–1996) conducted a historical analysis of science and, in 1962, published his famous book The Structure of Scientific Revolutions. His research suggested that science reflects two types of activities: normal science and revolutionary science. Normal science is governed by a single paradigm or a set of concepts, values, perceptions, and practices shared by a community that forms a particular view of reality. A paradigm, therefore, is a framework of thought or beliefs by which you interpret reality. Mature sciences spend most of their time in “normal science.” However, over time anomalies and criticisms develop, and revolutionary science occurs. During this more brief period (compared to normal science), the old paradigm is replaced by a new paradigm. Replacement of one paradigm with another is a significant event because the belief system that governs the current view of reality is replaced with a new set of beliefs. After a revolutionary period, science enters a new period of normal science, and this process, according to Kuhn, has continued throughout history.
A development within the field of psychology of learning provides an example of what Kuhn would have called paradigms. In the early 1930s, a mechanistic paradigm had developed in the psychology of learning. The basic set of concepts and beliefs or the fundamental principle of this mechanistic view was that learning is achieved through the conditioning and extinction of specific stimulus–response pairs. The organism is reactive in that learning occurs as a result of the application of an external force known as a reinforcer.
A competing paradigm at this time was an organismic paradigm. The basic set of concepts and beliefs or the fundamental principles of the organismic view were that learning is achieved through the testing of rules or hypotheses and organisms are active rather than reactive. Change or learning occurs by some internal transformation such as would be advocated by Gestalt theory, information processing, or cognitive psychology (Gholson & Barker, 1985). Piaget’s theory of child development is an example of the organismic view. Other paradigms or research traditions (Laudan, 1977) in psychology include associationism, behaviorism, cognitive psychology, and neuropsychology.
Feyerabend’s Anarchistic Theory of Science Paul Feyerabend (1924–1994) was a philosopher of science who looked at the various methodological approaches to science that had been advocated and was not surprised to see that each had been criticized and was lacking. For example, both the verification approach advocated by the logical positivists and the falsification approach advocated by Popper floundered because of the logical problems mentioned earlier. As a result of the failure to identify any single distinguishing characteristic of science, Feyerabend (1975) argued that there is no such thing as the method of science. According to him, science has
many methods. Most psychologists would argue, however, that Feyerabend went too far when he claimed that the single unchanging principle of scientific method is that “anything goes.” Feyerabend also argued that science included many irrational practices and was partially the result of the operation of power. He concluded that scientific knowledge was not nearly as secure as scientists would have the public believe. As you can see, Feyerabend offered a relatively severe critique of normal science. Perhaps the key conclusion to draw from his critique is that science might not be as simple and formulaic as it sometimes is made to appear. In short, it is true that scientific practice includes many complexities. Nonetheless, in this book, we will do our best to explain some of the complexities and provide a clear explanation of the current best practices in psychological research.
What exactly Is Science?
Philosophers have, for many years, been trying to provide an exact demarcation of science from nonscience. The logical positivists had hoped verificationism would be the criterion. They also hoped a single, universal method could be identified. Popper claimed the criterion was falsificationism (i.e., attempting to falsify hypotheses and determine which ones remain). For Kuhn, it was the values, interactions, technical language, key concepts, and activities of scientists that identified science. Some philosophers of science seek a relatively secure basis for science in experimentation or what Robert Ackermann (1989) calls “the new experimentalism.” According to this approach, experimentation can have a life of its own independent of theory, and scientific progress is seen as the steady buildup of experimental knowledge (Chalmers, 1999) or knowledge acquired from experimentation. In many ways, the experiment is the strongest and best of the scientific methods. It is probably better to conclude, however, that the multiple methods and practices used by successful scientists can contribute in complementary ways to the development of secure scientific knowledge.
Scientists must be skeptical, creative, and systematic. They must identify problems, question current solutions that are not working, creatively and systematically come up with new solutions, and, most importantly, subject these new solutions to empirical testing. When researchers subject important beliefs, observations, hypotheses, and claims of authority figures to repeated empirical testing, they will obtain the most reliable and valid knowledge possible.
Still, one needs a working definition of science. According to Chalmers, “a science will consist of some specific aims to arrive at knowledge of some specific kind, methods for arriving at those aims together with the standards for judging the extent to which they have been met, and specific facts and theories that represent the current state of play as far as the realization of the aim is concerned” (Chalmers, 1999, p. 168). This is consistent with our view of science as the preferred way of acquiring reliable, valid, and practical knowledge about the natural world, but to continue to be successful, it must always conduct research ethically, must critically self-examine its practices to determine what is working and what is not working, and must engage in ongoing learning and improvement. If science does this, scientific knowledge also will continue to advance.
• What is science, and how have the methods of science changed over time?
• What is the difference between induction and deduction?
• What is naturalism?
• What is Kuhn’s approach to science?
• Why has Feyerabend argued that there is no such thing as a method of science?
basic assumptions underlying Scientific Research
In order for scientists to have confidence in the capacity of scientific research to achieve solutions to questions and problems, they make several working assumptions so that they can get on with the day-to-day practice of science.
uniformity or Regularity in nature
Determinism
The belief that mental processes and behaviors are fully caused by prior natural factors
Probabilistic causes
A weaker form of determinism that indicates regularities that usually but not always occur
Reality in nature
The assumption that the things we see, hear, feel, smell, and taste are real
Science searches for regularities in nature. If there were no uniformity or regularity, science would only amount to a historical description of unrelated facts. B. F. Skinner (1904–1990) put it well when he stated that science is “a search for order, for uniformities, for lawful relations among the events in nature” (1953, p. 13). If there were no uniformity in nature, there could be no understanding, explanation, or knowledge about nature. Without regularity, we could not develop theories or laws or generalizations. Implicit in the assumption of uniformity is the notion of a rather strong form of determinism—the belief that there are causes, or determinants, of mental processes and behavior. In our efforts to uncover the laws of psychology, we attempt to identify the variables that are linked together. What we have found thus far are probabilistic causes (i.e., causes that usually produce outcomes), but the search for more certain, fuller, and often more complex causation will continue. You should construct experiments in your attempt to establish the determinants of events. Once you have determined the events or conditions that usually produce a given outcome, you have uncovered probabilistic causes.
Reality in nature
A related assumption is that there is reality in nature. For example, as you go through your daily lives you see, hear, feel, smell, and taste things that are real, and these experiences are real. We assume that other people, objects, or social events like marriage or divorce are not just creations of our imagination, and we assume that many different types of “objects” can be studied scientifically. Stating that something is true or real “because we said it is real” does not work in science. In science, researchers check reality in many ways to obtain objective evidence that what is claimed is true. In short, researchers interact with a natural world (that includes social objects such as attitudes, beliefs, institutions), and, in science, this reality must have primary say in our claims about reality and truth. This is why we collect data. Again, scientists make the assumption that there is an underlying reality, and they attempt to uncover this reality.
