Today we will look at the differences between experimental and nonexperimental methods and find out why scientists prefer to avoid nonexperimental methods where possible.
REMEMBER, ANOTHER WORD FOR NONEXPERIMENTAL IS NON-SCIENTIFIC
Some non-experimental approaches to psychology have led to errors and abuse
Insulin Shock Therapy
The 1942 study, carried out at the SAME Hospital,
had an improvement rate of 63%, with 42% of the patients
still well after two years of follow-up.
The rate is still not "more than 70%" and is surprisingly close to the 63.5% found in the first study.
They used a rating scale for "not improved, slightly improved, etc." Does this give you any insight into the "scientific" value of rating scales?
Famous People who have undergone ECT
In a 1954 follow-up, other researchers could not
find improvement in Bender's children: "In a number
of cases, parents have told the writers that the children
were definitely worse."
In spite of this, today's researchers interpret Bender's study as evidence that shock works, at least temporarily.
New studies are again reporting great success.
A UCLA study had 100% success in nine adolescents.
The Mayo Clinic found 65% were better.
At Sunnybrook Hospital in Toronto, 14 who received shock spent 56% less time in the hospital than six who refused the treatment.
In addition to the history of abuse(and continued abuse)
Look at the following example.
Published in InfoWorld August 6, 2001
There is an opinion that the right brain is something of a parallel processor, integrating information far faster but just as reliably as the left brain's linear approach to thinking.
Tain't so. Want proof (or at least strong evidence)? Here's a quick experiment: Consider a chess game between equally matched opponents. Give one player five seconds to make each move -- plenty of time to use his or her judgment but not much time for logical analysis. Give the other player five minutes per move, so that he or she has time to think. Assuming the players are evenly matched, is there any doubt who will win?
This is strong evidence or proof that the right brain is the sole operator during the 5 seconds per move and the left brain is incapacitated and contributes nothing.
EXPERTISE or AUTHORITY
Are experts or authorities objective presenters of scientific truth or do they perhaps have idiosyncrasies or biases in their observations?
Look at the following example.
Correlation Between Bipolar Disorder and Reactive
Attachment Disorder by John F. Alston, M.D.
Historically, mental health professionals have long associated Attention Deficit Disorder with Reactive Attachment Disorder. Experts have put this correlation of ADD and RAD at between 40% and 70%.
In my experience as psychiatric consultant to
the Attachment Center since 1977, as well as within
my own private practice and consultations with other
attachment programs and adoption agencies in which
I supervise psychotherapists who work with attachment
disorders, I have come to realize that correlations
between Bipolar Disorder and Reactive Attachment Disorder
are indeed much more common than believed and the correlation
between ADD and RAD is much less common than believed.
Additionally, In my experience, this miscorrelation between ADD and RAD is international. This conclusion has led to different, and in my experience, much more effective medical treatment plans for these children.
Brain hemisphericity and academic majors: a correlation study by Amany Saleh
The results appear to confirm that students choose to study subjects that accommodate their cognitive/learning styles.
The knowledge from this study is vital to educators since the literature on learning styles in the last three decades shows that teachers cater to only one learning style that correlates only with left hemispheric skills.
Personal Experience & Testimonials
BUT do we always interpret experiences correctly?
Look at the following example.
This is the story of a man who had cancer and
was told by his doctor that the two months of radiation
therapy he had been getting had done no good and that
he had two months to live.
Nine months later, the man is "alive and well" because a spiritual healer treated his cancer with Therapeutic Touch (TT).
Here is how TT works.
The healer directs his energy to the patient's body without touching it. In this case, the healer would place his hands over the man's chest and move them in a slow motion around his lungs for about 30 minutes.
What are the alternatives to the TT explanation?
1) The doctor was wrong in his prediction of how
long his patient had to live.
2) Maybe the man misunderstood the doctor, or the doctor may have qualified his prediction, such as, "my best guess is..." or "based on similar cases, I would estimate...."
3) It is possible that the doctor was wrong about the effectiveness of the radiation therapy and that it had worked better than he or she thought.
4) It is possible the cancer went into spontaneous remission.
5) It is possible the man was misdiagnosed and mistreated and he's alive only because his doctors gave him up for dead.
All we know for sure is that the man lived longer than a doctor predicted and that both radiation therapy and TT were administered.
This is where nonexperimental and experimental methods clash.
Look at the following example of interpreting data.
A careful analysis of traffic accidents showed that about 70% of car accidents happen when the driver is within 10 miles of home. This was interpreted to mean that people are careless when they are close to home (familiarity breeds complacency).
While the statistic is true, the interpretation is not. The reason why most people have accidents close to home, is that most of the time, they are close to home.
Look at the following example of two different points of view seeing the same data differently.
At the University of California, women sued the
university claiming they were being discriminated against
by the graduate school.
In countering this suit, administrators showed that there was actually a positive bias for women.
It turns out that BOTH the women and the administrators were right (depending on how you look at the data).
Suppose there were only two departments in the
graduate school, economics and psychology.
70 of 100 men (70 percent) who applied to the economics department were admitted
15 of 20 women (75 percent) who applied to the economics department were admitted
5 out of 20 men (25 percent) who applied to the psychology department were admitted
35 of 100 women (35 percent) who applied to the psychology department were admitted
(a)The administration noted that a higher percentage of women were admitted (75 to 70 and 35 to 25)
(a)However, overall, 75 of the 120 male applicants (62.5 percent) were admitted whereas 50 of the 120 female applicants (41.7 percent) were admitted.
For example, one researcher hypothesized that
men's financial stability would be preferred more by
women than women's financial stability would be preferred
by men. This was believed to be so because previous
research had shown this relationship.
However, when the data didn't come out supporting the hypothesis, the researcher decided that it actually did and reported ñ "though my hypothesis was not fully supported it was marginally significant."
"Marginal" or "nearly significant"
statistical outcomes (which up to now have been called
"non-significant") are a big controversy
in psychology among the experimentalists versus the
One faculty member at RIT has taught the idea "near significance", in classes, suggesting that we should be using the term "marginally significant" because it advances science.
The American Psychological Association (APA) disagrees, but what do they know?
Look at this example.
A computer generated a randomized 100-item list
of the digits 1, 2, 3, 4, and 5. 179 student participated.
Each student received a list in a sealed envelope and
had to guess which number was in each of the positions.
By chance, 20 guesses out of the 100, should be correct.
When told beforehand that ESP was beneficial, Ss averaged 20.66 correct out of 100. When told that ESP was harmful, they averaged only 19.49 correct.
This was an experimental method, but no journal would publish this study without replication.
Similar nonexperimental studies HAVE been published.
Therefore the researchers repeated their experiment.
The result of the replication showed NO statistically significant effects were present.
Therefore, the misinformation was not published as indicating a truth about ESP.
This is the value of using experimental vs nonexperimental methods.
Surveys are among the most inaccurate and poorly constructed research tools in use and are unlikely to get any worthwhile data just because of the way they are constructed.
Even given a perfectly constructed survey, there
is no reason to believe any data obtained from surveys
because people lie on them.
One reason they lie on surveys is because they are encouraged to lie on surveys.
Wired Magazine, DC Newspress, and other sources are pointing out that surveys are:
getting not only demographic data from you but also information on personal habits, tastes and opinions. This information is stored, processed, analyzed and traded.
Their advice is, if you are not already lying on surveys, please start.
Even without being encouraged to intentionally
lie, many people are liars to begin with. A Christian
survey found the following results.
How honest would you say you are:
1% lie constantly
4% mostly lie
27% lie a lot
57% lie sometimes
11% never lie
By these numbers shouldn't many of them have been lying on this Christian lie survey?
Another survey found the following:
If you could live in a world where everyone told the truth at all time, would you want to
65% said no.
These results probably aren't accurate either, so do we really know anything based on this result (other than we can't trust survey results).
Then there are lots of topics people just don't
seem to be honest about.
Sex is one of those issues.
The evidence is that people just don't tell the truth about their sexual practices on surveys.
For example, in most surveys the average number of sex partners reported by heterosexual males is significantly greater than the average number reported by heterosexual females.
The number of men reporting impotence on surveys is typically less than 10%, while the number of men surveyed who say they want Viagra to treat impotence is much higher.
Additionally, there is evidence that people don't
tell the truth even on surveys as innocuous as ones
on midlife satisfaction.
This is what was found:
The responses to the MacArthur Foundation study on midlife satisfaction didn't compare with other response measures. For example, self-protective and jocular responses like "Oh, we're happy and everything's fine" were frequent, while more serious responses like -- "My job is often boring" were infrequent.
The evidence is that colleges don't tell
the truth on surveys
In Money magazine's 1994 college guide, New College of the University of South Florida was ranked No. 1 overall. Listed among the school's strengths was the freshman class's average SAT score -- an impressive 1296. This placed New College among the most selective schools in the nation.
The score - as well as the pretense of exclusivity
- was false.
For years, the Sarasota-based school concedes, it deliberately inflated its SAT scores by lopping off the bottom-scoring 6% of students, thereby lifting the average about 40 points. Admission Director David Anderson describes the practice, which he says he recently discontinued, as part of the college's "marketing strategy."
Even when people are trying to be honest on surveys, there is the problem that people don't really know themselves and tend to overestimate their worth and abilities.
Take a look at a couple of examples
Ninety-four percent of university professors think
they are better at their jobs than their colleagues.
Twenty-five percent of college students believe they are in the top 1% in terms of their ability to get along with others.
Seventy percent of college students think they are above average in leadership ability. Only two percent think they are below average.
Additionally, surveys are often reported in ways that make them seem better than experimental data.
Consider this psychotherapy survey
A Consumer Reports nonexperimental survey (Consumer Reports. Mental health: Does therapy help? 1995, November, pp. 734-739) of 2900 readers who had received psychotherapy concluded that patients report substantial benefit from psychotherapy.
This is in spite of the fact that over 500 scientific studies show psychotherapists' credentials and experience are not related to patient improvement from counseling.
Experimental evidence tells us that psychologists
and psychiatrists are no better than intelligent, minimally
trained people at diagnosing mental problems or predicting
which people will have problems
Psychologists and psychiatrists with 30 or 40 years of experience are no better at helping people than are intelligent, minimally trained lay people.
Self-help groups often do as well as professional psychologists, and many members of the clergy do good counseling with little or no training.
Friends do just as well as professional therapists in both the long and short run.
Katie argues that -- We all know that serial killers
don't wake up one morning and say, "Today I think
I will kill someone." Something has to lead to
their fascination of suffering and death.
She claims -- Studies show that there is a definite cycle of abuse that starts with animals and leads to humans. Here is her evidence:
The FBI has found that a history of cruelty to animals is one of the traits that regularly appears in its computer records of serial rapists and murderers (Goleman, 1991).
The future killer's childhood centration on violence will lead to an adulthood violence-focus" (Anderson, 1994). (correlation/causation)
One confessed he had killed so many cats he'd lost count (PETA, 1998).
"If an animal is being neglected in a house, there's real good proof that a child is probably being neglected too" (Fisher, 1997).
One in four women admit to staying in the abusive home because of their pets and 57 percent of the total women admitted that their partner had harmed or killed their pet (Rizk, 1997)
Katie Linnes CLAIMS that -- These gruesome examples
obviously show a direct link between animal abuse and
The cycle of abuse can potentially be prevented during childhood, as long as the signs are caught early. Schools, parents, communities, and courts who shrug off animal abuse as a "minor" crime are ignoring a time bomb (PETA, 1998). Instead, they should be aggressively punished and counseled.
EXPERIMENTAL EVIDENCE SHOWS
However, not all cancers rely on angiogenesis.
Sharks do get cancer, even cancer of their cartilage.
Experimental evidence does not support the efficacy of shark cartilage in treating cancer.
Do you believe the nonexperimental evidence, the experimental evidence or both?
As you can hopefully see, there is a lot of garbage
out there masquerading as scientific research.
You can get anything published if you just work hard enough at it.
Look at the following example.
The study used 27 waitresses from the Elite Diner
and Merry Ann's Diner. The waitresses had strong opinions
about what different types of soup-eaters were like.
Wansink then conducted a random telephone survey across the 50 states, assessing people's opinion of 12 common soup products. The survey included adults over 18 with 602 women and 401 men.
Wansink took the people with strong preferences and then went back to the waitresses, asking which profile went with which soup. He said the waitresses found the task easy.
He gives an example of Lisa Moreno, a waitress
at the Elite Diner in Urbana.
She has eight years of waitressing experience and said she often has an idea of what people want when they come in.
When asked about a typical tomato-soup eater, Moreno said it will probably be a woman who is middle-aged or older
In the experimental method, psychologists manipulate
variables and observe how subjects respond to the manipulation.
This manipulation of variables allows the establishment of cause and effect.
Nonexperimental methods cannot establish cause and effect.
In the experimental method, psychologists apply the manipulation to one group (the experimental group) while another group does not receive the manipulation (the control group). This allows the psychologist to know if the manipulation actually causes some effect.
The nonexperimental method typically examines relations between variables. Thus, groups are often left as they are and are merely observed.
In the experimental method, participants are randomly assigned to all groups. Thus all groups are considered equivalent.
The nonexperimental method uses already self-assigned groups, thus the groups cannot be considered to be equivalent. The outcome is that biases can make the results meaningless.
In the experimental method, all variables are controlled. This means that opportunity for biases and error are basically eliminated.
In the nonexperimental method, the relations between variables usually already exist, there is no manipulation or control of variables, thus biases and errors are not eliminated.
The nonexperimental method is considered to be "non-scientific".
1) Experimental methods can only be used when
it is practical and ethical for the researcher to manipulate
For example, a psychologist might want to know if parent's method of disciplining their children has an effect on how their children behave.
It would be neither practical nor ethical to make parent's discipline their children in a certain way just to see if that effects their child's behavior.
2) Experimental studies are usually done in the
highly controlled setting of the laboratory.
These conditions are artificial and may not reflect what really happens in the less controlled and infinitely more complex real world
3) Experimental methods can only be used when
it is possible to manipulate variables.
For example, if you want to study deaf/hearing, male/female, psychologist/engineer, etc. differences, then you couldn't use the experimental method.
You can't randomly assign someone to be deaf or hearing or male or female or a psychologist or an engineer.
FINALLY: No research (including Experimental Research) proves anything.
A scientific proof is not known with absolute
A controlled study can never be completely controlled-there are just too many possible variables. Findings cannot be known in the sense of known beyond any logical or conceivable doubt.
The best that science can offer us is that we can distinguish between what is "conceivably" true and what is "reasonably" true.
DON'T USE THE WORD "PROVE" IN ANY OF YOUR PSYCHOLOGY WRITE-UPS.
Go over assignment