7. Fact or fallacy

15BRODY-superJumbo.jpg
source

Introduction 

We live in a world where the line between fact and fallacy can often be blurred. This can happen due to scientific results being accidentally or deliberate poorly communicated and misrepresented. I think that of all the topics this is the one that underpins the rationale of this course, scientific literacy. It is more than regurgitating facts but rather being able to use knowledge to critically evaluate claims made by others and organisations. The links below will hopefully stimulate discussion and could be used as part of a depth study or an extended response question.

Glossary 

Click on this link to see the glossary using the Quizlet app.

Testing Claims

Inquiry question 1: How can a claim be tested?

A.Plan and conduct an investigation based on testing a claim, and consider:

  • the validity of the experimental design
  • reliability of the data obtained
  • the accuracy of the procedure, including random and systematic error

B.Using examples, evaluate the impact that sample selection and sample sizes can have on the results of an investigation

  • sample selection
  • sample size 

Some links below regarding sampling

C. Compare emotive advertising with evidence-based claims, including but not limited to:

  • health claims on food packaging
  • claims about the efficacy of a product

Some interesting articles to get a discussion going

Impacts on Investigations

Inquiry question 2: What factors can affect the way data can be interpreted, analysed and understood?

A.Using examples, justify the use of placebos, double-blind trials and control groups in order to draw valid conclusions

Background information and definitions

1.placebos and valid conclusions

In medical studies, a treatment, medicine, or therapy given to study participants that is known to have no therapeutic effect on the condition of interest. Placebos are used in clinical trials because they help control an important variable: whether or not the participants know that they are receiving treatment for a condition. With a placebo, neither the participants receiving an experimental treatment and nor those in the control group know whether they are receiving the treatment. source

2.double-blind trials and valid conclusions

An experiment designed such that neither the participants nor the researchers observing them know which participants are in the experimental and control groups until after the observations are complete. Double-blind experiments are particularly important in the field of medicine because they control for both the placebo effect and unconscious bias on the part of the researchers — two factors that can make the results of a medical study. Source

3.control groups and valid conclusions 

In scientific testing, a group of individuals or cases matched to an experimental group and treated in the same way as that group, but which is not exposed to the experimental treatment or factor that the experimental group is. Control groups are especially important in medical studies in order to separate placebo effects from outcomes of interest. Control groups are sometimes also called control treatments or simply controls. This can be confusing since this use of the term is slightly different from what we mean when talking about controlled variables. source

B.Evaluate the impact of societal and economic influences on the collection and interpretation of data, including but not limited to:

1)Predicting variations in climate

2)Suggesting remedies for health conditions

3)Manipulating statistical data

Misuse of statistics, including causes and examples

Further general background information that covers more examples including the ones above.

Evidence-based Analysis

Inquiry question 3: What type of evidence is needed to draw valid conclusions?

Evaluate how evidence of a correlation can be misinterpreted as causation, including but not limited to:

  1. The Hawthorne effect
  • A youtube video that outlines the history, uses and limitations of the effect when viewing results in social experiments.
  • The Hawthorn effect

2.1991 study that linked hormone replacement therapy to coronary heart disease

3.The Mozart Effect on child development

General information about correlation and causation

Whenever a study purports to show causality, it’s important to ask questions before simply accepting the conclusion as fact. Was the study observational or experimental? Was it even a formal study, or just anecdotal? What have other studies shown? Is there any bias in the findings? Correlation CAN indicate causality, but by itself is not enough.

Reading Between the Lines

Inquiry question 4: How does the reporting of science influence the general public’s understanding of the subject?

A.Examine a contemporary scientific debate and how it is portrayed in the mainstream media, including but not limited to:

Some contemporary scientific debates examples 

General links to the validity of scientific information and the media

B.Evaluate the use and interpretation of the terms theory, hypothesis, belief and law in relation to media reporting of scientific developments

C. Compare the difference in reporting between a peer-reviewed journal article and a scientific article published in popular media

D.Analyse how conflicts of interest can result in scientific evidence being suppressed, misinterpreted or misrepresented and discuss measures to counteract such conflicts, including but not limited to:

The tobacco industry and lung cancer

Fossil fuel industry and climate change

Commercial industries researching products for market

Asbestos mining and lung cancer

General links relating to conflict of interest in communicating scientific results and ways this can be mitigated.

E.Describe the halo effect and, using examples, explain how the influence of positive perceptions can result in the rejection of valid alternative perspectives, including but not limited to:

1. Celebrities endorsing products or viewpoints

2. Popular brand companies making misleading advertising claims

General articles about the Halo effect

  • What is the Halo effect?
  • Unconscious bias, the Halo effect.
  • This video lesson talks about threats to internal validity in investigations. Internal validity has to do with the investigations ability to actually show that the independent variable is having an actual effect on the measured or dependent variable and is not been subjected to bias, such as the Halo effect. The sections in the Halo effect is about 5 minutes in however, the whole video is very informative and also includes some previous sections in this Module including the Hawthorn effect and placebo effect.

F. Using examples, analyse a pseudo-scientific claim and how scientific language and processes can be manipulated to sway public opinion, including but not limited to:

1. astrology

2. numerology

Criticisms of numerology

3.iridology

 

General information

Science as Self-correcting – the Issues

Inquiry Question 5: Can the scientific community and the process of peer review find ‘the truth’?

Listen to this podcast as an introduction Science is supposed to be an unbiased way to uncover nature’s secrets. Through blinded experiments, rigorous peer review and replication—we’ve been told that by using the scientific method we’ll find trustworthy facts. But, with many scientific findings largely regarded as ‘wrong’, is science broken? To find out, science journalist Wendy Zukerman speaks to Dr. Neil deGrasse Tyson, Prof. Ivan Oransky, Prof. Barton Zwiebach, Ass. Prof. Alex Holcombe and Dr Alice Williamson.

A.Conduct an investigation using secondary sources to research a scientist who has falsified their scientific experimental results, and discuss the process used to uncover the fraudulent research

Some examples and articles below including many examples from all over the world.

B. Analyse the scientific debate surrounding ‘publication’ and discuss the implications of scientists’ need to ‘publish or perish’

  • The forensic implications of predatory publishing. Predatory and unethical publishing on such a global scale is a relatively recent phenomenon designed to bypass normal peer-review, to make money for the “publishers” and to create instant CV’s for authors.

C.Evaluate the increasing volume of scientific papers being published and assess the feasibility of science to effectively manage, review, replicate and validate investigations, for example: