The singularity—the point at which machines will surpass humanity in intelligence—is near. Technology “will provide tools to effectively combat poverty, clean up our environment, overcome disease, extend human longevity, and undertake many other worthwhile activities”. But beware! It “can also be used to amplify and enhance our destructive side”. It could spell the end of humanity.
The lines above appear to refer to artificial intelligence; however, they are taken from the 2005 book The Singularity Is Near, by Raymond Kurzweil, now a Google advisor, and were actually discussing nanotechnology. This field, like so many others before and since, dominated headlines, promises and warnings at the start of the 21st century. Twenty years on, its contributions are real, but more modest than the cure for cancer or the destruction of humanity that were predicted.
Exaggeration undermines public trust in science. The media have a power to temper expectations, to avoid false hopes and panic, as well as to rigorously explain the potential of scientific advances. If it fails to do so, journalism runs the risk of becoming an uncritical mouthpiece for press releases, companies and researchers, rather than acting as a filter for information. It is not a question of dismissing real achievements, but of contextualising and weighing them up. Here we have compiled some tips on how to spot the hype and what to do next.
1. Learn how to read the paper properly
A basic task is to assess the significance of a piece of research, usually based on a paper published in a journal. To do this, there are numerous questions to ask:
- Has this been studied in humans or in mice?
- Is it a controlled clinical trial?
- Is it an observational study?
- If it is an observational study, is there a risk of implying a causal link where none exists?
- Is the data derived from self-reported surveys?
- Is the statistical analysis robust or flawed?
- Are there wide confidence intervals in the results?
- Is it an individual study that merely analyses certain factors within a topic with many other factors to consider?
- Is it not an individual study, but a systematic review or a meta-analysis?
- Do they make clear the potential confounding factors and limitations?
- What are the conclusions regarding real-world applications?
- Are the conclusions generalisable to contexts other than the study itself, for example, to other populations or countries?
- Do the communication tools (press releases, press conferences) prepared by the organizations responsible for the project take into account their potential limitations?
In short: could this seemingly revolutionary breakthrough be accompanied by a dose of hype?
For example, a recent study suggested that a week without social media could improve mental health in young people. Upon analysis, it became clear that, despite being a good study in its field, the data were affected by subjective perceptions, there was no control group, and the self-selection of participants was a limitation... Ultimately, a categorical headline, however many clicks it might promise, could not be sustained by those results.
2. Make sure you fully understand the field and its limitations
Just as individual papers have specific limitations, there are fields built upon studies that share those limitations. Understanding them is essential when communicating new findings and determining whether the foundations are sound or not.
The steps outlined in point 1 are useful when dealing with an individual study, but it is not so easy to prepare ourselves when we are talking about entire fields of science that dominate the media and scientific journals. The list of trending topics is long: nanotechnology, graphene, gene editing, artificial intelligence, the microbiome, epigenetics, applied neuroscience, quantum computing, nuclear fusion, psychedelics, microplastics, the effects of mobile phones on mental health…
For all these reasons, the first step is to shift the focus. Here, we no longer need to check whether the methodology of an individual study is weak or whether a particular clinical trial has failed to produce the expected results. It is about understanding:
- Why is there so much talk and so much published material on this topic?
- What is its real potential?
- What are its weaknesses?
- What remains to be discovered?
- What problems do the studies published in this field have in common? (And back to point 1).
- Is the lack of evidence being exploited to make leaps of faith?
- What exaggerations surround it?
- What interests lie behind it?
- Are there criticisms or controversies within the research community?
For example, in this recent article, microbiome researchers highlighted the inaccuracies and exaggerations that are becoming widespread. They were not questioning the importance of microbiome studies or their impact on human health, but rather seeking to raise awareness of misconceptions to avoid unproductive research projects and preserve public trust. When a $50 million programme was launched in the United States to investigate the link between the microbiome and autism, a group of researchers published a paper stating that the idea was “a house of cards” whose “hype” went “beyond the evidence”.
3. Assess the balance between promises and achievements
This piece of advice, which applies to any journalist, usually holds true in science: if it seems too good to be true, it probably is. Unfortunately, drastic scientific revolutions are rare, and the advancement of knowledge usually involves far more work and time than we would like.
When a scientific field starts to gain popularity, it is not always something new that is still in its infancy; it often has decades of history behind it. Sometimes, many decades. Thanks to this, it is possible to check whether all those papers and expectations have translated into results, and to what extent.
- Is there any established result with broad consensus?
- What changes has this achievement brought about in your discipline?
- Is there an approved product or one already on the market?
- What applications does it work for?
- Has it met initial expectations?
- If there are no conclusive results yet, how long is it estimated they will take to arrive?
- What investments or research efforts are being made in this regard?
This is particularly important in cases where a panacea is promised that is supposedly capable of revolutionising everything from medicine to materials.
For example, twenty years after the sequencing of the first human genome by the Human Genome Project, some voices point out that all that data has had a negligible impact in clinical practice.
4. Look after financial interests
The phrase “follow the money” is a recurring catchphrase in journalism. If a field generates a lot of buzz in the form of studies and media attention, it is because it has potential—including economic potential. Consequently, there are often organisations behind the scenes with a vested interest in presenting their advances as revolutionary.
- In some cases, it is very easy to spot: pharmaceutical companies allocate millions to clinical trials which, if they result in approved treatments, will yield commercial returns. The AI industry has received such significant funding that there is increasing talk of a bubble that could trigger an economic crisis if it bursts.
- In other cases, it is not so obvious. Perhaps the researcher has set up a spin-off or secured a patent, so, logically, they have an interest in their field being perceived as groundbreaking. Perhaps the breakthrough is marketed as a milestone, but is merely a cosmetic change aimed at helping a company grab headlines and, as a result, secure funding. This is the criticism levelled at Colossal Biosciences, a company famous for making headlines by claiming to be closer to bringing back extinct species.
- It is also possible that expectations within a field are exaggerated in the hope of securing funding in an ultra-competitive system. A recent article analysed the attitudes of quantum physicists towards sensationalism in scientific communication. The participants acknowledged their role in creating hype as a strategy to secure funding and justified it by blaming the academic system.
- Caution: the existence of financial interests does not in any way imply that science is less valuable, nor that such interests are illegitimate; it does imply that they must be taken into account when reporting on such research, because they may influence communication and further investigation is needed to gauge their significance within context.
5. Identifies less obvious conflicts of interest
Scientific sources may have conflicts of interest, and this should not be problematic per se, provided they are known and disclosed, as we explained in this article.
- It is easy to take into account whether a researcher works for a company, has received funding from industry, or holds a patent in their name.
- However, conflicts of interest are sometimes less obvious: they may be ideological, relating to the way we view the world and what kind of solutions to its problems appeal to us.
- Or, sometimes, simply because we are human and research work is long and hard, a source might not be able to recognise that the field to which they have devoted their efforts for years is less robust than they would like.
6. Look for reliable and independent sources
The solution to this lies in seeking out other independent sources. Relying on reliable, reputable and diverse sources is always sound advice, and is particularly important when hype and uncertainty go hand in hand.
- In this regard, a diversity of voices helps us to understand where the consensus lies, where the dissent lies, which researchers hold differing views, and why.
- The complexity of this exercise lies not in remaining neutral, nor in giving a voice to people outside the field or lacking the necessary knowledge to evaluate it; rather, it lies in considering the viewpoints of other sources with epistemological proximity to the subject, which can provide a broader and more enriching perspective.
7. Approach every story with curiosity and humility
Stories are rarely written in black and white. The real challenge, ultimately, lies in exercising healthy scepticism whilst maintaining our curiosity and humility.
- If we err on one side, we run the risk of becoming uncritical groupies who simply parrot what a press release, a researcher or a paper says.
- If we err on the other side, we may end up embracing nihilism and disbelief to the point of rejecting and questioning any progress.
- And one more thing. How many times have we applauded upon reading a press release for a paper that says what we want to hear and that we’d like to cover with a bold headline? And, conversely, isn’t it frustrating to accept evidence that clashes with our own worldview? The science journalist must not only be wary of others’ biases, but also biases of their own, which are harder to detect, accept and tackle.
This list of tips is not a set of commandments set in stone; on the contrary, we intend for it to be a living document, so if you work in science journalism or science communication and have anything else to add, we’d be grateful if you could let us know: info@sciencemediacentre.es.
