Saturday, January 27, 2018

Testing the Experts

A reader recently asked some important questions about how a non-expert can evaluate the opinion of the experts and determine whether or not the expert opinion has real validity. For instance, should the non-expert simply agree with the majority opinion of the experts? (You can find the discussion in the comments section within my post on The Grand Design.)

I have thought a lot about this question as has the reader who asked the questions, but I still don't have a definitive answer. So in this post, I will try to link together some diverse thoughts on this topic. Maybe my credentials as one of those "experts" in particle physics has helped me develop a worthwhile perspective.

The first principle I can think of is that everyone should understand some principles of basic logic. There are times when the experts present arguments that should simply be dismissed because they violate rules of logic. Dismissing the argument does not necessarily mean the conclusion is false, but it does mean the conclusion does not follow from that particular argument. (The previous sentence is, itself, a rule of logic.) I'll give you a few examples where an expert has presented a poor argument that violates rules of logic. In many cases, though not all, this occurs when the expert is speaking outside his or her field of expertise,

A few years ago I heard Richard Dawkins give a lecture at the University of Oklahoma. He began his remarks by comparing belief in God to belief that a stork brings babies to their parents. He was suggesting that neither claim had any supporting evidence. Of course, this is the logical fallacy of "false equivalence." The evidence for belief in God is nothing like the evidence for belief in the stork bringing babies. There are no discerning people who have investigated the evidence for the stork who actually believe that tale, but there are many discerning, intelligent, and thoughtful people who have carefully investigated the evidence for God and believe the evidence for his existence is overwhelming. Many of the latter group started off as skeptics and became Christians themselves based on the conclusive evidence for God and the Christian message. There is no credible evidence that the stork brings babies, while there is abundant evidence for the existence of God. Dawkins also used the fallacy of a "straw man" argument many times in his introduction. Most of the enthusiastic audience at the event seemed to lack an understanding of basic logic because they simply cheered and applauded as Dawkins presented one fallacious argument after another. I found it ironic that many of these "free thinkers" did not seem to be able to think for themselves at all.

A second example of poor logic presented by an expert is found in the book The Grand Design by Stephen Hakwing and Leonard Mlodinow, and described in my post linked above. In that book the question is posed, "Are there any exceptions to the laws of physics?" or "Are miracles possible." The answer given is, "…the modern scientists answer to question two [exceptions to the laws of physics]…is…a scientific law is not a scientific law if it holds only when some supernatural being decides not to intervene." This is a clear example of the logical fallacy of "begging the question."

A reasonable grasp of logical fallacies will go a long way toward knowing that certain views presented by the experts do not necessarily have the validity claimed.

The second principle I can think of is that the consensus of experts is not necessarily correct. That principle was nicely stated by the aforementioned reader who wrote, "that consensus is really a faithless mistress in the quest for truth." There are so many past examples where the consensus of experts was not correct. In the arena of archeology, the consensus of experts has often claimed that the biblical account is false because there was no archeological evidence for a particular person, location, or group of people, only to change when evidence was eventually uncovered. This has happened many times including the skepticism about the existence of a Hittite civilization, the existence of Pilate, that possibility that crucified people could be buried in a private tomb, and many more.

Of course the expert consensus should carry some weight and be considered carefully but not uncritically. Instead, ask questions like, "What fraction of the experts hold the majority opinion?" "What are the presupposition of the experts that may limit their conclusions?" "Why do other experts in the same field hold a different opinion?" "Are the group of experts objective and unbiased or do they have a stake in the outcome?" And of course, "What is the evidence that has led to the majority opinion?"

The last question above hints at the third principle I can think of. Always read the experts with different viewpoints to try to scrutinize their arguments and their reasoning. In this blog, I have discussed books that I have read by people who disagree with my conclusions such as The Grand Design by Hawking and Mlodinow, A Universe from Nothing by Lawrence Krauss, and The Fallacy of Fine Tuning by Victor Stenger. Reading arguments on all sides of a disputed issue allows you to discern the strength and weaknesses of the arguments and come to an informed opinion. After reading different opinions for yourself also read the critiques of those opinions from both the pro and con perspective. This exposure to multiple views will help you determine which ideas are based on sound principles and actual facts and which are not.

One strength of scientific inquiry is that the data usually leads to majority conclusion with strong supporting evidence from a variety of sources. It may take some time for the data to provide an answer with enough statistical significance, but when it does, in general, the majority expert opinion can be trusted. Of course, for many scientist there is the constraint that only natural causes can be admitted as an explanation, which may mean that the "scientific" conclusion is incorrect if, in fact, God intervened in a particular case outside of the usual natural order. One of the key phrases above is that there must be "enough statistical significance." Until that occurs, I think it is wise to hold the expert opinion loosely. I think that in certain fields of study, majority conclusions are formed when the level of confidence is still quite low. I believe that is the reason why there have been archeological conclusions such as those listed above that have been shown later to be false. By reading different viewpoints about a subject, it is possible to get some idea of how significant the "signal" is. If there is a large minority opinion, then most likely the view held by the majority does not have a huge statistical significance.

Often when I speak about science and Christianity someone will ask me about global warming and climate change. Of course I'm not an expert on that subject so I must apply the principles listed above if I want to develop an informed opinion. The consensus of most climate experts is that humans are a major cause of warming and climate change, but is that correct? I've looked at the logic used by proponents of different views, read those who disagree with the majority, and formed my own opinion about the level to which humans are contributing to climate change. Although I am not sure my conclusions are valid they have been developed based on an informed overview of the different perspectives.

As a general rule I try to give both a scientific and Christian perspective in this blog. So far, the discussion has not specifically appealed to a Christian worldview, but there may be a few insights as a Christian on this topic. First, finding the truth is of utmost importance as a Christian. Christianity, at its core, is about truth. Jesus claimed to be "the truth" and we serve a God who is always truthful. It is valuable to hone our ability to determine the truth among competing hypotheses, including the role of expert opinion in that determination. Second, by applying these principles to the biblical text and the archeological evidence that supports it, I have become convinced that the biblical record is basically a very reliable historical record of events. Third, Jesus said that the way that leads to life is narrow and only a few will find it. I think that implies that some aspects of a Christian worldview will always be part of the minority opinion and not the majority opinion.

I don't know if this discussion gives any insight into the topic of how to evaluate an expert's opinion in subjects which you are not an expert yourself. At the beginning I said that despite the fact that I have thought about this I don't have definitive answers. Maybe some of my random thoughts will be helpful as together we try to discern what is true about the universe we live in.


  1. Dr. Strauss,

    Is it fair, in testing experts, to also consider their field of expertise and how much certainty that field renders to their opinions? Some fields of expertise allow measurement, precision and testing, and others don't but rather seem more a matter of mere opinion based on only a smattering of data.

    For example, a qualified chemist's opinion on the results of a particular chemical reaction seems to be much more trustworthy than that of an English expert's opinion on whether Shakespeare was one man or multiple authors. Similarly, a mathematician's opinion on a mathematical subject seems to be more trustworthy than a sociologist's opinion on sociological matters.

    It seems to me that distinction is true whether or not it is the opinion of only one expert or a "consensus" of experts in the various disciplines.

    Finally, it would also seem to be worthwhile, in any discipline, to determine how much of the expert's or experts' conclusions are based on hard data and how much is mere opinion and/or extrapolation from hard data. The more extrapolation or the more opinion, then the less trustworthy the experts' conclusions are.

    1. Yes, I agree that certain fields can have different levels of confidence in their results and that an expert is most reliable in their own field, and in other fields they may have anywhere from none to very reliability.

  2. Hi Dr. Strauss,

    I am humbled that you thought questions worthy of an entire blog post. I appreciate the time and effort you put into your response.

    To put my question in context, I am a Christian with a professional degree in the health sciences who has recently gone through a devastating crisis of faith. For the better part of the last two years I have been trying to sort out what is true and what is not, and earnestly have tried to evaluate the best evidence (and my own life experience) with an approach similar to what you have suggested above, with the goal of establishing some sense of certainty. I find that most people are able to cope adequately with life within a framework of basic certainty that gives a sense of order to their existence. It was exactly this which I have been lacking.

    Exploring evidence for and against a position is a necessary part of diligent truth-seeking. However it is a formidable task for someone in an existential crisis, regardless of their intellectual capacity or academic ability. Eventually you reach a point where your ability to weigh argument and counter-argument is surpassed. I remember early on in my struggle reading Stephen Barr's Modern Physics and Ancient Faith, being very encouraged by what seemed to be reasonable arguments, only to read a review by the late Mark Perakh on his Talk Reason page that seemed to capably attack many of Barr's points. When the conversation becomes a refutation of the statement "in science, order comes from order" with references to Belousov-Zhabotinski reactions, or about Benard cells, my ability to evaluate has been exceeded.

    I find at this point many people retreat to "if God existed, he would make things more obvious", if not in argument then in attitude. But that is just to demand God do what we want, or perhaps plead laziness. I don't wish to do either, but I am struggling with the journey.

    1. Nathan,
      Thanks for your candid comments. I have not been able to respond due to a busy schedule but I'll try to make a few comments. I think it is valuable in times of crisis to remember that the Christian God has a perspective that we can never fully understand or appreciate. Consequently, many things that may seem hard to understand from our perspective are perfectly clear to God. My posts on "A Transcendent God, Parts 1 and 2" may shed some light on that. For some reason, God has made himself obvious enough for even the most intellectual and skeptical person to believe in him. There are many examples of people who did not believe in God but looked at the evidence and changed their mind. But he is not so obvious that that there is no room for some doubts. I don't know why God set things up this way but I do know that his ways are totally incomprehensible in their fullness to me since I am such a finite being. I am fully convinced that when all the evidence is weighed, from multiple disciplines including science, history, sociology, etc. that the Christian God is the best explanation to explain the universe we live in.

      We could discuss particular details as you see fit.

  3. So what is your personal conclusion about climate change? =)

    1. I think it is abundantly clear that the climate is getting warmer. We also know that the climate has fluctuated much in the history of the earth. However, given that we know the physical mechanism in which greenhouse gases can cause warming, I think it is clear that human actions are, at least in part, responsible for climate change.

      But on a more pragmatic note, it really doesn't matter if human activity is responsible for climate change or not. It just seems wise and prudent to try to develop clean, renewable energy for future generations. Non-renewable resources can't last forever.

  4. I appreciate the levity Cara.

    As it relates to the conversation, I'd say I think so, but I'm not sure what it signifies. I think the experts mean different things when they use the term climate change, so that depending on who is talking we could be referring to anything from the effect of humans on the carbon cycle over the last hundred years or so, to the changes that occur on geological timescales, some of which humans have no effect on whatsoever. And the line is blurry - there have been many cases where an organism or system of organisms have become dominant on Earth for long periods of time, dramatically changing the structure and chemistry of the surface of the Earth. If humans do the same, it's hardly without precedent.

    The bigger question for me is, why care? On Christianity, I am commanded to love my neighbor and steward and exercise dominion over creation, so I have an obligation to cultivate the world in a way that demonstrates love for everything in it, particularly my fellow humans, and to treat it with reverence. My actions are subject to the judgement of a divine being who has authority over life and death, the universe and everything in or beyond it. On atheism, any sense of obligation I have is an illusion integrated into the greater illusion of "self", and I will simply execute whatever my biological programming dictates. Probably that means helping myself or those with whom I share close kinship, so burning coal, or burning down the coal mine, or burning all of the people using coal; all is equally justifiable in the amoral cosmos, and my illusion of self will find an acceptable way of rationalizing whichever option my biology chooses for me. And in the end, what does it matter anyway? My species is simply a placeholder for whatever species takes over next, whether it be dolphins or cockroaches. The entire world and any descendants I may ever have are doomed to incineration when our sun runs out of hydrogen. A massive Carrington event or an impact could take us out much sooner. So why would I worry about buying a Tesla versus a vintage Cadillac?

    1. Nathan,
      I think you have articulated a reasonable philosophical argument about the nature and existence of God. Without God, why care about the environment? With the Christian God, humans are valuable because they are the only spiritual beings on the earth and we are commanded to take care of the earth. A Christian worldview provides the incentive to care.