Applying the Scientific Method to Lead Conversion

Hey there! You’ve reached this book midway through. To read from the beginning, click here. 

Chapter 11: Applying the Scientific Method to Lead Conversion

“Okay,” I said. “Time to get nerdy.” 
“Now?” Carole said. “So up to now we haven’t already been nerdy?”

“She has a point,” Chuck said.

“Anyway,” I said. “Let’s talk about the scientific method.”

“You mean that thing I had to memorize in middle school science?”

“Yes,” I said. “It’s a step-by step-method for using math and observation to determine if something you think might be true is, in fact, true.”

“And this has something to do with marketing my company?”

“Trust me. It has everything to do with that.”

If you’re like most people, you remember the concept of the scientific method from middle school science class. Also, if you’re like most people, you probably need a review of the specifics.

Well, this method is not just for the lab. It can be applied to your business development funnels, which enables you to get more leads, focus on qualified leads, and increase sales.

The scientific method is a systematic approach to answering questions about the world. It consists of six steps, each of which will probably feel familiar to you as we walk through them in this chapter.

They won’t just feel familiar because you flash back to middle school. A/B testing, which people in your position will have done repeatedly in their careers, runs through the same steps:

• Observation
• Question
• Hypothesis
• Experiment
• Analysis
• Conclusion

Let’s start by examining each step in detail and looking at A/B testing through this lens.

The Steps of the Scientific Method

The power of the scientific method is that it offers a template for exploring the universe. It serves as both a guide and a reminder for how to keep what you want to be true from getting in the way of finding out what’s actually going on. It begins with observing something you want to know more about, and ends with a specific, quantified conclusion about that observation.

Observation

Isaac Asimov once wrote “The most exciting phrase to hear in science is not Eureka, but That’s funny.”

The scientific method begins with somebody noticing something about the universe that’s odd, important, or challenging. Archimedes noticed that a crown’s volume was hard to measure because it had such an odd shape. Salk noticed that the polio virus was both deadly and vulnerable to the human immune system. A coder notices that every time somebody right-clicks an icon in his program, the computer crashes.

In all of these examples, the observation inspires the scientist to learn more.

A/B testing begins with the observation that two marketing messages are likely to bring results. This observation inspires profit engineers to find out which brings the best results.

Question

The next step is forming a specific question related to the observation. Though it’s possible to just randomly apply stimulus to a subject, it’s very hard to derive any meaningful conclusions from that.

Instead, scientific (and profit engineering) experimentation demands a question that is as specific as possible:

• Archimedes asked, “How can I measure the volume of an irregular object?”
• Salk asked, “How can I keep people from being crippled and dying from the polio virus?”
• That coder asks, “How can I keep my program from crashing?”

When it comes to A/B testing, the question is always the same: “Will message A or message B produce the strongest response in this media channel?”

Hypothesis

In the hypothesis stage, scientists turn that question into an informed guess. It’s a little like the opposite of Jeopardy: You ask the question again, this time in the form of a statement.

Archimedes turned his question into the statement “I can measure the volume of an irregular object by submerging it in water and measuring the volume of the water it displaces.”

Jonas Salk’s hypothesis was “I can stimulate immunity to the polio virus by vaccinating subjects with a small amount of the virus itself.”

That coder’s hypothesis is “I can keep my program from crashing by changing lines 101 to 103 of my code.”

In A/B testing, you fudge things a little with a hypothesis that keeps you from favoring one or the other: “Either message A or message B will perform significantly better than the other, thus giving us a clear path for our marketing.”

Experiment

This is the fun part of science, the part where people get to tinker with the source code of the universe and see what’s what. It’s also the most complex, and the easiest to screw up.

In this stage, scientists set up situations in which they can test the hypothesis and prove or disprove it. In the next chapter, we’ll look deeply at the elements of a good experiment.

Archimedes submerged a crown in the bath, measured the volume of water it displaced, and then measured its volume the “old-fashioned way.” Then he did the same with a variety of other objects.

Salk vaccinated subjects with his vaccine, then observed whether they were immune to polio. This was the final step after multiple iterations of experiments in petri dishes and on test animals.

That coder changes lines 101 to 103 of his program, then runs it again and clicks on that icon.

With A/B testing, profit engineers make a limited run of both marketing messages and compares the results.

Analysis

The second-to-last stage of the scientific method is looking at the results of the experiment to see what it tells you about your hypothesis.

Archimedes compared the results of submersion and regular measurement for each object he tried, to see if they matched.

Salk tracked how many test subjects contracted polio after being vaccinated, compared to subjects who had not received the vaccine.

The coder runs the program again, clicks on the icon, and sees if the computer is still crashing.

The profit engineer looks at how many leads message A got, and how many leads were received from message B.

Conclusion

This part is why the scientific method is so powerful. By formulating a highly specific hypothesis, then testing it with a careful experiment and analyzing the results, a scientist is led to a specific (and hopefully irrefutable) conclusion. Because the method is so exacting, the scientist should only be able to draw a single conclusion about the hypothesis once the data comes in.

In Archimedes’ case, every single measurement matched. If even one of the objects had showed water displacement with a different volume than measurement and math, his hypothesis would have been disproven.

Salk was able to conclude that his vaccine was effective by the finding that a significantly lower number of vaccinated people contracted polio under the same conditions as those without the vaccine.

The coder knows whether his changes fixed the program by whether it still crashes when right-clicking the icon.

In A/B testing, you can tell which message works best by which produces the most positive response.

The Scientific Method and Conversion Management

You will use the scientific method in A/B testing and other profit engineering applications by asking and answering questions about which steps in your sales funnel are working well. Some sample questions you’ll need to investigate by experiment include:

• What do our potential clients value?
• What messaging resonates with them?
• What are the fundamental pains of our potential customers?
• What are they looking for to help them solve their problems?
• Is there a loss leader opportunity to start the conversation and build trust?

Just as in scientific exploration, some of those questions are closed-ended yes/no or A/B questions. Others are more open, “I wonder what happens when I….” questions.

Both are valid questions. The former is more useful for forming a plan of action. The latter are best for initial exploration to help you create an actual hypothesis.

As an engineer — profit or otherwise — active experimentation and careful recording of results is part of your job. In the next chapter, we’ll look at how to devise an  effective  experiment that gives you the most accurate possible results about what’s working and not working in your conversion tactics.