The B2B SaaS Marketing Blog

Boosting B2B SaaS Marketing Results With Hypothesis Logs

by 

Mike Smith

Last updated: February 22nd, 2024

Continuously improving your B2B SaaS marketing results means running tests, learning from those tests, then running more. However, you could lose time, money, and opportunity if you test the wrong things for the wrong reasons, or don’t test at all.

We’ve noticed a similar pattern among marketers who consistently struggle to see predictable growth over time.

These marketing teams either aren’t testing, or they’ll test randomly selected variables and strategies that produce a lot of activity and data, but no useful outcomes..

And if a test doesn’t work out, there’s rarely any analysis to determine why it didn’t work, meaning the team didn’t learn any insights to apply to future tests.

While there will always be uncertainty in marketing, you can improve the success rate of your tests by using a documented hypothesis framework.

In this post, we’ll introduce you to the hypothesis framework we use here at Powered By Search to incrementally scale progress and performance for our B2B SaaS clients. We’ll show you how to create a hypothesis, document it, and analyze test results to draw helpful insights and apply them to future tests.

Do you need to apply a deeper testing and learning approach to your B2B SaaS marketing efforts? Reach out today to get your Free Marketing Plan.

What Is A Hypothesis Log?

You can use any medium or software you’d like to start working on your own hypothesis log. At Powered by Search, our hypothesis logs are usually managed in a simple spreadsheet where we write out exactly what we’re testing, the hypothesis itself, and other factors, such as:

• Impact: How much will this test move the needle for the business if it proves true?
• Confidence: How likely is it that the hypothesis of the test will be proven true?
• Ease: How easy is it for us to implement this test?
• ICE Score: This is the weighted average of the other three variables that helps us prioritize what to test first.
• Start Date: So we prioritize when to take action rather than it being just another idea.
• End Date: So the test doesn’t run in perpetuity and we intentionally stop, learn, and refine our hypothesis.

As you’re filling out your hypothesis log, we recommend that you prioritize first testing the variables with the highest ICE score.

How Can A Hypothesis Log Improve The Success Rate And Impact of Your Tests?

Using a hypothesis log to document each test ensures that:

1. No hypothesis is redundant or duplicated,, so you never repeat the same efforts for the exact same learnings time and again.
2. Your experiments are always in a controlled environment (e.g., people aren’t running overlapping tests at once, which could skew the test results).

Hypothesis logs also help you build a valuable database of information on marketing messaging, channels, and strategies that do and do not work with your audience so that the magnitude of success for future tests is higher. This way, every single hypothesis and test no matter the outcome provides valuable data and learnings.

For example, let’s say you want to improve the conversion rate of your LinkedIn ads.

If you already discovered in a previous test that messaging around Pain Point A was significantly more effective at driving clicks than Pain Point B, you can use that insight to test only variations of Pain Point A in your ads.

• Testing without a hypothesis log: The trigger for the test is a rough idea that’s guided by opinion. It sounds like “I wonder what I should test?” then time, effort, and money is spent to simply see what happens if the test is run. There may be a performance lift, but it’s often by default and not by design.
• Testing with a hypothesis log: The trigger for the test is defined based on prior learnings. It starts with. It sounds like “We learned that the pain point messaging in last month’s LinkedIn  ad copy refresh converted 1% better than the previous copy. Our hypothesis is that if we A/B test variations of this winning pain point ad copy for the next 2 weeks, we can gain another 0.5-1% lift in conversion.”

As you can see, these tests aren’t based on random guesswork – they’re high-conviction bets that can improve the likelihood and magnitude of success of your next test.

Additionally, because we’ve created hypothesis logs for all our clients, our team has unique proprietary insights into what works for most B2B SaaS companies when deploying similar strategies.

So, even when we begin an engagement with a new client who currently doesn’t have a hypothesis log, we can still draw from the insights gathered from our other hypothesis logs to increase the likelihood of the first test’s success.

When Should You Use A Hypothesis Log?

It’s important to find a balance between testing absolutely everything, and testing nothing at all.

One way to set boundaries is to avoid hypothesizing and testing something that is already empirically true.

For example, if your website is down, you don’t need to test if getting it back up and running will improve conversions from the website.

It’s more or less certain that getting your website back online will help you drive more sales.

We only add tests to the hypothesis log if we’re testing a question where there isn’t absolute certainty of the outcome.

On the other end of the spectrum, your marketing team will also have to be comfortable with hypotheses and tests that are not certainties. Otherwise no test will ever be given the green light to run. Some hypotheses will prove true, others will not and that’s ok. That’s the point to testing – to learn from the uncertainty.

The hypothesis log is also helpful for prioritizing tests.

There are often multiple tests we’d like to run at any given time, so the ICE score helps us prioritize which test to run first by identifying which ones would be most impactful if they proved to be true.

How To Create Effective Hypotheses

There are three core frameworks we use for creating a hypothesis. Below, we’ll walk you through each one and provide examples.

Each level of hypothesis adds more detail and fidelity to the statement, and you can use the one that best fits your situation and level of buy-in needed to get approval and validation on the test.

Level 1

The conditional statement hypothesis is the most common and is phrased as:

“If We [Action] Then [Outcome]”

In this case, the action is the variable you’re testing, and the outcome is the result you expect. The outcome is usually the growth of a specific metric you’re tracking.

An example of the conditional statement hypothesis might be:

“Our hypothesis is that if we reduce the number of form fields from 5 to 3 then the conversion rate will increase by 5%.”

This is a great framework to use when you need to explain the logical outcome of performing a specific action.

Level 2

If you must get buy-in from stakeholders to run a specific test, like increasing your Google ads budget, you’ll have to answer why you should run this test and the outcome you’ll likely achieve.

In these cases, use the suggestive hypothesis framework.

Here’s how to structure the suggestive hypothesis:

“We Think You Should [Do This] Because [Why] So That [Outcome]”

For example, let’s say you think you should add an additional $10,000 to your ad budget for the month.

In this case, you’ll obviously have to justify to stakeholders why you need that budget. So here’s the suggestive framework in action:

“Our hypothesis is that you should increase your Google Ads budget because of low (50%) impression share. This is so that you can drive at least ten more demos per month by winning more bids for converting keywords.”

Level 3

This type of hypothesis is similar to the Level 2 hypothesis, though there’s also an additional layer because you’ll know it is true when you achieve a certain metric.

Here’s the framework:

“We Believe That [Action] Will Achieve [Outcome]. We Will Know This Is True When [Evidence]”

This is a great framework to use when you need to prove the outcome of a hypothesis with evidence and data.

Here’s an example of this in action:

“Our hypothesis is we believe that by updating the content on your home page we will increase the engagement of high intent traffic. We will know this is true when the conversion rate on the home page demo form increases by at least 2%.”

Example Hypotheses Logs

Here are a few examples of hypothesis logs in action for our B2B SaaS clients.

Headless CMS SaaS

Cybersecurity SaaS

Audience Management SaaS

Fundraising SaaS

Creating A Marketing Strategy That Incorporates Continuous Testing And Learning

One of our core values is having a growth mindset, and testing is a key aspect of growth.

Creating and testing strong hypotheses throughout marketing programs is how we keep our clients on track toward achieving Predictable Growth.

If your current approach to testing needs a refresh, or just needs to get started, hopefully the framework that we’ve covered here is something you can start applying to your own marketing efforts today.

If you want to work with a marketing agency that will continue to systematically test and improve results, reach out to us today for your Free Marketing Plan.