Growth Hacking Blog How to evaluate marketing efficiency with the use of machine learning? 
How to evaluate marketing efficiency with the use of machine learning?
1 Star2 Stars3 Stars4 Stars5 Stars (32 ratings, average: 5.00 out of 5) Loading...

Half the money I spend on advertising is wasted; the trouble is I don’t know which half.

John Wanamaker,
founder of the world’s first chain of department stores.

Almost 175 years have passed since the birth of the famous American entrepreneur, the founder of the world’s first chain of department stores (Wanamaker’s), who introduced the concept of the fixed price system in his stores, preventing sellers from haggling with customers. Moreover, John Wanamaker was also the first near-contemporary advertiser to buy newspaper columns to advertise his stores; it was for this reason that he was later called the father of modern advertising. His quote about the advertising budget is a perfect description of the current situation in the advertising market.

The advent of the internet has undoubtedly reduced the waste of advertising budgets: analysts, marketers and advertisers have almost learned to calculate the return on investment (ROI) for online advertising campaigns and to evaluate the effectiveness of online marketing. Nowadays, statistics systems such as Google Analytics are actively used; systems of key performance indicators (KPI) are designed for websites and advertising campaigns; indicators, such as cost per lead (CPL), cost per order (CPO) and others, are calculated. But how can effectiveness of advertising be evaluated, if your website is nothing more than just a business card of your campaign, if visitors can’t order anything there, nor leave a request? What if, in addition to your advertising campaigns on the internet, you also place advertisement in three magazines and on two radio stations? What if the main purpose of your online (and any other) advertising activity is driving visitors not to your company’s website, but to your department store in the city center? How do you avoid dumping half of your advertising budget into advertising agencies and spaces in this case? In this article, without going deep into technical details of using mathematical modeling and machine learning in marketing, I will provide an example of how you can get answers to all these questions.

Unfortunately, the famous John Wanamaker doesn’t have the opportunity to read this article and to find out which half of his advertising budget he was wasting; however, let’s give him credit for raising the question. Now, imagine that you are the owner of a department store. Your main task is to increase the number of visitors of your shopping center through advertising in order to justify the rent increase in your department store afterwards and to receive more money from tenants.

This year, you have already tried various channels, including online advertising, to acquire visitors, and now you are pondering over the optimal set of channels in your media mix and the advertising budget allocation for the next year. Last year, you placed banners on thematic websites, wrote and published articles in various online media, hired SMM agencies for special projects. In addition, you bought blocks of ads in popular printed press, and once you even placed a commercial on the radio. Several times during the year, you organized sales and BTL activities in your department store.

Now you are facing a thick pile of papers with annual data on the footfall and sales of your department store, cost, time and placements of your advertising (Internet, newspaper, radio, BTL). You will have to use all of these to come to a conclusion about the effectiveness of each advertising channel and create the optimal media plan for the next year. You know that each marketing activity this year has influenced the footfall of your department store to this or that degree, the question is — which activity and to what degree. In addition, you keep in mind such factors as seasonality or weekends and holidays.

Visually, it looks like this:

machine learning model

Basically, you have a certain amount of factors that affect the dependent variable, that is, the traffic of your department store. Here is our task again: to understand the degree of influence of each factor on the footfall of your department store. At first, let’s build a simplified real-life model, where your department store traffic is only affected by the cost of advertising in one single magazine. That is, the increase in traffic is directly proportional to the advertising cost, and each issue of the magazine is read by the same number of readers. Then, the dependence of the traffic on the cost of advertising in printed press can be represented as a regular linear function:



Y — the number of visits to your department store;
X — the cost of advertising in printed press;
b — a certain constant number of visits to your department store (which would be preserved even without any advertising);

a — a coefficient showing the relationship between the traffic and the cost of advertising.


You placed your advertisement in the magazine two times: the first time you bought an ad unit for $5 and acquired 10 visitors, the second time you invested $30 in advertising and acquired 100 visitors.

Let’s visualize this by putting these points on the plane:

With historical data on two placements of advertising, it is easy to solve a system of linear equations with two unknowns and find coefficients a and b. Knowing the coefficient a and the constant traffic rate b, it is not difficult to find out how many visitors would come to your shopping center if you invested N dollars in advertising. Dividing N dollars by the number of visits to the shopping center, you can calculate the cost of one visitor (including your investment in advertising in printed press).

Now let’s make the situation a bit more complicated: let’s imagine that each issue of the magazine is read by a different number of readers.

You are quite satisfied with the ratio of advertising costs and the number of new visitors, so you place your ad in 2 more issues.

Let’s visualize this by plotting the points:

As you surely remember from your school course in math, a straight line can be drawn through any two points. However, it is glaringly obvious from the graph above that a straight line cannot connect these 4 points, which means that you cannot graphically find coefficients a and b of your linear equation. This, in turn, means that you cannot be sure what footfall you will receive next time you invest in advertising.

Let’s look at the real world again. In addition to the large number of placements for each marketing channel, there are also many factors influencing the single dependent variable — the number of visits to your department store. In the real world, the equation can be formulated like this:

linear regression

The task of mathematical modeling remains the same: we need to understand how each of the factors affects the footfall. In the simplified example with a linear function, we have already found that the degree of influence of a factor on the traffic determines coefficient a for variable X. There can be many more variables and coefficients in the equation above, and historical data will not help to find coefficients β1, β2, β3 … βn.

In our company, that’s where the methodology based on econometric analysis of historical data with mathematical modeling steps in. This methodology uses historical data ​​and analysis of data fluctuations (deviations from the normal value) to build a mathematical model and find each of coefficients β1, β2, β3 … βn. This, as we remember, determines the degree of influence of each factor on the dependent variable (in our case, the department store traffic). The methodology is based on regression analysis. Regression modeling is one of the simplest machine learning algorithms.

The methodology of mathematical modeling is universal: with historical data on placements of advertising (their types, costs, volume, impressions, reach, etc.) for a significant period of time (e.g., a year or two, depending on the amount of marketing activities), it is possible to find the degree of influence of each advertising channel on important business metrics (including sales).

Knowing the influence coefficient of each factor on KPI, you can optimize your media plan (ROI for each channel, budget for each channel) and choose the optimal marketing mix for your future actions.

Let me give an example of how mathematical modeling can help to optimize your media plan and to finally understand, which half of your money spent on advertising is wasted.

The first thing to be determined in the modeling process is the statistical significance of the model. Since we model a certain function based on historical data, we can compare the constructed model with real data. It looks like this:

athematical model statistical significance

Here, R2 shows the significance of the model. The value of 86% indicates that we are 86% sure the constructed model is not a random result.

Next, we find the degree of influence of each factor on KPI; we also find variable a — the ground level of the modeled indicator, which would be preserved without any advertising (reminder: in our case, this indicator is the department store traffic):

As can be seen from the figure above, some factors (marked by light green color) had a negative effect on the traffic. It is not difficult to guess that this factor is seasonality.

We can also decompose the resulting graph and have a detailed examination of the effect of each factor on the department store traffic. For instance, the next graph illustrates the effect of various promotional activities inside the department store on the footfall:

It can be seen from the figure above shows that Promo 7 (pink colour) had the greatest positive effect on the traffic.

If we know advertising costs for each channel and the influence of each channel on KPI, we can calculate the approximate cost of one KPI item (in our case, of one visitor) for each channel, and thus optimize the media plan:

machine learning optimization results


Even the simplest machine learning mechanism which is the mathematical linear regression modeling allows us to use historical data on placements of advertising to model and forecast the influence of the same or similar advertising and marketing channels, as well as independent factors, such as seasonality, on KPIs of a business. In this article, we have given an example of how this methodology can be applied to assess the degree of influence of all possible factors on the shopping center traffic. As a result of modeling and media plan optimization, in the studied case, the average cost per visitor was reduced by 18%, which, given an advertising budget of $150 000 , saves 30 thousand dollars.

By using this site you agree to the use of cookies by the company in accordance with the Privacy Policy