Less Numbers – More Math
Numbers drive Media and Marketing today. People don’t like it, but the “numbers” people usually control budgets, so numbers are endured as a necessary evil.
Professionals work through their excel sheets, make some graphs (usually too many) and try their best to back up their ideas with “hard data.” Then, they hope, they can get back to doing the creative thinking that was the reason for choosing their profession in the first place.
This is a ridiculous situation. The numbers are frequently wrong (or at least misinterpreted). Everybody hates them and, truth be told, not many people actually believe in them. Most simply find the numbers that will support the case they want to make. As the new digital world creates ever more data, the problem is only getting worse.
What we need is less numbers and more math.
The Power of Assumptions
Let’s imagine the end of the world. There has been some calamity and the human race has been reduced to two people on a desert island.
If both remaining members are of the same sex, the human race is surely doomed. However, if we assume a man and a woman, the human race would have some chance, albeit with some serious genetic issues.
What if there were three people? Again, if they were all the same sex the extra person wouldn’t do us any good. Two men and a woman wouldn’t be much of an improvement over two people, and could actually be worse if the men got violent and killed each other. Two women and one man would improve the situation considerably (especially for the man!).
Notice how we don’t need to change the numbers to change the math. The numbers actually have very little to do with the substance of our analysis – whether the human race will survive or not.
How Mathematical Models Work
We all use mathematical models every day, without even knowing it. They are a way of saying “what if the world works like this?” In other words, they are basic assumptions and the assumptions we choose, as we saw above, will have a powerful effect on our analysis. Math lets us formalize our guesses and test them.
Here are some common models that describe how things around us work:
Linear model: This is the basic “rise over run” that we learned in high school. When we use linear models we are assuming a constant, unchanging relationship between two variables.
When we estimate how long a trip will take, we usually simplify it by using a linear model. We assume a certain average speed for a certain time. We know it isn’t the truth, but it’s close enough to make a rough estimation. Linear models can be very useful.
Polynomial Models: These models reverse themselves to form arcs. The simplest kind is a quadratic model, which can either go up and then come down or go down and then come back up. They are useful for describing things like tossing a ball.
By increasing the degree of the equation we can make it reverse itself as much as we want to. Imagine a ball bouncing across a floor. To be honest, the assumptions that quadratic models make usually aren’t very useful. However, because polynomials are so easy to work with (just a click of the mouse in excel and some algebra) they are often be used to approximate other, more complicated models.
Exponential Model: In this type of model, the numbers grow or fall in proportion to their size. The bigger they are the faster they grow. The smaller they are they slower they decay.
Bacteria growing in a Petri dish and radioactive decay are both good examples of exponential models. The core assumption here is that the rate of change (e.g. growth or decay rate) stays constant. In the business world, compounding interest and growth rates are common uses of exponential models.
Exponential models can be useful, but they’re dangerous. Many new industries go through a period of exponential growth and people who are tied to that industry assume that the rate will continue, which it never does. However, by tracking real data against an exponential model you can clearly see when an industry is starting to mature.
Normal Distributions: These are also known as “Bell Curves” or “Gaussian Distributions” after the man who invented them. We use them often because the science of Statistics is based on Normal Distributions. Whenever we expect things to “average out” or say that “the sample size is too small,” we are using a normal distribution whether we know it or not.
However, Normal Distributions are probably overused, because they assume that numbers are randomly generated. The world doesn’t work that way. True randomness is actually very rare, which is why Statistics can cause a lot of mischief. Using Statistics is a good way to perform some basic tests, but if one takes the models too seriously, the world is grossly misrepresented.
The problem with Bell Curves is that they tend to underestimate extreme values and volatility. That’s why statistical analysis never accounts for a boom or a crisis, it assumes that the world will stay “normal.”
Power Laws: These are the “evil cousins” of normal distributions. They look a lot like exponential functions and probably best describe the world we live in. Chris Andersen made Power Laws famous in his book, “The Long Tail.”
Yet, “long tails” fail to describe the beauty of Power Laws. They show us change with a constant order that continually repeats itself. Like angry old men who send back soup in restaurants, they erupt unpredictably, but always with the same pattern. (See Chaotic Social Networks).
Creative types who hate math should take a look at Mandelbrot Sets that Power Laws produce and they will know the meaning of mathematical beauty.
How Numbers Lie
When we work with numbers, we make assumptions whether we know it or not. A good example happened to a friend of mine. He was running an ad agency and wanted to measure the correlation between what his large, sophisticated telecom client was spending on advertising and their sales.
He had the best of intentions. He wanted to show that he cared about his client’s business and was concerned about their ROI. Nevertheless, his numbers lied without him even knowing it. He was generating numbers without understanding the assumptions that the model was making.
Correlation is linear. When we use a linear model, we are assuming trees really do grow to the sky. For instance, if we spend $1 we get one sale and for $1 million we get a million sales. A billion for a billion, a trillion for a trillion, etc. – A ridiculous notion that any reasonable person would reject immediately.
Nevertheless, he presented his analysis to his client and they were quite impressed. After all, he had the numbers to back him up! It’s easy to get lost in numbers if you don’t know the math.
Some Simple Rules
Numbers lie, but good math doesn’t. As a matter of fact, math gives us some common sense rules that are easy to follow.
Keep it simple: A basic mathematical rule is to always use the simplest model that fits the data. This important rule is broken way too often. Use as few numbers and as few charts to describe the situation accurately. If you can’t explain your analysis to your mother, it is probably too complex.
Null Hypothesis: Always assume that numbers mean nothing unless you have a clear reason to believe otherwise. I always tell my staff to “lean back” when they are working with numbers. If you look for a pattern long enough, chances are you will find one that isn’t really there.
Correlation is not Causality: Whenever we analyze numbers, we use assumptions and those assumptions can be, and often are, wrong. Quantitative analysis is a good way to test how things might work under certain conditions, but it doesn’t tell us what will actually happen or why.
Data is an Input, not an Output: The point of analysis is to gain understanding of how the world around us works. There is no point in using numbers just to produce more numbers. Math is about understanding, numbers confuse.
More Math – Less Numbers
As we gain access to more and more information about the world around us, we can drastically increase our knowledge and understanding. Old superstitions can be discarded and would be shamans can be unmasked. However, to do that we will need to learn some math.
A good place to start would be the basic statistical functions in excel, which provide powerful tools for us to test our ideas in different ways. I’m always amazed how much companies spend for software, training, etc. and how few marketing people can use Microsoft Office effectively.
Yet, what is really needed is a basic curiosity – the desire to learn things rather than to know things. All too often, we strive to rattle off facts rather than promote understanding. With a little imagination and some thought we can all do our little part to increase the overall knowledge of the world.
True knowledge begins with wonder.
– Greg
Sounds like an interesting thought process in theory but the article lacks example. Have you been able to apply these models in day to day work?
Viral,
It’s tough to include practical uses of the more complex models in the space of a blog, but I’ve worked with all of them at one time or another. Are you interested in any particular model?
– Greg
I understand the limitations of blog. I would be keen on seeing any example using say Bell Curves or Exponential models to understand you are trying to make in the article.
Viral,
I don’t actually plot bell curves, but anytime you use Statistics the are implied, for instance when you calculate Max, Min, Mean, Median and Standard Deviation you are generating a bell curve. Another example is when you calculate sample error (or say a sample size is too small) you are using a Bell Curve.
Compounding interest or growth rates are examples of exponential models.
My point is that we refer to mathematical models all the time, often without realizing it. The result is that often people are working with numbers that they don’t understand.
– Greg
Greg,
Thanks a lot for your inputs. I guess now on I will be more observant about these things!
Viral,
Thanks for the great input. I’ll update the article. Comments like yours are really helpful.
– Greg
Dear Greg,
I really enjoyed this post, especially that you put such a complicated and avoided things by many as “math models” into a very practical perspective. Trees indeed don’t grow to skies, a good thing to remember when making a next analysis.
Thanks,
Olga
Thanks, Olga!
– Greg
Greg,
A great article! I often work with entrepreneurs and startups, and I must remind people about GIGO (Garbage In – Garbage Out). You can have wonderful pro forma financial models that show your compound growth rate and rising net income, but if the numbers that you base your model on are not valid (and often they are not) it won’t make a difference.
I recommend two ways to avoid such problems: first, do proper research into the financials of your competition as a comparison, and then be a bit modest about your own selling ability!
Kevin
Kevin,
Thanks. Good point about benchmarking.
– Greg
Greg,
a great post! You make it sound so obvious and so practical and I agree, math is about understanding rather than complicated science especially in business environment. The numbers need to be simple but understanding should be very deep.
Kind regards,
Katya
Спасибо!
– Гриша
Greg,
Hi Greg,
Yesterday, I was getting ready to write an article titled “Analytics are for cowards” when I read your article “Less Numbers – More Math.” It augmented my thoughts so I quoted you and linked to you. I hope your readers and you think it adds value to the discussion.
Thanks for the excellent information and the unwitting help.
John Ribbler
http://media-proinc.com/ribblog/blog/
John,
Thanks. It’s a very good point you make in your article, that not only are numbers often wrong, but wrong for the worst of reason – cowardice.
I encourage everybody to read it http://media-proinc.com/ribblog/analytics-are-for-cowards/
– Greg
Good article, Greg. I know big organizations that base most of their business decisions on metrics that involve so complicated data processing that numbers are surely far from accurate.
I personally would be very interested if you can share in a future post interesting examples how numbers lie in online marketing.
Stan
Stan,
Thanks for your comment.
I really don’t see much difference between online and offline. The main point is that you need to know how your data are collected and what the assumptions of the model you’re using are. Otherwise, you’ll run into problems.
One point about Digital media is that they don’t know enough about how traditional media metrics are used, so there are a lot of misconceptions about what needs to be done. I’m publishing an article about that tomorrow.
– Greg
Thanks for suggesting this article to me, Greg. Excellent points.
I’d be interested to also get your thoughts on the following example. A client of our hired a statistician to work on finding if there was correlation between media spending and retail sell-through.
– The modeling was quite tough. We’d used a relatively flat, consistent media schedule to build momentum – planning media by response tends to lead away from front-loaded schedules in my experience. But the noise at retail from appearance in circulars and other market forces made it difficult to detect any on-going repeatable media impact.
– This required some transformations (don’t recall the specifics). And, after the transformation work to damp out the noise, there was a clear correlation between media and retail sell-through.
– It showed a roughly 2 week delay between spending and the start of impact.
– And, it showed a 3 week sweet spot where the majority (60%?) of the sales impact was seen.
Fine to this point. Then, the company asked him to “quantify” the impact.
– Being a good statistician, he ran the equations and came up with estimates of the impact size being quite small.
– Market research interviews of purchaser influence showed that a media impact of 15 to 20 times the impact.
– My sense is that the statistics were conservative – especially after the transformation machinations required to damp out other market forces. They also had no ability to detect the very long term impact of a media flight because the numbers were small in any given week (but significant when looked at over a 6 month period).
– On the other hand, literally accepting the research interviews was likely a very aggressive estimate due to an entire variety of challenges from consumer recall, vagaries of responses, etc.
My working hypotheses have been:
– It is easier for statistics to determine whether or not media has an impact. It is much harder for statistics (or any other method) to accurately calculate the specific size of that impact.
– Statistics and consumer interviews probably bracket the true impact of a campaign.
Would love any thoughts…
…Doug
Doug,
It’s a bit tough to explain in just a few words, but here’s some pertinent points:
– Correlation is not the right word. I’m not trying to be nit picky, but correlation only holds for a linear relationship. What he was trying to do is fit a model.
– It’s pretty easy to show that advertising works, the real question is how it works, which is why you try to fit a model to varied data points.
– Usually the model doesn’t explain much more than 50% of the data, so it’s hard to pin down exactly what the impact is. This type of modeling is much more suitable for optimization.
In general, it seems like a pretty crappy study. This type of modeling only makes sense if it’s ongoing. Doing it short term almost guarantees to confuse as much as explain.
– Greg