Skip to content

Your Best Guess is as Good as Mine

October 2, 2014

Interesting article, I think NRO pointed it out to me (or maybe their Twitter feed, or maybe I clicked a mail… it’s been open for a couple hours now).  I’ve often thought that an over-complicated model throws in as much error as a simple guess, and this article agrees with that assumption.

This resistance to state-of-the-art statistical models has frustrated the academics. So, a decade ago, marketing professor Florian von Wangenheim (now at the ETH Zurich technical university in Switzerland) and his then-student Markus Wübben (now an executive at a tech incubator in Berlin) set out, in Wangenheim’s words, to “convince companies to use these models.”

To do this, Wübben and Wangenheim tested the predictive accuracy of Pareto/NBD and the relatedBG/NBD model against simpler methods like the “hiatus heuristic” — the academic term for looking at how long it’s been since a customer last bought anything — using data from an apparel retailer, a global airline, and the online CD retailer CDNow (from before it was acquired by Amazon in 2001). What they found surprised them. As they reported in a paper published in 2008, rule-of-thumb methods were generally as good or even slightly better at predicting individual customer behavior than sophisticated models.

I know a bunch of data geeks who are probably wincing at this, but I don’t think that’s the point.  Data is always good, and validating that data in any variety of ways is a huge need in any strategy.  But the point is that a lot of good business people have an innate understanding of measurement that works for their business.  Using data to validate that is a great way to certify instinct.

One point, though… once you get enough data to justify your pre-conceived conclusion, keep digging for a while longer.  I’ve seen a lot of validation justify a practice that doesn’t take in all the factors, so I’m a strong proponent of using all the data available to find all the right conclusions.  There, now the data geeks can relax.

No comments yet

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: