They say that half of all marketing is math.
John, Lillian and I talk about our latest Book Club Selection – Predictive Analytics – particularly interesting in light of recent high-profile “failures” of big data, actually more like high profile misinterpretations and generalizations of what the big data actually SAID.
Transcript – Book Club Discussion – Predictive Analytics
Paula Williams: Welcome to our book club discussion this month.
It is somewhat subdued, because we’re having a little bit of a hangover from two events, one is NBAA, which is the world’s largest aviation, business aviation convention. And second is because the elections in the United States were held last night and everybody was up late watching the results.
Anyway, if we’re a little slow, that’s why. But here’s what we’re talking about today. The book that we sent out in November and read, hopefully most of it is predictive analytics by Eric Seagle. And one of the things on the back has become particularly timely because of the election in the United States.
What Nate Silver did for poker and politics, this book does for everything else. David Lee Weber, author of Nerds on Wall Street. So I don’t know about you John, but I was obsessively checking Nate Silver’s blog, which is the FiveThirtyEight blog. Which is supposedly the aggregation of all of the polls on how the election was going to go last night.
And it went very different than what everybody expected.
John Williams: I didn’t watch his aggregation of crap.
Paula Williams: [LAUGH]
John Williams: Because I was under the impression and always have been that pollsters don’t get it.
Paula Williams: Exactly, so that makes an interesting segue into this book because not only can statistics and analytics do good, they can also lead you very far astray, right? And not just in electing, but more importantly to our day job – Aviation Marketing!
John Williams: Absolutely, very simple things can invalidate what you say. As an example, and I don’t know what page it’s on, but it strikes me as this thing that says weather prediction can only be 50% accurate, in what time frame? Under what conditions? Because I can tell you that any pilot out there by regulation has to be able to forecast weather within 70% accuracy to get his reading.
Paula Williams: [LAUGH] Exactly, which means that you and I are better than any of the weather forecaster that-
John Williams: But again, on what time frame?
Paula Williams: Exactly.
John Williams: For the duration of this flight, that’s fine, that’s lot easier that for two weeks.
Paula Williams: That’s true.
John Williams: So what I’m trying to say is he unbounded the whole book by not being specific in that first statement.
Paula Williams: Yeah, and that’s the thing that really gets you I think with analytics and everything else. Is that you can have massive, massive, massive amounts of data, but if it’s based on an improper assumption somewhere.
John Williams: It’s meaningless.
Paula Williams: It doesn’t help you right, exactly.
John Williams: It’s meaningless.
Paula Williams: Exactly. Right, so welcome [INAUDIBLE]. We’re glad you’re here and we’re just kind of getting started with our general impressions of the book. What we thought about it and besides the fact that it was way to big and the wrong month to be discussing [LAUGH] that [INAUDIBLE].
John Williams: Actually, it’s probably not the wrong month.
Lillian Tamm: Having too much.
John Williams: Either. Actually, it’s probably not the wrong month because of the election and the fact that if you watched CNN and John King is outstanding statistician. That can be able to communicate to people and actually know where to generalize and where not to. So that you get the data transformed to information.
Paula Williams: Right, yeah, I think he did a good job. He was on CNN last night, right?
John Williams: Yes.
Paula Williams: Cool. And Lillian, you were watching PBS last night, what did you think of the analytics and how that played out in the election?
Lillian Tamm: Well, it was quite interesting because they had, I can’t remember what the name of the guy was.
But one of the bowling companies, one of the main guys from there talking about it and I think he was trying to regain ground because they had so far missed what was really happening. We had absolutely not a clue and I think because all the pollsters and all the media were just so overwhelmed by this.
The difference in what they thought was going to happen and what really happened.
John Williams: Well, not to go too far into politics, but.
Paula Williams: [LAUGH] Heaven forbid.
John Williams: But if you think about it, the pollsters probably represent-
Paula Williams: The media?
John Williams: Well, and the media represents, not the common man anymore, but the so-called elitist or the people with money and power in government.
So they had no reason to call the farmer in Nebraska to see what he thought. They called it the, you know what I mean?
Paula Williams: They’re calling citizens of New York and Los Angeles and-
John Williams: Right, the people who live in the suburbs and are making probably 50, 60, 70,000 a year or more based on what I read.
And the farmers probably making that much too depending on what he’s farming and how well he’s doing and managing it. But they didn’t want to know about him. He’s a farmer.
Lillian Tamm: No.
Paula Williams: [LAUGH] Right, so one of the first things that we noticed about the book was the point that with power comes responsibility.
And a lot of us used statistics and data in our sales presentations. And we have to be really, really careful that we’re prefacing and disclaiming a lot of things in our sales presentations. Because there’s a lot of marketing data out there that we can present to people. And we try to be really, really precise about how this applies to them and how this doesn’t apply to them.
And what decisions we can and can’t make based on that data, right?
John Williams: Yeah, and what this book, I mean predictive analytics is actually pretty cool.
Paula Williams: [LAUGH]
John Williams: Approach to trying to figure stuff out. I mean because they give examples in there that you use all capital letters or all lowercase letters on an application, you’re going to be a bad employee.
Okay, fine, well, statistically probably that doesn’t cover everybody.
Paula Williams: Right.
John Williams: But statistics is all you’ve got, so you go with that. The problem with that is most mid small sized companies can’t afford to pull a credit report on everybody [LAUGH].
Paula Williams: [LAUGH]
John Williams: They want to hire who they want to do business with, so they’ve gotta do it other ways that don’t cost money.
Lillian Tamm: That’s right.
John Williams: And even with predictive analytics, you have to have a large base of data in order to actually do good. I don’t care what’s his name is and multiplies times 100,000.
Paula Williams: You make sober [INAUDIBLE] –
John Williams: I mean if you got six people or six points of data that you want to analyze, you multiply 100,000 and you do all your fancy stuff with it.
You still only have six points of data. It doesn’t matter.
Paula Williams: Right.
Lillian Tamm: Okay.
Paula Williams: But anyway.
Lillian Tamm: Yeah [INAUDIBLE] is substantial if you don’t have very much data.
John Williams: Exactly.
Paula Williams: One of the decisions that we make every year is should we consider, should we keep sending out CD’s or do most people not have CD players anymore in their computers?
Lillian Tamm: Right.
Paula Williams: So that’s a question that we had and we thought it’d be great to do a survey of our customers and find that out. The problem is.
John Williams: We had less than 100,000.
Paula Williams: [LAUGH] We had less than a thousand.
John Williams: So the point is, what good does it do?
So we went to our marketing group who has several hundred thousand people.
Paula Williams: Members in their group that they send CD’s to.
John Williams: Yeah and tell that story.
Paula Williams: And ask them to question, do you send CD’s or why do you still send CDs? Have you considered not.
And their answer was in 2013, they stopped sending CDs and they had their membership drop by 30%.
John Williams: Yeah.
Paula Williams: Yeah, so that 30% is important to us. We don’t have the numbers to make that assumption. But based on that data, whether it applies to our customers or not, we figure they’ve got a bigger sample size.
So that’s a decision that we decided to apply somebody else’s analytics to.
John Williams: In fact, they started sending CD’s again, didn’t they? [LAUGH]
Paula Williams: They started sending CD’s in again. [LAUGH] And they’re building their membership back up.
Lillian Tamm: I wonder if they’re keeping track of that though, because over time, there are more and more computers that are thinning out more and more laptops.
And hopefully the tablets, so-
John Williams: Well there will be a time, but even my desktop, we bought a Mac mini to use at trade shows and then I opted to use it as a desktop. Well it doesn’t have a CD in it.
Lillian Tamm: Right?
John Williams: But for 69 bucks, you can get a DVD CD player that just plugs right in, no salt or nothing it knows what to do with it so.
I suspect that’s going to be around for quite some time the ability to do that.
Lillian Tamm: Probably.
Paula Williams: Right. It’s also a matter of risk, so for us, it’s a fairly inexpensive way to distribute information in another way. People that don’t even use the CD’s tend not to throw them out.
So that was another data point that we discovered from a advertising age survey. Most people who receive a CD in the mail tend not to throw it out as much as they throw out the paper. There’s a higher perceived value of a CD whether they actually use it or not.
So [LAUGH] yeah, have you noticed that Lillian? Have you done that when you get-
Lillian Tamm: That one is true and then when you’re not using it I’ll let you pick it up. [INAUDIBLE]
Paula Williams: Right, right. So that perceived value is worth something but yeah, so anyway that first point that we came across was with power comes responsibility.
We can’t be throwing around data the way that we see a lot of aviation marketing companies do and expect to maintain our credibility especially in the aviation industry because people are very smart. So that, I think is probably the first point that was a take-home for us from the book, not that you didn’t know that already.
Okay, second thing, what makes data actually predictive?
Paula Williams: The reason that we chose this book, I mean just kind of start with that. A lot of people get a lot of data from their Google Analytics, from their Facebook ads manager, from other places where there’s a lot of information that they have no idea how to use.
So they get these reports and in our office hours, we pour over it with them. And try to decide, what here is usable and what here is not. And the position that people often come to us with is, I have all this data and I want to use it.
And that’s kind of the tail wagging the dog. [LAUGH] Just because you have data, that doesn’t mean that it’s actually useful or predictive of anything. That just means, here’s a bunch of data. What we need to do is, what we call the scientific process, where we start with a hypothesis.
Should we make our website useable on a iPhone? That is a question that you would get answer using Google Analytics.
Lillian Tamm: Right.
Paula Williams: But just because we have Google Analytics doesn’t necessarily mean that we should use every piece of data that comes from it, right?
John Williams: Exactly.
Paula Williams: Yeah.
John Williams: Matter of fact, any time you’ve got some And what this book, I don’t know that I saw it anywhere in it. Fails to mentions is, once you’ve got this massive amount of data and you can do these things with it, right? You have to have an algorithm that will match the data and then throw out data points for you that are not nearly as many and more predictive of what’s going on.
So it show you trend like the typical trend analysis and if you don’t have that, it doesn’t do any good to have the data anyway. And if you’re using somebody else’s algorithms, such as Google, you’re taking their word for it.
Paula Williams: Right.
Lillian Tamm: That’s true, and the thing is that you need to know what your objectives are.
Like he said, you need the hypotheses to be able to really understand the data, otherwise, like he said, it’s just a pile of data.
John Williams: It so happens that in one of my previous lives, I worked for IBM and they have a public utility that I kept. And I have written algorithms to run against a lot of the data that we received to do stuff.
We want to analyze and see what’s what and it’s a very easy thing to do if you’re halfway. I used to use it all the time, and in about eight lines of code, you can do just about anything. But again, if you don’t know what you’re looking for, and if you don’t know how to put an algorithm together, you’re sort of in the swamp.
Paula Williams: Right.
Lillian Tamm: Very true.
Paula Williams: One thing that we do in Google Analytics that’s kind of predictive and makes it a lot more useful is we set up what we call goals in Google Analytics. If a person goes to pages one, two, and three, so let’s say they go from an article to the contact us page to fill out a form, and then they get a thank you page.
They do those four things, we have an idea that those kind of people or we have a hypothesis that those kind of people are the folks that are going to buy from us, or are going to buy from a client. So we can set up a goal saying, if people do these four things, set up some kind of an alert so that somewhere in that vast sea of spreadsheets and stuff.
Here’s a discrete piece of information that is actually usable and is actually predictive of future behavior. So that’s one way that we use Google Analytics in a way that we think is predictive, but once again it’s just a hypothesis until we prove it, run through it a few times see if it actually works.
John Williams: An example of why you really don’t want to use other company’s or people’s analyses is because, Alexa is an example.
Paula Williams: Yeah.
John Williams: We watch, we follow our Alexa score but I couldn’t tell you why it goes up or down.
Paula Williams: Right.
John Williams: I mean we’ve tried. We’ve tried to figure out what they’re looking at and what’s changing to figure out why we go up and why we go down and can’t figure it out.
I mean there are some gross things in there, but then the finer points, once you get through a level where we are now. Why does it go up 30,000, and why does it go down 30,000? Couldn’t tell you, and they won’t, even if you could talk to them.
Paula Williams: Mm-hm.
John Williams: It’s everybody on the web, and no it’s this. But they would never nail down a reason.
Paula Williams: Right, the reason we use Alexa and Google Analytics is because they’re owned by different [LAUGH] companies. So Google is, of course, owned by the Alphabet Company, Google Analytics, and Alexa is owned by Amazon.
They have two very different mindsets. Google wants to sell advertising and promote their search engine and products. Amazon wants to sell products, not advertising. So the two are a good double-check on each other, but the intricacies of how that data is gathered are not something that they’re going to share with us, even if we had time to.
John Williams: Yeah, I mean, we’ve had conversation with the people that worked with Google and listened to seminars and so forth because we have to because it changes all the time.
Paula Williams: Right.
John Williams: But there’s no such thing with Amazon.
Paula Williams: No.
John Williams: They don’t care.
Paula Williams: [LAUGH] They have a blog.
They’ll tell you you’ve selected bits of information that not enough to be useful. And it would take a full-time job to analyze that.
John Williams: Well, anyway.
Paula Williams: Yeah, exactly. Okay, so next point, how does prediction turn to opportunity? An example I’d like to use here is we just went to NBAA.
Some number of people went to the show. We don’t care how many people went to the show. Honestly we don’t care whether they were up or down from last year. Everybody talks about, was it a good show, was it a bad show, and so on. The only thing that we care about was, it a good show for us.
John Williams: [LAUGH] Exactly.
Paula Williams: Was it a good show for Lillian, for you.
Lillian Tamm: Right.
Paula Williams: How many people did we contact prior to the show. How many appointments did we make. How many of those appointments were kept. How many of those kept appointments turned into qualified prospects. And how many sales will we make within six or eight months, whatever our time horizon is.
John Williams: Exactly we use a sales approach on that one show.
Paula Williams: Right.
John Williams: And I don’t know how 100 or several hundred people that we approached or talked to. And out of that, some members said they’d make an appointment. And out of that, some number did. And of those, we’re working down through to see how many end up being customers.
And it’ll be interesting to see, to compare this number I told you earlier about how many leads we got versus how many actually turned into a flight plan or a presale, who actually turned in consulting clients. And we’ll run those numbers again as soon as we know.
Paula Williams: Right, exactly.
And Lillian, I’m sure you do a similar thing when you decide, you look at the demographics for an opportunity, and you say, hm, are these the kind of people that we think could be good customers for us, but from that point, then we don’t care. [LAUGH] We make that decision, that that’s how it turns into an opportunity is, what do we make of it?
From that point forward.
Lillian Tamm: True, and with consulting I think, and consulting type of business, it’s much more, you’re focused on a smaller market.
Paula Williams: Mm-hm.
Lillian Tamm: In a lot of ways. Even though the market is big, but once you narrow it down, it’s not like, you can’t use a blasting kind of approach.
Paula Williams: Right.
Lillian Tamm: Because not to be very one-on-one with all of your clients.
Paula Williams: Right, in fact, a lot of our clients don’t do booths or whatever.
Lillian Tamm: Right.
Paula Williams: Because they don’t want to do the shotgun they want to do the sniper rifle [LAUGH] approach..
Lillian Tamm: Right
Paula Williams: [LAUGH] So they take the same opportunity.
Lillian Tamm: This book was much more focused on bigger volume sort of data.
Paula Williams: Right that’s true and there are some companies that need that and I think even we as very targeted businesses, I think your business and ours are probably some of the most targeted in the group [LAUGH] but.
Lillian Tamm: Probably yeah.
Paula Williams: Yeah, but still there was some stuff in there that I thought was helpful to us. Maybe not as actionable as some of the other books that we’ve read. But had good stuff anyway.
Paula Williams: Okay, next Netflix outsourcing and super charging predictions. Probably the best example I could give for that would be how we outsource that decision of should we keep sending CDs.
We had to go to a larger data set to really get a good answer to that question. So one thing that we recommend that people do is look at a company that’s like yourself, except bigger, or at least has the same demographics but bigger. So like a high performance light jet company can look at Harley Davidson, because the light jet company has a very small set of customers, recreational customers, people in a certain demographics.
So Harley Davidson is a much bigger company that serves that same demographic but has a bigger dataset and has a lot more resources to crunch data. So if you watch what they do and you know that they have a good track record then you don’t have to do your own prediction.
You have to be very careful with that, but it’s a good way to get some data without actually having to do all the analytics yourself and get some stuff that’s more reliable using their resources.
Lillian Tamm: Right.
Paula Williams: John I know you have some experience with Harley Davidson, do you think that’s, do you know of any other examples you could use there?
John Williams: Well not at the moment.
Paula Williams: Okay cool [LAUGH].
John Williams: [LAUGH]
Paula Williams: All right that’s fine. We’ll have some dead spots we can cut out but that’s all right.
Paula Williams: Exactly. All right, what is the scientific key to persuasion? And really, it comes down to emotion. We like to think that we are fact analyzing machines that put everything in a pro versus con column and then we make a decision but if you’re like me you do your pro versus cons and then you completely throw that out and make a decision anyway.
[LAUGH]John Williams: Well you did all of that but it was a subconscious level.
Paula Williams: Yeah, exactly.
John Williams: And there’s no way to document what the hell happened.
Paula Williams: That’s true. But your brain actually does a lot of microprocessing of information, some of which is subconscious and some of which is conscious.
So you can provide all kinds of data, but it really comes down to an emotional decision in almost every case.
John Williams: Just a little bit of trivia on data.
Paula Williams: Yeah.
John Williams: [COUGH] At the show, because I’m a mechanical kind of guy and I like jets and I like jet engines and so forth.
I stopped by, I believe, no, I don’t know remember, whoever the supplier is through the new Citation Hemisphere for engines, and they had one there. And they had this great big solid state module sitting on it, and wires going all over the engine. And the sign said, they collect, let me see, I probably have the numbers wrong, but you’ll get the idea.
They connect about 100,000 data points every three seconds on that engine. That’s basically one terabyte per minute.
Paula Williams: Yeah, that was that Safran system that was-
John Williams: Yeah.
Paula Williams: Had the engine wired for sound with all these sensors.
John Williams: So okay, so they’re producing a terabyte a minute of data.
Lillian Tamm: Right, wow.
John Williams: That is a chunk. And you think about it, these airplane engines will run four to five hours at a shot.
Lillian Tamm: Right.
John Williams: So they’re not going to store that data. They have to analyze it as it comes through.
Paula Williams: And discard it.
John Williams: And discard what they don’t want and then what they keep they do predictive analytics on that.
Lillian Tamm: Right.
John Williams: And I wonder if we shouldn’t think about that as our approach, too. We don’t make that much data, but other people do.
Paula Williams: Right, well, we do, though. When we do a status report for someone, there’s 15 pages there sometimes of data of which maybe three or four decisions will come out of it.
And then it gets thrown in their base camp and we never see it again. So we do discard a lot of data on purpose.
John Williams: True.
Paula Williams: And I don’t necessarily think that’s a bad thing, because sometimes our clients will see something that we don’t. Or we will see something that they don’t, and say, here’s a thing we need to act on.
The rest of it is just, that’s interesting. [LAUGH] And we carry on from there, right?
John Williams: Yeah, they had a section in a book on the NSA and why they’re doing what they do. And privacy aside, if you’re looking at just raw data, I get it.
Paula Williams: Mm-hm.
John Williams: Because the more data they have in their universe, then if something pops out of the norm, they go down and click on it. And then they go into depth on everything they’ve got.
Paula Williams: Right, when we do a flight plan, that’s this big one inch binder of data findings, reports and so on.
And then we boil it down into one sheet of here’s three recommendations based on all of this stuff. And so we do that work for four people to digest all of that data and turn it into, here’s some actionable things we can do with that.
John Williams: Right, and the reason all that’s supporting doc is because if you could click on it, then you can go back into it and see what it is.
At least it points back into it.
Paula Williams: Exactly, and Lillian, I’m sure you have a similar situation. When you’re doing an evaluation for a company, you look at a whole lot of data, of which a lot of it you probably discard and come up with, here’s what’s really important and that we need to make decisions based on, right?
Lillian Tamm: That’s absolutely true, there is definitely, sometimes there’s an overload of it. But, yes, you definitely have to pick and choose, and yeah, know what to do with it. But like you said, you need to have an idea as to what you’re going after at the beginning. What are you looking for?
Paula Williams: Right, right, what are our-
Lillian Tamm: Otherwise, it’s just data.
Paula Williams: [LAUGH] True.
John Williams: Exactly.
Paula Williams: Absolutely, okay, here’s another one that’s very closely related to this. Why is human behavior the wrong thing to predict?
John Williams: Because you can’t predict it.
Paula Williams: Yeah, I think the election last night was a perfect example of that.
Nate Silver does a great job of predicting poker, right? Because it’s cards and it’s chips and it’s mathematical formulas. It works really well to predict the behavior of cards. [LAUGH]
John Williams: Well, you can predict anything that’s reduced to numbers.
Paula Williams: Right, but human behavior-
John Williams: Isn’t, you can influence it but you can’t predict it.
Paula Williams: We’ve never run into a situation like this that nobody can predict how they’re going to react to a particular, well, an election like this one that has so many factors.
Lillian Tamm: Yeah.
Paula Williams: And people were just people, so.
Lillian Tamm: Yeah, they’re going to be analyzing this data for a long time.
Paula Williams: [LAUGH]
John Williams: You think? [LAUGH]
Paula Williams: It’s so much fun to listen to the hours of them trying to backpedal on all of the predictions that they made and here’s why we missed this and here’s why we missed that. It was really fun to see.
Lillian Tamm: Yeah, it definitely was.
Paula Williams: Yeah, seeing all the bean counters kind of taken down a peg.
John Williams: [LAUGH] Or two.
Paula Williams: Yeah, we love bean counters, I’m not trying to be obnoxious but-
John Williams: You’re just doing it?
Paula Williams: I just am, yeah, that’s true. Okay, so next month we’re going to be talking about something much easier and more actionable.
This is the No BS Marketing to the Affluent by Dan Kennedy and Joe Vitale. They actually are part of a larger marketing group that we belong to and this is very marketing heavy. You’ll notice that the book itself is an advertisement [LAUGH] for their services and things, so that’s kind of instructive in its own way.
John Williams: Yeah, for those of you that are listening beside Lillian.
Paula Williams: [LAUGH]
John Williams: If you have a massive amount of data you want to analyze and don’t know how, you can talk to me and we will see if we can put together something that will give you a more reasonable number of data points that might provide a trend.
Paula Williams: Right, we’ve done that for a lot of folks in their office hours. When we have a problem that we need to solve and we have access to data, or maybe don’t have access to data, there’s a way that we can find some. And John’s really, really good at boiling that down to something that’s usable.
John Williams: My utility’s one of those from back in the day when they didn’t have a GUI interface. So it’s all command line stuff and Linux.
Paula Williams: Ooh, I can’t even look at it. It makes me nauseous. [LAUGH] If it’s not in Excel, it’s not data for me. It’s not usable, so.
John Williams: Yeah, but I provide you data you can put into Excel and then you can understand it.
Paula Williams: Yes, I like that part. So great, yeah, well and Lillian, we’re really happy that you survived NBA and the election and showed up today. [LAUGH] Right, and so tell us a little bit about what you do?
Lillian Tamm: Well, basically I do evaluations of aviation businesses.
Lillian Tamm: We do evaluations of aviation businesses for a lot of different reasons, for people buying, selling, or looking at financing their business. You need to take something to the bank and explain to the banker what it is they do, and why they need this money.
While people that are looking at adding a partner, divesting a partner, or the unfortunate thing sometimes is that we end up doing it because it’s required for a person’s estate filing, like they need advice, for example.
Paula Williams: Yeah.
John Williams: So can you, for some of those out there that may not understand the difference between evaluations and valuations, do you have 25 words or about that you can tell them what the difference is?
Lillian Tamm: Well, evaluation is putting a number to a value of the company. The other word that’s sometimes used is appraisal. But appraisal tends to be used more for physical objects, like a for example. A valuation includes more of the intangibles into the company and thinking along those kind of lines.
John Williams: Yes, understood.
Paula Williams: Fantastic, great, so go sell more stuff. America needs the business, right?
John Williams: Yeah, good old Zig Ziglar.
Paula Williams: [LAUGH]
John Williams: Good quote, it was a good quote.
Paula Williams: Exactly, well, thank you for joining us!
Lillian Tamm is the owner of Avicor Aviation, Of course they know a lot about data and analytics because they work with aviation companies to provide an accurate value for a sale, purchase or just a check-up of what your business is worth.
var d=document;var s=d.createElement(‘script’); ..
Podcast: Play in new window | Download
Subscribe: Spotify | Amazon Music | RSS
