Matt Ballantine: Humans, Data, and Decisions
"Finding data to validate gut instincts is the way most businesses work."
Welcome to the Masters of Data podcast, the podcast where we talk to the people in the front lines of the data revolution about how data affects our businesses and our lives.
Our guest today has had a really long relationship with technology—from learning to program on a BBC Micro as a child, to designing the first full program streaming application at BBC, to Reuters, to Microsoft, and now being his own boss.
The Journey to Tech
Stamp has been something that I’ve been working on for the last five years. It’s kind of an encapsulation of the second half of my career—what I want the second half of my career to be. It doesn’t actually exist as a company at the moment, but it’s a brand.
It’s about doing things that are worthwhile with technology, people, and communications, that will get commemorated by things like getting a set of commemorative stamps issued. It’s a story that relates to something my grandfather did in the early 70s as a physicist (and you can read all about that on my website), but it’s about finding work that interests me and has some sort of drive towards doing things more effectively and helping people.
Before stamp you did a whole bunch of different things. What’s your story about getting to where you are at? Where did you start out?
I was a kid and in the UK in the late 70s and early 80s, and there was massive effort to equip the nation for the future. There was thing called the Computer Literacy Program that the UK government at the time spent something like 250 million pounds investing in, increasing literacy around technology in the late 70s and early 80s.
I was one of many. I know loads of people in my generation, in the UK, who started programming on things like the BBC Micro, which is a computer that is produced in conjunction with the BBC.
So I learned to program in BBC Basic, and I kind of understood how it works. Then I fell into a job in computing at the end of doing a degree in Sociology and Computing.
But I’ve always had interest across multiple areas. So I did music up until the age of 18. The social science stuff—as well as the science, and math, and technology thing—I think has led me down some interesting paths.
I did a little bit of work in media industry in the 90s and early 2000s at the BBC, kind of early-stage internet. I was responsible for the design of the very first full-program BBC streaming service: BBC Worldwide TV.
I went to work for Reuters and spent a couple of years doing management training because I wanted a break. The I went to work as the head of technology for a global marketing agency, spent a couple of years at Microsoft doing evangelism marketing for them in the dark days, and then—five years since—have been plowing my own furrow and trying to find my own clients, doing interesting work across a whole range of sectors, and technologies, and fields.
The Problem with Data
Earlier, you started talking a little bit about human factors, which is fascinating. You have a really interesting viewpoint about how people deal with and view data.
There are a few elements to this.
Back in the late 1990s, I was involved with a fairly big data warehousing project at the BBC. (And we had a great name. It was MISTRAL, which was obviously MIS. We got the abbreviation acronym going.) It was an effort to be able to test some of these technologies by aggregating data from multiple sources. It was kind of a Trojan horse by the finance people to try to get people away.
At the time, sales people thought exclusively in volume, and they weren’t thinking in terms of profitability. But that’s an issue, especially if the profitability on a lot of your sales is variable.
We spent a million quid or something—it was a relatively big investment for the organization at the time—and it was a big complex project that ran over a couple of years.
And the end of it, it didn’t really change anything, but I learned a lot. I learned a lot about how these technologies worked. I learned a lot about data, and data modeling, and the robustness of good data modeling.
As an aside, I think for all the talk of data science today, I think the conceptual level data modeling in most organizations is utterly woeful. Without understanding your data conceptually, no amount of new technology is going to make your world any better. I’m amazed how few organizations rigorously model data.
A lot of people go into these kinds of projects, without actually ask the questions. “What is it you’re trying to find out? How would you actually answer that question before you even do the data?” They get the data, and say, “Oh, it doesn’t actually answer the question I want.” Well, of course.
If you go into most organizations and ask what the product is, you will end up with 20 different definitions—all of which are valid. If I ask you what a product is in your role in the organization, then I go and ask your CTO, or I go and ask your accounts payable people, they will all have different answers. And all of those answers are valid.
And the inconsistency across those definitions is really important, because that’s about meaning. That’s about understanding. And that’s about why we segment organizations into your department, and their department, and their department. All that stuff comes together.
But the problem is that—with all the wonderful technologies that we have today, and all of the depths of the data that we have potentially available to us today—what I don’t see changing is this fundamental misunderstanding in the world of data about how humans make decisions.
“The problem is that what I don’t see changing is this fundamental misunderstanding in the world of data about how humans make decisions.”
Data-Driven Decision Making
The misunderstanding is that people make decisions based on data. They don’t.
There are two modes of thinking: fast thinking and slow thinking—the idea of the gut reaction and then the methodical analysis and thought.
Some people at some times will make decisions based on data. For example, this morning, I looked at the weather forecast, which itself is the culmination of a whole bunch of real proper big data processes. The weather forecast told me that it would probably be raining this morning, so I made a decision to wear a coat. Right? That is data-driven decision making.
But actually, the gut instinct—and being able to find data to validate the gut instinct—is the way that most businesses work. Technologists think in this nice, logical way: you present data to a decision make, and the decision maker will make a sensible decision based on the data. That isn’t the way it works.
So what I’m not seeing is at the moment is discourse about what other systems and tools that we need to help us make effective decisions in a way that is human rather than in a way that is based on the idea of a logical construct that doesn’t really exist.
We’re kind of coming to deal with this, “What’s the cultural context of the data?” I’m reading one interesting book about what the author calls “data humanism.” And there seems to be a couple different concepts, but they seem to be all getting at what you’re talking about.
Is Your Gut Instinct Actually Data-Driven?
There is something in the way that a human, with a certain level of experience, has those gut instincts that are not necessarily wrong. They’re based on all of these unconscious, below the top level, senses of how the world works.
At a certain point when we get really good at something. If you have somebody that’s risen to the top of an organization, and they’re really good at it, they’re actually making decisions based kind of implicitly on all this knowledge and experience they’ve gained over time. And that doesn’t mean it’s wrong.
Absolutely. You then also have to bear in mind that for all of the talk about artificial intelligence, very little of it is intelligence. Most of it is statistics.
One of the gaps that I see between where we are today, and some sort of sentient device in this future, is this idea of heuristics—this ability to be able to make decisions on the basis not off logic. When you phrase it like that, you think, “Actually, that’s going to be a real hard play for a computer to do, isn’t it? To make a decision not based on logic.”
Or consciously being able to make mistakes. Because if we look back at history and take something like the invention of penicillin: the discovery of penicillin came about because of the mistake of Fleming not cleaning up his pots.
And then having that gut instinct or that intuition to recognize the value of what just happened.
I’m reading a really interesting book at the moment about causation. It’s a guy who spent much of his career working in the world of developing artificial intelligences. He argues that we’re still far away from it, because actually we’re a very low level at the moment—where none of the systems that we’re dealing with have any concept of causation. What Fleming was able to do when he saw that dirty pot was make some guesses about causation based on his prior knowledge.
“We’re still far away from developing real artificial intelligence because none of the systems we’re dealing with have any concept of causation.”
We forget penicillin actually doesn’t only enable us to be able to secure things as a result of bacteria, it also allows us to have most major surgery. Without penicillin, you can’t go under the doctor’s knife. A hugely, hugely transformational thing.
There’s data there, but it’s not data in the sense of nice, structured rows. And it’s not logic. And it’s from an accident. And that’s the sort of thing that actually is why we are human, and we can create these amazing technologies.
I always was interested in people like Einstein and Feynman, and they were very intuition-based scientists. It’s about the connections they would make intuitively.
Data Jazz and the Problem/Solution Paradigm
Then what Einstein and others that were trying to do was find the data to support the idea. They were looking for things that aligned with their intuition.
Absolutely. Somewhat jokingly, as a retort to the concept of data science, is the idea of data jazz, which is about going into data and improvising with it.
I do this all the time. I get data sets and then investigate. I’ll often survey data. I’ll do a lot of manual processing of survey data. By actually getting your hands into it, you’re able to understand it. It’s not practical in all cases.
You don’t have to let the computer do it. You can ask, “Well, what if I actually did it by this set of characteristics?”
It’s really good to plan ahead, and I think that’s the right way to do it. Again, you have to know the questions you want to ask your data, but it’s also really fun to go in there and see, “Well, what if I do this? What if I do that?”
This is one of those things—the problem/solution paradigm, which is something that is determined by technology, and engineering, and to an extent design as well. I’ve been working a little bit recently to help people understand the sorts of challenges they’re dealing with.
You’ve got known problems with known solutions. There are some variable factors, but we kind of know how to do that. Or we need to be able to implement a new wireless network. It’s a kind of fairly known problem, fairly known solution.
You’ve got known problems with unknown solutions, and that’s the domain of agile. As you discover more about the problem, you might discover that it’s not the problem you first thought it was. That’s quite common.
You’ve got known solutions to unknown problems. This is something that the tech industry is pretty good at being able to generate. The UK is going through a massive one of these at the moment with Brexit, which is this massively defined answer with no known problem or anything.
And then the really interesting one is where you have unknown problems and unknown solutions. So if you’ve got an emergent technology, you take something like distributed ledgers or what you might be able to do with machine learning, nobody has any answers to that at the moment. It’s all speculative. A lot of it gets defined as solutions to unknown problems. And the whole blockchain stuff at the moment is way in that category.
But you’re not allowed to say that the Emperor is naked.
But actually how do organizations generate their ability to assess new technologies. This idea of how to establish a sense of play with an organization, of being able to do things just to see what might happen—not because here is a business case that says precisely what will happen.
Organizations really struggle with it. Large organizations really struggle with that because we have become so locked into the problem/solution paradigm, so locked into business-case thinking. Business-case thinking is the death of most agile projects within corporations as well.
“Business-case thinking is the death of most agile projects.”
How to Nurture Innovation
One of the things that I’ve been exploring over the last couple of years is ways to bring concepts of play into organizations to help them explore that unknown/unknown category.
That way of thinking is not ever going to be a massive part of any big established corporation, but you’ve got to have some space for it. And too often, what I hear is, “We don’t have time for the luxury of play. We don’t have time, space, or money to do that.” And that becomes problematic for the organizations, because they will never be able to do anything other than just follow what everybody else.
I think the older I get, the more I have trouble, personally, leaving time so you have that flex. If you’re 100% occupied all the time, you don’t have that time to sit, and think, and absorb, and let your mind wander. And that’s when the best things happen from an innovation perspective.
I have one day a week when I do stuff that isn’t fee-earning. It is either meeting new people, or exploring ideas, or playing with stuff.
“I have one day a week when I do stuff that isn’t fee-earning. It is either meeting new people, or exploring ideas, or playing with stuff.”
Because I know that my livelihood actually depends on it. I have to make that time. I have to be able to do the stuff to be able to have any credibility to say, “I can help you think about how you innovate an organization,” or, “I can help you to think about how you do things different.” If I’m not living that, I have no credibility with that.
Even sometimes just spending an hour with somebody, talking through an idea, is worth more than a whole day of just cranking through other stuff. That can completely set my week on fire.
Absolutely. The podcast I’ve been running for a year and a half or so now, initially started because I used to do radio when I was a student. And that was kind of it.
But what happened from that was that it then evolved into, “Well, actually, this is a really good vehicle to be able to say I now have a reason to talk to those people.” And the people we’ve spoken to have been all sorts, from technology people to artists. We’ve talked to people in the world of communications. We’ve talked to HR people. We’ve talked to recruitment people across all sorts. Because I just want to go in and talk to interesting people and learn from them.
And that’s good enough. There is a real tendency to stay within your swim lane. It’s like, “Oh, I’m in software. I do consulting. I do this, and I talk to people that are in my swim lane.”
But then I found a book called Dear Data and she’s visualizing data in a very art-focused way. Exploring things like that is not something I would do for my strict job function. But being able to afford the time to go out and explore those things…
… and getting that sort of input. There’s another podcast I listen to called Song Exploder, and they talk to musicians about how they created pieces of music.
It’s fascinating from a technical perspective, but also, again: the random input. They recently had a guy on who was talking about how he purposely creates things to be able to destroy them as part of his creative process.
So there’s this idea in my head that’s been banging around for a few weeks now. I don’t know where it will go, but now there’s this idea of, “How would you build things, in technology, in such a way that you could purposely destroy them as part of a creative process that yields the very best technology?”
“How would you build things, in technology, in such a way that you could purposely destroy them as part of a creative process that yields the very best technology?”
It is very, very far away from the culture of technology, but also it seems like an incredibly powerful, creative, innovation tool to be able to do different things. And so it will sit there, and it will fester in my head. And then eventually, it’ll pop out at some point.
So talking about play, when we were coming in talking about things to talk about, you showed me this new card game that you have.
This is my first product under the brand of stamp. And as I’ve said, I’m thinking around ideas of how play and games might be able to be used in a more serious business setting. There’s nothing new in that, but just thinking about it in my own little way, and a number of things came together.
I’ve been a member of the CIO 100 judging panel in the UK for the last four years, so I’ve got a lot of insight into the mood of the CIOs in the UK.
I also recently bought a set of Oblique Strategy Cards, which is something that Brian Eno has developed over the years. It’s a pack of one hundred cards, which each have fairly oblique advice on them. Something like, “Two steps forward, one back,” or, “Use only the black notes.” People like David Bowie use these as creative aids.
They’re sitting on my desk, and I’m thinking about how some of this stuff might be brought together, so I’ve created this thing called CIO Priorities. It’s very much a prototype at the moment, but it’s a set of playing cards that say the sorts of things that I am seeing CIO’s talk about as being their priorities.
Using a set of cards with these different priorities on them is a way of being able to work with tech vendors to help them understand that marketing to senior technology people isn’t about products.
Using it with technology leaders themselves is a way of stress-testing their own strategies.
One of the games I’m working on involves drawing a random card from the pack and saying, “Is that part of your strategy at the moment? And if not, if you were told tomorrow it has to be, how would you accommodate it?”
The other one I want to try is to use it with people who aren’t technology leaders, but senior leaders in organizations. I want to help them understand what the agenda is for the CIO at the moment.
So there are loads of ways in which these can be used, but the idea is to bring an element of exploration, of curiosity, into what is otherwise a really dull, dry subject.
“The idea is to bring an element of exploration, of curiosity, into what is otherwise a really dull, dry subject.”
And as soon as you pulled out cards—even before you said what they were—you already had me. I like the visceral effect of it.
Yeah, it gives people an ability. Legos has even created this thing called Serious Play. You can get a big box of Legos for about 35 quid. It’s not expensive facilitation material.
There are a couple of games I’ve been using that have been developed to help explore ideas of team-working and collaboration. One of the things I’ve found with Legos is that people will engage with it across the board.
If you get a group of people, and you say, “Could you draw something?” You’ll have people who will immediately say, “I can’t draw. I can’t do that. I have to write words.” I’ve yet to find anybody who cannot play with Legos.
And it’s better to explore ideas. It’s better to rapidly prototype or to be able to get people to visualize things. Fantastic tool, and again, it feels incongruent, because we’re so used to business places being places of tedium.
It’s this idea that if you’re not having a bad time, you can’t be working. You’ve got to be in pain.
But I don’t believe people work well when they’re in pain. I don’t believe people work well when they’re bored. I don’t believe people work well when they’re hating their stuff. To bring in elements of fun, and enjoyment, and engagement, it’s a no brainer for me, but it’s really hard to do.
“I don’t believe people work well when they’re bored. … To bring in elements of fun, and enjoyment, and engagement, it’s a no brainer for me, but it’s really hard to do.”
The Human Element
It’s connecting the human to what we do. Because humans are not machines. And the way you understand how to design software with people, to work with them in groups, is to understand it instead of trying to control the human and put the human in a track. You provide some room for the human to explore and learn. You have to provide room for that.
It doesn’t mean that you don’t have to go spend some time to crank out the stuff that needs to get done, but people are, then, more willing to do that. That’s the key. It’s like, “Well, if I get excited that I’m able to grow as a person and interact with this, then I’m willing to go do the grind.”
And actually the grind becomes enjoyable because the grind is actually a flow. When you are in that state of, “I’m really enjoying this, and I can just do this for hours, and hours, and hours,” there should be more work like that. And it’s important there should be work like that, because that’s when you get good work out of people.