In this episode of the Expert Insights Podcast, Craig Martell, Chief AI Officer at Cohesity and former Chief Digital and AI Officer for the U.S. Department of Defense, breaks down what modern AI truly is, and what it isn’t.
We explore how AI is being applied to real-world enterprise challenges, the risks of over-relying on “human-sounding” systems, and why Martell champions smaller, purpose-built models over the pursuit of general intelligence.
He also shares Cohesity’s vision for transforming backup data into an “accidental data lake”, along with key lessons from deploying AI at scale in critical, high-pressure environments.
Q&A:
Note: This transcript has been edited for clarity and length.
Joel Witts: Craig, thank you very much for joining us on the Expert Insights Podcast today. It's great to have you. It would be great if we could kick off with an introduction to yourself, your background, and what drew you into the world of AI?
Craig Martell: That's a really interesting question. My name is Craig Martell. I'm the Chief AI Officer for Cohesity. Cohesity is the world's largest data protection company measured by the number of terabytes under management. We have hundreds of exabytes under management. In my immediately prior job, I was the chief digital and AI officer for the US Department of Defense. As you can imagine, that was a fascinating job as well.
And I've held a number of positions in the AI field, starting back with LinkedIn, Dropbox, Lyft, and I was a professor at the Naval Postgraduate School for about 15 years. Your question about what drew me into AI is a really fascinating question, which might lead us far afield here, but I started off studying philosophy, and I thought that artificial intelligence might be a useful way to do what I called testable philosophy. Like what does it mean for humans to behave in certain ways?
And maybe we could build these artificial systems where we could test human behavior. But what I learned very, very quickly is that humans, number one, are not as special as we think we are, because simple statistical patterns is what solved most of AI's problems. And even today, modern AI is just statistics at scale. You just gather massive amounts of data from the past, you use it to build a model and use that model to predict the future.
And so even though there's lots of stories floating around that modern AI is thinking or it's reasoning, and we say things like mixtures of experts, and we say reasoning model, none of that's true. That's all just naming conventions. Like we're just picking names that sound good, just like artificial neural net. That's a name that sounds good, but it's got nothing to do with the brain. It's got zero to do, it’s so far from the brain. And so we just sort of pick these terms that make them sound sexy and cool and like they're thinking things, but they're not.
They're just massive statistical pattern recognition systems. But the fascinating thing from a philosophical point of view is, isn't it amazing how much pattern recognition and pattern generation seems like intelligence? So, this view of ourselves as these special things that are generating brand new thoughts that no one's ever had before and brand new strings of words. It turns out that if you just gather all the data on the internet, count it all up and then regenerate, it sounds an awful lot like you anyway, Joel.
Joel Witts: I think this is probably a record for how deep we’ve gotten so early in the podcast. I love that. That’s fantastic. Because there was a story, right before Gemini—Google Gemini was released, I think it was Bard at the time—that one of the engineers thought it was real, sentient.
I wonder, do you think we’ll ever get to that point, or do you think it’s always just going to be statistics?
Craig Martell: Joel, it’s not clear that those two things are separate. Sentience might just be statistics. We don’t know the answer to that. I don’t think it is, but it might be. Will we get to a thinking, conscious, sentient machine? That’s a fascinating question. I don’t think so if we stay on the current path. The current path of statistical modelling, I can’t imagine that getting us to sentience.
There’s a really great philosopher, John Searle from Berkeley, I think he’s emeritus now. He wrote this great book, The Rediscovery of the Mind. Essentially, what he argues—and I’m aping that argument—is that if we really want to build artificial conscious beings, it’s likely that we have to build actual artificial brains. What we’re building now are simulated brains. Simulated brains are not the same thing as artificial brains. So maybe we should start looking at brain science and thinking about how brains actually work if we want to get to consciousness and how brains give rise to conscious machines. I doubt very seriously that just counting up stuff from the past, building a massive model that uses all the electricity on the planet, and then generating strings is going to get us to consciousness.
Which, by the way, Joel, since we’re in a philosophy class now, is why I dislike the term hallucination. The term hallucination makes it sound like a cognitive being gone awry, like it’s a kind of being that just made a mistake. It misinterpreted its own internal signal. I don’t think anything could be further from the truth. It’s not a cognitive being. So I just prefer the term error. The thing got it wrong in that context.
Joel Witts: Yeah, a machine has made a mistake. It’s a bug or something like that.
Craig Martell: Yeah, it’s just a statistical system, and it’s just wrong.
Joel Witts: Yeah, we met a couple of weeks ago, just for listeners, at the HumanX conference. This was something you mentioned a couple of times in our conversation and also in the panel you did, around this issue of hallucinations and how humans aren’t really, I guess, evolved to—when something sounds like a human, you want to trust it. I wonder if you can dig in and explain that perspective.
Craig Martell: Yeah, this really leads back to what I think we have to do in modern AI, which is to think harder about how we do error evaluation and how we let the end user know that the system might be wrong. That’s extremely important, and I don’t think we do that well enough as a field. Look, before, say, three or four years ago, the only things on the planet that spoke fluently were cognitive reasoning beings called humans.
You can argue the degree to which they’re cognitive and reasoning, but let’s assume human beings are cognitive and reasoning. These humans spoke fluently, in a conversational and sometimes authoritative manner. We reacted accordingly. We, as other humans, reacted accordingly when someone was speaking. Then suddenly, out of nowhere, comes this other thing that seems to speak—I’m putting speak in quotes because it writes, but they speak now too.
It seems to speak fluently, cogently, consistently, in a way that makes a consistent argument, and speaks with authority. So, I think our own cognitive science by default is predisposed to interpret that as a cognitive being because the only other beings we’ve met who speak fluently, humans, are cognitive. So, we sort of have a mapping of fluent speech to cognitive being.
So, I think it’s really important that we escape that. The way we need to escape that is to realize that these systems are statistical systems. These statistical systems make errors. By the way, they’re guaranteed to be sometimes wrong because they’re probabilistic, and it’s not a probability of one. So, they’re guaranteed to be sometimes wrong, and that sometimes could be pretty big. Some research suggests for some fields, they’re wrong 17% of the time. For other fields, they’re wrong 30% of the time.
By wrong, I mean they generate sentences which are false. Which, by the way, for some use cases might be great. For marketing, that might be a great way for you to get started. For your first draft, that might be a great way for you to get started. But if it’s question answering, well, maybe that’s a problem, right? The thing that I think we’re missing—and we’re spending a lot of time thinking about this at Cohesity—is how do you know when you’re using these systems if it’s getting it right or wrong? If you’re using an autonomous car, or you’re driving a car that drives itself, how do you know when it’s wrong? It does something that you don’t like, right? You know when it’s wrong—it’s too close to the car in front of you, or it’s too close to the car beside you, or it tries to steer into the wrong lane. So, you can easily detect it. Your cognitive load, if you’re paying attention—and you should be—is pretty minimal because you already know how to drive. Your input system to change it is very easy. There’s a steering wheel, there’s a brake, and there’s a gas pedal, and you already know how to use those. So, you can quickly correct this system if it gets it wrong.
We haven’t quite figured that out for gen AI. For recommendations for online shopping, if it gets it wrong, you don’t care—you just don’t buy the shirt, right? You don’t have to correct it; you just don’t buy that shirt. For older-fashioned blue-link search results, you look at them, and we’ve gotten really good at being able to say, yeah, those aren’t very good results. I’m going to search again and try new words or a different way of asking the question.
But when it generates a paragraph, the cognitive load necessary to determine whether that paragraph’s right or wrong is really difficult. One thing I do like is in some of these new systems, you can click show thinking. It’s giving you output of its state as it’s going along. That’s really helpful because you can think, okay, that’s a path I don’t want to go down.
So that’s the beginning of being able to see if this thing is generating output down a path that’s effective for me. I was going to say thinking, but I don’t want to say that, right?
Joel Witts:Yeah, I’ve seen that as well. I have to be honest; I wonder how many people would click on that. I guess it’s a bit like reading the terms and conditions, right? I think maybe 10%, 20%, I wonder. So how you build in those safeguards, as you say, is such an interesting question.
Craig Martell: Yeah, I don’t think we’ve solved that yet. I don’t think the field has solved that yet. In order for these things to become as widely useful as they are going to become, I think we need to solve that. We need to make it clear to folks that it could be wrong here.
Joel Witts: Yeah, I suppose there are two things as well. One thing I’ve been thinking about a lot since our conversation is that trust goes both ways. A user trusts the output as the baseline, but also trusts it to input data, right? To think I’m in this one-to-one chat with this chatbot. The default position, I suppose, isn’t that the data I share is going to be accessible to others, that this conversation is not isolated. I can trust it to upload this sensitive data so it can write an email quicker for me. I wonder, do you agree that there’s maybe some psychological trust there and that it feels like you’re talking to a person?
Craig Martell: Yeah, I had never thought about that, but I think that’s right. By default, we are treating it like just someone else we’re talking to. It’s probably the case that you still have to read the terms of service, I guess is the answer, because it’s probably the case that the data going up is being used to train the system unless it expressly says it’s not, right? We’ve already exhausted all of the data on the internet, so anywhere else it can get new data, right?
Joel Witts: Yeah, absolutely.
Craig Martell: But that’s one of the reasons I moved to Cohesity, because Cohesity has hundreds of exabytes of data. All that data belongs to individual companies; it’s not accessible to anybody else but those companies. That’s the trust that those companies put in us. For that company, the general internet data is probably not the best data to build their model.
The historical data of their company is probably much better data for them to build models. You normally keep in expensive primary storage just about a year’s worth. But if you’re going to, let’s say, the world is moving towards a recession, you can’t look at the last five years and say, let me model the recession, because the economy has been pretty good. If you go all the way back to 2007, 2008, then maybe you have a good model of how your business did during that recession. Having this longitudinal data that’s specific to your company, I think, can be a real windfall for companies. We’re working really hard at Cohesity to make that backup data also accessible to the model building for your company.
Joel Witts: Wow, that’s really interesting. Can you give a bit of a background on Cohesity, what Cohesity does, the services you offer, and what your role looks like within Cohesity?
Craig Martell: Sure. Cohesity is a backup company. If you say that generically, it sounds boring. Backup sounds boring. So, what does backup actually do? Before you can even back up your data, a lot of really cool things from a nerd perspective have happened. One, you’ve gotten funding for data resilience. So, you have funding in your pocket to be able to do the right thing with your data. You’ve identified the important data for your company.
So, you already know what matters to your company, and you’ve identified how that data is relatively important because you say this data only has to be backed up once a month, but this data has to be backed up every 15 minutes. So you have this notion of how the data changes over time with respect to its importance for your business. Okay, so you’ve done all that, and then you’ve done the hard work of connecting whatever backup system you’re using. You’ve connected the connectors, the system, to all of the apps that generate your data. Okay, so that’s backup. You do all of that, and then you put it in a corner. Okay, and then Cohesity does another—that’s called data protection. We think that’s more than just the corner.
It should also have a data vault in an air-gapped place, so even that backup can’t be attacked. It should be immutable. There’s a whole set of really great practices about how to make sure that data is protected. We do all of that, and we’re excited about it. But the next thing you do is data security. As that data is sitting there, you ask really interesting questions like, is there any malware? Is there any PII? Are there any credit card numbers? As that data is sitting there, our data security tools scan through the data, identify which data has PII, which data has credit card numbers—we call it sensitive information—which data is sensitive.
We mark all of that data as sensitive. So now you know, and you might not have known that, right? You might not have known that in some unstructured document file, somebody put a bunch of credit card information, right? Or, worse, from a GDPR or privacy perspective, somebody put a bunch of social security numbers, right? Knowing that would really matter to you.
Because you have different governance rules for different kinds of information. Okay, so we scan all of that, we find all that out for you. Then we do things like scan for malware. We scan for malware via signatures, but we also scan for anomalous behavior. We look—are any of these files behaving anomalously?
Then we do this really interesting thing, which is ransomware detection. We do that by looking for an increase in entropy, because the higher the entropy, the more likely something is to be encrypted. So if we look at the bits and there’s an increase in entropy, and it’s an anomalous increase in entropy, meaning it’s more backed up than normal, and—look out—it’s the stuff that’s credit card information or PII or whatever that’s being encrypted more quickly than we expect.
Then we can warn you and say things like, hey, look out, there might be a ransomware attack. Now, if there is a ransomware attack, one of the nice things about Cohesity is your data is in a file system that’s directly accessible. So we can do something called instantaneous mass restore. We can go back to the last known good of your data, and you can point your computers directly to our storage layer. You can run your computers off of our storage layer while we’re restoring your hard drives to where they need to be. So that’s data protection, data security, data resilience—being able to get the data back where it needs to be. But think about it from an AI perspective. That’s just how a backup system works. From an AI perspective, think about what we have. We have essentially an accidental data lake. We have all of the data that matters to your company in one location with an accessibility layer.
That means you could—and it’s immutable, so you can make copies of it for model building. For each experiment you do, you might want slightly different sets of data. So per folder, you could add different sets of data, each folder being a different model-building experiment. I say model builder and not just AI because this idea is valuable for AI, ML, business analytics—anyone that’s using past data to make predictions about the future.
So, we already have that accessibility layer. What we’re working on now—and we already have the ability for you to select some chunks of that data to have a ChatGPT-like interface with it. That’s called Gaia. We’ll talk about that in a second, but we’re also working on a discovery layer, which means our goal would be something like, show me all of the data about this topic between these two dates and include conversations between Craig Martell and Joel Witts.
Right, so that we could actually get that data back. Maybe we’re doing an e-discovery, trying to find out, did Joel leak some information that he wasn’t supposed to, or it’s compliance—did we make sure that we all did the right behaviors? There are lots of reasons you might want to query your data. Now we already have this Gaia product, which we’re very excited about, where you can already select.
What you do now is you select the files you want to ask those questions of. So you say, I want to look at the following 10 email boxes over the last two years. You select that two-year period, and then you can have that conversation, like, show me any emails where there was an attachment and where someone from within Cohesity sent it out to someone in the press or someone at a particular press outlet, for example.
Joel Witts: Incredible. So this product released in March, right? So it's a new…
Craig Martell: No, Gaia released in March a year ago. it's so Gaia is a year old now. We've had some really fascinating uptick uptake. And one of the things we've learned is we want to make the discoverability process a little bit better. Right now, you have to select the files that you want to have a conversation with. We would like to make it so you could just ask, show me all files. Now, there's a trade-off there because every one of these files that you want to have a conversation with you have to do embeddings, you have to index differently, and that increases costs. So we’re figuring out with our customers now the best way to maximize value. Does that make sense?
Joel Witts: Yeah, absolutely. I’ve seen legacy archiving products where that e-discovery feature is such a headache. It takes hours of time, even with a great search tool, to find specifically what you’re looking for. So, you’re saying you can actually have a conversation and say, hey, I’m looking for this—no, that’s not quite right, I’m more looking for this sort of thing—and get really granular with it. That’s awesome.
Craig Martell: It is, but let’s be clear, for things like e-discovery and compliance, you still need a human in the loop to validate that the things the system returned are the right things. Our goal is for the first stage of e-discovery, which is the information-gathering stage, we want to make that as easy as possible.
Joel Witts: Okay, fantastic. You mentioned earlier about making that more widely accessible, is that something on the roadmap? I guess for other business processes, you can leverage that data for whatever other use case you might want to have access to all of that information.
Craig Martell: Well, at this point, I’m just telling you what we think is important. Then I will tell you to stay tuned for any particular announcements, right? Right now, we have Gaia; it’s a great product. We’re very excited about it, and people are loving it. There are enhancements that you should probably stay tuned to see what and when.
Joel Witts: Very exciting. Okay, fantastic. I wanted to circle back to your time at the Department of Defense and talk about some of the challenges you faced there with AI and AI governance. What were your big takeaways from that role? I can imagine it being an incredible time of change in the industry and what an amazing position to be in.
Craig Martell: Ironically, I took the Cohesity job in part because of the challenges of that role. The story I just told you about Cohesity and what we want to offer is something that would have been amazing to have had in that role. I was the chief digital and AI officer, and I believe what they wanted was magical AI solutions yesterday. But that’s not how it works. When you walk into a new job, you have to say, okay, what are the fundamental blockers to getting to where you want to be? At the DOD—and I’ve said all this publicly, so none of this is new; you can find this on YouTube or in the press—the fundamental blocker was getting the data right, making the data discoverable, clean, and accessible in a way that could be accessed by anyone, anywhere, who had permission to do so. There were serious governance issues as well.
You can imagine data at the DOD has even stricter governance than inside your company. We spent most of our energy working on getting the data right. My argument was the following for the organization: the IP of the American people was the data, and the real intellectual work for the DOD was the metrics. What would qualify as success? You have a set of problems. What are those problems, and how do you evaluate whether those problems are being solved or not?
Forget technology, like forget which technology you use. If someone could write a Python script that solved your problem, you’d be happy. It doesn’t have to be a large language model. What’s the problem? What data do you believe will solve that problem? And how do you evaluate whether it got it right or wrong? In this hierarchy of needs, we had data, then metrics and analytics—like, how do we actually decide we’re doing a good job? Finally, at the top, we had AI. That doesn’t mean AI is the least important or even the most important.
The actual AI tools—there are thousands of vendors who are going to knock on the door and sell you an AI solution. Let me say it differently: they’re going to sell you a technological solution. The current hot way of describing it is AI, right? Eventually, this will just become the norm, like large language models will become the norm, and it won’t be called AI anymore. AI will be the magical next thing. They’re going to sell you a technological solution to your problem.
But how do you know whether that technological solution is going to be effective? Well, you need to know how to measure its effectiveness. You need the metrics. That’s your job, DoD. If you just take that vendor and throw them at your pile of data, what they tended to do was take that data, scoop it up into a stovepipe, massage it, make changes to it, and get paid money to make those changes to the data. That’s their job; they’re in this to make money.
But then it was locked in a stovepipe for a single problem, a single solution, a single problem set. If you sold it again—like you built it for the Army and then sold it to the Air Force—that same data had to be rebuilt again, and the American people were being charged again. I argued we need to have a separation of the data layer from the app layer. We get the data layer right, and then the app layer can use any data anywhere in that data layer and not just stuff that’s in the stovepipe that you’re going to charge us for over and over again.
That was a pretty big mental shift. When I entered, commanders, four-star generals, were looking for a single pane of glass. They wanted to have a view of their environment as quickly and readily as possible. We switched from a single pane of glass to a single plane of data.
If you think about data as a plane of data, then you could have any apps running on top of it. My argument then—I still hold it—is that that also opens up the ingenuity of the American people because our industrial and academic complex is completely innovative. Two folks in a garage might solve this problem. But if they’re not big enough to actually tackle all that data, they’re not going to be able to compete with the big guys.
If you have this separation between data and apps, then even two folks in a garage could build an app that can consume that data and deliver some real value. That change has continued. The people who followed on from me have continued to push that change. We’ll see what happens under the new administration. Evidence seems to indicate that they’re not changing that direction.
My argument was to go slower to be more effective later. Nobody in a warfighting environment wants to go slower to be more effective later. They have to go fast now. That was a tightrope to walk. But let me say this: why did I join Cohesity? That story I just told you, where Cohesity is an accidental data lake—the act of backing up gets your data ready to do AI, right? That’s what we want to build. We want to build on the fact that you have to back up. You can get funding to back up. There’s going to be a board mandate or a congressional mandate or some government mandate, depending on the industry that you’re in, to back up. The act of backing up, if we do this right, will create that data layer that allows for easy accessibility for all of these apps. That’s the goal.
Joel Witts: Absolutely. Yeah, we’re definitely seeing more regulation for backing up. There’s been some talk around federal regulations like GDPR, but that seems very unlikely in the US from my perspective anyway. On the topic of regulation, and I suppose more on the AI side, do you think we need more regulations around how AI is used within companies internally?
Do you think that’s likely to happen? From your perspective, from your previous roles, do you think there’s any more needed there? Or are we at the stage of prioritizing innovation and pushing innovation first? There’s a lot of debate around what Europe’s done on AI and whether that’s gone too far.
Craig Martell: That’s right. I go back and forth here. On the pro-regulation side, we have to keep people’s data safe. That has to be the case. Also, as an author myself, we should figure out the copyright issue. Is it the case that my copyrighted stuff is consumable by these large language models? Maybe it is. There’s an argument that says copyright was designed to help the American people, not individual authors, right? We’ll have to see where that goes, but we have to answer that question. We have to figure out a way where individuals are not endangered in any way by this data being gathered. On the other hand, Europe may overregulate, and their default assumption may be to overregulate.
We need to allow for innovation. So what’s that middle path? I’m not 100% sure. I go back and forth. It’s probably not as robust or onerous as Europe, but it shouldn’t be as Wild West as where I think we’re going now. It has to be somewhere in between. I know that’s sort of a weak answer, but I don’t actually know where I fall.
Joel Witts: Some middle ground. I think, to be fair, that does seem to be—it’s such a fast-moving area, isn’t it? How can you actually regulate in an effective way where you’re going to…
Craig Martell: Do you want to regulate away the innovation? That’s the fear, right? If you say you can’t move unless you’re safe, well, then we’re not going to build anything new ever.
Joel Witts: Yeah, absolutely. You’ll end up with the innovation happening elsewhere. Okay, we’ve been talking for around 30 minutes, so I’ll ask you one final question, Craig. Looking five years into the future, what’s one advancement in AI? I suppose my original question was going to be AI safety or governance, but more broadly, what’s one advancement in AI that you’re excited to see, and what’s one thing you’re maybe concerned about?
Craig Martell: How about two sides of the same coin? What I really want to see is better theory about how these models work so we can figure out ways to make them smaller. I think a federation of small models—and there’s lots of work pushing in this direction, and I’m very excited about that—but a federation of small models allows for a number of things.
One, more competitors. Right now, to build these models, you have to be one of the big players, and you have to have hundreds of millions of dollars that you can use to try and experiment, right? But smaller models, federations of smaller models—universities, smaller companies, folks in a garage maybe—can start building some models, because compute, you can rent compute, right?
Compute’s not the problem anymore. It’s the amount of compute that you need that’s the problem. I would like to see smaller models federated that are targeted at particular use cases. I’m not a fan of even the goal of AGI. That’s fine if we get there, that’s great, but I don’t really care about the general. We have a whole bunch of problems that we want to solve with this technology. We should focus on solving the problems.
I don’t actually care what technology solves the problems, right? The scientist in me finds it interesting from a technological perspective, but the business person just wants to solve these problems. Let’s be clear on our problems and focus on smaller, more targeted ways and not try to solve—like we’re boiling the ocean, trying to release the next N+1 model that now can do everything all at once. Why? Why do we care that it can do everything all at once? It’s good marketing, but I’d rather have you show up and say, here’s 10 things we do great, and we’re going to solve these business problems for you, these personal problems for you, as opposed to having one thing that’s going to solve all of your problems. I don’t like this monolithic view of AI. That’s the flip side. The dangers of that—besides, I just don’t think it’s very effective; it’s not becoming as effective as quickly as we hoped—the danger is the costs, right?
Never mind the potential environmental impacts of all the electricity. We’ve kind of ignored—we seem to have given up on the environment because we enjoy talking to these bots. We’ve seen this: well, it’s worth the carbon if we can talk to these bots. It’s really weird the way we’ve switched on that. But only a few companies—it’s monopolistic, it’s an oligarchy of a few companies that can actually do this work. I’d rather see it broader and have competition.
Joel Witts: Yeah, I suppose it’s a bit like the general internet, right? Where you started off with everyone can build a website, everyone can own their own business. Then suddenly it’s like, well, there’s only three, four, or five big internet companies that have dominated. You’ve got Amazon and Microsoft, and I guess the danger is the same thing may happen in AI.
Craig Martell : Except, I think that’s 100% right, Joel, but let’s go back to our governance question. The governance of the internet is such that they’re not allowed to block other folks from using it and being innovative. So governance like that, like net neutrality, things that allow—when the backbone of the internet moved from the NSFNet to the internet, the folks who got to own pieces of the backbone were required to make sure they transported all traffic and couldn’t block any of it. So that kind of governance, which allows for innovation to flourish, I’m on board.
Joel Witts: Absolutely. Craig, thank you so much for joining us today. It’s been an absolutely fascinating conversation.
About Expert Insights: Expert Insights saves you time and hassle by rigorously analyzing cybersecurity solutions and cutting through the hype to deliver clear, actionable shortlists. We specialize in cybersecurity. So, our focus is sharper, our knowledge is deeper, and our insights are better. What’s more, our advice is completely impartial.
In a world saturated with information, we exist to arm experts with the insights they need to protect their organization. That is why over 1 million businesses have used us to inform their cybersecurity research.
