Third Party Threat Hunters
A dialogue with leaders in Cybersecurity and Third-Party Risk Management led a leader in the field: Gregory Rasner (author of three books in TPRM and one in PAM)
Third Party Threat Hunters
Beyond Questionnaires: AI Agents, Zero-Day Breaches, and TPRM with Clarence Chio
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
In this episode, Clarence Chio, CEO of Coverbase, discusses the evolving landscape of AI in cybersecurity, third-party risk management, and the implications of autonomous AI agents. We explore real-world incidents, innovative solutions, and strategic insights for security professionals navigating the AI-driven future.
key topics
- AI's role in cybersecurity and risk management
- Implications of autonomous AI agents in security breaches
- Innovative solutions for continuous third-party monitoring
- The evolution of security culture in tech companies
- Strategic insights for security professionals in an AI era
Hello. Welcome to Third Party Thrunt Hunters Season Two. This is my interview with one of my favorite authors at Clarence Joe. And uh Clarence is a uh your co-founder, Clarence of of the Okay. I'll let you do the introduction. I want to make sure I didn't do it screw it up. So uh Clarence, why don't you do the rest? Why don't I make it worse?
SPEAKER_00Thank you, Greg, for having me here. I'm Clarence. I'm the co-founder and CEO of Coverbase. And we're building an AI-native third-party risk and security platform that uses agents and automation to help to elevate the jobs of third-party risk professionals outside of the admin and the paperwork to help them focus on risk and security and compliance. And I think that's what they all are in the industry for. So I'm excited to share more about myself and the company today.
SPEAKER_01Great. Thanks. Thanks, Clarence, for bailing me out on that one. All right, so let's do the five quick questions to get to know the guests better. So, Clarence, you spent some time in unconventional environments, including I I I watched one of your videos where you hacked a you were hacking a session in an abandoned warehouse near the Kremlin. I I I wish you had video of that. What is the most surreal or memorable hacker experience you've had? Completely shifted how you view physical or digital security.
SPEAKER_00So the way I got into these communities was actually kind of uh kind of weird. But when I was working my first job, I was I was a fresh grad out of college. I was really interested in security. You know, I was like working on side projects, and I had a colleague that was that would go to these security conferences all around the world. They were they were really like, you know, some were more above ground, some are more underground, like hacker conferences. And I found out that if you submitted a talk and uh you know they liked what you had to present, they would pay for your flights and hotels and fly around the world. And to me, that was a really interesting thing.
SPEAKER_01So I was like, yeah, I you sold me right there, Clarish. Yeah.
SPEAKER_00Yeah. So I was like, let's go, let's go everywhere. So I I I did a bunch of research. That was the main reason I I got into security. I was like, wow, this is fun, interesting. And someone would pay my flights to travel the world. I did that for a couple of years. I went to over 200 conferences in like 50 plus countries. And part part of these were these these events. The one that I think was the most interesting. I mean, Russia and Ukraine always had the most interesting hacker conferences because there was a little bit. Yeah. There was a little bit more of a counterculture air there. But uh I remember in each of these conferences they would have these interactive areas. You know, they would have these booths that would like teach you how to do things. And the the two that I went to in one of the first conferences I was I was there was uh that there was just there was this like ATM machine there. It wasn't the latest ATM machine, it was one of the previous generation ones, and they were teaching you how to how to hack them. And there was a previous talk where the person running the running that that workshop was on stage giving a talk about how he managed to hack one of the previous ATM machines. All these are, of course, like they're above ground, he reported it, and and all the all the previous versions of these machines have been patched by now. But it was really interesting and educational, I think, for people to see how some of these machines that people generally think to be infallible can pretty easily be hacked.
SPEAKER_01Yeah, we call them cash i in the in a banking business, we call them finally call them cash pinatas. Because if you hack out them enough, you can you can get cash out of them. In our industry, people I'm sorry, I didn't talk about real classes, you're just gonna go to the next question. In our industry, people often talk about calculated risk, but you've mentioned uh the little bit of delusion might be necessary for entrepreneurial persistence. You're definitely an entrepreneur. Can you tell us more about your time when you when you had that delusion that pushed you, helped you push through that wall, what seemed kind of insurmountable? Like normal you in a normal mode, you would have just said, okay, I'd said I'm done. But you you were able to say, I can see the end, I can see where this is going. I was able to delude myself enough to get past that that that hump, right?
SPEAKER_00Yeah, I think uh d delusion is is uh kind of the the key here because it's not really about whether you can see the end. It's like you think you can see the end. And that's a good one. Therefore, yeah, you you keep going.
SPEAKER_01You don't really see the end, is what you're saying, Clarence. You're just you're trusting the process. Is that what you're doing?
SPEAKER_00I mean whenever you hear these success stories of entrepreneurs out there in the press, you know, I remember watching an interview of Jeff Bezos back in the day when Amazon was just exploding, and you know, recently when you talk when you hear interviews from Jensen Huang and Nvidia, you you look at their stock prices, you look at their growth at their companies over the last 20, 30 years. And imagine just being in that flat of the curve for so long when all your other peers are are like succeeding and doing lots of amazing things. You just wonder what's going through their their minds, right? And like, you know, my my experience is is is not nothing like that, of course, but it's I I think it it takes a certain type of delusion, I think, to go through that journey. And uh I I I don't think I don't think it's it's like a particularly interesting or glamorous one. I mean, I currently it's it's it's glamorized, but there's a lot of the parts of the job that's like re really uninteresting.
SPEAKER_01So of course, yeah, yeah. It's it it it it it it yeah, it it uh the glamorous part is the visual part, but a lot of the stuff that you'd probably do is is it it's it's it's what we would consider and sometimes consider mundane, but it's just part of being an entrepreneur and starting a business, right, and and growing one. And and speaking of that, you've built and scaled multiple companies. You said letting go is one of the hardest leadership uh lessons that you've learned. What specifically did you have to let go of to move from being sort of an engineer operator to a CEO? And and how did that how did you do that?
SPEAKER_00Yeah, I think different people start companies and build things for different reasons. For me, I'm an engineer, I'm a software engineer, and and the the reason I was so attracted to starting companies and building products was because I really enjoyed just taking something from non-existence to putting it in someone's hands and seeing them get value from it, like seeing them like it. And I think there's always a a controlling element to that, right? You you want to you want to have control over what you build, you want to know that every decision that's being made about the thing is going to be something you agree with. And I think you know, without fail, once you start growing, once you start to have to go beyond what you can do personally as a as a single person, these things no longer hold true. You can no longer build things that you personally just agree with. And then you start realizing that actually what I thought I needed to do was not necessarily all right. And there's other people that you know, rightfully we bring in that have differing opinions that I don't agree with, that actually may be the right thing to do. So I think you know that that process of letting go and realizing that you know I'm not all all right. I think that's that's uh like a process of maturity, maybe.
SPEAKER_01That's a really so you you you talk about humility. Uh you're you're sort of bleeding into this the next question, which is it it's great. Um thank you for that. It helps me as the interviewer, but you you're talking about humility there, about being uh able to understand that hey, I am not always the smartest person in the room or the rightest person in the room, that other people have great ideas. I've I that's one of the things I think that made me uh successful at my previous roles is was stepping back and letting people share their opinions. And you've you've done that as well. Speaking of humility, we often see ego, ego driving tech leadership. Yeah, you emphasize humility as a core part of building successful engineering control cultures. How do you cultivate that hum humble environment in a field that thrives on competition and winning?
SPEAKER_00Yeah. I think you know that my company is in San Francisco, it's it's in Silicon Valley. I think there's this for a long time now, this pretty unhealthy culture of hiring quote-unquote rock stars and building a rock star culture. Unicorns, yeah, unicorns. And whenever you see this portrayal online, I think it's like not very inaccurate what you see in the Silicon Valley HBO series and all these like failure mini mini-drama series. It's not unfamiliar and not unrealistic as well. And I think, you know, when you create an environment of people that have high egos, you know, like diva cultures, then you inherently are gonna operate the company accordingly. And I feel like the the magic to the magic to building companies and organizations is actually you don't really just want individual people that are all gonna be you know individually competent, but the only way to scale is if you can build a culture that attracts people that want to work with each other and like working with the other people that they are being being put together to work with.
SPEAKER_01Even the open positions, parents, you sound like a great person to work with.
SPEAKER_00No, well, yeah. That's I mean, it the the the thing is like you surround yourself with people that you can do. And that's kind of the magic I'm sure, you know, Greg. You you you do the same as well. And and and like you really just want to find people that you like working with, and so like humility and low ego, it's it's they're they're big traits that we look for. That's great.
SPEAKER_01Well, let me throw you a slight curveball as a fifth question. What uh you you you've traveled quite a bit as you as you said earlier. What what are you what was your favorite country to visit and and why? I know it's tough to do a favorite, but if you if you could only go back to one of those countries, which one of those would you go back to? And why would you want to go back to it? Let's put it that way.
SPEAKER_00There's no relation to security or uh, no, no, no. I just this is just fun.
SPEAKER_01This is a fun question.
SPEAKER_00Yeah. But it's probably chili. So my mother-in-law really likes Chile. And we've probably been like three times in the last five years just because it's such a big and diverse country because of the the latitudes it spends. And I feel like if there was one country like I we you know, we we love nature. I spend you know, like 10 hours in front of a screen all day, and any opportunity we we find, we just want to try to get get out there. And yeah, so if I could just pick one country to be at, I want to pick the one of the most diverse nature and sceneries, and Chile is definitely uh hard to beat.
SPEAKER_01That would yeah, that would tick the box, I could tell that for sure. That's for sure. Uh well, great. Thanks for sharing that, Clarence. All right, let's get into the the meat of the matter, as we say. So the the hook start today is the hugging face AI agent breach. So uh we were recording this a few days just after that incident where the AI agent, internally tested by OpenAI, autonomously bypassed security measures and exploited zero-day vulnerabilities and hugging faces. I still can't get over the name, that's what it's really the hugging faces infrastructure. We've spent years building defenses against human attackers, but now we're gonna see AA models actually just do it on their own. What how are we going to uh if we have our defensive AI models being weaponized against us, how are we gonna fundamentally redefine third-party risk for neuroization that's now dependent on agentic AI?
unknownYeah.
SPEAKER_01It's probably more than five minutes, but you know, do your best. This is right.
SPEAKER_00It's it's a deep deep question, right? Because there's such a big economy built around agents right now. And I think the the proxy around agents is that there's people doing work and there's people building this whole economy of software engineering and operations and financial back office and everything. And the idea is that agents are this these autonomous pseudo-people that will be able to do the job of people and more with superhuman AI intelligence capabilities. And of course, right, if you set a bunch of these while they don't have to rest, they don't have to sleep, you just have to throw money at server firms and they will continue token maxing and continue living out their lives. Things like these are bound to happen. I gotta acknowledge because there's a lot of top in this on X and on threats and everywhere.
SPEAKER_01But like Yeah, please ground us if you think it's a little bit overblown or overwrought, maybe.
SPEAKER_00I don't think it's overblown. I just think like there's a big element of this, which is you gotta be clear that OpenAI is trying to do a bit of fear-mongering here, especially on the heels of anthropic doing that whole Fable Varming the federal government blocking it.
SPEAKER_01Yeah, that makes sense. That does make sense, actually, Clarence.
SPEAKER_00Yeah, yeah, all all this, all this is it's great marketing for them because it's like this our autonomous AI agent went to hack this other team. And and it's it's interesting. I I don't doubt it it's true, but it it's also, you know, you guys are you also got a question like how did this even get to that point in the first place?
SPEAKER_01And yeah, that's that's that's the big one right there. I don't understand how it jumped over the guardrail so so so so simply. And that's not just to me, in my mind, and may maybe I'm over sim simple simplistic in it, but it's not a simple just jump over the guardrail. There's gotta be it had to have some mechanism to have done that, to evol elevate privileges, what whatever, right? I mean, I don't know, you're you're you're you're better you you wrote the book on the this this matter. I I didn't, but that's that that I'm a I'm with you on that. So so uh I and I I think I'm with you too on it it's not overwrought. We we do need to be uh aware of that risk. But if we if we do this things that like your your book, your your your your course, other courses, my book, others, other suggestions, other other risk management techniques for AI out there, it it is a it is a vertible, or at least prov you can lower the risk of it happening. But none of every all of us go to this knowing risk is never zero.
SPEAKER_00Yeah, exactly. I think that that specific incident is is interesting because there was uh there was an event happening. So it it wasn't that there was, you know, open AI just like somehow managed to to hack that. There was a there was an event called Exploit Gym. It was kind of a cyber capability task, and actually things like this have been have been happening for a while. I remember one of the first DEF CONs that I went to in 2016, that there was this like first event. Actually, maybe it wasn't even the first one, but DARPA held this cyber grand challenge, and it was like a at that time was like kind of futuristic, you know, it was like all machine hacking tournament. There were a bunch of teams, mainly from universities, that would go in there and try to launch these like small programs that would build to find exploits, found vulnerabilities, and see like capture the flag style. And this was a very similar one, except that now you actually have trillions of dollars, no exaggeration, of extra training costs put into these algorithms, and it actually can work much better now. And then I think there were two problems here, which is like in this exercise, these these models actually managed to reach the open internet, break through the safeguards, and then it broke into having face to steal the benchmark answer key. And it was mainly because, yeah, it's it's kind of crazy.
SPEAKER_01My dog's doing a cameo, so I apologize for everybody.
SPEAKER_00But it's adorable.
SPEAKER_01Sorry, clear.
SPEAKER_00No, but uh, I mean, the the failure mode is actually kind of uh kind of elementary because the the way that the AI model got got access was actually the age-old problem of standing credential left there. So you remember like a few weeks ago there was this up.
SPEAKER_01It's funny, we we talk about the credentials are gonna be the one that AI agents, one of the biggest weak points is gonna be access issues.
SPEAKER_00Exactly. Yeah. People always talk about how you know AI or Fable 5 or all these crazy new models are able to find this like very specific path in the code that is so obscure that no one has ever looked at before is like the most ingenious way in. But no, the most common ways in are like the most obvious things that there's a key right there. Maybe not maybe everyone missed it. And yeah, hey, there's this obvious key right there. I can just use it, it found it, and they used it. Like it's super simple. Yeah, yeah.
SPEAKER_01It always is. All right, I I we're gonna this one's gonna probably go over the 30-minute time limit, but but we'll uh we'll we'll do our best, Clarence. And I I don't not because I I don't because I want to give you a chance to answer. I think these are great answers in them. So I apologize if this goes a little bit over. A big question number one for you is from financial crime to vendor risk, right? So you spent your career at the intersection of AI and security, specifically high-stakes environments like fighting financial crime at Unit 21, which is really cool. How does the threat hunter mindset that you develop about tracking money laundering and fra fraud translate into a world of third-party risk? And are we currently treating third-party risk too much like set a compliance exercise and and not enough like a threat hunting mentality?
SPEAKER_00Yeah, I think absolutely. I mean, the the reason why I'm interested in third-party risk is because I feel like we're at this inflection point now, especially due to due to AI, but also due to many other factors around how companies use third parties and use software, where you start realizing that the questions you ask vendors and the questions you ask people to attest to, they they provide some level of, they provide some level of certainty, some level of assurance that the people who are, you know, answering questions truthfully and with with with genuineness, they're answering these questions that reflect the reality. But I think for a long time now, every other adversarial model in security, in fraud, doesn't have this assumption, right? You don't ask someone who is withdrawing money from an account, are you withdrawing this for legitimate purposes? And if so, wait, here you go. Because you know that there's always this incentive and a very understandable human nature incentive to game the system. And I think as people start seeing third parties as a primary source of entry to game the system, which it's not a new thing. I think we've been seeing this for years and years now. It's like most of these breaches have some part of a third party and part of the attack chain. Then you gotta think about what does asking the question really do? And what can I do beyond asking the question? And I think there's the inflection point in third-party risk is not caused by changing attacker behavior, but it's caused by an increasing realization in the defenders, like you know, the security teams, the risk teams today, that hey, all this stuff is being automated away. What's my role next? Is third party risk solved just because AI can analyze my questionnaire? No, I don't think anyone thinks that. It's like so so I think this is really like a moment of reckoning and inflection point where people are forced to think about what they should be doing with their expertise and time and their access today. And that's why I feel like this is this field is going to be so interesting to be in right now. It's gonna change dramatically in the next few years.
SPEAKER_01Well, what I what I like about the tools that are coming out, like yours and others, is is these uh TPR Times have have always been small by by their very nature. Um it it it's a risk, it's a risk organization, and risk organizations typically are not overly, overly resourced. And and that's oh understandable. You're they're not selling things or not producing revenue, they're revenue burgers. And then so that that's part of just our our existence, and we have to sometimes justify that existence. But it it if you're if you're if you're if you're running an organization that that is that small, having productivity tools that these risk management tools now that really simplify some of those rote tasks that really just sucked up a lot of our time. Now you can you you're your tools will tell you this is your highest risk or these are your highest risks. And and that's what I was always trying to do as a manager in my space was was not trying and spread everybody out like chunky peanut butter, but was to focus them in on what were the real issues and not focus on the things that weren't risks or low risk for very for that very reason. Clarence, I think you're you're exactly right. I think the the tools, there's two I see a problem with two two issues with this is I think you're gonna get some senior leaders who are gonna think that the tools will force the elimination of a third-party risk team because it can now automate everything, which of course is you and I both know is is is hooey. But uh what I prefer to think of is that it it allows those those minimally staffed teams to be more productive and better able to identify what is really risky. That that's what my hope is. But hopefully we don't get the other ones. That's that's those that's after I mean. All right, big question number two is the future future of active, and I think it's future of active risk defense. Uh we're seeing a shift where static annual vendor questionnaires are fine to uh are failing to capture dynamic nature of modern supply teeth, right? And I've I've never been, you know, you and I you I've had to you and I have conversations about this, and I've certainly talked about how much I'm not a big fan of the questionnaires. But if we move away from point-of-time assessments, what does a truly continuous or active risk defense model look like for a CISO? Yeah.
SPEAKER_00So another big question. So I think the the biggest things that interest me around third-party risk around third-party security is that it's not just around a technical solution to a problem. So it's like in any third-party risk or security problem, it's not that as long as you have the software or some ingenious technical solution, you can solve the problem. Because there's this inherent incentive structural problem when you're working with parties that you don't have control over. And third parties and vendors and suppliers are these parties that you don't have control over. You rely on them to share information with you. And I think every interesting security problem that doesn't have a direct, obvious solution to it has something like that about it. If everyone had direct access to all the code base and internal operations and, you know, the infrastructure of all your vendors, third-party security wouldn't be a problem. You could just write all the monitoring code and software in the world to monitor all of this, all your vendors at all times. You don't have to ask a single question. You can just look at everything and verify them, just like it was your own infrastructure. And I think what makes it really challenging here is that, you know, if you look at the evolution of security, InfoSec in companies, I think it's gone through a very similar evolution over the last 10, 20 years, right? It's it's like it used to be this cost center, no one really wanted to invest in it. There used to be a ton of a ton of apprehension in, you know, should I be even like buying this antivirus software, this endpoint security software, this firewall, this email security software? Today it's not a question, right? Because of all these high-profile failures, CISOs are in a boardroom, CISOs are becoming CIOs and CEOs. And I think it's it's it's like a very obvious thing. And you know, the converse is also true. If you don't invest in it, then people are asking you, what's wrong with you? Kevin, you've seen all the failures that have been happening. Do you want us to go bankrupt? And I think the same thing is happening here. The transition of, you know, where we go from questionnaires and documents and this like one time ask you, give me these things, I'm gonna check you once and let you through, is very similar to what happened in security back in the day. There used to be this like one perimeter defense in the I think 80s and 90s that people used. To used to talk about like the firewall. The firewall, as long as it's strong enough, then you're safe, right? But of course, over time, this has been proven to be untrue over and over again. Like people have always shown that no matter how strong your firewall is, the most motivated and tailored attack access attackers are going to be able to get through. And that's why you start getting into all these different types of security solutions that said, hey, your firewall is great. No one's arguing that, but someone's going to get through. Can you find what's happening after someone gets through? And can you detect what's happening in your environment so that you have multiple layers of defense?
unknownRight.
SPEAKER_00And that's how you think.
SPEAKER_01Yeah, you assume a breach and and you say, okay, here are my most important assets. I'm going to I'm going to make sure that I hyper protect those and monitor those. And the intranet, infranet site, which certainly you shouldn't have important information on because it's an infranet site by nature. If it gets hacked or and deleted, I I'm not really not going to sweat it. I'm going to go back to it back. But my my IP, intellectual property, financial records, credit cards, that kind of stuff. That's the stuff I make sure I put extra guard towers around and all that kind of stuff. And mostly monitoring. Like you said, it's it's it it it it APTs, advanced persistent threat actors, those folks, they have enough time and energy and money. If they want to enter your system, a lot of solar winds, they'll get into it. The question is, do you do do you find out soon enough and can you lock it down and make sure that they don't do any damage before yeah that and and AI is going to change that that cycle time, I think, quite a bit, unfortunately, for for a lot of organizations. That's gonna be some messy, messy, messy Fridays, I think. Because they that the attacks never happen on a Monday morning. They always happen on Friday after as happy hours starting, usually, right, Clarence?
SPEAKER_00Yeah.
SPEAKER_01Usually get the call on Friday afternoon, right? Yeah. There's a preacher. Great. That's that's a good that's a good answer. I uh let me let me let me uh we're we're coming actually we're we're we're good on time. Let me let me follow up with the after action report. We we've talked about some of these issues as we as we've gone in this conversation. What's the one thing you want listeners to take away from the conversation today? The single tactical action that when they go back to their desk, they should be looking at to help lower the risk of some of the things we've talked about today.
SPEAKER_00I think the number one thing here is mu most of the the listeners of your podcast here are are they're gonna be experienced third party with professionals that are, you know, you didn't you didn't enter the industry to answer questionnaires or to review questionnaires and documents. I think everyone that I've spoken to in industry thinks of their their role as a higher, higher level calling. And I think rightfully so. It's like there's so much risk that companies are taking on that cannot be solved with questionnaires and the status quo. And consequently, on the other hand, I think there's also a lot of interest, apprehension, curiosity about AI and what it can do on the attacker side and the defender side. So I would highly encourage everyone listening to this to embrace what AI can do by reading your book, by attending our courses, by like learning more about AI, because I truly believe that the industry is going to get transformed, is being transformed right now. And the people that are gonna be the most secure in their positions, the people that are gonna be the most productive and be the leaders of the next generation, are the people that are prepared to think about not the surface, superficial things like how can I analyze this document using AI, how can I analyze this question using AI, but think about what does AI allow me to do that I wasn't able to do before, not just in terms of freeing up my time, but in terms of what level of access to information, to raw data to help me to fulfill the first principles of my third party that exists in the first place. And I think that kind of thinking is like is is is a little lacking, but it's also what's gonna drive the future evolution of this space. And I think that's that's exciting. I agree with you.
SPEAKER_01I my my my my trailer's take would be that it's similar to yours. The AI isn't uh it's not a point solution. You look look at your strategy about how you want to implement it to get to an ultimate goal of like I said earlier, you have you have you probably have smaller teams. So so find ways that you can leverage it to identify the riskiest issues and uh and attack those. Like you said, nobody wants to be spending time looking through SOC2 reports. I mean, it it it it isn't it isn't very rewarding at times. It's necessary to learn what a good SOC2 looks like necessarily, but you know, for 20, 30 years, after a while you've figured out what it is. So now you can use the AI to help you kind of identify what the issues that are missing in the SOC 2, as opposed to having to read through the whole SOC 2. That's a much more productive use of your time, I think is what you're saying, Clarence, right? I do I love that as well. I think that's great. Any any parting thoughts, Clarence? Again, thanks for thanks for being a uh a guest today.
SPEAKER_00No, thank you, Greg. I've been uh admirable admiring of your work, all the effort you've put into educating the space. I love your books, love your courses and your podcast. So thank you for having me on. Love to continue interacting and learn from you.
SPEAKER_01Thank you so much. Well, that is a wrap for our session. Make sure that you, if you want to, we're gonna have Clarence stay stay on and do uh uh some bonus material where he's gonna uh talk about his product and and and some of the solutions that it it helps provide. Uh but if you want, uh subscribe and and so you can get more cyber risk insight updates and hopefully see you at the next uh episode. Thank you.