Uncapped #39 | Daniele Perito from depthfirst

[00:00] There is this saying in security circles that in order to survive a bear attack, you don't need to outrun the bear, but you need to outrun the person running next to you. That's the way that the business has been operating for a very long time. But with AI, you can think about the fact that there isn't just going to be one bear, there's going to be a thousand AI bears. That's terrible. So we're really trying to secure the whole software from AI bears, really. Today, I'm here with Daniele Perito, who co-founded FAIR. [00:30] Square and you also ran data and security there. And then most recently, you've now become the founder of Depth First, which is an awesome AI security company. Really excited to be doing this podcast with you today. Thank you for having me. I want to start by learning about FAIR and sort of like what your experience was like there. But maybe if you could take us back to sort of the founding insight or what sort of led to the creation of the company. I would say that FAIR was probably a little bit of a contrarian bet. [00:52] People at the time didn't think that brick and mortar retail was this place where there was going to be a lot of growth, like fair proof. But at the time, Max, Marcel and I were talking about ideas on companies to start together with Jeff Golubson as well. And Max was introducing a high-end umbrella from New Zealand to the U.S. market. He was seeing, that was a side gig, you know, he was working at Square, but he had a little bit of a side gig. And he was seeing how... [01:21] Getting sales on Amazon was extremely hard. Getting into Nordstrom or Walmart was also extremely hard. And working with hundreds of thousands or millions of retailers was just impossible because there were many, many regional sales reps and things like that. So we thought that there had to be a better way. And then from Square, we knew that sort of taking risk on behalf of your customers was always a good way to create value because Square is in the risk management business, in a sense, and we have learned that there.

[01:51] the ability to order and not have to pay for 60 days and be able to return anything that they don't like. So, and taking the discovery risk off of their balance sheet and then not even asking brands to offer that value prop, but us sort of trying to... [02:06] use technology to offer that money. So that was a big insight. Yeah. Starting there. Did it go sort of the way you expected from the beginning? Like how linear was it from like that concept to just like the company, you know, taking off and going the way that it ended up going? Yeah, I would say that through product market fit, there was a little bit meandering. I think at the time we were unsure exactly what was going to [02:28] give enough value to retailers to order unfair. We were experimenting with a lot of things. We were experimenting with something called consignment, which is a little bit of a technical term in the business, but it's just a bit easy to like put something on someone's shelves without actually, without them having to actually buy the merchandise, right? And we were experimenting with consignment that was extremely capital intensive for us and very, very risky. We were experimenting with like points programs and we were experimenting with other things. And I remember [02:58] July or August of 2017. [03:01] Max was at a trade show in Atlanta. Every single day he was at the trade show, and Marcel and I were in the background, like, coding changes so that he could, like, sell them the day after. And he was just like, "I think we need to go with, like, try before you buy," which is, [03:14] The same thing we've been experimenting around, which is like net terms plus the ability to return. But I'm going to talk about it before you buy. And so at night I coded that.

[03:24] behind the scenes. The day after he went in front of customers, they immediately got it. And that's when we knew we had something. Yeah. It's amazing how like those early days, the speed you can move, the way customers react when you do that is crazy. You know, like I remember we had some experiences like this early on at Lattice where you like take, you know, take some customers like bug and you fix it within the hour. And then it just like it completely changes the relationship. I'm sure you had a lot of that. Yeah, 100%. I think it's really, you know, [03:54] talks about, of course, but I feel like until you see [03:58] it now being there to then it being there. Yes. And that shift that is real. Because before that actual shift, you try to convince yourself. I think we have a real marketplace. Yeah, because you don't want to tell yourself you're wasting all your time. Right. Yeah. But after that, you actually say, oh yeah, I was putting myself before. And this is actually what it is. Yeah. I don't know. I mean, for us, we had something similar where it's like a product was built and all of a sudden it went from like no to yes. I think maybe there's some companies where it's more of like a gradual thing that happens. I don't know. [04:28] We can talk about this when we get to depth first, but I'm curious how that experience compares to the current experience. Before we get there, I want to stick with Fair a little bit longer. You talked about how it was an operationally intensive business that required a lot of rigor. Can you talk about what that looked like in practice? Look, one of our values of Fair is seeking the truth, and that is sort of necessary in operating a marketplace business. You are providing value to retailers by having more brands on the platform that can sell to them,

[04:58] insofar as there are retailers on the platform that buy from them. There are all sorts of balances you're trying to make sure that exists between supply and demand. You want to give retailers an amazing discovery. You want to manage risks so that retailers can get as much payment terms on the platform. They can order on terms as much as possible. [05:18] but without risking too much on your end. And to brands, you want to onboard them as fast as possible, but also making sure that in their first week, in their first month, [05:27] as many orders as I possibly can. So there is, [05:31] many, many factors in the marketplace business. And you're trying to make decisions within a system that is highly recursive, where small-- it's chaotic. Small changes can ripple out. Ripple, yeah. [05:45] intellectual rigor and data analysis [05:49] is crucial. But of course, that always needs to be paired with sort of intuition and sort of ambition, because otherwise you can be a little bit [05:59] to incremental, right? - Yeah. - So you want to balance out these two things, but in operating a marketplace business, you really need to be rigorous. - Yeah, I often think about that, where like a software business, [06:10] you know, another another cut on this software business has these high margins and that affords a lot of, you know, error underneath. When you're operating a business that has a different margin structure, let's say you have a 10 percent margin versus a 85 percent margin. There's just a lot less underneath there that you can operate within. So I imagine [06:27] the sort of daily workings of the company have to be more precise and measured. Yeah. I mean, one thing to say is that marketplace businesses are not, you know, depending on how you compute them, but usually you compute them over sort of gross revenue. So in general, they're more like 50% margin businesses. Yeah, yeah, yeah. I was even thinking of, you know, there's like a business like Amazon or something like, you know, it's like at the extremes, like how careful do you have to be day to day? There is a lot of rigor that you need to build in.

[06:57] and the business really relies on them to actually operate and flourish. The marketplace is really something that needs to flourish, where demand needs supply and really. I'd imagine that means you also need a culture of a lot of testing, rather than maybe in a B2B company, you can do a lot of just like, here's a plan, we kind of know this is roughly going to work. It takes a lot of effort and you do the plan once, versus I'd imagine a marketplace business, there's a lot more test things, [07:27] over time. I would say that operating fair has given me a healthy amount of epistemic modesty and humbleness because... About like how much you can really know? How much you can really know. And I come from academia, right, where... [07:40] I was trained in some sense to be skeptic of my own beliefs, but there is nothing quite like trying to [07:47] test your beliefs in the market, to actually know the limits of your beliefs. Where you will launch an A/B test and be like, "I am certain that this A/B test will land." And then you'll discover that there is a second, third order concern that you will never have anticipated, that completely sort of undermines the hypothesis that you had. And so this idea that there are limits to human knowledge and you kind of need to experiment your way into things, is really just beating to you [08:17] Yes. I mean, in general, I really appreciate when people can like express the confidence interval that they have on whatever they're saying. When someone's like, I'm really sure about this thing. I'm 85% sure I'm right. Like, I really like that, actually, when people can just like just sort of admit that, like whatever strong view that they have, unless it's like a piece of arithmetic, it's like you probably don't know for sure. Yeah, I have this little quip, which is I say that the market is an incredible truth seeking machine for the type of questions that it can investigate.

[08:47] or a certain way or another way with the market. But for the types of questions like, does a certain type of signup flow work best for retailers, or another type of signup flow work best? The market is really good at getting the answer to that question. - Right on the other side of product market fit, when you're like, okay, this thing's, you know, we've got something people want, [09:05] It's working. Did you experience ease from there? Like did... [09:10] employees and candidates did like investors like did people see what you saw at that point or was it like ah this is still niche you know the tam's not that big or was it easy to get people excited i think there was probably another two or three years i don't remember the exact dates but another two or three years of just pushing just to get people to understand even though you knew it was working even though we i mean i think we had a very good idea that it was working we had [09:40] about the time of this business, I think time is just a directional thing where for the first two or three years, we were trying to estimate the size of the fair market in China. [09:49] a thousand different ways. There were these signals that will tell us there are millions of stores across the globe, trillions of dollars of wholesale orders. And it was just hard to wrap our heads around it. It took really like two or three years for us to be like, okay, this is a gigantic market and we have a place in it. But investors didn't quite get that, or like many investors did. Of course, the ones that bet on us did. And I was in a lot of just one-on-one conversations with

[10:19] "Oh, this is gonna be big, trust me." But yeah, I think it took a while to really just get the word out. And I remember maybe like it was four years into starting the business where now people were starting to like repeat back to me things that I was trying to tell people like three years earlier. - You're like, "Oh, where'd I hear that before?" - Right, and finally it was just like, "Oh, finally, people get it now." - Yeah, when you look around like AI landscape right now and you think about like comparing this moment in time, starting a company today versus, you know, [10:49] qualitatively different and sort of like the psyche of founders or how people are thinking about these types of questions? Or is it similar? Very different. I feel like you could rely on an assumption of some type of steady state system underneath you. [11:03] nine years ago. And right now the assumption is that everything is about to change every three months in ways that are hard to predict. And you have to just stay alert to all the potential changes. And everybody's sort of trying to see where the puck is going and it's extremely hard. So I would say the level of energy, paranoia, the level, and I think it's also because the rewards are much bigger than ever. I mean, stuff growing faster than it ever has. Yeah, exactly. And so [11:33] Everything is just so intense all the time. And this was, I mean, don't get me wrong. Things were extremely intense, not fair too, but it was kind of like we had our market where we knew that it was there and we just had to figure it out within that market. But now things are just changing all the time. Right, both because it's like, you know, the market might completely change, what product you could build might completely change, competition might completely change, just like way faster. Yeah, or we might get the singularity in a month and everything changes. That's right, yeah. Can we talk about Cash App?

[12:03] at Square and sort of the beginnings there. So you were on the sort of funding team at Cash App and you talked to me about how like there was a certain mindset that you went into that with where you're like, you know, taking a bet [12:15] inside a big company. Can you talk about that? Yeah. So this is something that I like to tell people a lot. Just establishing [12:23] my frame of mind at the time. I joined Square as my first sort of corporate job. I was a researcher before, like in academia. I was doing a postdoc, and I joined corporate America. [12:37] - And at first... - Went into industry. Went into industry. And at first, my feelings were, "Oh my God, everybody's gonna be on top of it. I don't know." You know, sort of imposter syndrome. [12:50] Thank you. [12:50] But then I think right after that, I started [12:54] having this belief. And the belief was simply like, [12:58] Literally, it was stated in my mind as [13:00] individually in a company of a few hundred people, there has to be a way for me to to X the value of this entire business. [13:08] I don't know how, why I had that belief, but I did. [13:11] And I think that when you have a belief like that, it has a way of being self-fulfilling. [13:19] - Why? Because I think you, [13:22] Another way of saying it, if you knew the success was guaranteed, what would you do to achieve that success? If you knew there was a way, then your brain is just going to try to find a way through solution space to try to find the set of actions you can actually take. So the way that manifested itself for me was...

[13:40] Cash App was a Hack Week project. [13:43] It was spearheaded by Jack. At the time we were using a trick, which was sending an email, [redacted email]. And as you know, emails can be spoofed and things like that. So everybody was a little bit worried at Square, that things were gonna get weird with security. I was working in security at the time at Square. And I was just like, hey, put me in coach. I want to work on this problem. And I was like, I want to make sure individually [14:07] that this is implemented correctly. Now, a few months later, we moved away from the email trick, and luckily because we built an app and it was much better. But the other thing that then happened is that our risk losses, our fraud losses from sort of stolen credit cards and things like that were a little too high. [14:24] And I remember going to my boss at the time and being like, [14:27] I want to work on this problem. I think I can make a big dent. And it was still through this mindset that I had at the time, which was like, what is the biggest thing where I can have an impact? You know, I think my brain is very sort of anxious, paranoid, and I try to like always find ways in which things can go wrong. But that was very well suited to the problem of fraud and combating fraud. So I came up with a whole system, I implemented all these rules and these machinery models where I had like one or two people helping me at the time. And so we implemented the system and we reduced the risk losses by 80%. [14:57] sort of range that was actually like healthy. And then, you know, in some sense, [15:02] that allowed Cash App to thrive and survive and go on to become the massive business that is, I think, according to Cash App. [15:09] to public data and earnings calls. I think it's a $10 plus billion of revenue business. It's amazing. So in some sense, that did end up happening. You know, that belief ended up materializing. Of course, it was a large team. Like many, many people had. Yeah, but it's a good mindset. Because I also think when you, when you either feel like I could work really hard and nothing's going to come of it, that's super demotivating. Or if you're like, I can work really hard, but you know, the best I can accomplish just doesn't matter that much. Like that sort of mindset. It's hard to, it's hard to care

[15:39] Yeah, and I think it's actually a mindset that, [15:41] is [15:42] related to security in a sense, right? Because what do hackers do? Hackers find a way in where nobody else sees a way in. This suspension of this belief is similar. For the hacker. Similar to how a hacker would think. It's like, there has to be a way to create value. There has to be like a path for me, a set of actions, but a few words to whisper to the right people at the right time, a piece of code that I can write, an idea that I can have, a partnership that I can form, a customer, you know, whatever that may be, [16:12] reflect the business. [16:14] And there is. [16:15] I guarantee you, no matter who you are, whatever company you work at, there is a way for you to have just an outside impact. So let's talk about depth first. So you're doing it again. You got sort of the motivation to go back through the journey and you're sort of doing it with full force. What's the sort of idea? [16:35] behind it, what's the mission that you care about with UpFirst? I think I'm doing this again. For me, it's a very mission driven endeavor. Maybe a year and a half ago, I was listening to a podcast episode between Sam Harris and Max Degmark. And they had this point that really resonated with me, which was without much better computer security, [16:54] we do not get to play the AI safety and control game. If you think about it, AI safety and control are gonna be mediated by software. And to the degree that our software is not secure, which it isn't, and we need to make it a little more secure, then what are we even talking about?

[17:09] And so I was like, okay, if I can create a business that is both commercially successful, [17:15] but it's aligned with the mission of making the whole software more secure, then maybe I can create a flywheel there. And the flywheel is like helping secure open source software, building better AI to find vulnerabilities and fixing them in, you know, [17:29] the software runs the world, creating infrastructure, open source, anything, creating goodwill with that. And on the other hand, using the same technology that we build, [17:38] to create a [17:39] product that customers want. And here we're talking about corporations like Square, Fair, Lattice, you know, companies that are trying to secure their perimeter, making sure that their customers data secure. And I really thought that there was a way to create a massive business with a tremendous amount of positive impact by creating this like flywheel. I would say that I think we're starting to get a good way of the way there. And like the pieces are really falling in place. And I'm really excited about the mission. I could not be more excited. [18:09] before we get into the specifics, what is the sort of landscape for security with AI? Like what's the, if you had to sort of try to like, describe the most important parts of like the new territory, now that like, you know, there's, [18:22] AI, AI-generated code, you know, sort of the ability to sort of, you know, do reasoning, to look at, you know, if you're an attacker, like, what does this all mean for security? There are multiple lenses through which we can answer that question, but... [18:35] At the macro level, like at the mission level, I'll study at the mission level and then sort of the commercial side. At the mission level, software runs the world.

[18:43] There are billions and billions and trillions of lines of code and systems and configurations that make, you know, they turn the lights on and they operate the banks and all the things. Every sort of serious security professional will tell you that there is always a way in. I think AI is fundamentally changing the equation there. We can go maybe into that a little bit later, if you like. On the commercial side, I think people are figuring it out. Our take at that first is that [19:10] two years from now, a company like FAIR will operate pretty differently to the way that it operates today. Today, a company like Square or FAIR will buy a certain number of SaaS security products [19:22] This can certain subsets, [19:24] of their code or their infrastructure. They do so largely using old school techniques, like heuristics and rule-based systems. Those techniques necessarily have [19:36] higher false positives, lower detection rates, and can only discover shallower problems. With reasoning and AI, what we really see happening is a convergence of all these subcategories in security. And essentially what we're building is an AI security engineer. Think about a swarm of independent agents that are going through your organization, going through the lattice infrastructure, and they're saying, "Hey, there is a code bug here that allows someone to log in and [20:06] else and nothing before could detect that. That was not possible. It needed like the intuition and judgment of a human. But today we're starting to approach the point where we can do that. Or there is a misconfiguration in your cloud that will allow someone to get in into this way. The pieces were there, the detection rates were lower, the false positives were higher, the technology only before only allowed to like solve a little sliver of the problem. But with AI, we really think we can put it all together and make it feel like you have an AI

[20:36] driving to a degree where it's like, you don't need to name, okay, here's it for permissions, and here's what we care about for logins, and here's what we care about for API keys and whatever else. And you're able to just say, [20:47] I want this thing to just very intelligently say, like, what are all the possible vulnerabilities and just swarm and look at it all. I think a lot of that is true. I think there is probably the human element is still going to be something different. Like call somebody like, oh, I dropped my password. Like, can you give me your login? Yeah, I think the human element, I think there is always companies that need to, like, understand how to interact with the human side and make sure that they authenticate properly and they don't do. But on the software side, do you think that's basically where this is getting? I think there is going to be a great unification because the technology itself. [21:17] I mean, to me, it's just a mechanistic claim. Before, technologies could address small slivers of problems, and now the technology is actually able to generalize a lot better. Do you think in theory, at the end state, like, let's go ahead four or five years and just assume things kind of stay [21:33] what we expect, which we expect, whatever that means. Do attackers or defenders have the edge over time? So I think it's a dynamic system. [21:42] I'll use an analogy. Like perfect security is not achievable. And I think this might seem like a scary claim if you're not into security, but everybody understands this intuitively. Like everybody knows that there is no such a thing as a perfect bank vault. That a bank vault is only as secure as two things. Number one is how difficult we can make it to attack it, to like get in. And that's a matter of like cost, equipment, expertise to actually like drill into it or lock the lock or things like that.

[22:12] The second aspect is how likely it is that you're gonna get caught, and what are the disincentives there. So you do the equation and then thieves and attackers do [22:24] Pretty rational math there where they say it's not worth it or it is worth it. You know, in software, it's similar. There is no perfect vault. That's just impossible from a purely theoretical perspective. And attackers are making the same judgment call. And it has to do with how hard is it to get in. And that has to do with like how good your protections are and then how good enforcement is. But the reality is that online enforcement is last. It's much more difficult. Like it's much more anonymous. [22:54] in a state where, in a nation where, you know, there is no bell-lighter treaties for things like that. So the equation has the same factors. Enforcement is a lot of a smaller factor. [23:04] - Okay, now, so therefore it's really about cost. How much does it cost to get in? With abundant intelligence, that cost is bound to go down. And I think what's gonna happen because of that is that we're going to see a lot more frequent [23:19] attacks for organizations. And on the other side of that, I think we need to get far ahead of the attackers. And so, but I'll get to the actual question, which is, so I think, [23:29] A company like that first is going to have to get in front of the problem. Health organizations secure themselves for this coming wave. And I think we're doing that. [23:39] On the other hand, though, I think that the balance of attackers and defenders is not going to change drastically because...

[23:45] defenders still have [23:47] a certain advantage, which is they have full context. At that first, we spend [23:52] hours and hours of compute on one of our customers' code bases, to fully understand how it works. The AI spends hours and hours in their understanding, "Oh, those are the ingress point. Those are egress points. This is the inputs and the outputs. This is how everything works together." And using that knowledge, the AI helps secure the business. Attackers need to fly blind. Now, there is another advantage that the attackers have, which is, defenders need to find every attack. [24:20] attackers need to find one. But I do think that with the technology that we're building at that first, we can, [24:27] tilt the scales in favor of defenders. You also, I guess, have some advantage as a defender because you can know everything about your own systems versus an attacker, you can't know everything about the system. Exactly. So yeah, this is the context point that I was making. Our AI spends hours and hours just mapping out everything. I guess every time you push new code, though, it exposes a potential for new vulnerabilities too. Yes, exactly. And that's why we have a [24:49] one of our products is one that scans all of your pull requests as they're being written. And, you know, I think one thing that I've noticed at both Square and Fair is that [25:00] There's always been a little bit of a... [25:02] difficulty between [25:04] prioritizing security versus productivity. And I really think that this is a false dichotomy at this point. I think another big thing that I think AI will enable is a great reunification piece between the infosec teams and the security teams, and they'll be able to achieve that security that they want without impacting productivity. - Yeah, it's like a drag.

[25:25] Yeah, security engineers had to sort of sometimes say, hey, folks, we need to [25:30] "Look at this, give us a day or two when we need to..." And that was totally rational. It was the way to do it. But if you move at the speed of AI, you can sort of do those reviews much faster. And so I think we're also gonna see a little bit of, [25:44] reunification i think we will be able to achieve security with productivity as well yes it's funny also because like outside of engineering i think there's like uh people understand security as like password protection and random stuff you know like hey if somebody like sends you a phishing email be careful but like and that is part of it and there's probably a whole separate you know approach needed there but for the software piece it does seem like everything's about to probably look very different that's why i'm here too because i want to sort of tell how to people how cool security [26:14] I got into security originally in grad school because of, [26:18] how fantastical it is. You know, there's attackers and defenders and firewalls and bastions. It's really just like a fantasy world. It's really funny because like it was security, like the excitingness of it ranges from like password manager and, you know, just like somebody at your company telling you like, hey, you got to follow these protocols. Like that's one side. And then the other side is like Ocean's 11. Exactly. So people, I think the first thing, if I ever say, tell someone that I'm working on security, I think the first thing that they think about is just like, [26:48] And the other day I had to reset my password. That was painful. Like, I think that's the first thing they think about. But the reality is that what they should really be thinking about is those crazy hackers that are doing like daring things to get into systems. High level government agencies. High level government agencies. That actually is what security is at the limit. And it's incredibly intellectually stimulating. It's really just at the edge of technology.

[27:18] a little bit boring. And now with AI, it's like back to this like very fresh thing. Right. Because I think it goes back to that point I was making, which is security is relative to the level of attacks. Right. So we had reached sort of a steady state where a company like Federal Lattice could operate [27:33] and having a team of ex-security engineers and business will go on and the likelihood of attacks was relatively low. So you could just do your thing and put security a little bit on the back burner. There is a saying in security circles that in order to survive a bear attack, you don't need to outrun the bear, but you need to outrun the person running next to you. I think that's the way that the business has been operating for a very long time. But with AI, you can think about the fact that there isn't just going to be one bear. There's going to be a thousand AI bears. [28:03] It's terrible. [28:04] So we're really trying to secure the whole software from AI bears, really. Why does it seem like security is its own sort of ecosystem, echo chamber world? Like to me, I'm not like, I'm lucky to invest in you, but in general, I don't. [28:17] do security companies. And what I found is it seems like its own world. Why is that? Like, why is it not similar to just like other software categories? For context, I was in charge of Security Affair during my tenure there and had a team of folks that was super talented, super talented, as well as Cash App that was within the Square ecosystem. I would say that security is such a different market just because of how hard it is for both buyers and sellers to

[28:47] observability software or databases. Someone can try your database, you make a claim, I test it, I see it, and it's done. If I run a company like FAIR, [28:57] I don't necessarily know everything that is wrong, or like Lattice, or anything, or any other business, really. And a vendor, a security vendor comes in saying, [29:05] I think these things are wrong. [29:06] And I was like, are they really wrong? First, you don't know, because the claim may be partially incorrect, because many security issues may be false positives. Because it's almost true that someone could take advantage of it. But there's this one little detail that makes it not true, right? So first, you need to investigate every single claim, right? So that's the first part. Sort of really, both the buyer and the seller might have different opinions on what's really a false positive, what's really a true positive. [29:36] buyer or the seller know what the true positives are. So let's say that I go into your organization and I don't find anything. Did I not find anything because I'm not good? - Yeah. - Or did I not find anything because you don't have anything to find? Neither of us knows. [29:50] So the information of the ecosystem is, [29:54] Very hard to get. It's very hard to get. It's worse than a market for lemons in some sense. You know, we're not that the buyer nor the seller really know what's going on. And I think with AI, actually, we can overcome that because I think the reasoning abilities of these models can actually do that. You can trust that the AI will find the bug if there's a bug. Well, and also you can make sure you can trust that it will find a higher percentage of those bugs potentially. And I think this is why we're investing in technology. And I can tell you about our investments there.

[30:24] of bugs that you can actually find is increasing. [30:27] pretty rapidly. So that part of the equation is changing. And then the second part is that the AI can actually operate almost like a human in the sense that they can [30:36] verify their work. Yeah. And they can be like, oh, those were my assumptions. I thought that that was a real security problem because X and Y and Z. And actually one of the things our customers love about our product is that we tell them all, we list out all the assumptions that we made to conclude that something was a real security issue. And we showed the work that the model has done to actually verify each assumption. Security is a different market because of those reasons. But I do think that AI hopefully [31:06] change the equation a little bit, where it's going to be much easier for buyers and sellers to be on the same page exactly on the value that is being exchanged. Getting into sort of the tactics of depth first, what are the important pieces of technology for you all to build to be able to accomplish this? Yeah, so maybe let's digress a little bit. So I'm one of the founders, I'm the executive chairman. Kasim is the CEO. He comes from Databricks. [31:36] he comes from deep mind [31:38] And he was one of the authors of Alpha Dev, which is the reinforcement learning algorithm that found a better way to sort than hash on Google. And I think this is almost like the perfect team [31:49] to go after this problem, because AI, so to answer your question, AI has a big infrastructure component, especially when you're doing AI for security. We're doing these technical things. I don't wanna go into too much technical detail, but we're spinning up Docker containers to like run code inside so that the LLM can test whether certain hypothesis are true or not. And having hired a bunch of folks from Databricks

[32:14] helped us a ton in setting up that infrastructure. People have been calling it the scaffold, the harness, but it's our intelligence layer that allows us to really repurpose the technology that we built on each new problem. So vulnerability discovery is one thing we've applied it to, but we also applied it to other things. And each one thing that we apply it to becomes easier and easier, because we built like a really solid AI infrastructure there. The second piece is more like the deep, [32:43] research side which andrea has done before at deep mind [32:47] fundamentally, [32:48] I believe that reinforcement learning plus large language models will allow us to sort of create a superhuman hacker. [32:57] for defensive purposes. [32:59] We're talking about the fact that [33:01] systems, we were only able to find a certain low fraction of the real problems that [33:05] existed. And that was because the other problems were deeper and deeper and deeper and more and more complex. And I think with reinforcement learning, we can teach these LLMs to go deeper, to like find those clever ideas that will allow them to put two small vulnerabilities together and combine them into something that is actually real. We have some security researchers on the team coming from Apple and security service like IDF. And the way that they work is phenomenal. [33:35] work differently. [33:38] Mabo on the team tells us that he's discovering Malabini with our LLM and how is like

[33:44] he verified it and how he actually sometimes pieces it together. It's wonderful to see. Is that way of thinking learned or is that like a certain brain type that exists from the beginning? Is it something that comes out of experience or is it something that's [33:57] I feel like just almost like everything is probably a combination of nature and nurture. Personally, I feel like my inclination of being a little bit of anxious, paranoid person that is always trying to see how things could go wrong helps. It's definitely like the background thread in my brain that is constantly seeing how catastrophizing and seeing how things can go wrong is definitely helping in that direction. [34:21] And there are definitely people that are more apt at sort of stepping out of the box and seeing things from a different angle, which is you clearly need. But I will say that it's probably just almost like every other thing. [34:34] It starts with probably a small talent [34:38] which then [34:40] tells you, "Oh, I'm good at this." And then you invest more in that, and then you get better at it. But if I had to guess, the seed was actually [34:50] It's quite small and they blossom because you invest a lot of time because you were good at it. So this is sort of like some of the technology underlying, you know, the product. And then I guess is the idea with the product itself, should it get to a place where, you know, a customer can basically just install depth first and they just know that you're constantly exposing vulnerabilities at a way higher rate than people and you're doing it more thoroughly, faster, cheaper? Is that basically it? Yeah, that's the goal. And I think by training our own post training, our own LLM.

[35:20] which we're experimenting with right now. What they hope is that we will have a technological edge [35:26] tell customers, I think it's two things. One is the technological edge of our [35:31] sort of AI stack. And then two is really thinking about the problems the right way. So for example, we started with code. [35:38] But we are now telling customers, hey, if you link your staging environment, we can test [35:45] the findings against your staging environment, to tell you whether something is real or not. So I think expanding into other areas that our customers care about is going to be crucial. And really giving them an interface, whatever that may mean. You know, right now we have a web app, but I'm also thinking that at some point you need to be able to talk with this thing as if you were talking with a security engineer, being like, hey, can you double check this thing, please, for me? And then giving the AI access to the components, giving them the context. [36:15] important. Another thing about security is that it's really context specific. [36:20] If you're creating a... [36:21] social network, the fact that people can see a customer's profile is the way that it works. Yeah. But if you're creating, you know, corporate, [36:30] Slack platform, you know, a messaging platform or corporate, you probably don't want the profiles to be public. So that's context specific. So our systems, again, as I said earlier, spends hours in a code base. Sometimes going into the [36:44] old commits, is if you think about it, whether something was done differently earlier,

[36:51] than it is done today may tell you, "Hey, we actually had an assumption a year ago about how this thing was supposed to work, but now it's not like that anymore, why?" And so it's really about building a centralized repository of context about the security posture and organization, and that's what we're building, then adding agents, [37:09] that can go in and say, let me look at your code. Let me look at your infrastructure. Let me look at your configurations. Let me look at this. Let me look at that. Will your system learn as a result of, you know, like, is it the kind of thing where the more customers you have, the more you will learn to be safer for the next one? Or is it, is each one its own new instance? So as an enterprise company, business, [37:29] - Yeah, of course data can't be shared. - Data cannot be shared. The customer's business never makes it into the weights. - Yes. - Never makes it. We just don't do that. We cannot and we will not. But what I will say is that there is an outer loop [37:43] as the folks in AI like to say, there's an outer loop, which is like we learn from the types of issues that are not-- - You, the people at depth first. - Yes. The people at depth first learn. So what do we do? When we see that we're not quite doing as well in the type of issue, we will take some open source software, find the issues that are similar, [38:02] and then train on that. [38:04] So that's the way in which you're sort of participating in an ecosystem. - Yeah. - That's not that different from how any other product becomes better because more people use it. Because as you join a platform or a SaaS offering, you're probably benefiting from the fact that other people have discovered ways to use it that have been built into the product. So in that sense, there is some. - Do you think of depth first, when it's a security engineer operating on the team? Is it like,

[38:33] Is its role to like help manage the human security engineers or are the human security engineers managing the AI? I think it's going to be a collaborator. I think it's going to be like the humans will probably have the ultimate amount of final say in context. I think I think you still need that. [38:50] for now. Probably one day you don't. At that point, it's not this company that changes. I think the whole society changes. And I don't know what happens. By the time that's happening, it's like all these rules don't apply anyway. Yeah. And then I think we need to have an entirely different conversation as a society about what's going on. But before that, I think that the security engineers would be the ultimate judges of what's going on and making sure that everything works. Okay. I want to maybe kind of switch to [39:17] back to sort of like [39:19] generalities around building this company versus building FAIR. And we talked a little bit about sort of like the mindset of like, the grounds are shifting faster, the rewards are bigger than ever. So that kind of changes in things. I also imagine just that in some ways, like, [39:33] the types of [39:34] culture inputs that you want are a little bit different? Maybe the types of people are a little bit different? Like what have you found when you take that difference in the sort of environment? How does that apply to like building a company now in this era? Like whether it comes to recruiting or, you know, the way you manage the team or anything else like that? So there's this book called The Platform Revolution. And they talk about two types of businesses, platform businesses and pipeline businesses. Platform businesses are dual-sided

[40:04] like Fair. Pipeline businesses are businesses that produce a service of good and sell it to their customers. And there isn't much interactions between the customers or there isn't like a lot of interactions on the other end. I would say that in a platform business, a marketplace like Fair, I think you need to keep a tighter grip on the business just because everything is so interconnected that it's hard to just let people know. [40:29] completely run with things, because there is always going to be second and third order things that might happen. In a pipeline business, I'm noticing with that first, I think there is a little bit more of letting a thousand flowers bloom and seeing what works. [40:44] But so that's one potential. But it's a small difference. I'm not saying that it's huge. I mean, you're-- - No, it makes sense. So it's almost like you just, you need, you need, like, greater coordination of efforts in a marketplace business versus a pipeline business. You need, like, you basically need systems that, like, let the flowers bloom. - I think so. I think so. I think that's a fair characterization. So that's one thing. A lot of things are the same. For example, one of the things I tell folks, [41:13] that I work with is don't shy away from [41:17] putting 30 data points on a spreadsheet and look at them. [41:22] and see what's going on. [41:24] and data points here is a generic term it may be like 30 customers it may be like [41:29] Thanks. [41:30] 30 ish is a [41:31] chargebacks on your platform, it may be whatever it is. - 30 is like approachable. I'm like, I could do that in two days for most things. - Yeah, it could be two days, it could be two hours. You know, just spend some solid time. You're gonna, a few things are gonna happen. One, you're going to build so much intuition about whatever that is. You're gonna be like, oh, actually that is how that works. You know, and that was already incredibly valuable. But then I think it forces you to overcome this almost like anti-pattern that we have.

[41:59] as tech people, which is like, we want big data, because that's the only way to know. [42:04] Like a lot of data is the only way to know. But the reality is that with 30 data points, you're going to know whether something is 60% plus or minus 10% or it's 10% plus or minus 10%. And you can like know a lot from that fact alone. - Yeah. - You know, is your conversion rate, is your [42:20] chargeback rate, whatever that may be. - It's roughly good or it's roughly bad. - It's roughly good or it's roughly bad, and make a decision. - Yeah. - And then take the top three things you've learned, and try to, [42:29] address them. So even like in everything, like when it came to FAIR, like get actual a bunch of search results and look at them one by one and form an opinion about when is it that you don't think they're good enough and why? And is it because they're completely irrelevant or is it because they're just, yeah. Even though it's like theoretically a little bit less accurate, I also think when you spend time in 30 anecdotes versus like, [42:53] 3,000 like sort of unemotional pieces of data. It's just very different. - Yeah. - You learn more when you look at an anecdote. - Absolutely. And you know, this is, I feel like I want people to form like a deep intuition about the data, the customers, talking to customers, and things like that. And this is one way in which it happens. - Yeah. - And it's a little bit of a blind spot. And by the way, [43:12] I try to do this myself. I seek out times where I can just like put on my AirPods and put on some music and then just kind of like churn through a bunch of data in a spreadsheet because I find that to be an important thing. [43:26] avenue in which I can actually get through context about what's going on. - Yeah. What else would you think about, like, because I obviously like

[43:35] decision-making is such a central part of what, you know, [43:38] you're trying to create and what you're trying to do yourself. What else do you think about at this stage for decision making? So another thing with that first is that I'm trying to [43:49] help with all of the context that I've accumulated at FAIR, but trying to only take with me the good lessons. I feel like I could, [43:58] potentially overshoot my [44:00] role by trying to apply the same exact learnings, the same exact patterns. I think, I mean, it is an enterprise security SaaS company and using AI, and it's just a very different type of business. And so now being a pipeline business, I think we can make sure that people can experiment. And so this is giving me the freedom to sort of step back a little bit. But I think it is also an element of trying to take the learnings that I have [44:26] But... [44:27] allowing experimentation, I guess. Yeah. And I guess probably also with that, you're like, as long as the guardrails are safe, because obviously you need that as a security company, you probably want people to like try things quickly in general. For sure. And, you know, I think we have a lot of security expertise inside, so we really make sure to build things properly from the beginning in a secure way. But, you know, when I talk about experimentation, I also talk about things that, [44:53] Let's see if the AI is able to solve this problem. - Yeah. - You know, you can do that. You're gonna have someone spend two or three weeks on that. - I always thought like, as long as it's a two-way door decision, just like the faster we can try stuff, the better. - Absolutely. - Yeah. - I'm always about making

[45:11] three 90% confidence decisions every week, rather than one 99% confidence decision every quarter. Yeah. Well, Daniel, this was really fun. Thanks for making the time for this and super excited what you're doing at DevFIRST. Thank you so much.