EDGE AI POD
Discover the cutting-edge world of energy-efficient machine learning, edge AI, hardware accelerators, software algorithms, and real-world use cases with this podcast feed from all things in the world's largest EDGE AI community.
These are shows like EDGE AI Talks, EDGE AI Blueprints as well as EDGE AI FOUNDATION event talks on a range of research, product and business topics.
Join us to stay informed and inspired!
EDGE AI POD
From Stanford Labs to Edge AI Pioneers: FemtoSense's Journey
The quest to create artificial intelligence as efficient as the human brain is one of computing's most fascinating challenges. While today's AI systems consume megawatts of power in massive data centers, your brain accomplishes far more complex tasks on roughly 20 watts—about the same as a dim light bulb. This efficiency gap is what FemtoSense is determined to close.
In this illuminating conversation with Sam from FemtoSense, we dive into the journey of this Stanford spin-off that's revolutionizing edge AI with neuromorphic-inspired computing. The company's very name speaks to their mission—"femto" references the femtojoule, the incredibly tiny amount of energy required for a single neuron in your brain to communicate with another. Their goal? Create AI systems that approach this remarkable biological efficiency.
What makes FemtoSense's approach unique is how they've evolved from pure academic research to commercial viability. Rather than pursuing neuromorphic computing in its purest form, they've distilled its key efficiency principles—sparse computing and spatial locality—into manufacturable, reliable systems. This pragmatic approach is already bearing fruit in consumer wearables like AI-powered hearing aids that maintain 24-hour battery life while delivering sophisticated audio processing, and in smart home devices that offer on-device intelligence without cloud dependence. For battery-powered devices, their technology extends runtime dramatically; for plugged-in devices, it slashes costs by reducing silicon footprint.
As Sam puts it, "Efficiency is the new currency." The principles that make FemtoSense's technology possible aren't tied to any specific market or modality—they represent fundamental improvements in how computing can be done. With global operations spanning the US, Asia, and Europe, and active participation in industry groups like the Edge AI Foundation, FemtoSense isn't just building more efficient chips; they're helping shape a future where AI can be everywhere without consuming the planet's resources.
Curious about the future of efficient AI? Join us at upcoming Edge AI Foundation events and discover how companies like FemtoSense are making AI that's not just smarter, but fundamentally more sustainable.
Learn more about the EDGE AI FOUNDATION - edgeaifoundation.org
Sam, good to meet you. I know we haven't met in person yet. I met your team in Austin at our event recently, but yeah, thanks for joining us today, or joining me.
Speaker 2:Well, I'm very glad to be here, pete, and maybe we will soon meet at one of the upcoming Edge AI Foundation events.
Speaker 1:That's where the community is. You look like you're in a very sound proof kind of chamber when are you? Calling in from.
Speaker 2:I'm calling in from one of our little phone booths at our headquarters. We are in San Bruno, california. It's maybe one of the lesser known cities, but it's where sfa is near the, near the airport is it? Five minutes from the airport. It's where our office is yeah, exactly.
Speaker 1:So I think everyone has a subconscious idea of where san bruno is, because they've seen the signs near the airport of uh, where saint bruno is yeah, exactly, exactly so so you guys are so just kind of getting into it a little bit for our listeners, femtosense, who may have not heard of you yet, you're a Bay Area based company. You were spun out of Stanford. Is that, is that the? Or give us the, give us the origin story behind it.
Speaker 2:Yeah, yeah, so we are a Stanford product, I could say Myself, my two co-founders, alex and Scott. We are all doing PhD work in the same lab, the Brains of Silicon Lab at Stanford. It's part of the Norfolk engineering community. So, doing AI perhaps before it was cool or as cool as it is now, and we'll come to the conclusion of the phd program around 2018 and you know, we look around. It's like, oh, you know, this ai stuff is kind of kind of picking up out there right and uh, that's one and then two.
Speaker 2:Uh, it looks like silicon in particular is starting to become sort of an investable uh you know class of business, uh companies that are involved in silicon, whereas you know previously that business, uh companies that are involved in silicon, whereas you know previously that might have during like the heyday of sas, that might not have been true. So it's kind of this combo of you know opportunity. You don't see this uh technology that we were developing. Um, the market seems uh like it's interested in this and it seems financeable. So you know why not? That's the, that's sort of the pipeline, and the whole mission of the place is to get you know useful technologies products out to the world, and it was a good time for us.
Speaker 2:So that's what we did. We basically, you know, took at least initially, actually the research chips that we had made look to build a company around it. That's kind of where the origin of the company name came from. You know, femto is the SI prefix for 10 to the minus 15. So it's very small. And the Jeopardy question is is sort of how much energy does it take for one neuron in your brain to talk to another neuron? That little communication between there is on the order of femtojoules. So if you can make electronics that you know at their base level work on those that order of magnitude, energy for sort of communication, oh, it's just a matter of sort of architecting and engineering and programming it to get it to, you know holy grail of artificial general intelligence.
Speaker 1:Right.
Speaker 2:But yeah, that's kind of where it came from. Right, artificial general intelligence, right. But yeah, that's kind of where it came from.
Speaker 1:Right, and you're trying to. You're trying to create AI, edge AI platforms that are as efficient as the human brain.
Speaker 2:Yeah, there's a lot of. There's a few speaker, sip fix. You know there are a few orders of magnitude to go before what we currently have on the world is sort of anywhere near what you or I are sort of running off Like in the world is sort of anywhere near what you or I are sort of running off, like, you know, just at lunch, uh, we can go for, you know, probably a day plus, you know, with fairly coherent thoughts. Uh. So, yeah, there's, there's definitely some some plenty of headroom in terms of what you currently see in the world and what the world could be so that's kind of the angle here I heard uh, we had ai zip on and they were talking about kind of efficiencies of you know.
Speaker 1:I mean sort of, and you've probably seen this too. Like you know, the human brain is kind of running at about 20 watts or so or something like that you know relative to a giant data center that's, you know, in the, in the kilowatts or megawatts, uh, yep, and so yeah, it's definitely several or orders of magnitude. We have to go uh on that space.
Speaker 2:When, when you know sort of nuclear power, uh it starts to come into the conversation in terms of serving all the stuff, I mean, okay, that's maybe one way to handle it. Um, the other way to try to handle it is be a little bit more, um, you know, put a little more thought into yeah, things are being done, because if you can get the efficiency, yeah, there's plenty of savings there. And this is something that I think also popped into the news in the last month or so. You know, with the, you know like the deep seek models, right.
Speaker 2:It's like oh wow, you know, you really proved that while you can throw jewels and cash at the problem, you can also go. You also go build these capabilities in other ways and there's a lot of merits to that, because it becomes more scalable, more economical. And all of this has to sort of make sense for this AI fields to really be successful. You can't just be wasteful about everything.
Speaker 1:Right. Well, I mean, they say efficiency is the new currency. You know, so DeepSeek sort of proved that and, like you said, I think this is kind of where our community is really. You know, on the frontier of how to create hyper-efficient AI, and you know use power and cost and for maximum impact. You know, use power and cost and for maximum impact.
Speaker 2:Yeah absolutely Absolutely. Yep, you know you talk to the folks. You know integrating these systems, while they often have a very powerful AI, you know on the backend it does cost something to run that and while that's coming down, you know there is a. There are still discussions about how to make that sustainable. If this is a physical product you are buying, is there now a subscription with it? I don't know if I want to pay a subscription for every single physical device I have. However, there is a subscription that we are all paying, which is just our electricity bill. We are very comfortable paying that. In this case it's to a specific gas and electric. Um, but, yeah, if you can deliver those ai capabilities, right then, and there, yeah, you don't have to worry about that sort of uh new business model. And, of course, because it can run on your device, it must be very efficient so that overall is just a a nicer uh impact on, you know, the planet.
Speaker 1:Um, yeah, there's a, there's a gravitational and it's, you know, there's a gravitational pull toward the edge, you know, for computing, and it's like you know, running computing where the data is created is usually the most efficient, effective and secure way to get things done. Yeah, yeah, yeah. So would you characterize the work that you're doing kind of in the neuromorphic realm? Are you in the neuromorphic world? Yeah, you in the neuromorphic world. Yeah, it kind of depends on who you ask.
Speaker 2:I mean, we definitely came from there. That was the PhD work. You know we'd have some, let's say, technology, you could say, pivots along the way. The way that I cast it is that you know, what's good for research might not always be the best for commercialization. So yeah, in the PhD world, it was the full vision of how a neuromorphic might come to pass substrational analog circuits behaving very closely to what an actual neuron would do, at least mathematically.
Speaker 2:You know, you had no clock on the chip for asynchronous communication and compute for anything that was digital and you had spiking neural network sort of programming frameworks and algorithms. And yeah, we were like okay, for the first year of the company it was just trying to commercialize that. We found that there were some challenges associated with that. So the main two are programming. So that style of compute is a little bit hard to get to perform competitively with standard deep learning based back propagation. And then the other big challenge that we recognized with that approach was about manufacturability, variability, sort of liability when you're dealing with systems where every it's a real physical thing, which means physical things like process variation, how much dopant is in each part of the transistor, temperature, and if that can change your behaviors and make it non-repeatable. When you get to real economies of scale or commercial scale, that could cause some real problems in terms of decommercialization. It's nice to have an abstraction, a digital abstraction, to sort of absorb all that variability so that you can deliver something that's reliable. But more than that, we're thinking that, well, why are these systems very efficient and does it have to be implemented in that same way? And we came to the conclusion that, well, these systems are efficient largely for two reasons. One is the sparse compute, the spiking neural networks, all these neuromorphic systems. They don't do heavy operations unless they really need to, and those heavy operations are mostly communication, like point A to point B. Yeah, if you don't have an input then you probably shouldn't do anything. So, yeah, that's the basis there. But the question is, does it need to be in this sort of unary, binary mode of compute with these spikes? And well, you could get sparsity similarly from a rectified linear unit, so why don't you do that? And then, similarly, the other idea, which is or the other principle, I think you could say it that makes these neuromorphic systems very efficient is just that they don't move things very far. So there's sort of the time and space component of this. The time we just talked about sparsity, the space is. Well, when something happens, everything is local to that event.
Speaker 2:So whether you're doing this is like compute memory stuff, as you're not moving things between far-flung memory banks and logic blocks. Yeah, you're saving probably the largest cost in terms of where the energy goes for these modern processes. But the question again was like well, does that really need to be in totally analog, uh sort of fabric? And it's like, well, no, this is not a black or white thing. Uh, it's definitely shades of gray. Uh, you could use a standard segregation between memory and logic.
Speaker 2:But just chop up the structures so they're much smaller and closer together, right? So, as opposed to one giant block of memory over here, one giant block of compute over here, chop it all up and mix it together, make your compiler smarter to handle that. And now you've shrunk that distance and you know you've gotten like 90, 95 of what you can do otherwise. So all that is to say like, well, yeah, the ideas are good, the reasons why it's efficient, you know, they're true, these are just physical devices, same as everything. But you can make them in much more manufacturable, reliable, programmable instantiations, and so why don't you go do that, instantiations, and so why don't you?
Speaker 2:go do that, so that's kind of our journey from you know the sort of pure research that we've been doing to, the more you know how can we take ideas from there and inspiration there and actually make something that is, you know, easy to use, reliable, can be mass produced, can get to market quickly, all the things that kind of. You know a company also has to satisfy beyond just a, you know, single measurement or a single benchmark that you might be incentivized to do if you're just trying to have something to show.
Speaker 1:Right, yeah, no, I think that makes sense. I mean getting it at the end of the day it has to be commercialized to generate revenue, right. So I think that's that's pretty key. Where do you see, like one of your key industries, what are your, what are your hot industries that you're going after, or scenarios or canonical use cases where you think that your stuff would really shine?
Speaker 2:Yeah, everywhere really. You know it's like everybody has. I think on average there's like 18 to 20 devices or something on. All of us are in our, in our home, right. All that stuff could have in it More concretely. You have to start somewhere. So where we're starting is in sort of the consumer consumer curable space as well as smart home space, so in the consumer curable space, for example, beyond the right. After the Edge AI Foundation meeting in Texas last week, we had like a hearing aid conference so we were showcasing and we and our customers called New Sound we're showcasing their first AI hearing aid, Right.
Speaker 2:So that has our tiny little bit of silicon in there, uses all the principles I just talked about. It's a very demanding kind of product category. These things have to last the whole day, right. You know, 10 hour battery life isn't going to cut it, although that would be impressive for something like an AirPod For hearing it. You got to go like 24 hours, right. So if you deliver noise cancellation, speech separation, all those sorts of features on a 24 hour battery life in that form factor, yeah, that can make a difference. So that's one segment that we're in as an initial beachhead market. The other one is in home appliance. The previous iteration of this kind of product category was Internet of Things, where you kind of control things with your phones and devices.
Speaker 1:I may not want to do that, I may just want to talk to it directly, with or without connectivity.
Speaker 2:So, having the smarts on my device, like an air conditioner, I sit down. I may want it to be cool initially, but then, once I've been sitting for a while, I may want to have that off and I could just say, hey, air conditioner, turn off. Those kinds of convenience features is the other segment that we're initially in, but I'll go back to those pencils. We're talking about sprocing locality. Those are not, of course, tied to any modality or any market right. How you navigate the markets is really just a question of you know where you want to deploy your capital initially, yeah, where you think there's opportunity, and take the economics and grow from there.
Speaker 1:Build that flywheel, uh yeah just in one good, good impact although I guess the hearables, wearables Market where you've got that, you know, obviously the battery life is such a huge value proposition there.
Speaker 1:Yeah, so you know, power efficiency there probably has a even like a, even more of a superpower than than like a like I would say like a ring doorbell or other things like that, where it's you know, power is important, um, but with hearables or wearables, I mean just like even with my Apple watch and stuff, like the reason I bought the Ultra, the only reason I bought the. Ultra is because it the battery was bigger, so I could like last like two or three days on it as opposed to charge it every night.
Speaker 2:So um yeah, it's interesting being in disparate sort of markets where they care about maybe some things but maybe something's different. Totally agree with you that the wearables wearables space, battery and cost are sort of the prime drivers, and the other smart home space like if you're an air conditioner, you know nothing consumes so much power in its normal function. Yeah, that's right. Any of the power that we talk about doesn't really matter.
Speaker 2:However, what does matter to them is cost right, and so these ideas of sparsity. Where we talked about the time component, there is a space component as well. If you say that you know you only need to store, like deploy silicon for 10% of this model, well, you've just shrunk your silicon footprint and hence your potential cost by 10, right? So that's perhaps a little bit more important for them than I mean. The power is important. There are energy efficiency standards that people have to satisfy, but yeah, relative to the uh overall device operation, it tends to be more that the cost is the important thing then yeah, and that's a huge thing.
Speaker 1:I mean you can't underestimate. Uh, I've seen so many projects get. You know, they get to the POC stage and they just don't get to commercialization because the ROI is not there and the cost it adds up.
Speaker 1:And especially, I mean you're talking about consumer products, which have a notoriously low margin, and but you know, across across the commercial space, you know there's a quick serve restaurants. I mean all kinds of vertical markets too, where the cost of these solutions like every penny counts. And so I think bringing bringing the AI into, into something that fits inside an ROI envelope is like kind of required.
Speaker 2:Yeah, exactly yeah. There's some pretty good, you know, development platforms out there for edge AI that are, you know, sort of like mini, mini GPUs, um, and those are great. To get to that point, which is the poc, which is say, hey, can we enable this feature? Uh, then you have to do production, and that's when things become a lot harder yeah, that's right.
Speaker 1:Everything gets squeezed um. Yeah, cool. So are you, uh, focusing mostly on the us market now? Are you worldwide, or what's your?
Speaker 2:yeah, worldwide, uh, so while we're based here in the us, uh, we have a lot of activity in, uh, asia. You know, that's where a lot of electronics just happens. Yep, um, at least for now. We'll see. We'll see. With all the trade stuff going on, I don't know yeah, right yeah. And then there's um uh, we have a good amount of projects going on in Europe as well.
Speaker 2:Cool yeah it's definitely a global world and every geography is looking to add these smart features either to their products or to kind of be on the forefront of, I guess, this super cycle. Yeah.
Speaker 1:Some people call it and I guess also as a kind of a startup. In this space you kind of need to go where the business is, so you know if there's interest somewhere. You need to be there and and make it happen Right. So getting that voice of the customer into the product cycle is so critical so that you can, you know, build stuff that's, you know, fitting, fitting you some real, real needs out there, which is pretty critical.
Speaker 2:Yeah, exactly the product developers. While the wider audience may know the biggest companies in the world, the NVIDIAs, the Qualcomm, the Intels, the people who actually build the products, especially in this more consumer electronics space. They have a whole complicated supply chain. It's very important to be out there, engaging, known, so that the product developers have the best possible solutions to, of course, bring to their customers and, ultimately, ourselves as consumers.
Speaker 1:Right, right, yeah. And I would say also, I'll have to tip the hat, the Edge AI Foundation hat, to FemtoSense, because you guys joined recently and I know that you have a representative on our marketing working group. You also have a representative on our program committee for Milan.
Speaker 2:Yeah, that's very exciting.
Speaker 1:Who have not yet registered for the Milan event. You can go there. So, yeah, I really appreciate it. You guys have really leaned in to help out the community and get help with your business as well, so that's really appreciated.
Speaker 2:Yes, yeah, no, it's been great. We only joined recently, but I think we've gotten a lot out of being a member. The JI Foundation, you know, for all of the suppliers, vendors, customers here. We all have largely the same problems when it comes to just being known in the broader marketplace, being understood by totally a person Like how would you explain all this to grandma, grandpa? These things don't have to be solved independently by every single person. We can put some heads together to think about it too.
Speaker 1:grandma, grandpa, right, these are these things don't have to be solved independently by every single person. We can put some heads together to think about it too. Yeah, no, definitely the collaborative vibe. So, yeah, cool. Well, sam, I appreciate the time and, you know, look forward to more collaboration with, with FemtoSense, and certainly we'll. Maybe I'll see you in Milan or, if not, sooner. But yeah, best of luck. It sounds like you guys are on a really good path.
Speaker 2:Great, all right. Well, thank you, pete, and yeah, I look forward to meeting you and other HAI Foundation people in the near future. Sounds good, all right, thanks, sam. Thank you, pete.