EDGE AI POD
Discover the cutting-edge world of energy-efficient machine learning, edge AI, hardware accelerators, software algorithms, and real-world use cases with this podcast feed from all things in the world's largest EDGE AI community.
These are shows like EDGE AI Talks, EDGE AI Blueprints as well as EDGE AI FOUNDATION event talks on a range of research, product and business topics.
Join us to stay informed and inspired!
EDGE AI POD
Edge Computing Revolutionized: MemryX's New AI Accelerator
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
Ready to revolutionize your approach to edge AI? Keith Kressin, a veteran with 13 years at Qualcomm before joining MemoryX, shares a breakthrough technology that's transforming how AI operates in resource-constrained environments.
MemoryX has developed an architecture that defies conventional wisdom about AI acceleration. Unlike traditional systems dependent on memory buses and controllers, their solution features autonomous parallel cores with localized memory, eliminating bottlenecks and enabling linear scaling from small devices to powerful edge servers. The result? About 20 times better performance per watt than common alternatives like NVIDIA's Jetson platform, all packaged in a simple M.2 form factor that consumes just half a watt to two watts depending on workload.
What truly sets MemoryX apart is their software approach. While many AI accelerators require extensive model optimization, MemoryX offers one-click compilation for over 4,000 models without modifications. This accessibility has opened doors across industries – from manufacturing defect detection to construction safety monitoring, medical devices to multi-camera surveillance systems. The technology proves particularly valuable for "brownfield" computing environments where legacy hardware needs AI capabilities without complete system redesigns.
The company embodies efficiency at every level. While competitors have raised $250+ million in funding, MemoryX has built their complete hardware and software stack with just $60 million. This resourcefulness extends to their community approach – they offer free software, extensive documentation, and support educational initiatives including robotics camps and hackathons.
Curious about bringing AI acceleration to your next project? Visit MemoryX's developer hub for free resources and examples, or purchase their M.2 accelerator directly through Amazon. Whether you're upgrading decades-old industrial equipment or designing cutting-edge multi-camera systems, this plug-and-play solution might be exactly what you need.
Learn more about the EDGE AI FOUNDATION - edgeaifoundation.org
All right, here we are. Welcome everyone to Edge AI Partner. We're here to talk about some of the new and interesting partners in our ecosystem and part of the foundation community. Today I have Keith Cresson from MemoryX.
Speaker 2:Yes, nice to meet you. Good to see you. Thank you for the invite.
Speaker 1:Yes, keith and I actually know each other from a previous life, where he was at Qualcomm and I was at Microsoft. How long were you at Qualcomm for?
Speaker 2:A little over 13 years.
Speaker 1:Yeah, so pretty good chunk of time.
Speaker 2:Yeah.
Speaker 1:Yeah, good time, it was a good time, good times. So it's funny in this ecosystem for me I've been doing kind of where software meets hardware for like for decades, and so you kind of meet people over and over again kind of that are in this area and so that's kind of the fun of it, I guess, is you kind of reengage with folks over and over again in different phases. But but you've been with MemoryX now for a few years now Almost four years.
Speaker 2:This year will be four years, which is hard to believe.
Speaker 1:Cool. But you've been with MemoryX now for a few years now. What's the story? Almost four years. By this year it'll be four years, which is hard to believe Cool. So why don't we get like the? What's the origin?
Speaker 2:story here on MemoryX and what's the deal? Yeah, memoryx was founded by a couple of professors at University of Michigan and professors and a couple of their top grad students, so it was founded in 2019. And the goal was to create an edge AI accelerator different from everybody else. I know lots of people have different ideas on how to be different, but I was at Qualcomm at the time and I reviewed their architecture and technology, really liked it and decided to join and lead the company. So I've been there since late 2021.
Speaker 1:Cool, and are they mostly still Michigan Michiganders, I guess?
Speaker 2:Yeah, we've expanded. So the heart of the company and the headquarters is still in Michigan. We have a couple of stragglers like myself out in California, a small team in Taiwan, small team in. India and we're about to open offices in Saudi Arabia. So we have the address, we're just hiring now and had our first acceptances, and so by October I'll be visiting Saudi and we'll be inaugurating our office there.
Speaker 1:Okay, well, we'll have to have a chat offline about the kingdom, because I think there's a lot of really cool tech happening there and it's kind of a real hotspot these days for investment and innovation too. So that's pretty cool.
Speaker 2:They're going to be a major hub in the Middle East, no doubt about it. I think. Uae and Saudi Arabia, yeah, exactly.
Speaker 1:Cool, well, I must keep you busy. I guess you get a lot of frequent flyer miles then flying around yeah, now and again.
Speaker 2:Yeah, some good uh time zone action too. Uh trying to set up a meeting between all those sites yeah, tell me about it.
Speaker 1:That's one thing we didn't solve. Like we really flattened the earth in terms of, you know, meetings and other things, but time curvature of the earth tough one. But so memory X so as I understand it and I have a few questions about it, because it sounds really cool I mean it's really like I wouldn't say it's like a radical simplification of an AI accelerator where you have everything you need in an M.2 connector and the accelerator, the memory, everything is just in there and you just can. I guess one question is what makes it special and different? I'm curious about that and this term that you were using called at memory compute. I'm interested in that.
Speaker 1:And then also, where do you see this? Where are you starting to see traction? In my mind, kind of like in my mind, when I read about it, I was like, oh, this sounds like ideal for like brownfield computing or other platforms, where maybe you have a compute platform and you want to add AI into it, and then this is like a way of doing that without redesigning the whole system. That's true. But so what's the? What's the special? What makes it?
Speaker 2:so special Special sauce. Yeah, everybody's got something that's special. On memory acts, I think the architecture is the key. So it's a bunch of highly parallel cores and memory is local to each of those cores and the cores are all instructed as to what to do at compile time and they all work autonomously. So there's no bus that exchanges information between the cores. You can make a chip with, you know, 100 cores, a thousand cores, a million cores. Uh, memory is localized and you could scale so it all operates in parallel.
Speaker 2:So if you think of a cpu and a gpu, you have kind of a narrow bus that goes out, obviously. Gpus, you have HBM, you have multiple buses to HBM, but it's always kind of a bus to memory architecture. Here there's no bus, there's no memory controller. It can operate with any memory that's local to the compute and it's autonomous. So it really really has good scalability and complexity has been pushed as much as possible to the software layer versus the hardware layer. So it's very simple to use, very scalable. You know the compiler and everything is done in-house, very efficient from a power performance standpoint.
Speaker 2:But the scalability, ease of use. You know, run models out of the box without tuning them and we could build basically any design point. We chose to build, like you said, a form factor that goes into an m.2, so kind of single digit watts. But if someone said tomorrow, hey, can you build a, you know, can you build a server core? Uh, we have the foundation for it and we could, um, by just scaling up the design.
Speaker 2:So the architecture is really compelling. Of course it's expensive to do silicon, and so we have a single chip now, the MX3, and we're working on our MX4, but we can almost scale to any design point and the software and hardware is scalable for that. And then, what markets is for? We've initially targeted, uh, computer vision centric markets, and so we'll go into, you know, a device that has a single camera. That's possible. But we're really, you know, because performance is higher, it's really good for any multi-camera system. So surveillance or security system, you know, video system, retail or industrial use, or a machine with multiple cameras, those are all good, really good applications.
Speaker 1:Yeah, and do you see it like in those kind of brownfield environments where people are like I have a platform, I want to AI accelerate a platform, and then I can plop this thing in there? Is that like that?
Speaker 2:Often, yeah, often, and, by the the way, we get some really old legacy platforms like uh, really surprising the long tail. We have a you know someone who's a 32-bit platform, or someone has a you know an old you know one of the first pci gen 3 platforms. It's more than a dozen years old and they want to add ai to it and it's an industrial application. They don't want to rip it up, they don't want to change the os, they don't want to change the OS, they don't want to change the application processor. We just connect up to a USB or PCI Express port. You can offload all the AI to us and we support, you know, any Windows or Linux or Android, any Xe6, any ARM, and so it scales really well across different applications and frameworks and AI models.
Speaker 2:And so yeah that's the long tail kind of just update these ancient platforms. That work and they run and they're for industrial use, and also, obviously, there's a number of new platforms that we can do well, sure, yeah, a lot of people don't realize.
Speaker 1:Um, we were talking before we started recording about revenue, but, uh, there's a lot of stuff deployed out there and, um, nobody is really motivated to rip it up and re-validate everything again because, you know, probably the developer that worked on it is no longer there anymore and blah blah. It's like if it's not broken, don't fix it. But there are all these new scenarios and new things I was thinking about, like vending machines and other things like that, where there's some level of automation in there and some some level of compute, you know. But obviously what's happening now in terms of AI and edge AI is way beyond, you know, what people originally imagined. So there's a huge opportunity for people, especially in the industrial, like you said, retail environments, to kind of add more AI capabilities into it, so that must be interesting.
Speaker 2:That's right and we'll go into. You know, each chip uses kind of a half a watt to two watts depending on the model.
Speaker 2:So we can go into a very small system like an individual camera, all the way up to edge servers where we have, you know, multiple M.2 cards and someone, for example, has an NVID powered server with, uh, a6000 gpus or something else and we can go in and say, wow, we can be much cheaper, much lower power, go to a 2u form factor and we can process, you know, 50 or 100 data streams and that edge server for a lot less money, a lot lower power, a lot, uh, smaller footprint and uh so.
Speaker 2:yeah, so there's a number of compelling applications, but yeah really, by the way, we can run on one chip or we can attach basically any number of chips, uh to a platform and it just scales, you know, linearly yeah exactly in software you mentioned a couple of like universal truths around edge ai.
Speaker 1:There, like you know, being able to do things for less cost, less power, you know, more impact. I mean most people. When they get started with edge AI, you know they start with like a Jetson nano box or something like that, which is, you know, no harm, no foul, you know, but you know then it's like, well, actually do I need that? Do I need all that? You know like, so yeah, it turns out you can do a lot of AI with a lot less power and cost.
Speaker 2:Yeah, that's a, yeah, that's a. That Jetson's a very common starting point, because everybody happens to know NVIDIA and they have a lot of software. And if someone wants you know, I'd say, on average you're about 20 X better performance per watt. So if someone says, and lower and lower cost, obviously so. If someone says, and lower cost, obviously so. If someone wants a smaller form factor, lower power, lower cost, less maintenance, then we're a good option to go to production.
Speaker 1:Yeah, you mentioned the software layer, so you must have a pretty hot shot software team in terms of the compiler and optimization. And do you think about these paths, these developer paths that are coming from CUDA or coming from PyTorch or like? What are the different? How do you handle some of that?
Speaker 2:transition we start with. So we don't start with the CUDA optimized model, but we start with the trained model from you know any of the modern frameworks, you know PyTorch or Hectorflow or TensorFlow Lite. So we start with just a trained model and really our claim to fame is one click out of the box compilation without any model modifications. And everybody says, oh no, you can't do that. And everybody says, oh no, you can't do that. Uh, but if you look at our website, we have I think on our website now we have maybe 400 models that we show we can do that and uh, in our in our testing, we have more than 4 000 models where we just uh get them from customers. We scrape the web, all you know hiding face, you know all different sources and we do a one-click compile and then if someone you that gets the customer most of the way there then if the customer is ready to deploy in volume.
Speaker 2:Obviously we can work with them to optimize the model squeeze every bit out, but really out-of-the-box performance. Power, you know, is stellar with the platform.
Speaker 2:So, yeah, the compiler team is definitely, I would say, kind of the heart of the company and we do everything in-house. You know, the full ai design is including, you know, the state machines and macs, and everything is done in-house, as is the compiler and the entire software stack, so we don't rely on anyone else's ip it's, it's all done homegrown, uh, within memory apps that's pretty impressive yeah yeah, that's cool, and so where are you at in your your?
Speaker 1:you're obviously not public yet, so you're still private here, that's right. How many series are you in are you? What's? What's your funding situation these days?
Speaker 2:yeah, we completed the series b earlier this year, probably raises c next year, um, but we've raised a total of about $60 million to date, which sounds like a lot. But complete silicon, pcb, complete software stack supporting lots of customers working on the next generation architecture, everything in-house. If you look at our peers, everybody's raised, I think, more than $250 million plus. We try and be as frugal and efficient as possible in all respects, both in power and in fundraising.
Speaker 1:There you go. Well, you know you guys are a bargain, so that's good. Yeah, efficiency all the way through. You know operations as well. That's really cool, and so do you have any particular deployments that you'd like to highlight? You mentioned some of the markets that you were going into, but any kind of? You know you've got your cocktail party chatter on your case studies and stuff.
Speaker 2:I mean, what's stuff?
Speaker 1:that's really kind of gotten you excited.
Speaker 2:Yeah, I mean there's really a variety of different applications. So in manufacturing we have systems deployed for defect detection. We have a customer, although it's not a big focus. We have a customer that's working on a drone, someone that's doing a handheld medical device. We have a construction safety application. We have a number of video management server-based applications. We have someone that's deployed many, many thousands of camera streams and they're using NVIDIA-based servers and we offer a pretty compelling opportunity. So we're not deployed yet but we have the server in-house. We can show the advantage and that'll take a little while to get the deployment out there, but I think it's going to be a huge advantage for the customer. So there's a pretty wide variety of industrial manufacturing, vms services, surveillance, instruction, and we're all about kind of computer vision and, like I said, single to multi-camera and the more cameras someone has kind of, the more value they're going to get in using us.
Speaker 1:Yeah, yeah. Well, you mentioned, like workplace safety, construction sites. People maybe don't realize the amount of tech that's being put into those sites. I mean there's like OSHA and kind of a lot of regulatory safety things where people need to be monitored and, you know, there's even like noise level measuring. I mean there's all kinds of interesting things going on at construction sites visually and the sound and monitoring environmental conditions and stuff like that.
Speaker 1:So that whole space is really fascinating, because you'd think it's not a very techie thing. It's like a lot of dust and wood and stuff. But yeah, there's a lot of tech in there. It's pretty amazing.
Speaker 2:Yeah, yeah, safety and compliance is critical. A lot of tech in there. It's pretty amazing. Yeah, yeah, safety and compliance is critical. So making sure they're, they're the right people are in the right zones, and uh, they're, they're prepared. And uh, yeah, no one, no one, goes the wrong direction and uh, even even construction site progress is something, yeah sure, everyone's want to do. You know why have a 24 by 7 camera just staring at your construction site when you can uh use metrics right to evaluate?
Speaker 1:progress.
Speaker 2:So there's a lot of really interesting and creative applications.
Speaker 1:I had a kind of a funny sidebar. We have a house in Massachusetts it's kind of out in the boonies, out in Cape Cod and we had a septic problem. We had to get the drain field field replaced and so they did it under the.
Speaker 1:They did it. They had to dig up the driveway to do it and I had a ring camera out there. So I'm like in seattle, so I'm like monitoring this construction through my ring camera. Yeah, and uh, seeing these giant you know diggers and stuff like that and uh, it was funny for about a week like nothing happened, and I was like hmm. So I texted the guy chris. I'm like, hey, you know diggers and stuff like that, and it was funny for about a week like nothing happened. And I was like Hmm. So I texted the guy Chris.
Speaker 2:I'm like.
Speaker 1:Hey, you know what's up with the crew, you know, and I guess they had been assigned to some other jobs and so he's like, oh yeah, we'll get him back. We'll get him back. But I was like, Well, I'm glad I had my camera looking at my.
Speaker 2:Yeah, and I don't know if you were just looking at it, but yeah, if you enable some AI metrics, you know what time they show up, what time they leave, how many workers are there. Often they leave the area they're supposed to be working on. I mean, there's a lot of metrics someone can get kind of automated using AI these days. Yeah, yeah, that's fascinating.
Speaker 1:So how did you get connected with the Edge AI Foundation? Fascinating.
Speaker 2:So how did you get connected with the Edge AI Foundation? Well, you know, I knew about it from many years ago. The tiny ML effort, yes, and so I knew about it. And you know, philosophically, memoryx one thing is open source, community, broad based. All these are philosophies that we have that align really well with the Edge AI Foundation. We just finished a summer training with the robotics camp at University of Michigan.
Speaker 2:We're doing a hackathon here in September. We're doing another hackathon in Saudiudi arabia in october. Um, we publish everything you know publicly. We don't. We don't hide or keep things closed source. So someone can go to our developer hub and they can get, you know, 100 examples, 400 different models. They can get all the software downloaded for free. We explain how it works. We have architecture white papers. So we want to be as open and easy to use and free as possible and we thought, hey, that aligns really well with some of the goals of the EJI.
Speaker 1:True, true, yeah, no, that's true, and we do a lot of, we underwrite a lot of educational programs and I'm making a mental note to connect you with this guy, marcelo we're running, actually, a program in Columbia, latin America, I think at the end of this month. It might be too late for that, but yeah, I think that it's an amazing. It's almost insatiable the hunger that's out there to learn this stuff and to implement it. And that's one of the benefits of this type of AI is that you can do it hands-on, right, right, and no cloud required, exactly which is kind of cool and you know we really, really go out of our way to make it as easy to use as possible.
Speaker 2:So you know one of the programs on the robotics, these are high school students using ai accelerator and programming it using all different models, integrating into it's not just theoretical in the physical, you know physical objects that they're using AI acceleration and we want to be as easy to use as possible so we can scale to not just you know, the hardware, but scale in terms of people and that's the education aspect and giving back to the community is, you know, is really important to us yeah, cool, cool.
Speaker 1:So what kind of advice would you give? There's so many startups out there doing ai stuff. You know obviously you're coming at it from the silicon angle, um like, and you've been in the industry for a while, kind of know the deal. Like, what are your words of wisdom when someone comes up to you and says, hey, I want to start an ai company, do x, y and z? Oh, yeah well one.
Speaker 2:One is to survey the market, because there's a lot out there, a lot of creativity out there, and you need to have something special and differentiating is one. Two is realize you can't do everything yourself on AI, so partnership is extremely important. Just about everybody requires partners at some level, and so I think doing the upfront diligence and what I have, that's differentiating. What partners do I need? Ecosystem, what value can I provide for my customer and what really am I all about and is it sustainable?
Speaker 1:Am.
Speaker 2:I a product or a feature, or am I a company? These are the sorts of questions, uh, but I I think a lot of people kind of just jump into it very quickly without doing the diligence up front, and you need that to be successful. So yeah especially, especially these days. There's so many ai companies and, uh, if you're not in ai, most people can barely tell the difference, and so you need to put the go beyond chat GPT questions to due diligence for yourself on the right path to go forward.
Speaker 1:Yeah, no, that's good advice. That's good advice and doing that due diligence, being honest with yourself and your product, like you said, are you a product or a feature, and is this sustainable? Is it scalable? I mean, these are all questions obviously investors will ask you anyway. So good to sort of, uh, answer those first before you get in front of some uh, some sandhill road folks or whoever that all yeah, those guys are increasingly skeptical, you know, they've heard a lot of stories in ai land and uh, you need to bring more than just a good story, uh to them.
Speaker 2:Yeah, need to be buttoned up pretty good cool.
Speaker 1:Well, ke, it was really great to catch up again with you. Hopefully see you in person at some point soon. And wow, memory X sounds really cool and I encourage folks to go to your website and learn all about it. Anything else, any other call to action other than the website?
Speaker 2:Our website has a link to our developer hub and you know you can. You can, not that this is a commercial, but you can buy the product off Amazon. You can use all the software you know, plug it into any system with an M.2. All the software is free online and so I encourage you know. Like I said, high school students can use it. I would encourage anyone that's interested in learning more about AI and Edge AI and one of the tools. Hopefully we have all the resources that are out there and if we don't, we have an online community. We can answer any questions. Anyone has Awesome.
Speaker 1:Yeah, sounds very accessible, awesome, cool. All right, keith, great talking to you Thanks. All right, take care. Bye-bye. Okay, so I'm going to hit end and then the. It's actually recorded locally it.