Optimizing Memory on SoCs with Zephyr RTOS
When you move an AI model from a laptop to a 128KB microcontroller, everything changes. You don’t just run a model; you negotiate for space.
Most people look at the datasheet of a chip and see “256KB SRAM” and think they are safe. But the moment you boot Zephyr, that space starts disappearing. The kernel itself needs a slice. If you have Bluetooth, Wi-Fi, or any kind of security stack, they take their tax immediately. By the time you get to the AI model—the very reason you bought the chip—you might only have 60KB left.
This is the first place where projects stall. A data scientist might give you a model that needs 50KB just to sit there. But in the embedded world, you need Tensor Arena space — the working memory where the math happens.
An experienced engineer doesn’t just try to make it fit. They go into the Kconfig files and perform surgery. They shrink stack sizes, disable unnecessary drivers, and turn off chatty logging features that eat up RAM for no reason. It’s not about being a programmer; it’s about being a minimalist. If you don’t have someone who knows how to strip a system down to its bones, your model will never even load.
The CPU is Your Biggest Expense
In a typical software project, the CPU does the heavy lifting. That is what it’s there for. But in the world of battery-powered Edge AI, using the CPU for everything is a mistake that kills your product’s lifespan.
Imagine a sensor collecting data every millisecond. If the CPU has to wake up, grab that data, and move it into the AI’s memory, it’s constantly burning power. It never gets to sleep.
Professional engineering teams use a side-channel called Direct Memory Access (DMA). It’s like a dedicated conveyor belt. The sensor drops the data on the belt, and the hardware moves it directly into the AI’s memory without the CPU ever knowing. The CPU stays in a deep, low-power sleep until the buffer is full and it’s time to run the actual AI math.
Setting this up in Zephyr isn’t plug and play. It requires deep work with the Device Tree—the map of the chip. If this isn’t done correctly, your device might work, but it will require a battery the size of a brick to last a week.
The Problem with Jitter
Zephyr is a preemptive system. That means it is a multitasker. It is constantly deciding what is the most important thing to do right now.
Imagine your AI is halfway through a calculation to detect a critical failure in a machine. Suddenly, a Bluetooth heartbeat signal comes in. Zephyr might pause the AI to handle the Bluetooth packet. Now, a calculation that should have taken 100ms takes 130ms. That 30ms gap is called jitter.
In the lab, 30ms might not matter. In the real world, if you are monitoring a high-speed engine or a medical device, missing that 30ms window means missing the very event you are trying to detect.
Engineers solve this by being ruthless with Thread Priorities. You have to isolate the AI. You can’t let it run on the same level as a low-priority system check. You have to use tools like System Work Queues to ensure the math happens exactly when it’s supposed to, every single time.
Secret Weapons in Your Chip
Many modern chips have secret weapons—hardware components designed specifically for math, such as NPUs (Neural Processing Units).
Zephyr supports these, but it isn’t automatic. If your model uses a specific type of math that the NPU doesn’t recognize, the system will quietly switch back to the slow CPU software. You won’t get an error message; the device will just run 10 times slower and get much hotter.
Professional engineers spend a lot of time tailoring the model to the hardware. It’s a back-and-forth process. You change the model slightly, check if the hardware likes it, and repeat until the data stays on the fast path. This is where most internal teams get stuck for months.
Production is Harder than Prototyping
A prototype works in a controlled office. Production happens in the messy real world as explained in our previous post – Zephyr RTOS in Production Environments.
In the real world, batteries get weak, chips get hot, and devices stay on for months without being touched. AI math is intense, and it creates heat. If a chip gets too hot, it slows itself down to prevent melting (thermal throttling). If your code doesn’t expect this, your timing gets ruined.
Furthermore, a tiny memory leak that doesn’t show up in a 5-minute demo will crash a device after 5 days in the field. Production engineering means building in safety nets—watchdog timers that can restart the system if it hangs, and monitoring tools that check for memory fragmentation before it becomes a problem.
The Gap in Talent
The reason so many Edge AI projects fail isn’t a lack of effort. It’s a gap in expertise.
You likely have great AI scientists who understand neural networks. You likely have great firmware engineers who understand bits and bytes. But finding someone who understands how an AI model uses a Zephyr thread while managing DMA channels is incredibly rare.
This Integration Gap is where projects go to die. Companies spend a year trying to fix mysterious bugs that are actually just basic system engineering issues.
Why Smart Companies Partner Up
When you are building a product, your goal is to ship. You don’t want to spend six months learning the hard way why a specific DMA channel is clashing with your Wi-Fi stack.
The most successful companies realize they shouldn’t reinvent the wheel. They look for a partner who has a blueprint for success. Someone who has already shipped millions of devices and knows exactly which settings in Zephyr will cause a crash three weeks from now.
By working with experts, you stop guessing and start shipping. Your internal team stays focused on the product features—the things that actually make you money, while the experts handle the complex plumbing that keeps the lights on.
The embedUR Advantage
The industry is moving toward Zephyr because it is powerful and flexible. But that power comes with a steep learning curve.
At embedUR, we have spent over 20 years in the world of embedded systems engineering. We have seen every way a device can fail, and we know exactly how to prevent it. We don’t just write code; we build production-ready systems that are stable, fast, and efficient.
We help you bypass the trial-and-error phase. We ensure your memory usage is optimized, your power consumption is minimal, and your time-to-market is as short as possible.
If you are tired of the prototype cycle and ready to put a real product in the hands of your customers, let’s talk. We have the technical muscle to make your vision a reality.
