Stop talking about TOPS, start talking about TOPS per Watt
As edge hardware becomes more power-efficient and AI models get smaller and smarter, intelligence is moving closer to where data is created. The result isn’t just faster insights, it’s a fundamental shift in how IoT systems are designed, deployed, and operated.
This article is based on a conversation with Dave Kranzler, Director of IoT Services at AWS, and Jennifer Skinner-Gray from Avnet, during IoT Stars at Embedded World North America..
Together, they explored what this shift means in practice, from why TOPS per Watt now matters more than raw performance, to why it’s time for the industry to stop framing architectures as “edge versus cloud” and start thinking in terms of collaboration.
The New Metric for Success: TOPS per Watt
For years, edge computing conversations were dominated by raw performance. Faster processors. Bigger numbers. More TOPS (Trillions of Operations Per Second).
According to Dave Kranzler, that mindset is outdated as he argues that the metric that matters most today is efficiency, specifically TOPS per Watt. While edge processors have leaped from 1-5 TOPS to 50+ TOPS in recent years, the real breakthrough is doing this without draining the battery. Dave Kranzler used the human brain as the benchmark: a processing unit that runs on roughly 20 Watts.
For IoT engineers, the lesson is clear: you don’t want to carry a "power plant" on your device to run AI. Success lies in selecting hardware that balances high-performance compute with power efficiency.
Stop saying "Edge vs Cloud"
The industry often frames architecture as a binary choice between processing at the edge or in the cloud. Dave Kranzler calls on IoT engineers to view the edge and cloud relationship as symbiotic rather than adversarial.
In a modern architecture, the cloud acts as the "Orchestrator"; handling security, provisioning, software updates, and aggregated training. The edge handles tasks requiring low latency, high privacy, or bandwidth reduction.
This symbiosis enables state of the art capabilities like Federated Learning, where devices train on incremental data locally and send only the insights back to the cloud. This approach turns the edge into a distributed training network, reducing data transport costs while improving model accuracy.
Purpose-Built Robotics using Narrow AI
Humanoid robots may dominate headlines, but Kranzler expressed skepticism about their near-term practicality. The challenge lies in the massive “sim-to-real” gap; the difficulty of translating simulated training into reliable real-world behavior, especially for complex movement and manipulation.
Where real progress is happening is in purpose-built robotics. Examples already delivering value today include fulfillment center robots, and medical exoskeletons. These systems focus on narrow, repetitive tasks where constraints are well understood and reliability matters more than general intelligence. This is what Krazler refers to as “narrow AI”.
IoT is the nervous system
The definition of IoT is shifting. It used to be about connecting sensors to a centralized brain. Today, we are moving toward a world of distributed, autonomous agents.
Kranzler summarizes the conversation with a quote from Paul Williamson “IoT is the nervous system. GenAI is the brain.” When these brains move closer to the edge, the system starts to look less like a single organism and more like a community of intelligent agents.
This shift introduces real challenges around orchestration, security, and lifecycle management. But it also unlocks entirely new classes of applications.
Then, are we finally at a point where sophisticated AI can run on edge devices without compromising performance? The answer was a resounding "Yes," however, success in this new era requires a focus on power efficiency, a symbiotic cloud-edge architecture, and the wisdom to partner on the plumbing so you can innovate on the product.
Powered by AWS and AVNET.