Untethering the development of processors with AI

As the tech industry witnesses an unprecedented explosion of machine learning and generative artificial intelligence (AI) use cases, Canada continues to produce home-grown talent capable of rising with the tide of emerging expertise.

Among these leading players is Untether AI, a Toronto-based semiconductor firm that specializes in developing high-performance AI processors for various applications, particularly focused on deep learning and neural network tasks.

Listen to podcast interview with Chris Walker of Untether AI

These processors deliver exceptional performance while minimizing power consumption, making them suitable for deployment in edge computing devices, data centres and other AI-driven systems. Untether AI’s technology often involves innovative approaches to parallel processing and memory management, enabling efficient execution of complex AI algorithms. Overall, the start-up firm aims to push the boundaries of AI hardware acceleration, facilitating faster and more energy-efficient computation for a wide range of applications.

“Our firm was founded on the notion that as AI goes into deployment – as it becomes real in the world from training and inference – we’re purely focused on inference and making it run – making it work at scale,” said Chris Walker, CEO, Untether AI.

Walker says that in the early adoption days of AI, chips have been running on the multi decades old von Neumann architecture.

“What we do is called memory compute, as we have an array of more than 1,400 processors together with memory and a spatial array on the chip. What differs is, we can we put the memory and the compute together – where it matters the most for speed and accuracy. In contrast to traditional GPU based architectures, this dramatically reduces the amount of work and energy consumption involved with moving data from external memory, through what are a couple of bottlenecks in traditional architectures into the computation. This is where the real action happens.”

Memory & compute combined

Walker says by putting the memory and the compute together, while building in efficiency – ultimately delivers as much as an 80 to 90% reduction in wasted energy by pushing data through bottlenecks.

“So much energy is wasted just shuffling data around. If we’re going to be using AI in our everyday lives at scale deployment, it must be energy efficient, it has to be fast, and it has to be accurate,” explained Walker.

At present, the firm processes AI compute workloads at least 15 times more efficiently than the state-of-the-art chips offered by leading GPU manufacturers. From autonomous vehicles to enabling new enterprise class generative capabilities, demand for AI is an immense opportunity. This is where Untether AI’s ultra-efficient at-memory chip architecture is well-positioned to be a market leader, according to Walker.

Chris Walker, CEO, Untether AI. Source: Untether AI

The firm’s depth of engineering expertise is fortified by a blend of talent on the team – both on the hardware and software side. Untether AI draws its strength from staff that have worked with AI start-ups to large scale technology architectures, from FPGAs to microprocessors. Walker has spent most of his career at Intel Corp., and has access to machine learning experts, as well as senior advisors involved in four generations of the tensor processing unit (TPU).

“Most importantly, we have a very rich base of both experienced and new students from the University of Toronto and University of Waterloo. That’s a combination that brings together new ideas and experience,” Walker said.

Special time for AI

Founded in 2018, Untether AI has grown from its nascent years of employing 30 people to more than tripling its staff size today.

“I think it is a really special time for AI, as the world’s moving beyond just the training and getting AI developed – to actually making it happen. For us, we think our role is serving as an accelerating ingredient, to be that partner of choice – to enable other streams and applications,” Walker continued. “We think this technology is going to be impactful for the world in terms of energy consumption and usage. As a start-up, we see the spirit of innovation, the spirit of choice, in disruption, where things have been dominated by one big player (NVIDIA). It’s a very big market, so there’s a lot of room for more. We think we’ve got a great role to help our customers differentiate from a standardized solution – where there’s only been one vendor. Being a part of our customer’s success is really how I see us playing in the future, and what I think our role is.”

OEMs and on-premise data centres have begun using the firm’s AI inference ICs and AI accelerator cards to deploy artificial intelligence just about anywhere, noted Walker.

“I think from a design and implementation perspective, the semiconductor industry is often creating for use-cases and capabilities that the end-market doesn’t quite know they need yet. So, you have to be ahead of the curve of envisioning what the capabilities need to be, and what are going to be the important factors,” Walker said. “In our case, we believe it’s computing power, that’s energy effective, because if you want to do things at scale, you can’t be infinitely building out data centres and adding power plants, while cooling them as well.”

As a relatively small and nimble company, Walker says Untether AI operates within a very transparent internal culture – functioning as open and collaborative with customer issue resolution.

“In some customer-use cases, they might be a few years out before they deploy our technology. But, they’re looking at architecture today. They understand what it can do in the future. A big part of what we strive to achieve, is to be a great partner for our customers, to listen – to adapt. There’s fantastic new technologies and enablers coming out, like chiplets and standard interconnects, that are going to enable us to do that, and still retain our focus on being the most efficient and effective,” added Walker.

With customers in Canada and across North America, Untether AI partners in different ways. One is just on the delivery of its product, while others involve more in-depth engagement including collaboration on designing the test infrastructure, which is typical in the hardware ecosystem. The firm also teams-up with clients around the world on industrial, automotive, aerospace applications, which primarily focuses on software.

Run a test model

“We need to find what they want to do in their neural networks. We have an infrastructure that permits us to run it on today’s chips, or a simulation of the future chips. With that, we can start development with them in real time,” Walker stated. “So, we can run a test model, to gauge performance and determine what they might iterate. That’s the interesting part, as you start with your own ideas of the product. It’s really when you engage with customers, that you see the impact it has. And, it makes us better in terms of what customer support is necessary, such as network development, tools and flows. It’s an iterative and collaborative process to make it all happen.”

Moving forward in the next two years, Walker says his team will rapidly grow off the baseline it has established and continue to support markets anywhere from enterprise data centre to automotive and a chiplet approach.

Source: Untether AI

Untether AI is a member of the newly formed consortium behind the UCIe chiplet standard (Universal Chiplet Interconnect Express), an open specification that defines the interconnect between chiplets within a package. This enables an open ecosystem and ubiquitous interconnect at the package level. Other prominent board members include Alibaba, AMD, Arm, Advanced Semiconductor Engineering.

“Today, most products like ours are developed at the silicon level – which then gets put in a package – and then put on a board,” Walker said. “Chiplets are designed in a ‘Lego style’, which allows a customer to create their system with multiple different components in one package.”

As the (UCIe) standards are being established, and Untether AI remains at the forefront of those standards, Walker is confident that this will help his team stay on the cutting edge when working with others and how they want to imagine their products.

“This enables customers to achieve what they want, and not have to do the whole solution themselves. Together, including AI, brings the ‘best-of-the-best’ in terms of inference acceleration, which really opens the door for new applications and new deployments,” concludes Walker.

—————————————–

www.untether.ai/