Semiconductors and AI intersect

As artificial intelligence (AI) continues to gain a foothold in all technology sectors globally, semiconductor development is in lockstep with the revolution. Innovation in this space is accelerating at breakneck speed, as market demands evolve, while geopolitical factors and supply chain challenges nip at the heels of chip designers today.

In order to gain some insights to this topical issue, EP&T sits down with Raymond Chik, PhD and consultant with DRC Enterprises, providers of business solutions, from financial budgeting and planning, start-up guidance, institutional marketing, as well as governance, risk management and compliance (GRC) consultancy.

Adding to his relevancy in this conversation, Chik is a highly regarded semiconductor design engineer, serial entrepreneur and an angel investor. He completed his PhD in electrical & computer engineering from the University of Toronto in 1995.

Raymond Chik, PhD and consultant with DRC Enterprises

Q. Can you provide an overview of the current state of the semiconductor industry and its significance in the rapidly evolving field of technology?

The current state of the semiconductor industry is characterized by the interplay between technological innovation, rapidly evolving market demands, geopolitical factors and supply chain challenges – recent geopolitics has forced the once well-established global semiconductor supply chain into two separate supply chains – China and the West. This is highly inefficient and anti-capitalism in exchange for national securities.

From the market perspective, as we come into 2024, the global semiconductor industry is heading into a recovery from the downturn in 2023. This is partly due to the rebound of the memory chip market. Of course one significant driving force that is fuelling the semiconductor market is artificial intelligence (AI).

Q. How do you perceive the intersection of AI & semis shaping the future?

AI and semiconductors are extremely important to each other. In fact, I would say that the main reason for the rapid development of artificial intelligence is because of semiconductors. AI has been around for a long time, but only in recent years, with the advent of semiconductor compute chips, in particular GPUs, the amount of computational performance/ required to train and run AI neural network models became adequate.

This also partly triggered the explosion of LLM and generative AI in the past year, because now there are hardware infrastructures powered by these compute chips available for training and running these big models possible. You’d still need tens of thousands of GPU chips to train and run them; but at least it’s feasible.

Q. Can you highlight some key challenges and opportunities in the chip business?

Here are a couple of key challenges that I see:

• Supply chain inefficiency due to geopolitics; the de-globalization and on-shoring or supply chain will have a significant pricing effect on semiconductor products – we will see more expensive electronic products.

• Talent shortage: the need for skilled-workers in the semiconductor industry will be growing and companies often have difficulties filling qualified candidates with relevant experiences to fill their open positions.

Some opportunities that I see:

• Advance packaging technologies: while the industry is pouring tons of investment into keeping the Moore’s Law alive, the advancement silicon chip packaging technology has not been progress at nearly the same rate as silicon wafer processing technology. This has changed over the past few years where advanced packaging technologies are starting to provide a new and better way to improve chip performances.
• Techniques such as Fan-Out wafer-level-packaging (FOWLP), 2.5D and 3D die-stacking technologies allow new avenues for chip makers to create high performance products without only resorting the process node advancement through Moore’s Law.

• The rise of AI: this is a no-brainer. It has been predicted that by 2032, the AI HW market will grow from about US$30B from 2022 to >US$600B in 2023. This explosive growth will be a tremendous fuel to semiconductors made for AI.
• Automotive industry growth: this is another huge growth area for semiconductors. The shift towards EV (electric vehicles) and autonomous driving, particularly battery management systems and ADAS, will create an increased demand for semis designed for automobiles.

Source: Getty

Q. AI is transformative in various sectors. What role will semis play?

As mentioned, semiconductor chips are the basic building block of the hardware (HW) infrastructure that all AI models/applications are trains and run/deployed on, from the compute servers in the cloud/data-centers to the micro-power chips right at the extreme edge – sensors and such. So semiconductors not only play a pivotal, but an essential role, in advancing AI technologies.

Q. Can you elaborate on the specific areas that deep tech is making an impact?

Here are a few specific domains where significant innovations and advancement driven by semiconductor and AI:

• AI accelerators: this is the obvious one; AI, and therefore semiconductor chips running AI, everywhere is the trend

• Energy efficient semiconductors: power consumption by chips is an essential metric is paramount regardless of whether they are operating in the extreme edge all the way to data centre servers, using energy harvesting, batteries and grid power

• Advanced sensing technologies & Edge AI: In the realm of IoT, and autonomous systems especially for industrial applications, advanced sensing technologies such as LiDAR/radar/ imager/force/position, coupled with edge AI, very low power, AI chips capable of processing data right at the edge where the data is captured.

• Next gen memory:
• Start-ups are working in photonics chips, where light is used instead of electrons to transmit data and signal potentially offering faster and more efficient data processing. Many semiconductor start-ups are exploring integrated photonics for applications in data communications and processing.

Q. What role does angel investing play in fostering innovation in deep tech?

Building a semiconductor startup is a capital intensive venture, especially if you’re in the market where advanced processing nodes are required for computationally intensive markets, such as AI and HPC. I see Angel investors with a relevant industry background/experience, being able to provide some capital, while also sharing previous industry start-up experience.

In Canada, there are not enough Angel investors with deep semiconductor industry knowledge to help foster the deeptech start-up ecosystem. But, I believe that over the past 10-15 years, there are a growing number of successful Canadian semiconductor entrepreneurs who can be contributing to the angel or early stage start-up investment community/ecosystem.

Q. How do you assess the investment landscape for start-ups working on the cutting-edge?

While AI has been the hype and continues to be the darling for VCs to invest into, I’m not sure how sustainable the trend is. As mentioned, semiconductor start-ups are generally capital intensive and certainly the hype around AI has led to investors paying more attention to semiconductor start-ups with some form or shape of AI angle. In particular, the capital market has experienced a major downturn and re-calibration of startup valuations, which have been overly-inflated during the 2020-2021 boom.

This year will see a slight up-tick, with tech stocks performing well generally as well as the anticipation of the return of some IPO activities. However, no one is expecting a return to the ‘glory’ days of 2020-2021. The current state of minds of VCs are one of ‘back to the basics’ , like 5-6 years ago.

Dry-powder is still out there, but VC’s are putting more scrutiny in assessing the business cases, strategies and revenue potential of growth stage startups (Series B and beyond). However, AI and semiconductor related start-ups are still one of the hottest area where VCs are willing to invest in.

Q. As an angel investor, what criteria do you consider essential when evaluating potential partners in semiconductors or AI technologies?

First and foremost I look at the founding team. I don’t prefer a solo-led team. It’s very hard for one person to have all the skill-sets required to navigate the complex natures of a AI semiconductor start-up.

The questions I would ask is how well do the founders know each other. What is their track-record – do they have the relevant technical expertise and experience. The next thing would be the technical innovation. Does their idea solve a significant problem that is not already managing an existing technology? Is there an obvious market requirement for such as solution? Is there a real market need for it (or is it just a fancy science project)?

Obviously, there are many more questions that I would ask, such as business/revenue model, scalability, go-to-market strategy and financial projection, etc. The first three things are the most important ‘screen-test’ to for me.

Source: Getty

Q. The semi supply chain has faced challenges recently. How will that impact the deployment of AI, and what strategies can businesses adopt to mitigate?

Deglobalization of the supply chain and on-shoring of manufacturing are the most significant impacts I can see. On the one hand, it will significantly impact the cost structure coming from a sub-optimal use of resources which will likely result in increase of cost of semiconductor manufacturing, which in turn would increase the HW cost for deploying AI solutions.

We had been facing chip supply shortage over the past few years, but this is easing. The semiconductor industry is known to be cyclical, and inventory is already building up now. It is also expected that the global semiconductors fab capacity will be drooping back to the 70% level.

Q. With the global focus on sustainability, how are semis and AI aligning with environmental goals?

There has been a realization of the energy required to train and deploy AI models on compute HWs will create an energy consumption concern. It is predicted that up to 20% of the global energy consumption by end of the decade will be dedicated to datacentre compute HW, most AI being most of the workload.

Many AI researchers and chip designers are spending a lot of time in creating more energy efficient AI models and hardware. There will be a major driving force to deploy more energy efficient AI models, as well as semiconductor HW. Therefore, investors should be looking out for start-ups that are focused on creating energy efficient AI solutions.

Q. Can you share any success stories or breakthroughs you’ve encountered in the deep tech sector?

I started my semiconductor design career in the mid-90’s after I finished my PhD. Early in my career, as a chip designer, I had designed one of the first PRML Read Channel chip for Hard Disk Drive applications. Since then, while still doing a lot of hands-on design, I’ve also been building up my semiconductor start-up career as a serial entrepreneur

To date I either founded or was a member of a founding team of five semiconductor start-ups, with two exits. My latest start-up Untether AI (see cover feature), perhaps my favourite one yet, makes industry-leading energy efficient chips & hardware (HW) for AI inference.

Q. The field of quantum computing is gaining attention. How do you envision its integration with semis and its potential impact on AI applications?

The use of semiconductor-based qubits, instead of binary bits of a classical computer, in quantum computers, is a major R&D area. However, to-date most semiconductor based quantum computers are either superconducting qubits or quantum dots, both require cooling down the circuits to absolute 0 or a few Kelvin – which require sophisticated cryogenic cooling systems. This is one of

the main challenges in quantum computing – to be able to scale computing and AI applications to mainstream.

However, there are other ways to make qubits, such as through photonics. The photonic qubit based quantum computer has been gaining traction, as it operates at room temperature and photonic qubits inherently have lower error-rates. This is because photons do not react with each other as strongly as matter-based qubits, leading to easier quantum error correction.

Photonics are also intimately related to semiconductor technologies as in recent years, the integration of photonic systems with silicon technology has been prolific.

Whether it’s superconducting qubits or photonics qubits, quantum computing is expected to have a profound impact in future AI development, as it delivers a new compute infrastructure that enhances the speed and accuracy of AI systems.

Q. What advice would you give to semiconductor start-ups aiming to navigate the competitive market and attract investment?

For a semiconductor deep tech start-up, in fact as with any other start-up, one of the most important thing is to have a strong team of founders. Not only the technical backgrounds have to be relevant and solid, the founding team members, typically two to four, would be best to have complimentary skill sets and experiences. Then you want to be very clear on what problem you’re trying to solve and the solution you’re providing is unique and ideally no one else is offering a similar solution to the market (that’s the ‘story’).

Equally important, also, is your IP/patents. For any deep tech start-ups, it’s typically very important to patent protection because that’s your secret sauce and probably your market differentiator(s). Fundraising at early/pre-seed stage means that you’re selling the Team and Solution as a ‘story’ to the right investors who are familiar with the hardware/ semiconductor business – so they’re not scandalized by the ‘pre-revenue’ mode.

The main difficulty for semiconductor & hardware start-ups in raising early stage / pre-seed or seed rounds is the time-to-revenue is usually a very long road , possibly a few years – this scares away many software/SaaS centric investors. Therefore often times a semiconductor/hardware startups would raise it’s early stage funding only on a story or the founding teams track-records/ credentials and through corporate/ strategic VCs who are already familiar with your business.

That being said, with the current ‘recalibration’ of the capital market from the 2020-2022 ZIRP booms-day era, VCs are much more cautious about capital deployment. For early stage / pre-seed/ pre-revenue start-up founders, they must be laser-sharp focused on getting customer traction and interest as much as possible early on.

One vital point that I cannot stress enough is ‘get your first-silicon out the door’ as quickly as your can – forget about making it perfect. The key is to place a product on your potential customers’ hands; get their feedback and apply the learnings back to your next generation product.

Q. Looking ahead, what emerging trends or advancements do you foresee?

What I see as emerging trends in the forthcoming years withing the AI and semiconductor sectors are as follows, some if not all of those I had already touched on previously.

Compute-Chips & HW optimized for AI workload:

This is what I see as ‘what’s beyond GPUs’.

Advanced packaging:

Keeping up with Moore’s law through advancement in semiconductor process technology has become an uber expensive endeavor, advanced 2.5D and 3D packaging offer potentially a much ‘wider’ avenue for improvement chip performance and IO bandwidth especially many of AI compute applications (such as LLM and G en AI) are very much memory BW bound.

• Edge AI:
• As we see more and more AI chips and HW made to be highly energy efficient, integrating AI/ inference acceleration HW on edge devices will be prolific,reducing the heavy loading to cloud-based services.

These advancements will likely lead to a future where technological landscape where real-time smart processing becomes more integrated into everyday devices and services.

—————————–

Raymond Chik, PhD, is a consultant with DRC Enterprises, providers of a wide range of business solutions.

https://drc-ent.com