Hardware-Software Co-Design In The AI Era
Key Takeaways:
- Several hurdles must be overcome for a combined hardware/software development flow.
- Structurally, teams remain independent, use different tools and languages, and work at a different cadence.
- Technically, significant problems remain in enabling software to be executed virtually on hardware at different points in the development flow.
AI is already proven to help in software development, and it is beginning to make inroads into hardware development. But to enable a fully integrated hardware/software development flow, some big hurdles must be overcome, not all of which are within AI’s power to resolve.
The benefits of closely integrated hardware and software development have been known for a long time. Those efforts have been hampered by both structural and technical issues. On the structural side, development teams are not integrated. Engineers use completely different sets of tools, and development schedules are also not aligned.
On the technical side, the lack of performant hardware models minimizes the potential scope for continuous integration. While some virtual prototypes do exist, they are not early enough, and trading off abstraction versus accuracy is difficult. Emulation has attempted to provide a cycle-accurate hardware model, but it can struggle to run full workloads that are representative of the final system.
The first challenge is constructing specifications that are sufficiently rich and complete so they can be used by both humans and AI agents. Outside of Mil/Aero, few industries do this today.
With the promise of AI helping in both hardware and software domains, one of the big questions is how will AI learn about co-design when so few examples are available. AI may be able to run synthetic experiments to determine how certain software functions perform in terms of timing and power when they are run on an array of hardware architectures. Only then can AI start to make informed codesign decisions.
“The trade-off has always been between how flexible or general-purpose a workflow is, and how performant it is,” says Arvind Srinivasan, design verification solutions engineer at Normal Computing. “The most performant workflow will be entirely bespoke to the task at hand.”
But there are constraints to consider when making these decisions, including cost, form factor, and power budget. There are also thermal considerations, environmental factors, and a long list of other system-level constraints. So far, nobody has encoded them in a way that any AI algorithm could use as a goal, which means AI is not ready to take on the task.
Team structure
Two terms are frequently used when talking about hardware and software integration — vertical integration and shift left. Both involve bringing the teams and development flows closer together.
“If you look at decoupled hardware-software teams, there might be a couple of people who can cross the line,” says Sam Grove, head of software and tools at MIPS/GlobalFoundries. “The reality is the chip gets out, then the software team picks it up. They have to deal with everything that’s changed, every piece of errata. They’re saturated, coping with how to make this work so they can get the product out the door. In a perfect world, they would be participating in the design of the next-generation chip. This is a structural change that will have to happen in these companies.”
Many companies do employ continuous integration. “The overarching challenge is latency,” says Frank Schirrmeister, executive director of strategic programs for system solutions at Synopsys. “By latency I mean the time it takes to come to a conclusion. When considering hardware-software integration and starting with a spec, you can iterate more often, you can make quick decisions, and then everything afterwards goes faster. If you wait until you have silicon, the latency is the project timeline. If you use virtual prototypes, it then comes down to how fast you can iterate. The good news is the rapid development toward specification engineering, where you have a spec, you build your RTL, you build your model, and potentially build your software from that spec. The iteration could be lightning fast. Now it’s becoming a question: ‘Can you simulate it fast enough?'”
Some companies are looking at novel ways to implement this. “AWS explained its process, where it ports RISC-V IPs to FPGAs and makes them available in the cloud,” says Andrea Gallo, CEO at RISC-V International. “We are familiar with the concept of continuous integration on the software side, but imagine the same on the hardware side. You have your SystemVerilog design in a continuous integration flow. As soon as you modify a line of Verilog it will generate FPGA images and upload them into the cloud. You can have continuous integration. You modify the hardware, you modify the software, and a few minutes later you have it ready for testing on real hardware in the cloud.”
Developing and testing software before the hardware exists is not new. “This is enabled by virtual prototyping, where you can get a virtual model of a piece of hardware and start developing the software against that,” says Yaron Ilani, design verification solutions engineer at Normal. “Maybe AI will be helpful in abstracting hardware components so that you can more efficiently develop the software ahead of the RTL. The fact that we have an ontology, which is a representation of the spec, means we don’t necessarily need to have the RTL. We can develop software against the ontology, which is a source of truth derived from the spec.”
It is not an easy shift to make. “The people most likely to become leaders are the ones who can embrace this model the fastest,” says MIPS’s Grove. “They understand the workload, they can build the chip, they can customize it, and they can get it to market. Previously, we would build a chip and then start to enable the ecosystem. The problem here is trying to keep up with all the updates. How do I get different optimization passes in the compiler, or why is the code behaving this way? Instead, if you know how the code is going to behave, can you build hardware to better support that, or better execute that? That is a significant paradigm shift that will impact the entire semiconductor industry.”
This leads to the identification of another hurdle. “We are watching how our customers are utilizing AI,” says Andy Meier, principal product marketing manager for Siemens EDA. “They are primarily utilizing it for RTL creation and test bench creation. I’m not seeing it in architecture. The human has so much knowledge used in making those trade-offs. They are answering questions about how fast it could be, given a particular processor type, and how to hit other performance metrics. Human knowledge remains extremely valuable in this process of trade-offs.”
Gaps remain
Most advances come from incremental improvement to the existing methodologies, rather than wholesale replacement. A major contributor to this is risk minimization.
“A lot of that is unsolved within the EDA space,” says Normal’s Srinivasan. “People aren’t jointly optimizing for global outcomes. They look at a very small piece of the puzzle and ask, ‘Given the existing waterfall approach, how do I improve my productivity on this step?’ The challenge with black boxes is that you still need guarantees of correctness and auditability. To do that, you need systems of record that understand how collateral links together, and your verification loops need to be well factored against that system of record. That’s a hard unsolved problem, but a high-value one.”
Starting the verification process early in the hardware flow has a lot of value. “Today, we are seeing software defining the solution, and that refines the system-level architectural requirements,” says Siemens’ Meier. “We are seeing early adoption of virtual prototyping at a much more robust level. They are not using them for synthetic workloads, or a small subset of the system. They want to run the full stack, or true system-level workloads. That’s forcing continuity between different models that haven’t been there in the past. They are asking for the ability to execute the same workload from a virtual platform and prototyping perspective, through emulation, through FPGA prototyping, and into silicon. This presents a challenge beyond performance and accuracy trade-offs. How much real wall-clock time can they execute? If they need 10 seconds of silicon time, is that realistic in simulation or emulation? Probably not. But what percentage of that is possible?”
That does not mean that everything has to fit into an emulator. “There are several axes here,” says Synopsys’ Schirrmeister. “There’s the complexity of the workload, and then there’s a complexity of what elements you actually put together to simulate, to understand the impact on the workload. That’s why capacity in emulation is such a big topic. It requires things like modular verification, where you take individual pieces, replicate them, and connect them, each of them being their own subsystem in an emulator or across multiple emulators. Now you can run the workload on the full chip.”
That’s part of the solution, but not all. “Something is missing,” warns Grove. “It’s simulation, it’s micro-architectural models, it’s power analysis models — the pieces of a simulated world where the teams actually collaborate. While the simulation might be the artifact that you start with, if that artifact gets into the software team’s hands, we’re no longer talking about handing off RTL or a netlist. Those are just pieces that pop out of a design flow. If the virtualized or the simulated version platform is what we’re working on, which is providing the insights, you have this continuum from design to application support to end customer application support to work with.”
This needs to be at different places within the hardware development flow. “It may start with a virtual prototype that allows you to see the impact of software on the hardware architecture,” says Schirrmeister. “But if you want to look at coherency issues, it is just not possible. Later, emulation gives you performance and power analysis. Then they can do it on FPGA-based prototyping that gives you real-time interfaces and so forth. People need everything, each with a different latency. Each solution provides a particular sweet spot, and that defines which types of problems you might tackle in emulation or prototyping, which has a different cost metric than virtual prototyping in terms of execution.”
You don’t always need every detail of a system. “There’s a lot of heuristics,” says Grove. “Perhaps the most basic is if the system is functional end to end. But do I understand the power profile for these sequences? Do I understand what is happening in the pipeline for these sequences? It requires a collection of simulation technologies. They don’t all happen together. We struggle to compute these solutions in a meaningful time with the demand of the world, let alone simulating them on these platforms. There’s a lot of innovation in this space, where it’s the full system and then the handoff of little bits that collectively builds the required data set, to be able to make informed decisions and guide the hardware design, the hardware integration side of things.”
Teams must acknowledge the tradeoffs and benefits of each model and how they should be used. “You have to recognize that there is a difference between a virtual platform with a high-level untimed SystemC model and RTL,” says Meier. “There’s a timing accuracy component that is going to be different. How do you keep them in sync? That’s where people spend a lot of time. Industry data shows that people spend time in verification and validation, and in software bring-up. It’s an area ripe for improvement.”
All of this is just step one in bringing about integrated flows. “How do you bring the tools used by the hardware team into the software world?” asks Grove. “How do you get both sides talking the same language? Today, you have a couple of architects and a couple of microarchitectural designers who understand that interface. But if you can bring the army of software developers into the equation, where they don’t have to pull their code into a different set of tools, or a different flow to understand it better, that enables collaboration. That’s what we call software-driven hardware design.”
“I’m optimistic that some of the data analytics side of where EDA is investing will be able to help system-level architecture and performance analysis in a much more meaningful way, through the use of data analytics and AI,” says Meier. “I don’t think we’re there yet, but I know there’s a lot of investment that’s happening.”
Help from AI
It’s not clear where AI will provide the greatest benefit. Today, it is being used by both hardware and software development teams. “We and other companies are using AI, not just to code software, but to also code hardware,” says Sean Murphy, senior director of product at MIPS/GlobalFoundries. “We are making sure that we’re using agents for such purposes in very targeted and responsible ways, such that we’re taking advantage of this great tool. But understanding its limitations and understanding its faults is important. You cannot just take it as this perfect model that’s going to solve all the world’s problems.”
Software iterations make it difficult to treat co-design as anything more than a single point in time. “A big impact of AI for software is the ability of AI models — LLMs — to write conventional software,” says Steve Roddy, chief marketing officer at Quadric. “While we may know the algorithm set used in the hardware-software co-design virtual models at the outset of a project, two years later when silicon is ready, the software itself has been rewritten and improved/expanded countless times by both programmers and AI assistants.”
We may need to dig deeper into the methodology to find the biggest gains. “AI has shown some nascent ability to manage cross-abstraction-layer dependencies more effectively than individual developers,” says Srinivasan. “The more tightly you can couple and co-design what you’re doing, from the product surface and software down to the firmware and hardware, the more advantage you’re going to get. And that’s a gating factor to an even more ambitious north star – software and hardware that recursively improves.”
Decisions that were made years ago may now impede progress. “Take verification, for example,” says Normal’s Ilani. “It has standardized on a methodology called the UVM. It was intended for humans to enjoy reusable components using a standardized code structure. However, it may not be as important now. What is becoming more important is to generate an environment that is easy to maintain with agentic flows. Maybe we don’t need the extra complexity of methodologies whose sole purpose is to be human-readable or human-maintainable.”
Conclusion
The industry has long wanted to bring hardware and software development closer together so they can both participate in mutual optimization. While the goal is usually accepted as beneficial, it never really happened. The development cadences are different, the impact of errors is not equal, and the tools and languages are not consistent. For most of the life of the semiconductor industry, hardware development has been seen as a high-value element on which software is ported. Today, we live in a world where software defines almost everything, and the role of hardware is to execute it as fast as possible. None of this requires particularly close collaboration except between a few experts from each team. That is where we have always been, and it doesn’t really appear as if the future will be different.
The post Hardware-Software Co-Design In The AI Era appeared first on Semiconductor Engineering.