Power Corner: Vicor’s Maury Wood on Vertical Power Delivery for AI Data Centers
Description
Maury Wood, VP of Strategic Marketing at Vicor, joins Power Corner to explain why powering the processor is the central efficiency challenge in today’s AI data centers. He explains how compute accelerator boards and compute trays account for ~50% of total rack power, making processor-level power delivery the biggest lever for cutting AI data center energy consumption. He walks through the brutal transient demands of training workloads that swing between idle and full load, and how the several layers of hierarchy for energy storage are necessary, particularly decoupling capacitors as close to the processor as possible.
He covers how going from 48 V straight to the core minimizes conduction losses to the PoL while also mitigating any safety concerns. The conversation goes deep into Vicor’s factorized power architecture (FPA): how Vicor’s vertical power delivery (VPD) modules regulate voltage first and then transform it at a fixed ratio (the K-factor)—the reverse of many competing IVR approaches. This essentially uses a fixed K-factor current multiplier to bring 48 V directly to the point of load (PoL).
In other words, a fixed-ratio module takes in 48 V at some current and outputs ~1 V (48 ÷ 48) at 48x the current, because stepping voltage down by a factor necessarily steps current up by roughly that same factor, i.e., current multiplication, which is the flip side of voltage transformation. He also covers how Vicor’s current multiplier, being a fixed-ratio device, not a PWM regulator, can handle 48:1 or higher with no efficiency penalty because it’s not trying to regulate, just transform.
Wood argues that converter switching frequencies are already near-optimized (at 90+ percent), so attacking conduction (I2R) losses was the true answer to the problem. He explains his thought process, stating that optimizing multi-phase converters for a small step-down ratio can be seen as “penny-wise and pound-foolish,” driving up converter efficiency while increasing losses along the conduction path due to the high current density at 2 V or 6 V.
Wood also presents some of the challenges he sees within in-package IVRs, facing reliability and customization tradeoffs: “If there’s any failure…you’d have to scrap the whole package assembly with the processor…there’s no opportunity for rework.”
Looking ahead, Wood previews a major industry shift toward glass substrates and panel-level packaging (PLP) with chiplet-based processors—panels as large as 600 mm per side—expected around 2029. He argues this transition will make VPD effectively unavoidable, while introducing new thermal and current-density demands across the entire power chain.
Transcript
AS: How are you doing, Maury? Thanks for joining me.
MW: Thanks so much. Thanks for inviting us.
AS: No problem, no problem. We could just go ahead and jump right in.
We chatted a while back, and you mentioned that processors, the compute accelerator boards, and compute trays represent about 50% of total rack power, meaning efficiency gains made there attack the bulk of the AI data center energy consumption problem. Can you explain why you think efficiently delivering power to the processor is the most important power problem in data centers?
MW: Yeah, again, Aalyia, as I mentioned earlier, the losses associated with powering the processors are really the bulk of the challenge with respect to getting the entire data center’s power utilization as high as possible. So it’s absolutely a key element of addressing what is becoming more and more of a societal-level concern about the enormous amounts of energy these AI data centers are using.
AS: Gotcha. Maybe we could talk about the impact of marginal efficiency improvements in data centers—what does that mean on a larger scale, especially as data center power consumption skyrockets?
MW: Yeah, the focus for power engineers really needs to be looking at the complete chain. As you might know, there’s a big initiative to move the feeds to the racks from 48 V to 800 V DC, and that’s, from our perspective, running in the right direction—in terms of reducing dramatically those I2R losses in the cables.
There’s a lot of current passing through those, you can imagine, with these racks eventually on a path to hundreds of megawatts, and eventually perhaps even higher. So starting from that 800-V feed to the racks, it’s important to minimize the number of DC-DC conversion stages, and then make sure the losses in efficiency associated with each one of those stages is optimized to really attack this problem.
AS: You’ve described AI workload transients as brutal, with near-instantaneous swings from idle to full load. Can you expand on this, and how does it impact the characteristics of power delivery systems for training and inferencing processors?
MW: Sure. The workloads in both training and inference processors—particularly training processors—can effectively move from a no-load state to a full-load state, in the vernacular of power engineers. So the power delivery system, the last-stage data converters, need to have very high transient bandwidth.
There needs to be a hierarchy of energy storage, particularly decoupling capacitors, as close to the processor as possible, to supply energy across those transient events and make sure the voltage doesn’t droop excessively.
The rest of the power delivery system needs sufficient ability and bandwidth so that once that first level of energy storage is depleted, the regulators can come through rapidly to support the transient demand. By doing so with adequate bandwidth, you can avoid the processor doing any kind of throttling—any dynamic frequency or voltage scaling to wait for the power delivery system to catch up to these transients.
It needs to be very rapid, so the performance of these processors can be maximized.
AS: I know Vicor doesn’t focus only on the vertical power delivery solution, the VPD solution—there’s also a thought process behind delivering power to the VPD modules, which is the factorized power architecture. Can you describe the FPA that takes 48 V directly to the load, and what’s the penalty for stepping down to 6 V or 2 V prior to the final-stage buck regulators?
MW: So, a brief review of factorized power architecture—it’s something Vicor has innovated around for many years. The difference, essentially, is that FPA disaggregates the regulation stage from what we call the transformation stage. The VPD modules receive a stable 48 V from the regulator.
So our factorized power architecture does regulation and then transformation. Competitive solutions do the opposite—they do transformation, or a fixed-ratio step-down, to either 6 V or 2 V, and then regulation.
FPA turns that around: the vertical power delivery modules receive the 48 V and transform it using what we call a K factor. So if there’s 48 V coming in, and the K factor is 48, the voltage is divided by 48, and the current is multiplied by 48—you can think of that as current gain. That brings 48 V right to the VPD modules, effectively right to the point of load.
And by doing this at that higher voltage, you’re largely independent of the resistance across the printed circuit board, because the regulator is going to be somewhere not adjacent to the processor; there’s going to be some distance there, and therefore some resistance through those conduction paths.
So 48 V is still a safe voltage—no safety concerns. It’s been used in automotive and other platforms for a number of years. And again, 48 V minimizes the loss across the conduction path.
By contrast, if you move to 6 V, as some solutions have done, you’re talking about a much higher level of current across that last stage, which increases thermal losses tremendously. And 2 V at the point of load is even worse—2 V is only effectively 2x the final output, or maybe 4x if you’re delivering 0.5 V. So there’s only a very small amount of step-down, in our case, it would be transformation; in other solutions, it would be regulation.
So we see 48 V running in the right direction to minimize power, just like the transition from 48 V to 800 V at the cable level going into the racks is doing a similar thing.
That’s our perspective on this—we have a unique solution that we think really addresses the most pressing concern with respect to powering these processors.
AS: I’ve heard that adding a 2-V intermediate power bus segment, instead of going directly from 48 V to the VPD module, causes IVR solutions to struggle to scale up beyond a few hundred amps of thermal design current. Do you feel this is the case? That’s basically what you just explained.
MW: Absolutely. Some of the consequences of using that lower voltage with higher thermal losses is you’re creating additional thermal stress on the package—if these IVRs are mounted in the package, you’re multiplying the problem with respect to the heat that needs to be drawn out.
There are a number of other issues with embedding regulators within the package: the substrate of that package assembly needs to be customized, and if there’s any failure of these lower-output-current IVRs, you’d have to scrap the whole package assembly with the processor—particularly if the failure occurs over time, there’s no opportunity for rework or repair. There are a number of other issues we can discuss if you’d like.
AS: Maybe we could talk about the reasoning behind going down to 2 V. I’ve been told that going from 2 V to the core offers better efficiency than using higher input voltages—6 V, 12 V, or 48 V—to the core, and that this is the thought process behind many IVR solutions. There’s a balancing act between input voltage and efficiency, and you don’t want to sacrifice efficiency gains in that last mile of delivery, since—as we discussed earlier—that’s the most important power problem in data centers. What are your thoughts there?
MW: For multi-phase voltage regulators, you do need to avoid a large step-down, because of how those regulators are controlled—through a pulse-width modulation technique. You start to have a very low repetition rate of those pulses as you have a higher step-down ratio. So I think what you’re referring to is that with a very small step-down ratio—2:1, or 4:1 from 2 V down to 0.5 V—for that multi-phase topology, you’ll get better efficiency.
In our solution, that’s not an issue—the current multiplier can deal with factors of 48 or higher with no loss of efficiency. I’d like to make another point too: the thermal dissipation, the heat generated by those I2R losses, really dominates here. Power converters have relatively high efficiencies—90+ percent average efficiency, continuously—so there are relatively small gains to be made with respect to the efficiency of the converters themselves. The real win is attacking the conduction losses, the I2R losses, which generate the bulk of the dissipation.
So yes, I understand that with multi-phase you’d want that small conversion ratio, but there are alternatives, as we’ve talked about—you’re penny-wise and pound-foolish. You’re driving up the efficiency of the converter, but you’re then inducing all these other losses across the conduction path.
AS: Thank you. You mentioned earlier that most vendors use the multi-phase TLVR topology, but Vicor’s approach is different—it uses fixed-ratio current multiplication. Maybe you could talk about the merits of that current multiplication in the power chain, even though you’ve touched on it already.
MW: Sure. One thing I mentioned is that with this regulation-transformation architecture used by the factorized power architecture, the downstream current multiplier, which is the VPD module, inherits the regulation of the first-stage voltage regulator. So it’s coming in at a very tightly regulated 48 V. Then you’re doing the voltage transformation, which you can think of as current multiplication, and that’s fixed-ratio—so it can be whatever’s needed by the power-domain rail it’s servicing.
So you can, sort of dial in and, in fact, use current multipliers, as well as the voltage regulators, as a standard product family with a variety of output currents and K factors, so there’s the opportunity to use these in a VPD configuration without any customization of the substrate, without concerns about rework—because they’re on the back of the board—and without concerns about displacing the decoupling capacitor farm on the backside of the board, because the current density of the current multipliers is extremely high.
So you’re in a much better position from pretty much every perspective—lower heat, reworkability, repairability, very high efficiency, and no need to worry about step-down ratios to maximize multi-phase efficiency. There are lots of merits, and we’re very committed to this—we see it as the best path, again running in the right direction in terms of reducing the losses that dominate AI data center power concerns.
AS: Thank you, Maury. The capacitance multiplication via the current multiplier’s DC turns ratio, or K factor—Vicor has a lot of very specific terms, but I wanted to explain that in a bit more detail.
Can you explain how the transformer turns ratio ends up multiplying the effective power-chain capacitance? This is very relevant because in AI accelerators, power delivery is dependent on those capacitor farms—that hierarchy of energy storage we discussed earlier.
MW: So the capacitance on the input side of the current multiplier is multiplied up by the square of the K factor, and there’s a tremendous impact on that capacitance. And that appears on the output side of the current multiplier.
It’s a unique trait of current multiplication—it really helps with energy storage close to the processor. You’re talking about the output side of the VPD module, getting within a millimeter or so of the actual processor to the substrate, and up through the tens of thousands of microvias in these substrates—about half of which are power and ground—feeding up to the power and ground connections of the processor itself.
So you’re really getting this terrific attribute of capacitance multiplication, adding to the reserves of energy available during these frequent workload transients. It’s tremendously beneficial for dealing with them.
AS: Thank you, Maury. I have two more questions, and then you’re off the hook.
Maybe we could talk about that small ecosystem of IVR solutions emerging—these regulators that could be attached to the bottom side of the accelerator PCB in a vertical power delivery configuration, but could also be placed on the bottom surface of the package assembly substrate, or embedded within it. As you mentioned earlier, that comes with its own challenges—what are your thoughts on this approach, and is Vicor watching it?
MW: Yeah, we’re watching it. I think conceptually it makes sense. I won’t go back over what I already covered, but as far as we know, the output current levels of IVRs are quite low, so I think the fit might be for auxiliary rails that don’t demand as much as the core rail for these AI processors—there’s potentially a fit there.
It’s becoming evident that these solution providers are positioning it as something that can be done in traditional VPD on the back of the printed circuit board, on what people refer to as the land side of the PCB, or attached to the bottom of the substrate. One issue there is the height of those modules, in order to be compatible with the package assembly.
And then, with full embedding into the substrate—one point I haven’t made is that taking that approach means every single substrate for every processor variant would have to be customized. Furthermore, you’re also taking a passive substrate, which might have embedded capacitive elements, and introducing active power electronics—the regulators—into the substrate.
That’s a huge change in how people have thought about substrates, and it would require a remarkable level of cooperation, not only with the end customer and the processor developer, but also with the substrate maker, to successfully customize every substrate that would use this technology.
So, in concept, I think that it’s got some merits; in practical application, there are some shortcomings that need to be addressed.
AS: I remember you mentioning earlier that the vertical power delivery market itself is still quite new—it’s mostly been lateral up until now. I was curious about your thoughts—more forward-looking—what’s next for the roadmap, and anything I might have missed?
MW: I’d love to give a bit of foreshadowing on something I’ve been looking into, and others at Vicor have as well—the roadmap for these processors. The major suppliers of the substrates used in these package assemblies are looking at much larger panels, transitioning to glass substrates, and looking at panels that are 200 mm x 200 mm, 400 mm2, even 600—extraordinarily huge.
The thought process is that at the chiplet level, rather than the package assembly level, you can construct processors with enormous compute performance. And you also experience a tremendous increase in compute density, because the area required by chiplets is much smaller than the area required by a package assembly.
So in the future, you’ll see much larger panels with a lot more compute resources, a lot more I/O, and co-packaged optics. But my key point is that in the era—which will be around 2029 or so—of these panel-sized processors, you have to use VPD. There’s absolutely no alternative to it. And furthermore, you need solutions with very high current density, because of the concentration of energy in these panels—the idea of having any kind of embedded voltage regulator contributing to the thermal footprint of that panel is going to be unacceptable.
There’ll be much more stress on liquid cooling, and when that time arrives—there’s a tremendous amount of R&D dollars going into panelized processors—you’re going to have to totally optimize the thermals and the efficiency of the entire path. Along with the gain in compute density, you’re going to have some additional new thermal challenges, and we think 48-V factorized power architecture is the way to attack that.
AS: Thank you so much, Maury. I really appreciate your time—this has been a very enlightening discussion. I learned a lot myself, so I hope I got the right questions in. I do appreciate it.
MW: Aalyia, this whole topic at the end—this panelized processor thing—I think is extremely interesting. Number one, we’re going to have some good things to show you in that domain, but I think that foreshadowing, and the reality of it coming down the road, forces a lot of the issues we talked about much more to the forefront, and clarifies things that people are currently just sort of dabbling with, like VPD and other approaches—because you can’t mess around.
I mean, I did one study, and it was something like 30,000 A.
AS: I had no idea. So these panelized processors—200×200 mm2, is that what you said? Forgive me if I’m wrong, but I know TSMC is working on bigger and bigger CoWoS substrate packages. Is this an extension of that, or is it something else?
MW: It is. CoWoS-S and CoWoS-L have limits in terms of what’s possible—there’s a huge capacity problem around making those substrates. Intel’s got EMIB, for example, and they’re getting more business because people aren’t getting the CoWoS substrates or package assemblies they need.
And you’re moving from organic substrates to inorganic, with the glass—glass has great thermal properties. There’s no question it’s going to transition to glass, and once you’re in a glass substrate, the sky’s the limit. I mean—600 mm, that’s six-tenths of a meter on a side. It’s hard to imagine them being that size.
AS: One more question—the glass substrate, as opposed to what’s currently being used—just because I don’t really know.
MW: Yeah, as far as I know, it’s an epoxy material, much like what we’d use when we overmold our modules—materials that have been used for normal semiconductor packaging for a long time. The black epoxy you see is an organic compound.
AS: Gotcha.
MW: The fascinating bit—you’re mounting chiplets, which are silicon for the most part, maybe some gallium nitride, or whatever the co-packaged optics might be made of. But the vast majority of memory and processors are all silicon. And glass is also silicon-based, so all of a sudden you’re back to almost the entire thing being silicon-based—that’s why the thermal coefficient of expansion works out much better than with an epoxy approach.
AS: Okay, I have to research this in more detail—I’m very interested in learning more. You feel vertical power delivery solutions are non-negotiable at that point, but it also potentially makes integrating voltage regulators less feasible?
MW: Well, it depends on what the IVR suppliers and developers come up with. I think they’re staring down a really big challenge on this.
AS: Yeah.
MW: I don’t know—it’s sort of like, go talk to them, because I don’t know how interested they are.
AS: Okay, gotcha. Well, thank you for that, and for your time.
The post Power Corner: Vicor’s Maury Wood on Vertical Power Delivery for AI Data Centers appeared first on Power Electronics News.