AMD’s next generation RDNA 3 high-end desktop GPUs will have its launch event imminently. As a build-up to this, Angstronomics can reveal the first image of the Navi 31 silicon at the heart of these graphics cards. This brief article will not delve into the details, but you can read our previous article to find out more.

We can see the layout of the single GCD and 6 MCD’s flanking it, with the dark overmold covering the organic interposer that connects the chiplets together. This is all attached to a 55 x 47.5 mm ABF substrate and packaged with a wide stiffener ring and various decoupling capacitors.

Performance, power and price will be revealed at the launch event. Unfortunately, our A0 performance profiling results were not indicative of production samples, so we have refrained from detailing our findings publicly. Silicon and board cost is definitely lower than NVIDIA’s RTX 4090, so let’s see where the two Navi 31 SKUs will be priced.

Share

At Intel Innovation 2022, Intel presented a wafer labelled “Raptor Lake-S, 34 core“, which Angstronomics immediately identified to be identical to the monolithic 34-core Sapphire Rapids MCC (Mid Core Count) design. Now that images have surfaced publicly, it’s time for us to share what we know about the largest die Intel has made to date.

Intel’s Largest Single Chip Ever

Using a handy image captured by David Altavilla, we can do some image processing to derive the die size of this massive chip. From our edited image, we find a die size around the 770 mm² mark. This makes it Intel’s largest die ever!

2010 Tukwila: 699 mm²
2016 Knights Landing: 683 mm²
2017 Skylake-XCC: 678 mm²
2023 Sapphire Rapids-MCC: 770 mm²

Core Count Regression

Sapphire Rapids introduces a much larger, faster CPU core combined with more IO and acceleration features such as AMX, which all consumes area. This results in a regression in core count for a given die size vs the previous generation. A 10nm wafer can fit 84 dies of 40-core Ice Lake, but only 68 dies of 34-core Sapphire Rapids. So, despite having fewer cores, the new generation takes 24% more wafers to fit the same number of chips. This gap widens once factoring a lower yield for larger chips.

Maximum Possible Die Size

Not only is it Intel’s largest die, it also is the absolute maximum die size possible for monolithic Sapphire Rapids. The die has a mesh layout of 7 columns and 9 rows including IO, flanked by the DDR5 interface. The die is already larger than 30 x 25mm. The lithography machine’s Reticle Limit has a maximum field exposure area of 33 x 26mm. This means it is literally impossible for Intel to add another row or column without exceeding the Reticle Limit.

As such, Intel has absolutely maximized core count for a monolithic design, using an odd number of mesh columns which they have not done before, and trading IO on the bottom row for 3 additional cores. Having 34 cores allows Intel some margin for binning this massive chip into 32-core parts to put on the market.

Chip Features

We have done an annotation showing the layout and key features of the 34-core chip.

34 Server Cores + Cache
(2x 512b + AMX, 2MB L2, 1.875MB L3 per core)
68MB L2 Total
63.75MB L3 Total
2 Accelerator Units
8 channel DDR5 (640-bit wide with ECC)
3 UPI 2.0 Links
80 Lanes PCIe Gen 5
8 Lanes DMI to Chipset

What About Chiplet Sapphire Rapids?

Intel’s chiplet strategy for Sapphire Rapids is not to disaggregate functions, but to break the reticle limit and give more silicon per chip. More silicon = More cores = More Performance at each power level. 60-core Sapphire Rapids stitches four 15-core dies with EMIB. From the wafer, we can see each die is about 390 mm², and there are 148 chips per wafer. This makes just 37 full processors per wafer. The full chip has:

60 Server Cores + Cache
(2x 512b + AMX, 2MB L2, 1.875MB L3 per core)
120MB L2 Total
112.5MB L3 Total
4 Accelerator Units
8 channel DDR5 (640-bit wide with ECC)
4 UPI 2.0 Links
112 Lanes PCIe Gen 5
8 Lanes DMI to Chipset

Cores Per Wafer

Doing some multiplication, this wafer has 2220 cores printed, while the monolithic design has 2312 cores on the wafer. The chiplet design has lower core density as the chip connection interface takes area as well. We measure that the EMIB connections contribute a whopping 20% of total die area. Meanwhile, current 40-core Ice Lake has 3360 cores on a wafer, 50% more.

Balancing Act

Intel has to balance area (cost) with core size, core count and amount of IO. The final piece in this balancing act is the performance, which we will see when the parts arrive in 2023.

Share

In the last month, Sony has begun the rollout of their latest revision of the PlayStation 5, model CFI-1202. There has already been a teardown showing the model’s redesigned board and cooling that is smaller and lighter. Power consumption is also reduced while giving similar performance. To accomplish this, AMD’s Semi-Custom design teams have ported the PS5 silicon to TSMC N6, codename Oberon Plus. Angstronomics reveals exclusive images of the new, smaller die, and we discuss the how and the why behind this update.

How Die Shrinks Work

These console refreshes are the modern-day equivalent of what used to happen decades ago in the microprocessor industry, where existing designs would be ported to a new, more efficient process node with minimal rework. For this example, TSMC ensures that their 6nm EUV process is Design Rule Compatible with their N7 DUV process, meaning that customers can reuse existing designs made for N7 to use on N6, with a simple conversion process that does not alter the underlying high-level synthesis.

Along with an 18.8% increase in logic transistor density thanks to CPODE (which you can learn about in our article here), N6 also lowers power consumption for a given performance level vs N7. This is what enables the reduced wall power of the latest PS5 revision, with less requirement for heat rejection and hence a smaller, lighter cooler. As TSMC completes its transition to increase EUV adoption and move most products to N6 from N7, we get updates like Oberon Plus. While most fabless companies would just design a new product on N6, game consoles demand absolute low-level platform hardware and software compatibility. This is why we see the exact same design being ported with zero configuration changes, including keeping with the reduced Floating Point Unit in the Zen 2 CPU cores (which we will also see in Ryzen 7020 Series ‘Mendocino’).

Oberon Plus

With an exclusive side-by-side comparison showing 6nm Oberon Plus next to 7nm Oberon, we can see what has physically changed in the 2 years between them. Die size has gone from ~300 mm² to below 260 mm², a shrink of close to 15%. What this means in the end is that each wafer processed can produce near 20% more chips for a similar cost.

It’s All About the Money Margins

The game console business works on selling hardware cheap and generating revenue in game sales. As such, it is paramount to minimize production costs to generate meaningful profit margins. With the aforementioned smaller die and lighter cooler enabled by lower power consumption, we estimate that each new PS5 going forward costs Sony around 12% less to make overall, which helps to recoup program development costs sooner than if no die shrink took place.

As the PS5 is the most produced out of the 3 current generation consoles (PS5/XSX/XSS), it sees the biggest benefit moving to N6 first. Going forward, only Oberon Plus will be produced, and 7nm Oberon production stopped. And while one can make near 50% more PS5 chips per wafer than Xbox Series X chips, the latter will also get its 6nm update of Arden in the future.

Share

Yoji Shimizu of TechanaLye recently uploaded a teaser video featuring the latest Apple A16 Die. While the resolution is low, there are enough discernible features for Angstronomics to capture frames, compile a Die Shot, and do a simple annotation of the key features on the chip.

Unlike their M-Series Apple Silicon, Apple does not publish die renderings of the A-Series chips for iPhone. As such, we will be making some assumptions based on relative structure sizes to see what we can learn from the first publicly available die shot of the A16.

A16 vs A15

To start, we know that the E-Core Cluster L2 Cache size remains the same at 4MB between both A16 and A15. Therefore, we can get a rough scale of the relative areas if we assume the cache array area is similar between regions as it historically has been.

From this, we can infer that the P-Core Cluster’s L2 Cache has grown from 12MB on the A15 to 16MB on the A16, as the cache array area is about 4x that of the 4MB L2 array. Increasing cache size is an architecturally simple way of increasing energy efficiency by keeping more information closer to the CPU at the expense of more area. This is a 33% jump in cache vs A15, which had a 50% jump vs A14. Clearly there will be diminishing returns as cache scales.

P-Core Cluster L2 Cache
A14: 8MB
A15: 12MB
A16: 16MB

Conversely, it seems the System Level Cache (SLC) has regressed in size, from 32MB on the A15 down to 24MB on the A16 (~6x area of 4MB L2 array). Remember that A15 doubled SLC size from A14, so going back down seems surprising. However, we also note that A16 finally moved to LPDDR5-6400 after 5 years on LPDDR4X-4266 (A11 to A15). This 50% increase in memory bandwidth may be sufficient to offset the smaller SLC capacity.

System Level Cache (SLC)
A14: 16MB
A15: 32MB
A16: 24MB

Layout wise, the format stays mostly the same between generations with the memory, NPU, P-Cores, SLC and GPU all with a similar layout. However, the E-Core cluster has changed positions to be between the P-Cores and SLC instead of beside it.

Area: Slightly Larger

While we cannot accurately determine the area of the A16 chip from the video clip or other sources, we can see that the die size is indeed slightly larger than the A15. We will have to wait for further information to become available to determine the exact area increase. (If one assumes the 4MB L2 array on both A16 and A15 take the same area, that would also give a larger die size for the A16)

Factors supporting this are the 6% increase in transistor count combined with a lower amount of dense cache arrays on die leading to a larger die size to make up that transistor count. This may be enough to offset any transistor density benefits of moving to TSMC N4 vs A15 on N5. It would also make sense that the A16 costs more to produce than the A15, as for the first time ever, the latest chip is only available on the high-end Pro iPhone line, but the primary contributor of cost there would be from more expensive LPDDR5, as the die area differences are not large.

The Everest P-Cores on the A16 appear to be a new core with different internal layout to Avalanche on A15. Area seems similar to marginally larger.

The Sawtooth E-Cores on the A16 also appear to have a different layout from Blizzard on A15, while also taking a bit more area.

The GPU core design on the A16 is not discernible here, but area and performance seem identical to A15.

A16: Asymptote?

Independent testing of the A16 shows that performance improvements have been levelling off, as a direct result of having 3 generations on similar process nodes. Apple can increase die size to get more performance, but that increases costs. As such, A16 is a symptom of Moore’s Law slowing as we continue waiting for the next generation of 3nm-class nodes.

Share

The USB Promoter Group has just announced the new USB4 Version 2.0 specification, touting up to 80 Gbps data rates from the same Type-C connectors. While the major headline is the doubling of speeds over USB4, Angstronomics has dug up a neat feature about the new standard that enables additional functionality.

Keysight Documentation Details PAM3

6 months ago, Angstronomics discovered documentation of Keysight Technologies M8040A Bit Error Ratio Tester that listed input testing support for “USB4v2“.

In April, Keysight published an article in Chinese about their S94USBCB USB Type-C® Interconnects Compliance Test Software, that stated “support for doubling the Tx/Rx speeds in future USB implementations (for example the next generation USB standard USB4V2.0)”.

Also in April, Keysight released their PathWave Advanced Design System (ADS) 2022 Update 2.0 Software, where their website lists USB4v2 (PAM3) AMI Model Builder, and their W3081E PathWave USB AMI Modeler page “Builds USB4V2 AMI models for Tx and Rx“, “for USB with next generation PAM3 modulation“.

Thus, we can see that USB4 v2 achieves 80 Gbps through PAM3 physical layer architecture. This lines up with the deleted tweet by Gregory M. Bryant, former head of Intel’s Client Computing Group, that showed a poster with “USB 80G“ PHY based on PAM-3 modulation.

Special Trick: USB4 v2 is also 120 Gbps*

Standardized in 2014, the USB Type-C connector has 2 pairs of high speed data links, each with a transmit and receive. Thus, in a normal USB data configuration the 4 lanes are configured in a 2 Tx + 2 Rx symmetric setup. For USB4 v2 that does 40 Gbps per lane, that means 80 Gbps bi-directional bandwidth.

However, USB4 v2 also enables an asymmetric 3 Tx + 1 Rx configuration, meaning 120 Gbps out from the host and 40 Gbps back from device to host. This asymmetric setup then allows tunneling of the full DisplayPort UHBR20 rate that takes 80 Gbps of bandwidth out to a monitor while simultaneously supporting 40 Gbps bi-directional USB data transfer rates on top of that. This would meet the requirements of a single cable monitor + docking station solution that could handle 8K144 HDR with Display Stream Compression and no chroma subsampling with an integrated 40 Gbps USB hub at full bandwidth with no contention.

*Of course this asymmetric setup is not new to the USB standard. We already have examples of USB-C docks and monitor hubs that can do 4K60 + 10 Gbps USB3 concurrently on the same USB3 cable, with 2 lanes of DP 1.4 HBR3 out for 4K60 and the remaining 2 lanes for 1 Tx + 1 Rx enabling 10Gbps USB speeds. Thunderbolt 3, USB4 and USB4 v2 flexibly adjusts transmit and receive bandwidth as all data types are muxed and demuxed at each end of the cable.

When is it coming?

The deleted tweet was posted over a year ago, so we know Intel has been working on this for awhile. Hence, we expect them to be among the first to bring USB4 v2 capable host processors to market. As Intel’s future Meteor Lake and Arrow Lake CPUs share the same connectivity and IO capability and will not be succeeded into 2025, we believe it will bring USB4 v2 in late 2023 and into 2024 when Meteor Lake launches.

As for AMD, which has lagged behind in integration, they have to wait for industry partners to develop off-the-shelf design libraries before they can be integrated into their processors. We saw that with USB4, where Ryzen 6000 (Rembrandt) arrived ~1.5 years after Intel 11th Gen Core-U (Tiger Lake). And we should see that again with USB4 v2, as AMD’s 2024 “Strix Point“ processors do not integrate USB4 v2.

It is also a reasonable assumption that M-series Apple Silicon will integrate USB4 v2 in a similar timeframe between Intel and AMD.

Of course, having host support is not useful if there are no devices that take advantage of it. So we expect the external SSD, monitor and hub ecosystem to start building into 2024 and beyond.

Overall, we are excited that USB-C is finally doubling bandwidth. It will have taken the industry over 8 years to go from 40 Gbps Thunderbolt 3 in 2015 to 80 Gbps USB4 v2 in 2023+. However, we have concerns of cost and adoption over the need for active or even tethered cables for such high bandwidth over meaningful distances.

Update: Our assumption that Meteor Lake and its successor will integrate USB4 v2 is incorrect. USB4 v2 will be enabled through external Barlow Ridge controllers.

Share

Ahead of the official announcement of AMD’s Ryzen 7000 Series Desktop Processors, Angstronomics previews some key microarchitectural details and what this means for cooling and competitiveness.

Zen 4 Microarchitecture

AMD calls its Zen 4 CPU core design an “incremental update“ over Zen 3. This shows with the same CPUID Family 19h for products using either core generation. The next major revision arrives in 2024 with Zen 5 (Family 1Ah).

Transistor Count

• TSMC N5 Core Complex Die (CCD)

• Codename “Durango”

• 8 Zen 4 “Persephone“ Cores

• 8MB L2 + 32MB L3 Cache

• 6.57 Billion Transistors

Each 8-core 5nm Zen 4 CCD is made up of 6.57 Billion transistors. This packs 58% more transistors than the previous 7nm Zen 3 CCD (4.15B) despite the die being over 10% smaller. Thus the average transistor density across the whole die goes from below 50 MTr/mm² to over 90 MTr/mm².

Core Density

As we know that cache area scaling is smaller than logic area scaling moving from 7nm to 5nm, this tells you that cache takes up a higher percentage of total die area. What you get is that a 5nm Zen 4 core with 1MB L2 takes less area than a 7nm Zen 3 core with 0.5MB L2 while transistor count more than doubles. This means that logic area has also more than doubled in average transistor density.

Transistor Spend

Zen 4 has spent its transistor budget in 2 main areas. Modifying the Floating Point pipeline to handle AVX-512 instructions using the same 256b datapath as Zen 3, and improving the Front-End of the core to feed the execution engines better, which was more bottlenecked in Zen 3. The AVX-512 instruction support level is closer to Intel’s Sunny Cove than Golden Cove, with AVX-512F but no VP2INTERSECT. The Front-End now has a significantly larger micro-op cache with over 6K entries vs 4K on Zen 3. Funnily enough, that means the micro-op cache is 50% bigger than the L1 instruction cache, in terms of transistor budget. Of course, it is much more energy efficient to stay within the micro-op cache pool than going out to L1i or L2.

Consequences

The doubling of transistor density at the heat generating sections of the die has a major impact on the cooling of the chip. Cache has a comparatively low heat output per area, while logic is where the hotspots are located. Due to thermal limits at the logic hotspots, there is increasing use of “dark silicon”, meaning a lower percentage of transistors can be switching and producing heat at any one time.

Cooling Challenge

Ryzen 7000 Desktop will therefore represent the ultimate cooling challenge seen thus far. The die can approach 2 Watts/mm² if your cooler is up to the task. The smaller heatspreader area on Ryzen 7000 package design only makes things worse. We discussed the cooling difficulty and why the heatspreader is smaller in our Computex piece in May.

Due to the cooling difficulty and higher power limits on Ryzen 7000 Dekstop, the philosophy has changed to follow a more laptop-like approach. Instead of fixing a clockspeed or power limit, the processor is now temperature limited. This means maximizing performance for a given cooling capability. The processor will adjust to stay within the temperature limits. This also means a more significant performance difference that changes with cooling ability.

Competitiveness

AMD’s client desktop processors remain the weakest in their line-up compared to Intel. Outright performance for the money lags behind while being much more difficult to cool than Intel’s offerings. Ryzen 7000’s 5nm process significantly increases production costs and risks eroding margins as it is difficult to increase ASPs commensurately in such a competitive environment. Mandatory DDR5 on the new AM5 platform will also have some impact on uptake due to a much higher base platform cost.

Share

The much anticipated RDNA 3 graphics line from AMD has received numerous conflicting and changeable rumors over the past year, from multiple compute dies with hundreds of compute units to wildly different cache and die sizes. It is time for Angstronomics to clear the air, and detail just what RDNA 3 brings.

GFX11: Area, Area, Area

What AMD has officially detailed so far about RDNA 3 is yet another significant increase in performance per watt over RDNA 2, with contributions from process node and microarchitectural design choices. However, the design philosophy of gfx11 is all about area, area, area. What is the best way to achieve the performance target with minimal area? The rearchitected Compute Unit and Optimized Graphics Pipeline changes are mostly about trimming the fat in pursuit of the lowest area and cost (example: halving relative FP64 rate to 1/32). As a result of this focus, PPA is significantly increased. In fact, at the same node, an RDNA 3 WGP is slightly smaller in area than an RDNA 2 WGP, despite packing double the ALUs.

OREO

One of the features in the RDNA 3 graphics pipeline is OREO: Opaque Random Export Order, which is just one of the many area saving techniques. With gfx10, the pixel shaders run out-of-order, where the outputs go into a Re-Order Buffer before moving to the rest of the pipeline in-order. With OREO, the next step (blend) can now receive and execute operations in any order and export to the next stage in-order. Thus, the ROB can be replaced with a much smaller skid buffer, saving area.

Infinity Cache Updates

The Memory Attached Last Level (MALL) Cache blocks are each halved in size, doubling the number of banks for the same cache amount. There are also changes and additions that increase graphics to MALL bandwidth and reduce the penalty of going out to VRAM.

Navi3x dGPU Configurations

Now we will go through the specifications of each die configuration of discrete RDNA 3 GPU. To be abundantly clear, these configurations for Navi3x were done in 2019 and finalized sometime in 2020, with no changes since.

Navi 31

• gfx1100 (Plum Bonito)

• Chiplet - 1x GCD + 6x MCD (0-hi or 1-hi)

• 48 WGP (96 legacy CUs, 12288 ALUs)

• 6 Shader Engines / 12 Shader Arrays

• Infinity Cache 96MB (0-hi), 192MB (1-hi)

• 384-bit GDDR6

• GCD on TSMC N5, ~308 mm²

• MCD on TSMC N6, ~37.5 mm²

The world’s first chiplet GPU, Navi31 makes use of TSMC’s fanout technology (InFO_oS) to lower costs, surrounding a central 48 WGP Graphics Chiplet Die (GCD) with 6 Memory Chiplet Dies (MCD), each containing 16MB of Infinity Cache and the GDDR6 controllers with 64-bit wide PHYs. The organic fanout layer has a 35-micron bump pitch, the densest available in the industry. There is a 3D stacked MCD also being productized (1-hi) using TSMC’s SoIC. While this doubles the Infinity Cache available, the performance benefit is limited given the cost increase. Thus, the main Navi31 SKU will have 96MB of Infinity Cache (0-hi). This is lower than the 128MB in Navi21. A cut-down SKU will offer 42 WGP and 5x MCD (80MB Cache, 320-bit GDDR6).

The reference card appears to have an updated 3-fan design that is slightly taller than the previous generation, with a distinctive 3 red stripe accent on a section of the heatsink fins near the dual 8-pin connectors.

There were early plans for a version with 288MB of Infinity Cache (2-hi), but this was shelved as the cost-benefit was not worth it.

Navi32

• gfx1101 (Wheat Nas)

• Chiplet - 1x GCD + 4x MCD (0-hi)

• 30 WGP (60 legacy CUs, 7680 ALUs)

• 3 Shader Engines / 6 Shader Arrays

• Infinity Cache 64MB (0-hi)

• 256-bit GDDR6

• GCD on TSMC N5, ~200 mm²

• MCD on TSMC N6, ~37.5 mm²

Coming in 2023, Navi32 is a smaller version of Navi31, reusing the same MCDs. Navi32 will also be coming to mobile as a high-end GPU offering in AMD Advantage laptops. There were plans for a 128MB (1-hi) version, however it might not be productized due to the aforementioned costs. Thus Navi32’s 64MB is also smaller than Navi22’s 96MB.

Navi33

• gfx1102 (Hotpink Bonefish)

• Monolithic

• 16 WGP (32 legacy CUs, 4096 ALUs)

• 2 Shader Engines / 4 Shader Arrays

• Infinity Cache 32MB

• 128-bit GDDR6

• TSMC N6, ~203 mm²

Navi33 is the mobile-first push for AMD. They expect robust sales of AMD Advantage laptops with it, as the design is drop-in compatible with Navi23 PCBs, minimizing OEM board re-spin headaches. They aim to ship more Navi33 silicon for mobile than to desktop AIB cards. The first concepts showed Navi33 as a chiplet design with 18 WGP and 2x MCD, but this could not meet the volume and cost structure of this class of GPU vs a monolithic design.

As an aside, Navi33 outperforms Intel’s top end Alchemist GPU while being less than half the cost to make and pulling less power.

Conclusion

While Angstronomics has disclosed the hard specifications, we did not mention performance and power here. More importantly is how competitive AMD feels their RDNA 3 lineup is compared to market leader Nvidia’s “Ada Lovelace” line of next generation GPUs. We will cover this in a future article, and what AMD is doing about it.

Subscribe now

Share

Angstronomics presents the hard truths of the world's most advanced process node. We detail their claims vs real chips, how transistor density is calculated, show concrete measurements on the real dimensions of TSMC N5, and get technical on its transistor layout to explain area scaling. We will follow up with what this means in the context of their next N3 and N2 nodes.

TSMC has remained very tight lipped on technical details about N5. Information for calculating transistor density has not gone far beyond the ‘1.8x vs N7‘ that TSMC said 4 years ago. Their IEDM 2019 paper about N5 was accepted which contained no transistor dimensions. Scotten Jones and David Schor expressed that the paper was more marketing than technical. The paper seemed to have passed muster only because it was about the world’s most advanced process technology.

Since then, TSMC’s public disclosures left us with varying 1st party logic density improvement claims, from 1.7x to 1.84x, leading to many incorrect density assumptions from media and even industry. With nothing else to work with, the density claim was simply multiplied with known TSMC 7nm densities to arrive at numbers like 171 Million Transistors per square mm (MTr/mm²). We will explain how transistor density is calculated below. These assumed numbers have been with us for years, even ASML themselves showed just recently that foundry 5nm is over 180 MTr/mm² (relevant slide at the bottom of this article).

But the reality is completely different. The real density is far lower than that. The 1st sign of trouble came almost immediately with Apple’s A14 announcement, the world’s first N5 chip. 11.8 Billion transistors, about 88mm². The average transistor density for the whole chip is 134 MTr/mm². This marked a big divergence in the relation between calculated logic density and average chip density. Mobile AP (Application Processors) have historically just so happened to track closely with theoretical calculated logic density. Something was up. Apple’s design didn't change drastically. Then more and more chips came out. Kirin 9000. M1 Max. None reached the assumed density of 171. The reality is that the correlation still holds. The assumed density is wrong.

MTr/mm² Metric

The theoretical density is calculated using the metric pushed by Mark Bohr of Intel. This metric is used to calculate theoretical transistor density for logic circuits, and is better than previous attempts as it captures area scaling from transistor structure and layout improvements. SRAM and Analog is not used in the calculation nor is the ratio of logic to SRAM to Analog in real chip designs. Officially, the formula is:

To understand how to use this, we must know the important dimensions and structure of logic transistors in modern day circuit design.

Standard Cell Terms and Scaling Vectors

Modern process technology uses Complementary Metal Oxide Semiconductor transistors, mixing PMOS and NMOS to create logic circuits. Standard cells are organized into horizontal rows. Designers build standard logic circuits within these rows like Inverters, NAND gates, adders and flipflop cells that serve Boolean logic functions. The key dimensions to note in transistor area scaling is the Contacted Gate Pitch (CGP, historically poly pitch CPP) in the horizontal x-direction, and Cell Height in the vertical y-direction. This gives the unit cell area which can be used to determine density. Angstronomics will use the following descriptor for node dimensions:

2-fin TSMC N7 (H240g57)
H240 = 240nm Cell Height
g57 = 57nm Contacted Gate Pitch

Logic transistor density scaling comes from reducing these two dimensions. Scaling trends show that the actual transistors themselves (Front End Of Line FEOL) can shrink better than the metal interconnect to wire them (Back End Of Line BEOL). EUV is needed to shrink the lowest metal layers further. CGP is limited by contact width, spacer width, gate length and diffusion breaks. Cell Height is limited by minimum metal pitch, number of metal tracks, and less so fin count and fin pitch. Reducing these dimensions is difficult, and the pace has slowed down considerably. CGP width is barely shrinking at all. Cell height can still scale with Design Technology Co-Optimization (DTCO) techniques such as fin depopulation, Back Side Power Delivery, Complementary FETs (CFET) and others that reduce cell height independent of metal pitch.

Defining the Density Values

With CGP and cell height we can go back to the density calculations. For modern process nodes that have Single Diffusion Breaks (detailed below), a 4-transistor NAND2 cell has a width of 3x CGP. Intel presented 19x CGP for their Scan FlipFlop cell. We will go with that with a 32 transistor circuit inside that area. Angstronomics understands that the CGP width and transistor count of large cells may differ from node to node due to drive current and metal interconnect differences. Nonetheless, 4-in-3 and 32-in-19 is approximate to industry metrics for SDB. Add an additional CGP for each cell for Double Diffusion Break (DDB) nodes such as N7. Inserting these values into the formula, then adding values for Intel’s densest 10nm library (H272g54), we get 100.33 MTr/mm². Intel listed a density of 100.8. For TSMC’s N7 2-fin (H240g57), we get 90.64 MTr/mm². For fun, we can reverse the formula and find pitches that meet the incorrect density assumption for N5. H180g48 would give a density of 170.6 MTr/mm².

Measuring Actual N5 Transistors on Apple A15

Thanks to System Plus Consulting (now part of Yole Group), we can measure TSMC N5 logic transistors directly. From their Apple A15 Reverse Costing Sample, there are 2 crucial Scanning Electron Microscopy (SEM) images with scale bars that show logic transistors. We measure an average standard cell height of 210nm and average CGP of 51nm. Plugging in those numbers into the density formula shows the H210g51 of TSMC N5 2-fin achieves a logic density of 137.6 MTr/mm². These values match the industry disclosures if one knows where to look [1]. So N5 is 1.518x denser than N7. We also measure a minimum fin pitch of 28nm, which lines up with the 28nm M0 pitch of N5. TSMC disclosed that their N5 has a 30% smaller minimum metal pitch vs their N7 (40nm MMP) [2]. Additionally, a 6-Track cell at 210nm cell height reveals an M2 pitch of 35nm.

Why This Discrepancy?

TSMC’s density claims were never based on this density metric or any critical pitch in the first place. They compare using an ARM Cortex-A72 synthesis which is very flexible in layout and area. At a high level, the same core design on the same node and library can trade some area for increased frequency and performance by using larger standard cell variants with higher drive current strengths. An example with the simple logic inverter shows a higher drive strength INV_X2 standard cell can be made with a double-gate design with 2-fingers that takes more area than the standard INV_X1 for the same logical function [3]. This flexibility in design generates a Freq/area curve as presented by TSMC. The foundry can choose to compare any chosen point on one curve with another to derive a density claim ratio.

This kind of comparison is more 'real world' as the node benefits are extracted all the way to a complex CPU core level instead of just blindly estimating density based on raw FEOL pitch as in the Bohr model. As we detailed earlier, scan flipflop and large cells may take up different areas with the same FEOL dimensions based on BEOL aggressiveness which affects signal routing density. Design Rule Check (DRC) push can also provide overall density increase with the same FEOL size at the expense of increased design complexity. One example is reducing the margin width between logic and SRAM regions in silicon. The margin is needed to maintain different silicon strain values for Logic and SRAM to ensure device performance and reliability near region boundaries. Nonetheless, we are still going to call out TSMC here as the average chip density in real processors still lines up with the Bohr model and not their claims.

What Does This Mean for N5?

Nothing. Continue to enjoy your 20-month old iPhone 12 and brand new M2 MacBook. They are wonderful devices. An N5 wafer can still pack close to 10 trillion transistors. N5 has been the world's most advanced node for years. Just not as dense as assumed. Samsung’s 4LPE (H200g54) at 136.5 MTr/mm² is ever so slightly less dense than N5 but arrived 16 months later at vastly lower volumes and low reported yields. Density is only 1 metric in PPA (Power, Performance, Area). Samsung closed the gap in density but performance and power remain behind, with only a small improvement over their 7nm-class node.

Subscribe now

What About High Performance Cells?

So far we have only featured the densest 2-fin library as it gets all the density plaudits, but High Performance (HP) cells are also important for high performance computing and low latency serial applications. We saw with Qualcomm’s Snapdragon 855 on TSMC N7 use selective 3-fin HP (H300g64) cell region for a Prime CPU core over the standard 2-fin library (H240g57) to get more single thread perf at the expense of some area. Adding 2 more fins to the dense cell to get 3 PMOS + 3 NMOS fins, and adding 1 more Metal-2 track and relaxing pitches, we get a cell height of 280nm (7-Track 40nm M2). For CGP, the value is 57nm, which is the same as 2-fin N7. This tracks historically with TSMC’s HP cell using the same CGP as their previous generation HD cell (N7 HP = N10 HD = 64nm CGP). H280g57 gives a logic density of 92.3 MTr/mm² for 3-fin N5. Coincidentally, this just happens to be around the average chip density of NVIDIA’s H100 and Alibaba Yitian 710.

Now we observe that 3-fin N5 has a lower density than the 3-fin Intel 4 process. Intel 4’s sole 3-fin library (H240g50) has a density of 122.8 MTr/mm². Of course, Intel 4 products come years after N5 at much lower wafer volumes. Intel 4 is also an incomplete node, only offering a single 3-fin library and a minimal I/O library for chiplet-only interconnect to accelerate Time-To-Market (TTM). However, Intel 4 has a significant DTCO advantage over N5, as it achieves much higher density despite using a wider metal pitch and fewer EUV layers. They achieved aggressive cell height reduction by narrowing the space between n and p diffusion regions with a much narrower gate cut width. This significantly increased the percentage of active fins to patterned fins. We also expect Intel 4 to have better Performance/Power due to better metal interconnect material composition of Cobalt-wrapped Copper.

CNOD and Low Vertical Scaling

Readers with a keen eye will notice that cell heights have barely reduced going from N7 to N5. 300nm to 280nm for 3-fin and 240nm to 210nm for 2-fin. This is despite the vertical critical dimension (M0) reducing drastically from 40nm to 28nm. One of the reasons is that N5 2-fin now has 5 M0 signal lines between power rails vs 4 M0 signal lines on N7 2-fin. This allows for greater flexibility in contact placement. The other reason is that N5 uses an uncommon form of horizontal standard cell separation called Continuous Diffusion (CNOD).

To separate horizontally adjacent standard cells in the same row, dummy gates are used at the sides of each standard cell to separate one active diffusion area from another. In Double Diffusion Break (DDB), two dummy gates cut-off and separate the active areas. In Single Diffusion Break (SDB), space for a single separation gate is replaced by a deep etched insulator material which cuts the fins and isolates the active areas. This takes up less horizontal width than DDB, contributing to DTCO area scaling.

In CNOD, the diffusion is not broken at all. The fabrication process continues normally, but when standard cells need to be separated, the gate between them is designated as a dummy gate. This dummy gate is then connected to a Gate Tie-Down Via to the power rail. This dummy gate tie-down method of CNOD achieves the same horizontal width savings as SDB, and has the advantage of keeping the transistor diffusion unbroken and thus can achieve more uniform strain and performance characteristics. As a refresher, since 90nm, transistors have been made such that the channel where electrons flow is stretched or squeezed under great force which increases channel mobility in order to reduce resistance. SDB relies on the etched insulator region to maintain the physical strain which can have more variability.

The problem with CNOD is the need to tie-down the dummy gate to the power rails. This adds more Vias and connections in the critical region under the power rail, which is between vertically adjacent cell rows. This means the cell height has to increase to accommodate the space for additional contacts between standard cells. While EUV and self-aligned gate contacts do help, CNOD ultimately sacrifices some y-scaling to achieve x-scaling. CNOD is therefore uncommon as the overall area scaling is worse than SDB. SDB can also achieve better isolation as the active diffusions are physically broken and separated. SDB is the standard moving forward, and TSMC uses it in their later N6 and N3 offerings, known as CPODE (Common Poly on Diffusion Edge).

So Can We Trust Their Claims on N3E?

TSMC recently put out their N3E FINFLEX™ figures, once again in their A72 Freq Vs area plot. The rough guide given for combined area scaling contribution was 50% logic 30% SRAM 20% Analog. Logic would have to pull the weight as SRAM and Analog fundamentally cannot benefit from DTCO area scaling. In this case however, A72 core area scaling is much more inline with logic area scaling, giving us a result of 215 MTr/mm² for the densest N3E 2-1 Fin library. Angstronomics already has most of the critical dimensions for TSMC's 3nm offerings, and we will detail them when we can properly cover the debacle that is N3 and the walk back with N3E. As you can see, the actual logic densities are far lower than the 300 Million transistors per sq. mm that some investor presentations have shown for a foundry 3nm process.

The defining technology of TSMC’s 3nm libraries is FinFlex. Angstronomics sees that FinFlex seems to be misunderstood by some. It is not simply selective region cell rows as seen on the Snapdragon 855. Nor is it different ratios of PMOS to NMOS fins (FinFET physics allow the beta ratio to be closer to 1:1). TSMC’s FinFlex video illustrates the actual structure of the standard cell layout. N3 has a 3 fin cell row, 2 fin cell row and 1 fin cell row. That means each standard cell row will have 3 PMOS fins and 3 NMOS fins, 2P+2N fins or 1P+1N fin between power rails. The difference with FinFlex is the hybrid cell row structure.

3-2 fin means alternating rows of 3p3n and 2p2n. 2-1 fin means alternating rows of 2p2n and 1p1n. This alternating cell row layout provides more cell library density options before the move to nanosheets where library density no longer has to be quantized by fin count (although in reality the metal stack and cell rows provide some level of discrete units). Of course, this unique hybrid row layout means a complete rethink of standard cell designs. Engineers have to think about which transistors in a logic block to put in each alternating low and high fin count cell rows based on device performance characteristics. Design Rule Complexity gets even more complicated. Electronic Design Automation (EDA) software has to take the hybrid rows into account.

Single-Fin Library (N3S?)

TSMC VP Kevin Zhang mentioned the existence of another 3nm variant called N3S, arriving later in 2024+ and is said to be a transistor density optimized process. Historically, process variants do not normally change fundamental critical dimensions outside of a small relaxation in CGP to enhance yield and performance. Thus, Angstronomics believes N3S may offer a Single-Fin Only Library to squeeze out the last drop of transistor density.

Though an ultra-dense single fin library might grab headlines with its transistor density, there is a big catch. The practicality of such a library is subject to scaling bottlenecks by the metal stack. Each transistor still has to have its Gate, Source and Drain terminals connect to the metal interconnect network to function. Despite the best efforts of current EUV lithography, the lowest metal layers form the largest bottleneck in lowering cell height and reducing area. There is no point making the FEOL very small if your BEOL cannot keep up. If you shrink the metal stack through brute force, the wires and spacing become too thin to maintain good resistance and capacitance characteristics. An exclusive 1-fin library would thus not meet the performance requirements for many designs. Although N3S might bring that single fin only library, we believe it may be of limited use due to the sacrifices in transistor performance. Nonetheless, if the density benefits are worth it, designers will find a way to work around these constraints.

TSMC N2: >1.1x Chip Density

As for N2, our preliminary findings show a conservative 1st attempt at nanosheets, with low density improvements, instead targeting better power and performance by moving to Nanosheet transistors. Area scaling grinds to a crawl! Additional risk factors such as Back Side Power Delivery Network will be implemented carefully over the life of N2's derivatives and successors. This reminds us of TSMC’s FinFET transition with their 16FF process, that also had a low density improvement over planar. In addition, while the first devices on N2 are for 2026 onwards, the node can fall back on multi patterning low-NA EUV. Depending on industry execution, risk management may see High-NA introduction on N2 akin to N7+ dipping their toes into the EUV pool. High-NA is not a magic bullet and comes with many challenges from the anamorphic magnification such as directionality, extreme narrow depth of field for photoresists to work within, and alignment and integration with existing full-field processes used on everything but the critical layers.

TSMC N5 is Not as Dense as You Think

So that's it. The cat is out of the bag. You now have the true density of the world’s most advanced process node. The next time you see transistor density numbers from research and tool vendor roadmaps, just remember to dial those back by several notches. Angstronomics will cover the effect of slowing transistor area scaling on the long-term health of Moore’s Law in the Angstrom Era once we detail offerings from the other two remaining leading edge Logic foundries: Samsung and Intel.

Subscribe now

Share

References

[1] A. Nayak et al., "A 5nm 3.4GHz Tri-Gear ARMv9 CPU Subsystem in a Fully Integrated 5G Flagship Mobile SoC," 2022 IEEE International Solid- State Circuits Conference (ISSCC), 2022, pp. 50-52, doi: 10.1109/ISSCC42614.2022.9731604.

[2] J. C. Liu et al., "A Reliability Enhanced 5nm CMOS Technology Featuring 5th Generation FinFET with Fully-Developed EUV and High Mobility Channel for Mobile SoC and High Performance Computing Application," 2020 IEEE International Electron Devices Meeting (IEDM), 2020, pp. 9.2.1-9.2.4, doi: 10.1109/IEDM13553.2020.9372009.

[3] S. K. Marella, A. R. Trivedi, S. Mukhopadhyay and S. S. Sapatnekar, "Optimization of FinFET-based circuits using a dual gate pitch technique," 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2015, pp. 758-763, doi: 10.1109/ICCAD.2015.7372646.

[4] S. Badel et al., "Chip Variability Mitigation through Continuous Diffusion Enabled by EUV and Self-Aligned Gate Contact," 2018 14th IEEE International Conference on Solid-State and Integrated Circuit Technology (ICSICT), 2018, Fig.2,9, doi: 10.1109/ICSICT.2018.8565694.

In this second article on Angstronomics, we go beyond the Press Deck and discuss the interesting observations we picked up on during the week of Computex Taipei 2022 from AMD and NVIDIA. We will also add our own flavor into the mix with exclusive information we can reveal about some products. Enjoy!

But first:

A huge thank you to all those who have read and subscribed to our fledgling publication! In less than a week, with just one article, we have amassed over 10,000 unique visitors and received over 100 subscriptions, far exceeding our humble expectations!

A big tip of the hat to the Hacker News community, who were quick to share and discuss our article! While we do try to promote our articles on Twitter (which readers should follow for timely updates and insights), the HN post gave us a huge boost. Thank you!

We have big plans for Angstronomics moving forward, including process deep-dives, past and future events, and more exclusive info! Read on to the end to see what is in the works!

For our new folks, do Share and Subscribe to receive our articles right in your inbox!

- SkyJuice, Founder of Angstronomics

Share

Subscribe now

Socket AM5 Motherboards

As expected, Computex Taipei 2022 played host to the announcement of several Socket AM5 motherboards for AMD’s upcoming Ryzen 7000 Series Desktop Processors. This makes sense as these board makers are based in Taiwan and have the majority of their board manufacturing capability on the island. A wide spectrum of boards have been revealed, with many different configurations of expansion slots and ports on offer, indicating the vast optionality built-in to the I/O system of the CPU and the accompanying 600-series chipsets, which we detailed here.

From Thunderbolt and USB4 to 10GbE and even up to four PCIe 5.0 M.2 slots for storage, board makers are free to differentiate their designs through different PCIe lane bifurcation and ports enabled. Furthermore, AM5 600-series board prices seem to indicate more features for a similar or lower list price than the previous generation X570 boards, possibly contributed by the lower cost of the ASMedia chipsets compared to reusing the IO die for X570’s Bixby chipset.

AMD’s Computex Keynote presentation promoted 24 lanes of PCIe 5.0. The IO Die on the AM5 Processor does have the PCIe controllers and Physical Layer (PHY) for 28 lanes of PCIe 5.0 throughput. However, attached devices such as USB4 controllers and the ASMedia-made 600-series chipsets can only support up to PCIe 4.0 linkrate. As 4 lanes are occupied by the chipset downlink, this leaves 24 PCIe 5.0 lanes for the board makers to utilize.

From our block diagram, you can also know that ‘Up to 14 SuperSpeed USB’ really means an X670 daisy-chained solution, with four 10Gbps USB from the CPU Combo ports, and one 20Gbps USB and four 10Gbps USB from each of the two chipsets. That sums to 14 SuperSpeed USB ports, of which 2 are 20Gbps and 10 are 10Gbps. WiFi 6E support is through discrete PCIe M.2 Key-E cards from vendors such as MediaTek and Qualcomm. Finally, as we indicated earlier, the integrated graphics of the processor also supports up to 4 Displays, either HDMI 2.1 or DP 2.0 UHBR10.

The phrase ‘Up to’ is necessary here as from our article, 3 display outs are part of USB Type-C Combo ports. Mainboard designers can place a Type-C port that can switch between either 10Gbps USB or DisplayPort Alt-mode. If they choose to connect the combo port to a 10Gbps USB Type-A, then there will be one less display out option. Alternatively, they could assign it to a dedicated HDMI 2.1 port, in which case the combo port’s USB function drops to USB 2.0, removing a SuperSpeed USB option.

We did not indicate the die size of the ASMedia chipsets in our previous article, but thanks to a livestream by MSI, we can determine the die size to be ~40 mm². This is far smaller than the 125 mm² of the previous X570 chipset, which had many disabled regions but more I/O capability. Lower area reduces cost of fabrication.

Notice the chipset arrangement of the MSI X670 board, which groups the pair at the traditional bottom-right region of the motherboard. This contrasts with the ASUS X670 board, revealed in Phison’s E26 PCIe 5.0 SSD controller demo, which places the chipsets further apart in an East / West orientation. We explain why this is beneficial in our previous article.

Raphael Revealed

Shown on stage and in a Tweet, the chiplet configuration of AMD’s Ryzen 7000 series, codename Raphael, was revealed by AMD CEO Dr. Lisa Su, showing the individual dies and capacitor arrangement. The sample shown has been taken off the production line before the lidding process, as shipping units have an Integrated Heat Spreader (IHS) covering the dies. With this, we can do an estimate of the die sizes.

Ryzen 7000 series Desktop ‘Raphael‘ Processors:

• AM5 socket 1718-position Organic Flip Chip Land Grid Array: 40 x 40 mm

• IOD (Input Output Die) on TSMC N6: ~ 120 mm²

• CCD (Core Compute Die) on TSMC N5: ~ 72.5 mm²

As a side note, we notice that the CEO held the processor upside down. The corner triangular Gold indicator is always oriented in the top-left corner when installed on a standard Desktop motherboard.

Going for Gold

The most striking observation of Raphael’s dies are the Gold colour of the CCDs, while the IOD remains the usual grey of Silicon. This Gold colour comes from the silicon Backside Metallization (BSM) that contains a layer of Gold a few hundred Angstroms thick. The Gold layer is beneficial for silicon that is going to be soldered and lidded, as it prevents oxidation of the silicon and formation of tiny gas voids during heating, which is required to melt the Indium-based solder Thermal Interface Material (TIM) during the lidding process where the IHS is also attached.

Solder is used as the thermal link from the silicon dies to the IHS as it has low thermal impedance and can be melted and reflowed to fill any air gaps and rough surface finishes. The Gold layer also provides a better wetting surface for the melted solder to adhere to and flow across the silicon. While we are starting to see this Gold BSM process applied to more products such as Intel’s Sapphire Rapids and Ponte Vecchio Processors, this is not required for processors designed for direct die cooling like NVIDIA’s Hopper H100 GPU as there is no need to attach solder.

Likewise, a Gold coating is applied on the underside of the IHS for much the same reasons, shown below with Intel’s Clarkdale as a clear example.

If the CCDs have this Gold coating, why doesn’t the IOD also have it? It could be due to cost reasons. The BSM step is applied to whole wafers prior to dicing, so clearly the step was omitted on the IOD wafers, perhaps due to the low heat density of the IOD not requiring this additional, cost-adding process.

Packaging Engineer Tom Wassick indicates that all dies under solder TIM have this BSM process layer, so perhaps AMD may be mixing TIM materials. Solder for the CCDs and thermal paste for the IOD. This method also allows for differing die thicknesses to be attached to the IHS. As shown with Intel’s Clarkdale above, mixing TIM materials has been done before, with solder on the higher heat density 32nm CPU core die and thermal paste on the lower heat density 45nm Graphics and Memory Controller die. Alternatively, all dies for Raphael may be soldered but the IOD omits the Gold layer but keeps a Nickel coating that still appears grey.

AM5: More Cores or Cooler Compatibility?

One thing Angstronomics would like to clear up here is that 16 cores is the maximum for Ryzen 7000 Desktop. We note that there are no plans to design higher core count CCDs or introduce Zen4c Dense core CCDs on AM5 from AMD’s future EPYC Bergamo line. While 3 CCDs and 24 cores seems reasonable given it makes a quarter of the EPYC Genoa design, Angstronomics can confirm that this idea never went beyond strawman proposals that indicated the necessity of a larger package substrate and IHS which would increase cost and break cooler compatibility.

Given that the 40x40mm package and IHS stack height has been maintained by AMD ever since their original 64-bit K8 Opterons and Athlons from 2003, that was never going to happen! In fact, coolers using the latch mounting mechanism first spotted over 20 years ago at CeBIT 2002 for socket 940/754/939 will indeed support 2022’s AM5 processors! Of course, cooling capability is another matter entirely. Going from Ceramic 940-position micro-Pin Grid Array in the original Opterons to Organic ABF substrate 1718-position FCLGA after nearly 20 years while keeping cooler compatibility seems mighty impressive to us.

Performance Demo: 31% or 45% ‘Faster’?

As you can see, the Ryzen 7000 system completes the process 31% faster, which means creators spend less time waiting, and much more time creating.

- Dr. Lisa Su, CEO, Advanced Micro Devices (AMD) during Computex 2022

Much free marketing was made in Social Media over the percentage quoted in the performance demo shown at the Keynote. In a Blender render test, Intel’s Core i9-12900K took 297.23 seconds to render the frame, while a 16-core prototype Ryzen 7000 sample took 204.61 seconds to render the same frame.

Our stance is pretty clear: The demo chosen focuses on time to complete a set amount of work. Thus, the Raphael sample took 31% less time to complete the task. However, should they choose to demo a multi-frame render animation and compare in terms of renders per hour metric, they could then claim a 45% faster throughput in renders per hour. The preference of task completion latency or throughput rate depends on the demands of the chosen workload or workflow.

To add to this, Angstronomics can also confirm that this demo was run on a non-final stepping of Raphael, and run at some level below the maximum 230W socket power of the final retail model, so expect higher performance claims as we get closer to launch. We do have concerns over cooling difficulty at 230W and above though, given the higher heat density Zen4 CCDs, proximity of the CCDs to the edge of the IHS, and the reduced IHS surface area from AM4 due to the cutouts in the IHS to accommodate the additional capacitors on the top side of the package substrate.

Regarding frequency targets, the game demo showing 5.55GHz maximum frequencies was also not with the final version. While Angstronomics is aware of an Ordering Part Number (OPN) that is fused for a 5.85 GHz Fmax, we will have to wait and see what the retail stepping fuses will be set at.

As for the claimed Single-Thread Uplift of ‘greater than 15% expected‘, Angstronomics can confirm this is a conservative value, done at below final frequencies and using Maxon’s Cinebench R23 Single Thread Benchmark. We can independently confirm that the Performance Per Clock (PPC) targets for the Zen4 core are targeted at +7% Single-Thread PPC, +10% Multi-Thread PPC over their Zen3 core, with significantly higher PPC for memory sensitive workloads thanks to DDR5 while core execution bound workloads like Cinema4D have a lower PPC improvement.

We will be working on a proper deep-dive into the Zen4 core design, from power efficiency to seeing where all the transistors went. Subscribe to know when this article drops!

Subscribe now

We are also working on a deep-dive on Raphael’s IO Die, including memory and integrated graphics performance. AMD markets the integrated RDNA2 GPU as a display adapter, perfect for breaking through to the Commercial PC market. This is rightly so, as the performance is uninspiring. For now, Angstronomics can reveal that Raphael’s integrated graphics, GFX1036, codename Coral Bandfish, contains One RDNA2 WGP.

Mendocino

AMD's Mendocino, pronounced Mendo-SEE-no after the Californian region north of San Francisco, is the codename for their upcoming lineup of mainstream notebook processors. Designed for the FT6 socket that supports a 64-bit width memory bus, Mendocino is therefore the successor to Pollock, which has Two Zen1 cores and 3 Compute Units of Vega graphics.

Looking at the die presented at the keynote, we can derive the following:

• Mendocino Socket FT6: 24.5 x 24.5 mm

• Mendocino Die on TSMC N6: ~ 100 mm²

This die size is even smaller than the IOD in Raphael, even though Mendocino includes Four Zen2 cores! Naturally, the area savings come from featuring much less I/O. We plan to do a separate article covering Mendocino more closely, including why it represents a prime example of cost optimization in both fabrication and design.

Once again, AMD did not detail the RDNA2 GPU configuration for Mendocino, and as with Raphael, outright performance is not the key focus. Angstronomics can reveal that the graphics for Mendocino, codename Teal Grouper, contains One RDNA2 WGP.

NVIDIA Grace

NVIDIA’s Computex Keynote was next, presented without their Taiwan-born CEO. Of note is their announcement of reference server designs featuring Grace, NVIDIA’s take on a custom ARM-powered CPU as they charge ahead with decreasing their dependency on the current x86 incumbents, Intel and AMD. These reference systems will ship in the 1st half of 2023. NVIDIA’s plan is to design and ship servers using their ARM-based CPU designs in Odd years, while they will offer systems using traditional x86 CPUs in Even years.

The listed TDP of 500W for the dual Grace module and 1000W for the Grace + Hopper module represent the Total Board Power (TBP) of the modules, encompassing core, memory power and the losses in the onboard Power Delivery chips. This is in similar vein to listing the TBP of their graphics cards which represents power draw for the whole card. Specifically for the Grace CPU Superchip, this is different from how current CPU TDPs are listed, which only covers Socket power of the CPU, not including memory or power delivery. Thus, 500W for a Grace CPU Superchip module is inline with the power consumption of a Single Socket Xeon or EPYC CPU with accompanying memory.

Angstronomics notices that while Grace was announced in 2021 and marketed as having 2x the packaging density of current server CPU designs with traditional DIMM slots, the 2U4N reference systems arriving in 2023 do not bring any advantage in compute, memory or rack density compared to current designs. Despite this, Grace offers key advantages in high-bandwidth connectivity and higher memory bandwidth, thanks to employing a wide LPDDR memory configuration. This leads to much better power efficiency in memory bandwidth sensitive workloads, as seen on Apple’s M1 Max and M1 Ultra line of SoCs.

While we only have computer renderings of Grace, we can look at the size and features while we wait for proper photographs of actual Grace silicon. Using the Grace Hopper Superchip and the sizes of power delivery and H100 GPU, the latest renders would indicate a monstrous 700 mm² monolithic die featuring a 12 x 7 mesh grid with 76 cores and 8 memory controllers. Thus, 4 cores would be disabled per die to achieve 144 cores on a Grace CPU Superchip Module. Such a large die also indicates the use of larger ARM cores, most likely an improved generation to the Neoverse-V1 cores used by Amazon’s Graviton3 CPUs. Given the promoted performance numbers and power per unit area of silicon, this indicates a low operational frequency to maximize power efficiency. Consequently, Grace may not be as competitive in raw compute capability to Intel’s future Sapphire Rapids Xeons and AMD’s future Genoa EPYC CPUs, though many memory intensive HPC workloads may favor the bandwidth efficiency of Grace.

Spicing Up Computex

Some have found the new disclosures at Computex to be few and far between, so we hope this article helps add much more flavor to the event. Intel was basically absent from Computex, with no further updates on their Desktop Arc Alchemist discrete graphics cards that are supposed to launch this Quarter. To not leave them out of this publication, we will be presenting our findings on Intel’s most recent event, Vision 2022. While we work on that article, we recommend checking out the brilliant work done by Locuza and SemiAnalysis on their deep-dive on Meteor Lake, a paradigm shift in Intel’s SoC design.

For those who read till the end, here’s a bonus. A new Italian City visited by AMD, famous for the Piazza del Campo. Angstronomics presents the historic Tuscan City of Siena!

Subscribe now

As OEMs gear up for the announcement of their new AM5 motherboards at Computex Taipei 2022, more information begins to trickle out on a peculiar feature on these boards: their 600-series chipset.
In this premiere article celebrating the launch of Angstronomics, we will provide the full specifications, topologies and block diagrams of the AM5 platform and its chipsets, plus some additional bonus content on its history and development. Finally, after covering the what, we will also cover the why, detailing why this creative chipset solution was chosen and elaborate on the numerous advantages.

What is a Chipset Anyway?

Chipsets are companion devices on a computer motherboard that are paired with a main computing device such as a CPU and serve as a hub for Input / Output signals that interface with devices and connectors on the motherboard. This intermediate chip ‘bridged‘ the gap between I/O and processor, where the historical ‘Northbridge‘ handled high speed I/O to system memory and graphics cards, while a ‘Southbridge‘ handled low speed I/O to storage and networking. In this architecture, the CPU could be specifically designed to focus on compute and processing power, requiring only a single high speed link to the Northbridge. The chipset would focus on I/O distribution and management.

However, as technology advanced and system designs became increasingly integrated, the role of these chipsets diminished, where the main processor already has basically all the I/O interfaces onboard, from memory, PCI Express and storage devices, USB and even display outputs for integrated graphics. These System-on-Chips (SOCs) thus did away with discrete chipsets altogether.

In fact, AMD’s Ryzen line of client Desktop processors don’t even need a ‘chipset‘ as such to run, requiring only a tiny activator (Knoll) to operate. Branded as X300 and A300, these activators provide no additional I/O. Despite this, nearly all Desktop motherboards still contain a chipset, but in its modern iteration serve as I/O expansion hubs, not dissimilar to a multiport USB dongle. They allow one to plug in a greater quantity of devices, but all the data has to be funnelled through the equivalent bandwidth of a single port. This compromise is manageable so long as the attached devices are not all being used concurrently. The performance difference is negligible when you stay below the chipset uplink bandwidth to the processor. One can say that chipsets have been reduced to glorified PCIe mux switches.

With that out of the way, on to the fun stuff.

The AM5 Platform and Promontory 21

AMD’s upcoming client desktop platform for the DDR5 era was designed from the outset to rely entirely on chipset designed and made by a 3rd party supplier. This differs from the previous generation high-end X570 chipset from 2019 on the AM4 platform, where AMD repurposed their client I/O die chiplet repackaged for use as the aforementioned ‘glorified PCIe mux‘. The memory controllers, PHY and other unused functions just laid dormant in this chipset variant (Bixby). This resulted in a high-cost, high-power chipset solution that increased motherboard cost and sale price while also requiring an active cooling fan to cool the chipset on the first 2 years of X570 board designs. All this was done to get a chipset solution capable of PCIe Gen 4 linkspeed, which 3rd party manufacturers were not ready with at the time. Now that they are ready with PCIe4, AM5 returns to an exclusively 3rd party made solution, as was the case with the AM4 launch platform in 2016 with the original chipset made by ASMedia (Promontory), branded X370, B350, A320, X470, B450.

For AM5, there was also the intention to dual-source these chipsets from 2 different 3rd party suppliers, namely ASMedia and MediaTek. However, we note that more recent developments have exclusively focused on validation using ASMedia silicon, and are unsure when Mediatek will supply chipsets, if at all.

The AMD 600-series chipsets for AM5 are known as Promontory 21 (PROM21), and come in 3 configurations:

• Low-end, presumably A620, with a single defeatured PROM21

• Mid-range, known as B650, with a single PROM21

• High-end, known as X670, with dual PROM21 chipsets on the motherboard

The A-class low-end chipset configuration will launch at a later date, appropriate when the lower-tier Rembrandt APUs launch on AM5.
The high-end X670 chipset is unique, featuring 2 separate PROM21 chipsets linked in a daisy-chained arrangement. The first chipset connects to the CPU, and the second chipset connects to the first chipset.

Here are the specifications of a single PROM21 chipset (B650):

• 19 x 19 mm FCBGA package

• Maximum Power ~7W

• One PCIe 4.0 x4 Uplink to host

• Two PCIe 4.0 x4 downlink controllers (Total 8 Lanes)

• Four PCIe 3.0 x1 / SATAIII 6 Gbps flexible ports (4L PCIe or 2L PCIe + 2x SATA or 4x SATA)

• Six USB 3.2 Gen 2 10 Gbps ports (Two of the ports can be fused to form one USB 3.2 Gen 2x2 20Gbps port, with four remaining 10Gbps USB ports)

• Up to 6 additional USB 2.0 ports

For the A-class chipset, one PCIe 4.0 x4 downlink is disabled

For a typical motherboard with 4x SATA ports, B650 can support one PCIe 4.0 x4 M.2 NVMe SSD, while the A-class chipset supports none. The remaining PCIe 4.0 lanes are used for WiFi, 2.5G Ethernet and PCIe x1 expansion slots.

The X670 boards have their daisy-chained chipsets linked with PCIe 4.0 x4.

Thus, the specifications of a dual PROM21 daisy-chained board (X670) are as follows:

• Two 19 x 19 mm FCBGA packages

• Maximum Power ~7W x 2

• One PCIe 4.0 x4 Uplink to host

• Three PCIe 4.0 x4 downlink controllers (Total 12 Lanes)

• Eight PCIe 3.0 x1 / SATAIII 6 Gbps flexible ports (4L PCIe + 4x SATA or 2L PCIe + 6x SATA)

• Twelve USB 3.2 Gen 2 10 Gbps ports (Four of the ports can be fused to form Two USB 3.2 Gen 2x2 20Gbps ports, with eight remaining 10Gbps USB ports)

• Up to 12 additional USB 2.0 ports

The WiFi and 2.5G Ethernet can be connected to the PCIe 3.0 lanes, freeing up all 12 lanes of PCIe 4.0 downlink for M.2 devices and PCIe slots.

Additionally, there is plenty of I/O directly from the CPU:

• One PCIe x16 slot for graphics (or bifurcated to two x8 slots)

• One PCIe x4 slot for M.2 NVMe SSD

• One PCIe x4 link for discrete USB4 controller (Maple Ridge JHL8540 or ASMedia ASM4242) or M.2 NVMe SSD

• One PCIe 4.0 x4 for chipset downlink

• Three combo USB-C ports supporting USB3.2 Gen 2 10Gbps and DisplayPort 2.0 UHBR10 Alt Mode (each combo port can be split into discrete DP and USB 2.0 Type-A)

• One additional dedicated display output (DP 2.0 UHBR10 or HDMI 2.1 Fixed Rate Link or eDP)

• One USB 3.2 Gen 2 10Gbps port with BIOS flash capability

• One USB 2.0 port with SecureBIO support for Windows Hello capability

• SPI/eSPI, GPIO, HD Audio

“X670E“ branded motherboards will require the primary PCIe and M.2 slots to support PCIe 5.0 linkrate. The chipset itself is identical to X670.

With that, here are the I/O block diagrams for a typical configuration of B650 and X670E motherboard. These represent a maximum supported configuration, as lower cost and smaller form factor board designs may expose less I/O.

Why Daisy-Chained Chipsets are a Great Idea

This Daisy-Chained chipset solution drastically lowers design costs, since ASMedia only has to design and tapeout a single chipset, while still allowing AMD to offer multiple segments of increasing motherboard I/O expansion features. When restricted to designing only a single piece of silicon to span across multiple market segments, it is much more cost-effective to design for the mass market middle-end solution and double up for the high-end rather than designing a larger, more expensive die that fits the requirements of the top end and then disabling features for the mass market segments.

There are also two more intrinsic bonuses that come with placing 2 chipsets on the motherboard. They don’t have to be placed together in the same spot. In fact, it is advantageous to place the chipsets further away from each other. By doing so, the first chipset functions as a signal repeater for the daisy-chained chipset, eliminating the need for additional signal retimers to reach places on the motherboard further from the processor with high speed PCIe 4.0 signals, thus lowering cost. Lastly, splitting the high-end chipset apart also spreads out the heat generation sources, lowering heat density and allowing for designs with passive cooling of the chipset, eliminating the need for a fan that just adds cost and another point of failure.

What’s the Catch?

No such thing as a free lunch, and that applies here as well. A daisy-chained I/O system increases the complexity of the system, with higher chances of bandwidth contention and unoptimized installation of devices, not to mention adding a second point of failure. Daisy-chained solutions also have inherently higher risks when trying to split up the hardware through Virtualization, where port assignment starts to cause headaches with nested I/O hubs. While it might just all work out fine in the end, it certainly isn’t part of best practices to plug a USB hub into another USB hub and attach mission critical devices to that. Same applies here with this chipset situation, but you can use the ports directly from the processor for that.

AMD: Pushing the Multi-Chip Game

After reintroducing multi-chip processors with their original EPYC 7001 in 2017, AMD is now doing multi-chip solutions for something as seemingly innocuous as the chipset. The cost advantage for AMD and ASMedia has clearly been worth it for them to go ahead on this unique and quirky idea!

If you enjoyed this work, do Subscribe and engage with us! We will be back soon covering the upcoming Computex event with even more juicy facts!

Subscribe now

Share

Errata (24 May): Corrected error in block diagram of B650 not showing PCIe 5.0 storage, and updated USB port naming in block diagram for consistency.

• Technical analysis on products and their features, from hardware (eg. CPU / Accelerators) to software, services and ecosystems.

• Engineering Communications detailing the complex fabrication and design processes required to continue device performance scaling

• Highlight the economics of the Semiconductor Industry moving forward into the Angstrom Era, from a global level down to financial performances of individual business units in a company, looking out for design wins and competitive advantages

To kick-off the launch of Angstronomics, we have prepared some exclusive product information reveals, followed by coverage and highlights of the upcoming Computex Taipei. Subscribe now to keep up to date as we grow this project!

Subscribe now