[转帖]Introducing Exadata X10M: Extreme Scalability and Dramatically Improved Price Performance

https://blogs.oracle.com/database/post/exadata-x10m

Oracle is excited to announce the next generation of Exadata Database Machine platform, the Oracle Exadata X10M. Built on more than a decade of high-performance database hardware and software innovations, Exadata X10M brings extreme scale and better price performance to the ideal platform for running Oracle Database.

Overview

Exadata X10M delivers unrivaled performance, availability, and scalability for all database workloads. Starting at the same price as the previous generation, Exadata X10M also provides significantly more capacity and dramatically greater value than previous generations. Today, thousands of large and small enterprises run their most critical and demanding workloads on Exadata, including the majority of the largest financial, telecom, and retail businesses.

The latest Exadata generation features fourth-generation AMD EPYC™  processors and delivers dramatic performance improvements with up to 3x higher transaction throughput and up to 3.6x faster analytic queries on database servers. With X10M, High Capacity storage servers can now hold 22% more data, while all-flash Extreme Flash Storage Servers now offer 2.4x the capacity of the previous systems. Database servers support 50% higher memory capacity enabling more databases and larger memory-intensive workloads to run on the same system. The combination of improved performance and more storage and memory capacity enables greater levels of database consolidation and dramatically lower costs for all database workloads, with the price of Exadata X10M High Capacity systems remaining the same as the previous generation. The greater compute and storage density offered by Exadata X10M platforms also reduces the size of the systems customers require to meet their needs, lowering data center costs for power, cooling, and floor space, and improving data center sustainability.

Exadata X10M

Exadata X10M is built using the latest processors, networking, and storage technologies and includes numerous optimizations to the system hardware, Exadata system software, and Oracle Database software to deliver unparalleled performance and value for online transaction processing (OLTP), analytics, machine learning (ML), and mixed workloads. Exadata’s scale-out architecture, combined with the impressive scale-up capabilities provided by fourth-generation AMD EPYC™ processors that power Exadata X10M, enables organizations to meet their sophisticated data management needs in public cloud, hybrid cloud, or on-premises deployments.

Exadata X10M platforms for deployment in an organization’s data centers include the Exadata Database Machine X10M and the Exadata Cloud@Customer X10M. Exadata Cloud@Customer uses the same architecture, software, and infrastructure as the Exadata Database Machine to enable customers to run Oracle Cloud Infrastructure’s (OCI) automated Oracle Exadata Database Service and fully managed Oracle Autonomous Database in their data centers so they can benefit from OCI automation and economics. Exadata Database Service and Autonomous Database also run on AMD EPYC™ processor based Exadata Cloud Infrastructure in OCI.

Packed with tremendous OLTP and analytics performance and a substantial increase in consolidation capabilities, an Exadata X10M quarter rack still has the same price as an Exadata X9M - an amazing value for an incredible system. The enhancements found in Exadata X10M dramatically reduce infrastructure spending, lower system and database administration, and cut energy consumption compared to Exadata X9M platforms.

High-performance OLTP and modern workloads such as IoT, blockchain, gaming, financial trading, and eCommerce require the world’s fastest OLTP Database machine, which just got even faster. Exadata X10M database servers each have an awesome 3X the number of cores of the Exadata X9M database servers and a near meteoric 1 TB/sec of memory bandwidth. With each database server core in Exadata X10M running OLTP workloads faster than their equivalents in the prior generation, customers can achieve up to 3X the total OLTP throughput.

Exadata X10M storage servers each boast 2.8 million 8k read I/Os per second, an impressive 21% increase over X9M! Utilizing DDR5 DRAM in intelligent storage servers, Exadata RDMA Memory (XRMEM) read latency drops from 19 microseconds to an astonishing 17 microseconds with Exadata X10M. As with all Exadata performance metrics, these are real-world, end-to-end database workload numbers, not low-level I/O metrics using small-sized test workloads typically used by storage vendors.

Analytics workloads enjoy a healthy performance boost as well. Exadata X10M increases the number of processing cores in each storage server to 64, enabling greater parallelism for analytics queries. Smart Scan throughput offloads SQL processing to the Exadata intelligent storage servers, topping the 1 TB (Terabyte) per second mark in a single rack. 

Database consolidation on Exadata reduces the complexity and overhead of running more and larger databases. The immense increase in the number of cores and memory in Exadata database servers, flash and hard drive capacity, and SQL processing cores in Exadata storage servers means you can run more Oracle databases on less Exadata infrastructure than ever. Higher consolidation densities reduce CAPEX and administration and improve energy efficiency so you can run your Oracle Database fleet more efficiently.

The remainder of this blog details many astounding advances with Exadata X10M. We also recommend you read the Exadata Cloud@Customer X10M blog and the family of Oracle Exadata web pages for more information.

Exadata Hardware

Exadata X10M Database Server

Exadata X10M database servers use two 96-core 4th-generation AMD CPUs, delivering a stunning 3X increase in cores and greater per-core performance than the Exadata X9M generation. Memory size in Exadata X10M has increased to meet customer demands for increased database consolidation and to support increased CPU core density.

Memory configurations of 512 GB, 1.5 TB, 2.25 TB, and 3 TB DDR5 DRAM per database server are now available. Memory bandwidth in Exadata X10M database servers is 2.5X that of Exadata X9M, greatly accelerating memory bandwidth-sensitive operations.

Local storage on the database servers continues to utilize two 3.84 TB NVMe SSD drives, expandable to four 3.84 TB drives, which are all hot-swappable.

Exadata X10M replaces the PCIe 4 data interface for the network cards with PCIe 5 and adopts the latest generation of dual-port RDMA over Converged Ethernet (RoCE) network cards to increase throughput to the private network. In addition, X10M supports up to five client network interface cards for more flexible network connectivity.

Exadata X10M High Capacity Storage Server

Each Exadata X10M High Capacity Storage Server uses two 32-core 4th-generation AMD CPUs, doubling the number of storage server cores in Exadata Database Machine and increasing the core count from 24 to 32 for Exadata Cloud@Customer X10M when compared to X9M. Memory gets a massive upgrade in the storage tier to 1.5 TB DD5 DRAM, up from 256 GB in the X9M, to accommodate Exadata RDMA Memory (XRMEM). 

Like the Exadata Database Server, memory bandwidth in storage servers increases by up to 2.5X. Each storage server includes four performance-optimized Flash cards, each increasing in size to 6.8 TB from the previous generation’s 6.4 TB. The RoCE Network Fabric moves to the same dual-port RoCE network cards used in the database server. The 12 hard disk drives increase in capacity from 18 TB to 22 TB each, bringing the total raw capacity per storage server from 216 TB to 264 TB, up a healthy 22% from X9M.

Exadata X10M Extreme Flash Storage Server

All new with Exadata X10M, the Extreme Flash Storage Servers add a new storage tier with four capacity-optimized 30.72 TB flash drives for an aggregate, raw storage capacity of 122.88 TB, an impressive increase of 2.4X over X9M’s 51.2 TB.

As with the X10M High Capacity Storage Server, four 6.8 TB performance-optimized Flash cards are dedicated to the Flash Cache. Extreme Flash Storage Servers align with the High Capacity Storage Server, using the latest 32-core 4th-generation AMD CPUs, memory upgrades, and dual-port RoCE network cards.

Exadata X10M Extended Storage Server

The Exadata X10M Extended Storage Servers use the latest 32-core 4th-generation AMD CPU. Memory increases from 96 GB to 128 GB. Like the Database and other Storage Servers, the RoCE Network Fabric utilizes dual-port RoCE network cards. The 12 hard disk drives increase in capacity to 22 TB each, with a total capacity of 264 TB per storage server, up from 18 TB each and 216 TB per storage server in X9M, a 22% increase.

Exadata X10M Storage Expansion Rack

Exadata Storage Expansion Rack X10M starting configurations support the latest Exadata X10M High Capacity and Extreme Flash Storage Servers. Additional High Capacity, Extreme Flash, and Extended Storage Servers can be added to existing X8M, X9M, and X10M Storage Expansion Racks, ensuring customers can scale their storage needs to meet the most demanding database deployments. The Exadata X10M Extended Storage Servers use the latest 32-core 4th-generation AMD CPU. Memory increases from 96 GB to 128 GB. Like the Database and other Storage Servers, the RoCE Network Fabric utilizes dual-port RoCE network cards. The 12 hard disk drives increase in capacity to 22 TB each, with a total capacity of 264 TB per storage server, up from 18 TB each and 216 TB per storage server in X9M, a 22% increase.

Scalable Licensing

Capacity On Demand (CoD) has been available since Exadata X4 and continues on Exadata X10M to enable customers to reduce the required database software licenses. At installation time, a minimum of 14 cores must be enabled on each X10M database server. Oracle Exadata Deployment Assistant (OEDA) implements CoD during the installation process. You can then increase the active core count over time as the demand for compute resources on the database servers increases.

Exadata System Software 23.1

Oracle announced Exadata System Software 23.1 in March 2023. In addition to supporting X10M hardware, 23.1 adds several features to Exadata, including an operating system upgrade from Oracle Linux 7 and UEK 5 to Oracle Linux 8 and UEK 6. Exadata System Software continues to take the Exadata Database Machine to new levels with the following additional new features:

Oracle Linux 8 and UEK6

The core of the 23.1 release is the upgrade to Oracle Linux 8.7 and UEK6, which ensures platform maturity and strong ecosystem compatibility.

New compression algorithm for Database In-Memory Columnar Caching

Beginning with X10M, customers using Database In-Memory Columnar Caching in Storage Servers gain up to a 25% increase in the effective capacity by virtue of a new compression algorithm. This optimization increases the capacity and efficacy of In-Memory Columnar formatted data in the storage servers.

KVM Update with X10M

With the increase in max memory on the X10M Database servers comes an increase in memory available to virtual machines. The maximum amount of memory that can be allocated to virtual machines is 2.73 TB per database server on a 3 TB database server configuration. So, a single virtual machine or multiple virtual machines can utilize a combined footprint of up to 2.73 TB per database server. Remember, you cannot over-provision physical memory; the minimum is still 16 GB for each virtual machine. Exadata X10M continues to allow up to 2X virtual CPU (vCPU) over-provisioning on configurations with more than 512 GB memory and where Capacity-On-Demand is inactive. With two 96-core CPUs (two vCPUs per core) in Exadata X10M, you get a mind-blowing 760 vCPUs (four vCPUs reserved for the host) available per database server. 

Read the Exadata KVM Best Practices document for all the specifics.

Centralized Identification and Authentication of OS Users

The database and storage servers have added support for LDAP Identification and Kerberos Authentication. This improves the overall security posture of Exadata and, as the name suggests, centralizes identification and authentication functions of administrator accounts, standardizes password complexity, aging, and reset procedures, and improves the on and off-boarding experience.

Introduction of Exadata RDMA Memory (XRMEM)

Exadata RDMA Memory (XRMEM) in the Exadata X10M storage servers is leveraged as a shared read accelerator. The XRMEM Data Accelerator is a memory cache tier in front of Flash Cache, enabling orders of magnitude lower latency accessing remotely stored data. By utilizing RDMA to access memory remotely, XRMEM Data Accelerator bypasses the network and I/O stack, eliminating expensive CPU interrupts and context switches and reducing latency by more than 10X, from 200 microseconds to less than 17 microseconds. Smart Exadata System Software also ensures data is mirrored across storage servers for fault tolerance. Exadata’s unique end-to-end integration between Oracle Database and Exadata storage servers automatically caches the hottest data blocks efficiently between the buffer cache in database servers and XRMEM and Flash Cache in storage servers. XRMEM is a shared storage tier across all of the storage nodes, which means the aggregate performance of this cache can be dynamically used by any database instance on any database server. XRMEM is a significant advantage over general-purpose storage architectures, which preclude sharing memory and storage resources across database instances.

Supports Grid Infrastructure 19c and newer

Exadata System Software supports Oracle Grid Infrastructure 19.15 and newer and Oracle Database version 11g through 19c. My Oracle Support note 888828.1 details all the software version requirements.

Support for Exadata X10M Hardware

Exadata System Software 23.1, including Oracle Linux 8 and UEK 6, is the foundation for the Exadata X10M family and is fundamental to the extreme scalability of this generation of Exadata systems.

Summary

With each new generation of hardware and software, our goal is to push the envelope in the three pillars of our vision statement for Exadata:

• Provide the Ideal Database Hardware, optimized hardware that scales out by storage, compute, and network specifically for database workloads
• Build database-aware system software that takes full advantage of this hardware, providing unique algorithms and features that vastly improve performance, availability, and scalability for every kind of database workload, whether it be OLTP, analytics, consolidation, AI/ML, IoT, blockchain, gaming, and high-speed financial systems
• Provide a fully automated and optimized design, end-to-end, with easy management and monitoring across the entire software stack

Exadata X10M delivers stunning increases in CPU cores, memory speeds and capacity, flash capacity and performance, and disk capacity, starting from the same price as the previous generation. Now, more so than ever, customers can accomplish more tasks using fewer infrastructure resources in less time. With all these incredible innovations, Oracle Exadata X10M is a step change in scalability and performance and offers unparalleled cost-effectiveness to customers.

For more information, visit the Exadata Database Machine webpage and read the Exadata Database Machine System Overview document.