The Map of AI

Source: U.S. Department of Energy, “Clean Energy Resources to Meet Data Center Electricity Demand”.

Now, in response to transformations in technologies like artificial intelligence (AI), data center expansion, new domestic manufacturing, and electrification in different sectors, the United States is returning to a period of rising electricity demand, with total energy demand potentially growing ~15-20% in the next decade.

Data center deployment, partly driven by the need to power new AI applications, is a significant factor of near-term electricity demand growth. The Electric Power Research Institute (EPRI) estimates that data centers could grow to consume up to 9% of U.S. electricity generation annually by 2030, up from 4% of total load in 2023. At a national level, data centers are critical to supporting America’s economic growth by powering businesses and enabling continued leadership in innovation, including for AI applications.

Data center electricity demand has specific characteristics. It is growing rapidly and varies regionally. Data centers can impact regional grids given the steep increases in load size, may be geographically constrained due to latency requirements, and often require firm power sources to operate continuously. Projections of data center electricity demand growth continue to evolve due to developing use cases and demand for AI and the speed of improvements in energy efficiency.

A broad suite of tools can be used to meet and manage rising electricity demand and lower overall peak demand. Approaches that span the whole power system include deploying clean generation and storage technologies; leveraging existing nuclear and hydropower infrastructure; redeveloping retired coal power plant sites; enhancing and expanding grid infrastructure; and maximizing energy efficiency and demand resources. Complementary non-technology-based solutions for managing demand growth include proactive planning, innovative tariff structures, optimizing grid performance, adopting alternative financing structures to fund new energy projects, and supply chain and workforce development. Pursuing key enablers like interconnection and regulatory reforms can unlock the barriers to adopting these energy solutions.

Energy efficiency is a key tool in reducing energy consumption from data center facilities. DOE has long been a leader in developing improved cooling technologies, including for data centers. DOE national labs have built exascale computing facilities with a Power Usage Efficiency (PUE) of 1.03, demonstrating state of the art techniques for data center efficiency. DOE is also leading the Energy Efficiency Scaling for 2 Decades initiative, with a goal to increase the energy efficiency of the microelectronics that are needed for computation at data centers by a factor of 1000 over 2 decades.

Source: National Institute of Standards and Technology, “Why NIST Is Putting Its CHIPS Into U.S. Manufacturing”.

Silicon is the most frequently used raw material for chips, and one of the most abundant atomic elements on Earth. To give you a sense of its abundance, silicon and oxygen are the main ingredients of most beach sand, and a major component of glass, rocks and soil.

Silicon is a type of material known as a semiconductor. Electricity flows through semiconductors better than it does through insulators, but not quite as well as it does through conductors.

But that’s a good thing. In semiconductors, you can control electric current precisely, and without any moving parts. By applying a small voltage to them, you can either cause current to flow or to stop, making the semiconductor, or a small region within it, act like a conductor or insulator depending on what you want to do.

The first step for making a chip is to start with a thin slice of a semiconductor material, known as a “wafer,” often round in shape. On top of the wafer, manufacturers then create complex miniature electric circuits, commonly called “integrated circuits” because they are embedded as one piece on the wafer. A typical integrated circuit today contains billions of tiny on-off switches known as transistors that enable a chip to perform a wide range of complex tasks from sending signals to processing information. Increasingly, these circuits also have “photonic” components in which light travels alongside electricity.

Manufacturers typically mass-produce dozens of integrated circuits on a single semiconductor wafer and then dice the wafer to separate the individual pieces. When each of them is packaged as a self-contained device, you have a “chip,” which can then be placed in smartphones, computers and so many other products.

When we talk about chip packaging, we’re referring to everything that goes around a chip to protect it from damage and connect it to the rest of the device. Advanced packaging takes things to the next level: It uses ingenious techniques during the chipmaking process to connect multiple chips to each other and the rest of the device in as tiny a space as possible.

Advanced packaging enables our devices to be faster and more energy-efficient because information can be exchanged between chips over shorter distances and this in turn reduces energy consumption.

Measurement science plays a key role in up to 50% of semiconductor manufacturing steps, according to a NIST report. Good measurements enable manufacturers to mass-produce high-quality, high-performance chips.

Source: U.S. Department of Energy, “DOE Releases New Report Evaluating Increase in Electricity Demand from Data Centers”.

The U.S. Department of Energy announced the publication of the 2024 Report on U.S. Data Center Energy Use produced by Lawrence Berkeley National Laboratory, which outlines the energy use of data centers from 2014 to 2028. The report estimates that data center load growth has tripled over the past decade and is projected to double or triple by 2028. U.S. electricity demand is projected to account for data center expansion and the rise of artificial intelligence applications, domestic manufacturing growth, and electrification of different industries. The Department continues to develop advanced technologies and use its resources to meet rising electricity demand in the United States while maintaining a reliable, affordable, and secure national energy system.

The report finds that data centers consumed about 4.4% of total U.S. electricity in 2023 and are expected to consume approximately 6.7 to 12% of total U.S. electricity by 2028. The report indicates that total data center electricity usage climbed from 58 TWh in 2014 to 176 TWh in 2023 and estimates an increase between 325 to 580 TWh by 2028.

DOE resources span the entire power system, from new generation and storage technologies to enhancing and expanding the transmission system to maximizing efficiency and flexibility of demand resources. DOE’s key strategies for meeting data center energy demand include enabling data center flexibility through onsite power generation and storage solutions, using energy community opportunities to re-use infrastructure at retired coal facilities for data centers and associated power infrastructure, engaging with stakeholders on innovative rate structures, and commercializing next-generation geothermal, advanced nuclear, long-duration storage, and efficient semiconductor technologies.

Data center electricity demand is growing rapidly and varies regionally. Data centers can impact regional grids given the steep increases in load size, may be geographically constrained due to latency requirements, and often require firm power sources to operate continuously. Projections of data center electricity demand growth continue to evolve due to developing use cases and demand for AI and the speed of improvements in energy efficiency.

Energy efficiency is a key tool in reducing energy consumption from data center facilities. DOE national labs have built exascale computing facilities with a Power Usage Efficiency of 1.03, demonstrating state of the art techniques for data center efficiency.

Source: National Institute of Standards and Technology, *Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile*.

This document is a cross-sectoral profile of and companion resource for the AI Risk Management Framework (AI RMF 1.0) for Generative AI, pursuant to President Biden’s Executive Order (EO) 14110 on Safe, Secure, and Trustworthy Artificial Intelligence. The AI RMF was released in January 2023, and is intended for voluntary use and to improve the ability of organizations to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems.

A profile is an implementation of the AI RMF functions, categories, and subcategories for a specific setting, application, or technology, in this case, Generative AI (GAI), based on the requirements, risk tolerance, and resources of the Framework user. AI RMF profiles assist organizations in deciding how to best manage AI risks in a manner that is well-aligned with their goals, considers legal/regulatory requirements and best practices, and reflects risk management priorities. Consistent with other AI RMF profiles, this profile offers insights into how risk can be managed across various stages of the AI lifecycle and for GAI as a technology.

As GAI covers risks of models or applications that can be used across use cases or sectors, this document is an AI RMF cross-sectoral profile. Cross-sectoral profiles can be used to govern, map, measure, and manage risks associated with activities or business processes common across sectors, such as the use of large language models (LLMs), cloud-based services, or acquisition.

This document defines risks that are novel to or exacerbated by the use of GAI. After introducing and describing these risks, the document provides a set of suggested actions to help organizations govern, map, measure, and manage these risks.

EO 14110 defines Generative AI as “the class of AI models that emulate the structure and characteristics of input data in order to generate derived synthetic content. This can include images, videos, audio, text, and other digital content.” While not all GAI is derived from foundation models, for purposes of this document, GAI generally refers to generative foundation models. The foundation model subcategory of “dual-use foundation models” is defined by EO 14110 as “an AI model that is trained on broad data; generally uses self-supervision; contains at least tens of billions of parameters; is applicable across a wide range of contexts.”

Source: National Institute of Standards and Technology, *NIST SP 800-239: Security and Trust Considerations for AI Data Centers*.

An AI data center is specifically optimized for the training and inference of AI models. This optimization is evident in its hardware infrastructure, software stacks, workflows, and data storage systems.

The Computing Zone provides key services for preprocessing the training dataset, training AI models, and providing inference and other AI services. Compute nodes in an AI data center are often equipped with accelerators to accelerate AI model training and inference. These are often used in conjunction with High-Bandwidth Memory (HBM) for local data caching, key-value (KV) caching, model parameter caching, and related operations. High-speed networks are used for both scale-up and scale-out to build a unified AI supercomputing system. For instance, NVLink can connect multiple accelerators within the same rack to scale up computing, while an InfiniBand-type high-speed network connects computing nodes across different racks to scale out computing resources. This software stack, including AI libraries, AI software development kits, and tools, enables efficient AI model training and inference.

The Data Storage Zone provides a pivotal service for storing and accessing data since AI workloads consume and generate tremendous amounts of data. Like an HPC system, an AI data center storage system is multi-tiered to meet the extreme performance, capacity, and latency requirements of modern AI training and inference.

Data storage systems typically consist of local disks, block storage, object storage, and/or file storage. Databases and high-performance parallel file systems run on top of them to provide fast data retrieval services. In addition to structured data, AI model training also consumes large amounts of unstructured data, such as video, audio, and PDF files. Cost-effective object-based storage is often adopted to store vast amounts of raw data in its native format.

An AI data center storage system often acquires data from external storage systems for training and inference purposes and may share AI models and tokens with external parties. For example, retrieval-augmented generation (RAG) is an AI inference framework that improves the accuracy of large language models (LLMs) by retrieving data from external sources, such as company documents and live databases, in real time before generating a response. In the context of agentic AI, data storage systems may also retrieve and share data with external systems.

Source: National Institute of Standards and Technology, *NIST SP 800-239: Security and Trust Considerations for AI Data Centers*.

The Access Zone in an AI data center has significantly greater responsibilities than in an HPC system. In addition to the services provided by an HPC system, the Access Zone in an AI data center hosts an AI gateway to handle incoming AI inference and application requests. The AI gateway routes those requests to the appropriate servers, which promptly generate responses. Monitoring and logging user requests and AI-generated responses are often required for security and auditing purposes. The Access Zone provides an interface for data storage systems to retrieve external datasets and/or share internal datasets. All of these functions expose the Access Zone to greater threats and risks.

The Management Zone in an AI data center operates similarly to that of an HPC cluster. However, the Management Zone in an AI data center has greater responsibility for monitoring, logging, anomaly detection, and compliance checks. This is due to more frequent and diverse user access, increased data acquisition and sharing, more capable AI models, and heightened regulatory and compliance requirements for data and AI models. As a result, extensive monitoring, logging, and compliance checks become necessary. These functions are typically managed within the Security Operations Center (SOC), which is essential for protecting high-value assets, monitoring unique AI-related attack surfaces, and ensuring regulatory compliance. Overall, the Management Zone plays a critical role in maintaining the security posture of an AI data center.

While the composition and services offered by these zones resemble those found in an HPC system, AI data centers are specifically designed to support the unique requirements of AI workloads.

Source: National Institute of Standards and Technology, “Announcing the AI Agent Standards Initiative for Interoperable and Secure Innovation”.

The Center for AI Standards and Innovation (CAISI) at NIST announced the launch of the AI Agent Standards Initiative. The Initiative will ensure that the next generation of AI, AI agents capable of autonomous actions, is widely adopted with confidence, can function securely on behalf of its users, and can interoperate smoothly across the digital ecosystem. Working in coordination with other federal partners, including the Information Technology Laboratory (ITL) at NIST, CAISI aims to foster the emerging ecosystem of industry-led AI standards and protocols.

AI agents can now work autonomously for hours, write and debug code, manage emails and calendars, and shop for goods, among other emerging use cases. While the productivity promise is enticing, the real-world utility of agents is constrained by their ability to interact with external systems and internal data. Absent confidence in the reliability of AI agents and interoperability among agents and digital resources, innovators may face a fragmented ecosystem and stunted adoption. To address this concern, NIST, including CAISI, aims to foster industry-led technical standards and protocols that build public trust in AI agents, catalyze an interoperable agent ecosystem, and diffuse their benefits to all Americans and across the world.

CAISI, with ITL at NIST, will collaborate with the National Science Foundation and other interagency partners to advance the Initiative along three pillars:

1. Facilitating industry-led development of agent standards and U.S. leadership in international standards bodies. 2. Fostering community-led open source protocol development and maintenance for agents. 3. Advancing research in areas of AI agent security and identity to enable new use cases and to promote trusted adoption across sectors of the economy.

NIST will announce research, guidelines, and further deliverables for the AI Agent Standards Initiative. To support the interoperable and secure adoption of AI agents, NIST will leverage a full toolbox for public input, including convenings, requests for information, listening sessions, and other approaches.

Source: Ram D. Sriram, National Institute of Standards and Technology, “AI in the Doctor’s Office: How Standards Can Support Trustworthiness”.

When you go to a medical appointment, does the doctor look at you while you talk? Or are they busy typing everything you say into a computer? If it’s the latter, you may find it will change soon, thanks to artificial intelligence (AI).

Some doctors’ offices are using AI transcription services to transcribe your discussion with the doctor and automatically enter the results into your electronic medical records.

That’s a time-saver for doctors, who often spend hours filling out their patients’ records. It also allows them to look at the patient rather than their computer screen.

You may have also noticed AI chatbots asking if they can help you when visiting a company’s website. These chatbots are not as common in health care yet, but it’s possible they could assist you with basic medical questions in the future. This could free up the doctor’s time for more complex concerns.

These are just two ways AI may impact your future health care. But given the high stakes, it must be done with thoughtful standards.

If AI will work in the medical field, or any other field it's used in, we need to develop specific and useful standards. These will need to include characteristics that can be used to judge an AI model on its reliability and trustworthiness.

One way AI can prove its trustworthiness is by demonstrating its correctness. If you’ve ever had a generative AI tool confidently give you the wrong answer to a question, you probably appreciate why this is important. If an AI tool says a patient has cancer, the doctor and patient need to know the odds that the AI is right or wrong.

Another issue is reliability, particularly of the datasets AI tools rely on for information. Just as a hacker can inject a virus into a computer network, someone could intentionally infect an AI dataset to make it work nefariously. In many AI systems, which use large datasets to learn, people can introduce Trojans, similar to computer viruses. This can alter the AI system’s reasoning. This can be done at the level of the input (dataset), the model (the thinking) or the AI’s environment and how it interacts with the world.

For example, researchers introduced a Trojan by placing a sticker on a stop sign. This Trojan made the self-driving car run through a stop sign because it thought it was a speed limit sign. So, there are dangers we’ll have to face if AI is unreliable. My NIST colleagues are doing considerable work to help detect Trojans, which I hope will make AI more reliable.