S. 1633

119th CONGRESS 1st Session

To require the Director of the National Institute of Standards and Technology to establish a pilot program that uses testbeds to develop measurement standards for the evaluation of artificial intelligence systems, and for other purposes.

IN THE SENATE OF THE UNITED STATES · May 7, 2025 · Sponsor: Mr. Luján · Committee: Committee on Commerce, Science, and Transportation

Table of contents

• S. 1633
  • SEC. 1. Short title
  • SEC. 2. Definitions
  • SEC. 3. Pilot program on testbeds for development of measurement standards for evaluation of artificial intelligence systems

SEC. 1. Short title

• This Act may be cited as the or the .

SEC. 2. Definitions

• In this Act:
  • The term artificial intelligence system,artificial intelligence has the meaning given the term in section 5002 of the National Artificial Intelligence Act of 2020 (). 15 U.S.C. 9401
  • The term covered foreign country means—
  • The term Director means the Director of the National Institute of Standards and Technology.
  • The term Institute means the National Institute of Standards and Technology.
  • The term testbed means a facility or mechanism equipped for conducting rigorous, transparent, and replicable testing of tools or technologies to evaluate the functionality, performance, and security of those tools or technologies.
  • The term Working Group means the working group established under section 3(c)(1).

SEC. 3. Pilot program on testbeds for development of measurement standards for evaluation of artificial intelligence systems

• (a) Pilot program required
  • (1) In general
    • The Director shall, in coordination with the Secretary of Energy and after consultation with the Working Group, carry out a pilot program of testbeds to assess the feasibility and advisability of developing measurement standards for the evaluation of artificial intelligence systems used by Federal agencies through the use of testbeds.
  • (2) Process
    • The Director shall carry out the pilot program required by paragraph (1) by building and demonstrating an iterative process whereby measurement science is advanced and standards are developed using testbeds in accordance with subsection (d), and the results of the tests are reviewed by stakeholders from government, the private sector, and academia.
• (b) Memorandum of Understanding
  • Memorandum of Understanding
  • (1) In general
    • Not later than 180 days after the date of the enactment of this Act, the Secretary of Commerce and the Secretary of Energy shall enter into a memorandum of understanding to implement the coordination between the Secretary of Energy and the Director required by paragraph (2).
  • (2) Requirements
    • The memorandum shall be sufficient to ensure the Institute and other Federal agencies have access, as may be necessary and as determined by the Secretary of Energy, to the resources, personnel, and facilities at the Department of Energy, including the cross-cutting research and development programs to support the activities of the pilot program require by subsection (a)(1).
  • (3) Renegotiations
    • Not later than 2 years after the establishment of the pilot program required by subsection (a)(1), and every 2 years thereafter, Secretary of Commerce and the Secretary of Energy shall reassess the memorandum of understanding to ensure that it meets the needs of the agencies and adequately supports the activities of the pilot program.
• (c) Artificial Intelligence Testing Working Group
  • (1) Establishment
    • Establishment
    • (A) In general
      • Not later than 90 days after the memorandum of understanding required by subsection (b)(1) is signed, the Secretary of Commerce shall establish a working group to provide advice to the Secretary of Commerce and the Secretary of Energy on the development of measurement standards for the evaluation of artificial intelligence systems under the pilot program required by subsection (a).
    • (B) Designation
      • The working group established under subparagraph (A) shall be known as the . Artificial Intelligence Testing Working Group
  • (2) Membership
    • (A) In general
      • The Working Group shall be comprised of not more than 10 members that include the following:
        • (i) The Secretary of Commerce or a designee of the Secretary of Commerce from the Department of Commerce.
        • (ii) The Secretary of Energy or a designee of the Secretary of Energy from the Department of Energy.
        • (iii) The Director or a designee of the Director from the Institute.
        • (iv) Representatives from the private sector, as the members described in clauses (i) through (iii) consider appropriate.
        • (v) Representatives from academic institutions, as the members described in clauses (i) through (iii) consider appropriate.
    • (B) Prohibition
      • No citizen of a covered foreign country may serve as a member of the Working Group.
  • (3) Strategy for measurement standards
    • (A) In general
      • Not later than 1 year after the date on which the Working Group is established under paragraph (1), the Working Group shall develop a strategy for the development of measurement standards for the evaluation of artificial intelligence systems.
    • (B) Elements
      • The strategy required by subparagraph (A) shall—
        • (i) identify measurement standards that the members of the Working Group consider necessary, including standards with respect to an artificial intelligence system for—
        • reliability;
        • performance;
        • capability;
        • interpretability;
        • security, including the propensity of the system for leaking data;
        • privacy;
        • data bias; and
        • bounds for applicability and use cases;
        • (ii) include a proposed blueprint for developing the measurement standards;
        • (iii) identify the applications or artificial intelligence systems to which the standards should initially apply; and
        • (iv) identify metrics for the assessment of the pilot program.
    • (C) Publication
• (d) Development of Testbeds for measurement standards
  • (1) In general
    • Not later than 2 years after the date of the enactment of this Act, the Director shall, in collaboration with the Secretary of Energy and in coordination with the Working Group, develop, in accordance with the strategy developed under subsection (c)(3)(A), testbeds on which to demonstrate the development of measurement standards for the evaluation of artificial intelligence systems to be used by Federal agencies.
  • (2) Experts
    • In carrying out paragraph (1), the Director may hire any expert the Director considers necessary, including experts from academia, industry, and standards development organizations.
• (e) Report
  • Not later than 180 days after the date on which the Director completes the first testbed demonstration and development of measurement standards under subsection (d)(1), the Director shall submit to Congress a report that—
    • reviews the findings of the Director with respect to the initial demonstration carried out under this section;
    • recommends revisions to the pilot program plan, including by identifying any resources, funding, or personnel that is needed to improve and continue the development of testbeds and measurement standards for the evaluation of artificial intelligence systems for multiple applications; and
    • makes recommendations for legislative or administrative action to advance the development of measurement standards for the evaluation of artificial intelligence systems.

• The Secretary of Commerce shall publish on a publicly accessible website of the Department of Commerce the strategy for measurement standards required by subparagraph (A) to provide the information contained in the strategy to stakeholders, including code and model developers, researchers, and industrial users of artificial intelligence systems.