Je veux savoir à propos de…

6618

Federal Register / Vol. 86, No. 13 / Friday, January 22, 2021 / Notices
throughout the entire life cycle of an AI
system from development to retirement and assess those risks, as well as associated controls, on an ongoing basis. In designing and deploying AI systems, agencies should consider using relevant privacy risk management frameworks developed through open, multi-stakeholder processes.10
7. Security Agencies should consider the possibility that AI systems might be hacked, manipulated, fooled, evaded, or misled, including through manipulation of training data and exploitation of model sensitivities.
Agencies must ensure not only that their data are secure, but also that their AI systems are trained on those data in a secure manner, make forecasts based on those data in a secure way, and otherwise operate in a secure manner. Agencies should regularly consider and evaluate the safety and security of AI systems, including resilience to vulnerabilities, manipulation, and other malicious exploitation. In designing and deploying AI systems, agencies should consider using relevant government guidance or voluntary consensus standards and frameworks developed through open, multistakeholder processes.11
8. Decisional Authority Agencies should be mindful that most AI
systems will involve human beings in a range of capacitiesas operators, customers, overseers, policymakers, or interested members of the public. Human factors may sometimes undercut the value of using AI
systems to make certain determinations.
There is a risk, for example, that human operators will devolve too much responsibility to AI systems and fail to detect cases in which the AI systems yield inaccurate or unreliable determinations. That risk may be acceptable in some settings such as when the AI system has recently been shown to perform significantly better than alternativesbut unacceptable in others.
Similarly, if agency personnel come to rely reflexively on algorithmic results in exercising discretionary powers, use of an AI
system could have the practical effect of curbing the exercise of agencies discretion or shifting it from the person who is supposed to be exercising it to the systems designer.
Agencies should beware of such potential shifts of practical authority and take steps to ensure that appropriate officials have the
jbell on DSKJLSW7X2PROD with NOTICES

10 See
Natl Inst. of Standards & Tech. Special Publication SP80037 revision 2, Risk Management Framework for Information Systems and Organizations: A System Lifecycle Approach for Security and Privacy Dec. 2018; Office of Mgmt. & Budget, Exec. Off. of the President, Circular A130, Managing Information as a Strategic Resource July 28, 2016; see also Natl Inst. of Standards & Tech., NIST Privacy Framework: A Tool for Improving Privacy Through Enterprise Risk Management, Version 1.0 Jan. 16, 2020.
11 See supra note 10; see also Office of Mgmt. &
Budget, Exec. Off. of the President, M2106, Guidance for Regulation of Artificial Intelligence Applications Nov. 17, 2020; Natl Inst. for Standards & Tech., Framework for Improving Critical Infrastructure Cybersecurity Apr. 16, 2018.

VerDate Sep<11>2014

19:27 Jan 21, 2021

Jkt 253001

knowledge and power to be accountable for decisions made or aided by AI techniques.
Finally, there may be some circumstances in which, for reasons wholly apart from decisional accuracy, agencies may wish to have decisions be made without reliance on AI techniques, even if the law does not require it. In some contexts, accuracy and fairness may not be the only relevant values at stake. In making decisions about their AI
systems, agencies may wish to consider whether people will perceive the systems as unfair, inhumane, or otherwise unsatisfactory.12
9. Oversight It is essential that agencies AI systems be subject to appropriate and regular oversight throughout their lifespans. There are two general categories of oversight: External and internal. Agencies mechanisms of internal oversight will be shaped by the demands of external oversight. Agencies should be cognizant of both forms of oversight in making decisions about their AI systems.
External oversight of agencies uses of AI
systems can come from a variety of government sources, including inspectors general, externally facing ombuds, the Government Accountability Office, and Congress. In addition, because agencies uses of AI systems might lead to litigation in a number of circumstances, courts can also play an important role in external oversight.
Those affected by an agencys use of an AI
system might, for example, allege that use of the system violates their right to procedural due process.13 Or they might allege that the AI systems determination violated the Administrative Procedure Act APA because it was arbitrary and capricious.14 When an AI
system narrows the discretion of agency personnel, or fixes or alters the legal rights and obligations of people subject to the agencys action, affected people or entities might also sue on the ground that the AI
system is a legislative rule adopted in violation of the APAs requirement that legislative rules go through the notice-andcomment process.15 Agencies should consider these different forms of potential external oversight as they are making and documenting decisions and the underlying processes for these AI systems.
Agencies should also develop their own internal evaluation and oversight mechanisms for their AI systems, both for initial approval of an AI system and for 12 Cf. Admin. Conf. of the U.S., Recommendation 20183, Electronic Case Management in Federal Administrative Adjudication, 83 FR 30,686 June 29, 2018 suggesting, in the context of case management systems, that agencies consider implementing electronic systems only when they conclude that doing so would lead to benefits without impairing either the objective fairness of the proceedings or the subjective satisfaction of those participating in those proceedings.
13 Courts would analyze such challenges under the three-part balancing framework from Mathews v. Eldridge, 424 U.S. 319, 335 1976.
14 See 5 U.S.C. 7062A. Courts would likely review such challenges under the standard set forth in Motor Vehicle Manufacturers Assn v. State Farm Mutual Automobile Insurance Co., 463 U.S. 29, 43
1983.
15 See 5 U.S.C. 553bc.

PO 00000

Frm 00007

Fmt 4703

Sfmt 4703

regular oversight of the system, taking into account their system-level risk management, authorization to operate, regular monitoring responsibilities, and their broader enterprise risk management responsibilities.16
Successful internal oversight requires advance and ongoing planning and consultation with the various offices in an agency that will be affected by the agencys use of an AI system, including its legal, policy, financial, human resources, internally-facing ombuds, and technology offices. Agencies oversight plans should address how the agencies will pay for their oversight mechanisms and how they will respond to what they learn from their oversight.
Agencies should establish a protocol for regularly evaluating AI systems throughout the systems lifespans. That is particularly true if a system or the circumstances in which it is deployed are liable to change over time. In these instances, review and explanation of the systems functioning at one stage of development or use may become outdated due to changes in the systems underlying models. To enable that type of oversight, agencies should monitor and keep track of the data being used by their AI
systems, as well as how the systems use those data. Agencies may also wish to secure input from members of the public or private evaluators to improve the likelihood that they will identify defects in their AI systems.
To make their oversight systems more effective, agencies should clearly define goals for their AI systems. The relevant question for oversight purposes will often be whether the AI system outperforms alternatives, which may require agencies to benchmark their systems against the status quo or some hypothetical state of affairs.
Finally, AI systems can affect how agencies staffs do their jobs, particularly as agency personnel grow to trust and rely on the systems. In addition to evaluating and overseeing their AI systems, agencies should pay close attention to how agency personnel interact with those systems.
Administrative Conference Recommendation 20203
Agency Appellate Systems Adopted December 16, 2020
In Recommendation 20164,1 the Administrative Conference offered best practices for evidentiary hearings in administrative adjudications. Paragraph 26
recommended that agencies provide for higher-level review or agency appellate review of the decisions of hearing-level adjudicators.2 This Recommendation offers 16 See Office of Mgmt. & Budget, Circular A130, supra note 10; Office of Mgmt. & Budget, Exec.
Office of the President, Circular A123, Managements Responsibilities for Enterprise Risk Management and Internal Control July 15, 2016.
1 Admin. Conf. of the U.S., Recommendation 20164, Evidentiary Hearings Not Required by the Administrative Procedure Act, 81 FR 94,314 Dec.
23, 2016.
2 Recommendation 20164 addressed agency adjudications in which an evidentiary hearing, though not governed by the formal hearing provisions of the Administrative Procedure Act
E:FRFM22JAN1.SGM

22JAN1