S 7599: Regulates automated decision-making by government agencies; requires agencies to conduct impact assessments; requires disclosure of automated decision-making tools utilized by governmental agencies.

  
  STATE OF NEW YORK ________________________________________________________________________ 7599 2025-2026 Regular Sessions  IN SENATE April 23, 2025 ___________ Introduced by Sen. GONZALEZ -- read twice and ordered printed, and when printed to be committed to the Committee on Internet and Technology AN ACT to amend the state technology law, in relation to automated deci- sion-making by government agencies The People of the State of New York, represented in Senate and Assem- bly, do enact as follows: 1 Section 1. The state technology law is amended by adding a new arti- 2 cle 5 to read as follows: 3 ARTICLE V 4 AUTOMATED DECISION-MAKING IN GOVERNMENT AGENCIES 5 Section 501. Definitions. 6 502. Use of automated decision-making systems by government 7 agencies. 8 503. Impact assessments. 9 504. Submission to the governor and legislature. 10 § 501. Definitions. For the purpose of this article: 11 1. "Automated decision-making system" shall mean any software that 12 uses algorithms, computational models, or artificial intelligence tech- 13 niques, or a combination thereof, to automate, support, or replace human 14 decision-making and shall include, without limitation, systems that 15 process data, and apply predefined rules or machine learning algorithms 16 to analyze such data, and generate conclusions, recommendations, 17 outcomes, assumptions, projections, or predictions without meaningful 18 human discretion. "Automated decision-making system" shall not include 19 any software used primarily for basic computerized processes, such as 20 calculators, spellcheck tools, autocorrect functions, spreadsheets, 21 electronic communications, or any tool that relates only to internal 22 management affairs such as ordering office supplies or processing 23 payments, and that do not materially affect the rights, liberties, bene- 24 fits, safety or welfare of any individual within the state. EXPLANATION--Matter in italics (underscored) is new; matter in brackets [ ] is old law to be omitted. LBD11535-02-5 
 S. 7599 2 1 2. "Meaningful human review" means review, oversight and control of 2 the automated decision-making process by one or more individuals who 3 understand the risks, limitations, and functionality of, and are trained 4 to use, the automated decision-making system and who have the authority 5 to intervene or alter the decision under review, including but not 6 limited to the ability to approve, deny, or modify any decision recom- 7 mended or made by the automated system. 8 3. "Government agency" shall mean: (a) the state or civil division 9 thereof; (b) a county, city, town or village; (c) a school district, 10 board of cooperative educational services, vocational education and 11 extension board or a school district as enumerated in section one of 12 chapter five hundred sixty-six of the laws of nineteen hundred sixty- 13 seven, as amended; (d) the state university of New York; (e) the city 14 university of New York; (f) a public improvement or special district 15 including police or fire districts; (g) a public authority, commission 16 or public benefit corporation; or (h) any other public corporation, 17 agency, instrumentality or unit of government which exercises govern- 18 mental power under the laws of this state. 19 4. "Public assistance benefit" shall mean any service or program with- 20 in the control of the state, or benefit provided by the state to indi- 21 viduals or households, including but not limited to public assistance, 22 cash assistance, grants, child care assistance, housing assistance, 23 unemployment benefits, transportation benefits, education assistance, 24 domestic violence services, and any other assistance or benefit within 25 the authority of the state to grant to individuals within the state. 26 This shall not include any federal program that is administered by the 27 federal government or the state. 28 § 502. Use of automated decision-making systems by government agen- 29 cies. 1. No government agency, or any entity acting on behalf of such 30 agency, which utilizes or applies any automated decision-making system, 31 directly or indirectly, in performing any function that: (a) is related 32 to the delivery of any public assistance benefit; (b) will have a mate- 33 rial impact on the rights, civil liberties, safety or welfare of any 34 individual within the state; or (c) affects any statutorily or constitu- 35 tionally provided right of an individual, shall utilize such automated 36 decision-making system, unless such automated decision-making system is 37 subject to continued and operational meaningful human review. 38 2. No government agency shall authorize any procurement, purchase or 39 acquisition of any service or system utilizing, or relying on, automated 40 decision-making systems in performing any function that is: (a) related 41 to the delivery of any public assistance benefit; (b) will have a mate- 42 rial impact on the rights, civil liberties, safety or welfare of any 43 individual within the state; or (c) affects any statutorily or constitu- 44 tionally provided right of an individual unless such automated deci- 45 sion-making system is subject to continued and operational meaningful 46 human review. 47 § 503. Impact assessments. 1. Government agencies seeking to utilize 48 or apply an automated decision-making system permitted under section 49 five hundred two of this article with continued and operational meaning- 50 ful human review shall conduct or have conducted an impact assessment 51 substantially completed and bearing the signature of one or more indi- 52 viduals responsible for meaningful human review for the lawful applica- 53 tion and use of such automated decision-making system. Following the 54 first impact assessment, an impact assessment shall be conducted in 55 accordance with this section at least once every two years. An impact 56 assessment shall be conducted prior to any material change to the auto- 
 S. 7599 3 1 mated decision-making system that may change the outcome or effect of 2 such system. Such impact assessments shall include: 3 (a) a description of the objectives of the automated decision-making 4 system; 5 (b) an evaluation of the ability of the automated decision-making 6 system to achieve its stated objectives; 7 (c) a description and evaluation of the objectives and development of 8 the automated decision-making including: 9 (i) a summary of the underlying algorithms, computational modes, and 10 artificial intelligence tools that are used within the automated deci- 11 sion-making system; and 12 (ii) the design and training data used to develop the automated deci- 13 sion-making system process; 14 (d) testing for: 15 (i) accuracy, fairness, bias and discrimination, and an assessment of 16 whether the use of the automated decision-making system produces discri- 17 minatory results on the basis of a consumer's or a class of consumers' 18 actual or perceived race, color, ethnicity, religion, national origin, 19 sex, gender, gender identity, sexual orientation, familial status, biom- 20 etric information, lawful source of income, or disability and outlines 21 mitigations for any identified performance differences in outcomes 22 across relevant groups impacted by such use; 23 (ii) any cybersecurity vulnerabilities and privacy risks resulting 24 from the deployment and use of the automated decision-making system, and 25 the development or existence of safeguards to mitigate the risks; 26 (iii) any public health or safety risks resulting from the deployment 27 and use of the automated decision-making system; 28 (iv) any reasonably foreseeable misuse of the automated decision-mak- 29 ing system and the development or existence of safeguards against such 30 misuse; 31 (e) the extent to which the deployment and use of the automated deci- 32 sion-making system requires input of sensitive and personal data, how 33 that data is used and stored, and any control users may have over their 34 data; and 35 (f) the notification mechanism or procedure, if any, by which individ- 36 uals impacted by the utilization of the automated decision-making system 37 may be notified of the use of such automated decision-making system and 38 of the individual's personal data, and informed of their rights and 39 options relating to such use. 40 2. Notwithstanding the provisions of this article or any other law, if 41 an impact assessment finds that the automated decision-making system 42 produces discriminatory or biased outcomes, the government agency shall 43 cease any utilization, application, or function of such automated deci- 44 sion-making system, and of any information produced using such system. 45 § 504. Submission to the governor and legislature. 1. Each impact 46 assessment conducted pursuant to this article shall be submitted to the 47 governor, the temporary president of the senate, and the speaker of the 48 assembly at least thirty days prior to the implementation of the auto- 49 mated decision-making system that is the subject of such assessment. 50 2. (a) The impact assessment of an automated decision-making system 51 shall be published on the website of the relevant government agency. 52 (b) If the government agency makes a determination that the disclosure 53 of any information required in the impact assessment would result in a 54 substantial negative impact on health or safety of the public, infringe 55 upon the privacy rights of individuals, or significantly impair the 56 government agency's ability to protect its information technology or 
 S. 7599 4 1 operational assets, such government agency may redact such information, 2 provided that an explanatory statement on the process by which the 3 government agency made such determination is published along with the 4 redacted impact assessment. 5 (c) If the impact assessment covers any automated decision-making 6 system that includes technology that is used to prevent, detect, protect 7 against or respond to security incidents, identity theft, fraud, 8 harassment, malicious or deceptive activities or other illegal activity, 9 preserve the integrity or security of systems, or to investigate, 10 report or prosecute those responsible for any such malicious or decep- 11 tive action, such government agency may redact such information for the 12 purposes of this subdivision, provided that an explanatory statement on 13 the process by which the government agency made such determination is 14 published along with the redacted impact assessment. 15 § 2. Disclosure of existing automated decision-making systems. Any 16 government agency, that directly or indirectly, utilizes an automated 17 decision-making system, as defined in section 501 of the state technolo- 18 gy law, shall submit to the legislature a disclosure on the use of such 19 system, no later than one year after the effective date of this section. 20 Such disclosure shall include: 21 (a) a description of the automated decision-making system utilized by 22 such agency; 23 (b) a list of any software vendors related to such automated deci- 24 sion-making system; 25 (c) the date that the use of such system began; 26 (d) a summary of the purpose and use of such system, including a 27 description of human decision-making and discretion supported or 28 replaced by the automated decision-making system; 29 (e) whether any impact assessments for the automated decision-making 30 system were conducted and the dates and summaries of the results of such 31 assessments where applicable; and 32 (f) any other information deemed relevant by the agency. 33 § 3. This act shall take effect immediately, provided that section one 34 of this act shall take effect one year after it shall have become a law.