S 5668: Imposes liability for misleading, incorrect, contradictory or harmful information to a user by a chatbot that results in financial loss or other demonstrable harm.

  
  STATE OF NEW YORK ________________________________________________________________________ 5668 2025-2026 Regular Sessions  IN SENATE February 27, 2025 ___________ Introduced by Sen. GONZALEZ -- read twice and ordered printed, and when printed to be committed to the Committee on Internet and Technology AN ACT to amend the general business law, in relation to liability for false information provided by a chatbot The People of the State of New York, represented in Senate and Assem- bly, do enact as follows: 1 Section 1. The general business law is amended by adding a new section 2 390-f to read as follows: 3 § 390-f. Liability for chatbot responses. 1. As used in this section 4 the following terms shall have the following meanings: 5 (a) "Artificial intelligence" means a machine-based system or combina- 6 tion of systems, that for explicit and implicit objectives, infers, from 7 the input it receives, how to generate outputs such as predictions, 8 content, recommendations, or decisions that can influence physical or 9 virtual environments. 10 (b) "Chatbot" means an artificial intelligence system, software 11 program, or technological application that simulates human-like conver- 12 sation and interaction through text messages, audio, or a combination 13 thereof to provide information and services to users. 14 (c) "Companion chatbot" means a chatbot that is designed to provide 15 human-like interaction that simulates an interpersonal relationship with 16 a user or group of users as its primary function, or uses previous user 17 interactions when simulating an interpersonal relationship in future 18 interactions. An interpersonal relationship shall include, but shall not 19 be limited to, romantic, platonic, familial, adversarial, professional, 20 official, therapeutic, or stranger relationships that are between the 21 covered user and a fictional or non-fictional character or group of 22 characters. 23 (d) "Covered user" means a user of a chatbot in New York. 24 (e) "Human-like" means any form of communication or interaction that 25 approximates human behavior, including non-human behavior that could be EXPLANATION--Matter in italics (underscored) is new; matter in brackets [ ] is old law to be omitted. LBD01914-03-5 
 S. 5668 2 1 attributed to a human actor, including but not limited to, a human actor 2 role playing as a fictional non-human character, an animal, or other 3 interactive entity. 4 (f) "Minor" means an individual under the age of eighteen. 5 (g) "Proprietor" means any person, business, company, organization, 6 institution or government entity that owns, operates or deploys a chat- 7 bot used to interact with users. Proprietors shall not include third- 8 party developers that license their technology to a proprietor. 9 2. (a) A proprietor of a chatbot that is used as an alternative to a 10 human representative, or otherwise as an agent of the proprietor to 11 provide any substantive response, information, advice, or action may not 12 disclaim liability where a chatbot provides materially misleading, 13 incorrect, contradictory or harmful information to a covered user that 14 results in financial loss or other demonstrable harm to a covered user. 15 No such liability shall be imposed where the proprietor has corrected 16 the information and substantially or completely cured the harm to the 17 covered user within thirty days of notice of such harm. 18 (b) The proprietor of a chatbot shall be responsible for ensuring such 19 chatbot accurately provides information aligned with the formal poli- 20 cies, product details, disclosures and terms of service offered to 21 covered users. 22 (c) A proprietor may not waive or disclaim this liability merely by 23 notifying consumers that they are interacting with a non-human chatbot 24 system. 25 3. A proprietor of a chatbot or another person or entity that directs 26 the proprietor's chatbot to provide any substantive response, informa- 27 tion, advice or action may not disclaim liability of any kind where a 28 chatbot provides materially misleading, incorrect, contradictory or 29 harmful information to a covered user that results in bodily harm to the 30 covered user or any third party, including but not limited to any form 31 of self-harm. 32 4. Proprietors utilizing chatbots shall provide clear, conspicuous and 33 explicit notice to covered users that they are interacting with an arti- 34 ficial intelligence chatbot program rather than a human. The text of 35 the notice shall appear in the same language and in a size easily read- 36 able by the average viewer and no smaller than the largest font size of 37 other text appearing on the website on which the chatbot is utilized. 38 5. (a) A proprietor of a companion chatbot shall use commercially 39 reasonable and technically feasible methods to (i) prevent such compan- 40 ion chatbot from promoting, causing or aiding self-harm, and (ii) deter- 41 mine whether a covered user is expressing thoughts of self-harm and, 42 upon making such determination, such proprietor prohibits continued use 43 of the companion chatbot for a period of at least twenty-four hours and 44 prominently displays a means to contact a suicide crisis organization to 45 such covered user. 46 (b) Where a proprietor of a companion chatbot fails to comply with the 47 provisions of paragraph (a) of this subdivision, such proprietor shall 48 be liable to covered users who inflict self-harm upon themselves, in 49 whole or in part, as a result of such proprietor's companion chatbot 50 promoting, causing or aiding the covered user to inflict self-harm. 51 (c) Irrespective of the proprietor's compliance with paragraph (a) of 52 this subdivision, a proprietor shall be liable to covered users who 53 inflict self-harm upon themselves, in whole or in part, where such 54 proprietor: 55 (i) has actual knowledge that the companion chatbot is promoting, 56 causing or aiding self-harm; or 
 S. 5668 3 1 (ii) has actual knowledge that a covered user is expressing thoughts 2 of self-harm, fails to prohibit continued use of the companion chatbot 3 for a period of at least twenty-four hours, and fails to prominently 4 display a means to contact a suicide crisis organization to such covered 5 user. 6 (d) A proprietor of a companion chatbot may not waive or disclaim 7 liability under this subdivision. 8 6. (a) A proprietor of a companion chatbot shall use commercially 9 reasonable and technically feasible methods to determine whether a 10 covered user is a minor. 11 (b) Where such proprietor of a companion chatbot determines that a 12 covered user is a minor pursuant to paragraph (a) of this subdivision, 13 or has actual knowledge that a covered user is a minor, such proprietor 14 shall: 15 (i) cease such covered user's use of the companion chatbot until such 16 proprietor has obtained verifiable parental consent to provide a compan- 17 ion chatbot to such minor user; and 18 (ii) prohibit such covered user's continued use of the companion chat- 19 bot for a period of at least three days and prominently display a means 20 to contact a suicide crisis organization to such covered user if, using 21 commercially reasonable and technically feasible methods, such proprie- 22 tor determines that, or has actual knowledge that, a covered user is 23 expressing thoughts of self-harm. 24 (c) A proprietor shall be strictly liable for any harm caused where: 25 (i) such provider fails to comply with paragraphs (a) or (b) of this 26 subdivision; and 27 (ii) a minor covered user inflicts self-harm upon themselves, in whole 28 or in part, as a result of such proprietor's companion chatbot. 29 (d) A proprietor of a companion chatbot may not waive or disclaim 30 liability under this subdivision. 31 7. A proprietor of a companion chatbot shall implement and engage in 32 the ongoing implementation of commercially reasonable and technically 33 feasible methods to discover vulnerabilities in the proprietor's system, 34 including any methods used to determine whether a covered user is a 35 minor. 36 8. (a) The attorney general shall promulgate regulations identifying 37 commercially reasonable and technically feasible methods for proprietors 38 of companion chatbots required under this section. 39 (b) In promulgating regulations related to the commercially reasonable 40 and technically feasible methods for proprietors of companion chatbots 41 to comply with this section, the attorney general shall consider the 42 size, financial resources, and technical capabilities of the proprietor, 43 the costs and effectiveness of available (i) age determination tech- 44 niques for users of companion chatbots, (ii) techniques to prevent the 45 promotion, aid, or encouragement of self-harm, (iii) techniques to 46 determine whether a user is expressing thoughts of self-harm, and (iv) 47 techniques to discover vulnerabilities in the proprietor's system. The 48 attorney general shall also consider that prevalent practices of the 49 industry of the proprietor and the impact of the techniques listed in 50 subparagraphs (i) through (iv) of this paragraph on the user's safety, 51 utility, and experience. 52 (c) Such regulations shall also identify the appropriate levels of 53 accuracy that would be commercially reasonable and technically feasible 54 for proprietors to achieve in determining (i) whether a user is a minor, 55 (ii) whether the proprietor's companion chatbot is promoting, aiding, or 
 S. 5668 4 1 encouraging self-harm, and (iii) whether a user is expressing thoughts 2 of self-harm. 3 9. Information collected for the purpose of determining a user's age 4 under paragraph (a) of subdivision five of this section shall not be 5 used for any purposes other than age determination and shall be deleted 6 immediately after an attempt to determine a user's age, except where 7 necessary for compliance with any applicable provisions of New York 8 state or federal law or regulation. 9 10. The attorney general shall promulgate regulations identifying 10 methods of obtaining verifiable parental consent pursuant to subpara- 11 graph (i) of paragraph (b) of subdivision six of this section. 12 11. Information collected for the purpose of obtaining verifiable 13 parental consent shall not be used for any purpose other than obtaining 14 such verifiable parental consent and shall be deleted immediately after 15 an attempt to obtain verifiable parental consent, except where necessary 16 for compliance with any applicable provisions of New York state or 17 federal law or regulation. 18 12. Nothing in this section shall be construed as requiring any 19 proprietor to give a parent who grants verifiable parental consent any 20 additional or special access to or control over the data or accounts of 21 their child. 22 § 2. This act shall take effect one year after it shall have become a 23 law.