A 6767: Prohibits the provision of an artificial intelligence companion to a user unless such artificial intelligence companion contains a protocol for addressing possible suicidal ideation or self-harm expressed by a user, possible physical harm to others expressed by a user, and possible financial harm to others expressed by a user; requires certain notifications to certain users regarding crisis service providers and the non-human nature of such companion models.

  
  STATE OF NEW YORK ________________________________________________________________________ 6767 2025-2026 Regular Sessions  IN ASSEMBLY March 13, 2025 ___________ Introduced by M. of A. VANEL -- read once and referred to the Committee on Consumer Affairs and Protection AN ACT to amend the general business law, in relation to artificial intelligence companion models (Part __); The People of the State of New York, represented in Senate and Assem- bly, do enact as follows: 1 PART __ 2 Section 1. The general business law is amended by adding a new article 3 47 to read as follows: 4 ARTICLE 47 5 ARTIFICIAL INTELLIGENCE COMPANION MODELS 6 Section. 1700. Definitions. 7 1701. Prohibitions and requirements. 8 1702. Notifications. 9 1703. Enforcement. 10 1704. Severability. 11 § 1700. Definitions. As used in this article, the following terms 12 shall have the following meanings: 13 1. "Artificial intelligence", "artificial intelligence technology", or 14 "AI" means a machine-based system that can, for a given set of human-de- 15 fined objectives, make predictions, recommendations, or decisions influ- 16 encing real or virtual environments, and that uses machine- and human- 17 based inputs to perceive real and virtual environments, abstract such 18 perceptions into models through analysis in an automated manner, and use 19 model inference to formulate options for information or action. 20 2. "Generative artificial intelligence" means a class of AI models 21 that are self-supervised and emulate the structure and characteristics 22 of input data to generate derived synthetic content, including, but not 23 limited to, images, videos, audio, text, and other digital content. EXPLANATION--Matter in italics (underscored) is new; matter in brackets [ ] is old law to be omitted. LBD10829-01-5 
 A. 6767 2 1 3. "AI model" means a component of an information system that imple- 2 ments artificial intelligence technology and uses computational, statis- 3 tical, or machine-learning techniques to produce outputs from a given 4 set of inputs. 5 4. "AI companion" means a system using artificial intelligence, gener- 6 ative artificial intelligence, and/or emotional recognition algorithms 7 to simulate social human interaction, by retaining information on prior 8 interactions and user preference, asking questions, providing advice, 9 and engaging in simulated conversation on matters of personal well-be- 10 ing. "AI companion" shall not include any system used by a business 11 entity solely intended to provide users with information about available 12 commercial services or products, customer account information, or other 13 information related to a user's customer, or potential customer, 14 relationship with such business entity. 15 5. "Operator" means any person, partnership, association, firm, or 16 business entity, or any member, affiliate, subsidiary or beneficial 17 owner of any partnership, association, firm, or business entity who 18 operates or provides an AI companion. 19 6. "Person" means any natural person. 20 7. "Emotional recognition algorithms" means artificial intelligence 21 that detects and interprets human emotional signals in text (using 22 natural language processing and sentiment analysis), audio (using voice 23 emotion AI), video (using facial movement analysis, gait analysis, or 24 physiological signals), or a combination thereof. 25 8. "User" means any person who uses an AI companion within the state 26 and who is not an operator or agent or affiliate of the operator of the 27 AI companion. 28 § 1701. Prohibitions and requirements. It shall be unlawful for any 29 operator to operate or provide an AI companion to a user unless such AI 30 companion contains a protocol for addressing: 1. possible suicidal idea- 31 tion or self-harm expressed by a user to the AI companion, 2. possible 32 physical harm to others expressed by a user to the AI companion, and 3. 33 possible financial harm to others expressed by the user to the AI 34 companion, that includes but is not limited to, a notification to the 35 user that refers them to crisis service providers such as a suicide 36 hotline, crisis text line, or other appropriate crisis services. 37 § 1702. Notifications. An operator shall provide a notification to a 38 user at the beginning of any AI companion interaction and at least every 39 three hours for continuing AI companion interactions thereafter, which 40 states either verbally or in bold and capitalized letters of at least 41 sixteen point type, the following: 42 "THE AI COMPANION (OR NAME OF THE AI COMPANION) IS A COMPUTER PROGRAM 43 AND NOT A HUMAN BEING. IT IS UNABLE TO FEEL HUMAN EMOTION". 44 § 1703. Enforcement. Any person who was physically injured through 45 self-harm or was physically or financially harmed by another as a result 46 of a violation of section seventeen hundred one or seventeen hundred two 47 of this article may bring an action in a court of competent jurisdiction 48 for damages, equitable relief, and such other remedies as the court may 49 deem appropriate. 50 § 1704. Severability. If any clause, sentence, paragraph, subdivision, 51 section or part of this act shall be adjudged by any court of competent 52 jurisdiction to be invalid, such judgment shall not affect, impair, or 53 invalidate the remainder thereof, but shall be confined in its operation 54 to the clause, sentence, paragraph, subdivision, section or part thereof 55 directly involved in the controversy in which such judgment shall have 56 been rendered. It is hereby declared to be the intent of the legislature 
 A. 6767 3 1 that this act would have been enacted even if such invalid provisions 2 had not been included herein. 3 § 2. This act shall take effect on the one hundred eightieth day after 4 it shall have become a law.