S3224

	in section 106, in proposed section 1 of chapter 93M of the General Laws, by inserting after the definition of “catastrophic risk” the following definition:-
	​““Covered employee”, an employee responsible for assessing, managing or addressing risk of critical safety incidents.”; and
	in said section 106, in proposed section 2 of said chapter 93M, by inserting after subsection (c) the following subsection:-
	​“(c 1/2)(1) A large frontier developer shall clearly and conspicuously publish on its internet website a risk report that provides an overall assessment of the catastrophic risks posed by: (i) any frontier models the large frontier developer deploys externally; and (ii) any internally deployed frontier models with capabilities that materially exceed those of any frontier model that frontier developer has externally deployed.
	(2) A risk report that is required by paragraph (1) shall include, but not be limited to:
	(i) a summary of assessments of capabilities of the frontier models relevant to catastrophic risk, which shall address each type of catastrophic risk and describe any material changes to the capabilities of the frontier models relevant to each type of catastrophic risk since the most recently published risk report;
	(ii) a description of the key threat models the large frontier developer tracks to identify potential catastrophic risks, how observed capabilities of those frontier models relate to each threat model and key mitigations that the large frontier developer has put in place to mitigate any such identified risks; and
	(iii) an assessment of the residual level of each type of catastrophic risk posed by the frontier models after accounting for the mitigations implemented pursuant to clause (ii); provided, however, that the assessment shall provide sufficient information to demonstrate the evidence and reasoning behind the risk assessment and such information shall be sufficient to allow a reasonable person to reach a similar conclusion to that which the large frontier developer would reach in analyzing the level of risk posed by its frontier model.
	(3) A large frontier developer shall renew and update the risk report required by paragraph (1) not less than every 180 days and include in each update a comparison of the assessed level of each type of catastrophic risk to the level assessed in the previously published risk report.”; and
	in said section 106, in said proposed chapter 93M, by inserting after section 3 the following section:-
	“Section 3A. (a) A large frontier developer shall annually retain a third party to perform an independent audit of compliance with the requirements of section 2 except for subsection (c 1/2) of said section 2. The third party shall conduct audits consistent with generally accepted auditing standards and best practices and shall possess demonstrated competence to perform the audit, including experience employing or contracting with individuals who possess technical expertise in the safety of frontier models.
	(1) The third party shall be granted access to all materials reasonably necessary to comply with the third party’s obligations under this section, including, but not limited to, all unredacted versions of materials published pursuant to this chapter. To protect the large frontier developer’s trade secrets and confidential business information, cybersecurity, national security of the United States or public safety, a large frontier developer may impose reasonable security protocols on the third party, including, but not limited to, restrictions on note taking, copying, retaining, or removing materials, requirements for on-premises review and confidentiality requirements.
	(2) The third party shall produce a report that includes, but is not limited to:
	(i) a description of whether the large frontier developer has substantially complied with the requirements of section 2, except for subsection (c 1/2) of said section 2;
	(ii) when applicable, a description of material deviations from the requirements of section 2, except for subsection (c 1/2) of said section 2, an explanation of any deviation and its rationale and any recommendations for how the developer can improve its policies and processes for ensuring compliance;
	(iii) a detailed assessment of the large frontier developer’s internal controls, including its designation and empowerment of senior personnel responsible for such implementation by the large frontier developer, its employees and its contractors;
	(iv) a list of the third party’s personnel involved in the audit;
	(v) the third party’s procedures for managing conflicts of interest and any conflicts of interest of any personnel involved in the audit;
	(vi) the methodology of the audit and the nature of the information reviewed by the third party to conduct the audit; and
	(vii) the signature of the lead auditor certifying the results of the audit.
	(3) The large frontier developer shall retain an unredacted copy of the report for the duration of the frontier model’s deployment plus 5 years.
	(4) Not later than 30 days after receiving the audit report, the large frontier developer shall conspicuously publish on its website a high-level summary of the audit findings and a copy of the third party’s report with redactions as provided in subsection (f) of section 2 and transmit a copy of the redacted report to the attorney general.
	(b)(1) A large frontier developer shall engage at least 1 third party to conduct an independent evaluation of the developer’s frontier models with respect to each category of catastrophic risk. A large frontier developer shall engage a third party to conduct its evaluation not more than 30 days after publishing each risk report under subsection (c 1/2) of section 2 and in any event shall conduct an independent evaluation not less than once every 120 days.
	(2) An independent evaluation conducted pursuant to paragraph (1) shall include, but not be limited to:-
	(i) an independent assessment of each type of catastrophic risk posed by the large frontier developer’s frontier models, taking into account model capabilities, applicable threat models and the large frontier developer's safeguards;
	(ii) an assessment of the ability of the large frontier developer’s frontier models to automate artificial intelligence research and development and any increased potential for catastrophic risks or challenges to risk monitoring, assessment or mitigation resulting from such ability;
	(iii) a review of the large frontier developer’s most recent risk report published pursuant to subsection (c 1/2) of section 2 including an assessment of: (A) the adequacy and completeness of the information disclosed in the risk report; (B) the analytical rigor of the frontier developer’s risk methodology; (C) the appropriateness and materiality of any redactions made to the publicly available version of the risk report; and (D) whether the third party disagrees with any of the report’s claims, including the overall assessment of the level of risk for each catastrophic risk;
	(iv) a list of the third party’s personnel involved in the evaluation;
	(v) the third party’s procedures for managing conflicts of interest and any conflicts of interest of any personnel involved in the evaluation;
	(vi) the methodology of the evaluation and the nature of the information reviewed by the third party to conduct the evaluation; and
	(vii) the signature of the lead evaluator certifying the results of the evaluation.
	(3)(A) The third party shall be granted access to all materials reasonably necessary to comply with the evaluator’s obligations under this section, including, but not limited to, all unredacted versions of materials published pursuant to this chapter and the large frontier developer’s most capable frontier models.
	(B) The third party shall have the opportunity to ask relevant questions about the frontier developer’s frontier models, likelihood of catastrophic risks and related safeguards and the large frontier developer shall provide reasonable responses.
	(C) To protect the frontier developer’s trade secrets and confidential business information, cybersecurity, national security of the United States or public safety, a frontier developer may impose reasonable security protocols on the third party including, but not limited to, restrictions on note taking, copying, retaining or removing materials, requirements for on-premise review and confidentiality requirements.
	(4) The third party shall publish a public version of its report not later than 30 days after delivering the report to the large frontier developer and the large frontier developer shall conspicuously publish a link to the report on its internet website. The public version may be redacted only as provided in subsection (f) of section 2.
	(5) A large frontier developer may comply with this subsection by engaging multiple third parties focused on different categories of catastrophic risk or different aspects of the requirements of this subsection; provided, however, that: (i) collectively, the third parties perform all of the duties required by this subsection; (ii) each third party satisfies the independence and qualification requirements of this section; and (iii) each third party includes in its report a clear and specific statement of the scope of the assessment and the duties performed.
	(c)(1)(A) A third party engaged under this section shall have no financial, operational or management dependence on the large frontier developer or any of the large frontier developer's affiliates and shall be otherwise free from the large frontier developer's control in reaching conclusions or making recommendations, including through contractual safeguards and conflict of interest policies.
	(B) If no other source of funding has been established pursuant to clause (iii) of paragraph (1) of subsection (d), a large frontier developer may compensate the third party at reasonable market rates and shall not condition any payment or the amount of any payment on the results of the third party’s audit or evaluation.
	(2) Prior to accepting any engagement under this section, the third party shall certify in writing to the large frontier developer and the attorney general that the third party satisfies the independence requirements of this subsection. The certification shall include the third party’s sources of funding and remuneration for the engagement, any other current or recent engagements with the large frontier developer or its affiliates and any other facts that could reasonably be expected to bear on the third party’s independence.
	(d)(1) The attorney general, in consultation with academic institutions, nonprofit organizations and industry stakeholders, shall implement an independent evaluation ecosystem plan by: (i) developing and publishing standards for the qualification of qualified independent third party evaluators; (ii) exploring a licensing system to qualify third party evaluators; (iii) subject to government appropriation, providing government funding or arranging pooled funding to supplement other sources of evaluator funding; (iv) exploring the feasibility and benefits of licensing qualified independent evaluators to assess the adherence of artificial intelligence models to standards reflecting best practices for the prevention of personal injury, property damage and other harms that do not meet the definition of catastrophic risk, and reporting its findings and any recommendations regarding such licensing to the joint committee on advanced information technology, the internet and cybersecurity and the joint committee on economic development; and (v) providing resources and funding for nascent organizations seeking to become third party evaluators.
	(2) The attorney general may develop and publish a rating system for qualified third parties based on predefined criteria, including the rigor and quality of the evaluator's published reasoning and analysis, the thoroughness of the evaluator's methodology, the evaluator's track record of identifying material risks or deficiencies and stakeholder feedback, including from frontier developers, academic reviewers and the public.”; and
	in said section 106, by striking out, in lines 1526 and 1527, the words “has reasonable cause to believe” and inserting in place thereof the following words:- “reasonably believes”; and
	in said section 106, in proposed section 7 of chapter 93M of the General Laws, by striking out subsection (b) and inserting in place thereof the following subsection:-
	“(b) A frontier developer shall not: (i) enter into a contract that prevents a covered employee from making a disclosure protected under this chapter; or (ii) discriminate against any person that has made a disclosure under this chapter.”; and
	in said section 106, by inserting after the word “employee”, in line 1536, the first time it appears, the following words:- “, not later than 90 days after becoming a covered employee,”; and
	in said section 106, by striking out, in lines 1541 and 1542, the words “anonymously disclose information to the large frontier developer if the covered employee believes in good faith” and inserting in place thereof the following words:- “, anonymously or named, disclose information to the large frontier developer of the covered employee reasonably believes”; and
	in said section 106, by inserting after the word “fees”, in line 1553, the following words:- “ and court costs”; and
	in said section 106, by striking out, in line 1561, the words “or administrative proceeding”; and
	in said section 106, by striking out, in line 1566, the word “person” and inserting in place thereof the following words:- “frontier developer”; and
	in said section 106, by striking out, in line 1575, the words “this section” and inserting in place thereof the following words:- “paragraph (1) of subsection (d)”; and
	by striking out section 153; and
	in section 164, by inserting after the figure “72”, in line 2014, the following figure:- “, 106”; and
	in section 165, by striking out, in line 2016, the figure “106” and inserting in place thereof the following figure:- “105”; and
	by adding the following 4 sections:-
	“SECTION 166. Not more than 180 days after the effective date of this act or 180 days after the date on which a frontier developer first qualifies as a large frontier developer, whichever is later, a large frontier developer shall post its risk report required under subsection (c) of section 2 of chapter 93M of the General Laws.
	SECTION 167. The provision requiring an annual audit under subsection (a) of section 3A of chapter 93M of the General Laws shall take effect on January 1, 2027 or 180 days after the date on which a frontier developer first qualifies as a large frontier developer, whichever is later.
	SECTION 168. The provision requiring an independent evaluation of a developer’s frontier models with respect to each category of catastrophic risk under subsection (b) of section 3A of chapter 93M of the General Laws hall take effect on January 1, 2027 or 180 days after the date on which a frontier developer first qualifies as a large frontier developer, whichever is later.
	SECTION 169. Not later than 1 year after the effective date of this act, the attorney general shall develop the independent evaluation ecosystem plan required under subsection (d) of section 3A of chapter 93M of the General Laws.”