About the author:

Outset Legal Lens is led by Alice Frei, Outset PR’s head of security & compliance. In this series, she draws on years of experience in legal, compliance, and due diligence work across Web3 projects to show where teams most often get it wrong, and how to build communication that supports growth without quietly creating liabilities.

‍

Current laws don’t distinguish between content drafted by LLMs and content created by an employee simply because of its origin. When a business chooses to publish AI-assisted material under its own name, it becomes a corporate statement, advertisement, or other official communication. As a result, the company retains the same legal obligations it would have faced had the copy been written entirely by a human.

Whether developers of foundation models bear liability in certain circumstances remains an open question. In the United States, for example, courts and legal scholars continue to debate how Section 230 of the Communications Decency Act should apply to certain AI outputs. However, that debate offers little protection to a company that independently selects, approves, and distributes the final output.

The consequences of this principle become particularly visible when content that was never intended to be released accidentally reaches the public.

‍

A placeholder can become a legal claim

For example, a fictional customer review may be treated as a fake testimonial. Under the FTC's Consumer Reviews and Testimonials Rule, this includes endorsements attributed to people who don’t exist, including AI-generated reviews describing experiences that never occurred. 

The FTC relied on this approach in its case against Rytr, alleging that the service generated detailed testimonials containing specific experiences that weren’t based on real users.

The same principle applies beyond reviews. Draft customer stories, placeholder names, sample quotes, and test data should never be treated as harmless production shortcuts. 

Once published, the audience has no way of distinguishing placeholder content from verified facts. Organizations should clearly label these materials throughout the production process and implement safeguards to prevent them from getting published.

‍

The most dangerous content is not obviously fake

The greatest risk rarely comes from an absurd “hallucination” that anyone would immediately recognize. More often, it comes from information that appears entirely credible but was never vetted.

Some categories of content require particularly careful review because inaccurate information can lead to serious liability:

• Statements about people and companies. The highest risks arise where inaccurate information can affect a person's reputation, rights, or economic interests. False factual assertions may result in defamation, commercial disparagement, trade libel, unfair competition, or other forms of economic harm.

‍

The applicable standard depends on the jurisdiction and circumstances – for example, U.S. courts distinguish between public figures and private individuals – but knowingly disseminating false information or failing to exercise reasonable care can amplify legal exposure.

‍

• Financial, health and regulatory claims. Representations about investment returns, product risks, licenses, medical benefits, legal services, or professional qualifications often require a substantiated factual basis before they go live. Health-related representations also need competent and reliable scientific evidence under FTC guidance.

‍

Recent enforcement actions against Delphia and Global Predictions over misleading statements about their purported use of AI, as well as the FTC's action against DoNotPay, demonstrate that regulators increasingly focus on whether companies can justify their claims – not simply whether a model generated them.

‍

• Confabulation. NIST describes one of the core risks of generative models as false or unsupported information presented with confidence.

‍

Legal risks start long before publication

Uploading confidential information to a third-party provider like an AI developer may constitute a disclosure, depending on the service architecture, contractual terms, data retention practices, and the provider's handling of user inputs. 

Once information leaves a business's controlled environment, it can trigger obligations under NDAs, contractual confidentiality clauses, professional secrecy rules, data protection laws, or internal security policies.

Trade secrets require particular attention. Maintaining legal protection depends on whether a company took reasonable measures to preserve confidentiality. Uploading sensitive business information to a public service without appropriate controls weakens that position in a future dispute.

‍

Before using an external provider, organizations should understand what happens to the data they submit. Key questions include whether inputs are retained, used to train future models, shared with third parties, where they are processed, and what contractual safeguards apply.

‍

CNIL also recommends establishing internal rules governing AI use, limiting the disclosure of personal, confidential and strategic information, and considering local, secure, specialized or on-premise solutions for high-risk use cases.

‍

Does human involvement change the legal analysis?

Not always – but in some contexts, it matters significantly.

The level of human involvement can influence how responsibility is allocated within an organization, whether meaningful editorial control exists, and, in some cases, whether copyright protection is available. It may also help establish who ultimately made the decision to approve the final version.

Copyright is one of the most obvious exceptions. According to the U.S. Copyright Office, protection may be available where a person exercised sufficient creative oversight over the expressive elements of the work, made meaningful creative modifications, or produced an original selection and arrangement of AI-assisted material. Simply writing prompts will generally not be enough to establish authorship.

Human control is becoming relevant in Europe as well. From August 2, 2026, Article 50 of the EU AI Act introduces transparency obligations for certain synthetic content. For materials involving matters of public interest, factors such as editorial review and the identification of a responsible natural or legal person are of utmost importance.

‍

How companies can demonstrate human oversight

If a dispute, regulatory inquiry, or legal claim arises, a defensible governance process should include:

• Approved AI tools and clear usage rules. Organizations should maintain a defined list of services and establish rules on what information can be uploaded. Consumer tools, enterprise platforms, and private deployments present different levels of legal and security risk.
• Accountability. Every piece of text should have a designated owner responsible for verifying facts, sources, quotations, statistics, marketing claims, third-party rights, and granting final approval before release.
• Enhanced review for high-risk topics. Content involving financial, health, legal, security, or reputational matters should undergo additional assessment by an appropriately qualified specialist. Marketing claims should be supported by a substantiation file containing the evidence on which they rely.
• Documented editorial process. Organizations should retain prompts, draft versions, source materials, editorial revisions, comments, approvals, and workflow records. For copyright-sensitive copy, documenting the creative decisions behind the final output also proves valuable.
• Technical and procedural safeguards. The governance process should include controls preventing the disclosure of confidential information, similarity checks, verification of testimonials and attribution, defined escalation procedures, and a well-structured correction or takedown process.

‍

Guidance from the NIST Generative AI Risk Management Profile and CNIL provides a practical foundation for building these internal controls. Ultimately, the goal is to show that identifiable people remained responsible for the decisions that mattered.

‍

Technology has changed. Responsibility hasn't.

Generative AI has already altered how content is researched, drafted and edited. But the fundamental expectation that someone stands behind what is ultimately published remained the same.

A model can't decide whether generated material is accurate, sufficiently supported, or appropriate for release. That decision remains a human one – and it is that decision that creates liability.

‍

This article is part of Outset Legal Lens. In this series, we’ll keep unpacking the legal side of Web3 communication, with a focus on helping teams speak clearly, responsibly, and in a way that supports the long-term growth of the industry.