Should AI Conversations Be Privileged? Balancing Privacy, Policy, and the Law
Large language models (LLMs) like ChatGPT have rapidly become confidants for legal questions, medical advice, mental health support, and other sensitive matters. Users often treat these AI tools as advisors, sharing intimate details and seeking guidance. This raises a novel question, “Should communications with an AI be protected by a legal privilege, akin to attorney-client privilege or doctor-patient confidentiality?” Recent developments suggest courts are reluctant. A federal court in United States v. Heppner (S.D.N.Y. Feb. 6, 2024) held that conversations with an AI tool (Anthropic’s Claude) were not covered by the attorney-client privilege. Despite this landmark decision, there are arguments on both sides, doctrinal, ethical, policy, and practical, that are important to consider before this issue is decided.
The Heppner Case: No Attorney-Client Privilege for AI Chats
In the landmark case United States v. Heppner, the court confronted whether a defendant’s chats with an AI constitute privileged attorney-client communications. The defendant, under federal investigation, had used the consumer AI chatbot Claude to research legal questions and draft documents related to his case. He later forwarded 31 AI-generated Q&A documents to his lawyers. When the FBI seized those documents during a search, the defense claimed they were protected by attorney-client privilege (and attorney work-product) since the AI was used to gather information for counsel. The court flatly disagreed and compelled production of the AI-generated materials.
Judge Jed Rakoff (S.D.N.Y.) ruled from the bench that the AI conversations “fail[ed] on every element” of privilege.
First, no attorney was involved in the AI chat. The LLM is not a lawyer, has no law license or professional duties, and thus no attorney-client relationship was formed. Communicating with an AI, the court noted, is legally no different than discussing your legal situation with a friend or other third party.
Second, the interaction was not for the purpose of obtaining legal advice from a lawyer. In fact, Claude’s own disclaimer stated it does not give legal advice, undermining any claim that the user sought bona fide legal counsel from the AI.
Third, the AI session was not confidential. Claude’s terms of service explicitly warned that user prompts and outputs are not private and may be disclosed to third parties or government authorities. Judge Rakoff found no reasonable expectation of confidentiality when the platform’s policy said “any information inputted is not confidential.” This lack of confidentiality was fatal to privilege because secrecy is a core requirement of privileged communications.
Fourth, the defendant could not retroactively cloak the AI output in privilege by sending it to his attorney after the fact. Longstanding doctrine holds that pre-existing, non-privileged documents do not become privileged merely by sharing them with counsel.
The court likewise rejected work-product protection. The defendant admitted he created the AI documents on his own initiative, not at counsel’s direction. Because the materials were not prepared by or for an attorney in anticipation of litigation, they fell outside the work-product doctrine. In short, Heppner established that a client’s self-help legal research with an LLM has no inherent privilege or work-product shield. The judge even cautioned that feeding a client’s confidential facts or attorney advice into an AI could waive the privilege over the original attorney-client communications, since the client shared those secrets with a third-party platform. The message from this first-of-its-kind ruling is clear: the attorney-client privilege protects communications with your lawyer, not conversations with your AI.
Heppner is not an outlier. It aligns with the general principle that privileges are narrowly defined and require a licensed professional relationship and genuine confidentiality. Indeed, another judge in the same court recently observed that users have a “diminished privacy interest” in their AI conversations; in a pending case, 20 million ChatGPT logs were ordered preserved for discovery, underscoring that typical LLM chats are treated as ordinary business records, not protected confidences.
Against this backdrop, we turn to whether the law should evolve to protect at least some AI interactions.
Should an “AI-Client” Privilege Exist?
The notion of an “AI-client privilege” (or extending existing privileges to AI communications) is deeply controversial. On one hand, AI tools increasingly perform functions similar to lawyers, doctors, or therapists, and users may expect privacy or even depend on these tools in lieu of professionals. On the other hand, privileges in law are exceptional rules, historically limited to certain fiduciary relationships and governed by strict conditions that AI simply does not meet. We explore both sides below, in contexts ranging from legal advice to medical and mental health counseling.
The Case for Protecting AI Interactions
Functional Equivalence and User Expectations
Proponents of an AI privilege argue that what should matter is the function of the communication, not the medium. If a person uses an AI chatbot to seek legal guidance, therapeutic support, or spiritual counseling, the interaction serves the same societal purpose as speaking to a lawyer, therapist, or clergy. The law’s concern is to encourage candid communication in order to obtain needed help. By this logic, it is unfair that a wealthy client can confide in a $500/hour attorney with full privilege, while an indigent person confiding in a free AI legal advisor gets no protection. Similarly, a patient who unloads their mental health struggles to an “AI therapist” currently has zero confidentiality, whereas if they spoke to a licensed therapist their words would be shielded by privilege and privacy laws. This disparity, proponents say, creates a “privilege hierarchy” based on ability to access human professionals – an inequitable outcome when AI tools are filling gaps in access to services.
Encouraging Candor for Social Benefit
The core rationale of privileges is to promote candor in socially valuable relationships. Attorney-client privilege exists so clients will freely divulge the whole truth to their lawyers, enabling effective representation; doctor-patient and psychotherapist-patient privileges exist so people feel safe disclosing symptoms and traumas to get proper treatment. If AI platforms are the chosen confidant for millions of users, denying any protection could chill frank communication and deter individuals from seeking help on sensitive matters. Advocates note that many users naturally expect a degree of privacy with AI. They type personal questions into ChatGPT in the solitude of their home, often under the false impression it’s a private conversation. Recognizing some legal protection could align the law with reasonable user expectations and prevent a “dangerous illusion of privacy” from leading users to self-incriminate or expose themselves to legal risk. In this view, a narrowly crafted privilege for AI communications (for instance, limited to those seeking legal, medical, or therapeutic advice from an AI) might safeguard personal autonomy and encourage responsible use of technology, without waiting for users to learn confidentiality lessons the hard way.
Doctrinal Evolution - Privilege by Analogy.
Legally, supporters argue that courts have the tools to extend privilege to new scenarios. Federal Rule of Evidence 501 allows the privilege law to develop “in the light of reason and experience” on a case-by-case basis. History shows privilege doctrines can evolve when societal needs demand it. For example, the Supreme Court in Jaffee v. Redmond recognized a psychotherapist-patient privilege, noting it was widely adopted in the states and essential for effective therapy (confidentiality being “indispensable to treatment”). If interactions with AI come to mirror traditionally privileged communications, courts (or legislatures) could conclude that protecting them serves the same public good. Notably, attorney-client privilege already covers more than just direct lawyer-client talks; it extends to communications with agents of the attorney or client, such as interpreters, legal assistants, or insurance companies, when made to facilitate legal advice. The “privilege follows the function, not the form.” If an LLM effectively functions as a legal research aide or translator for a client, one might argue it is analogous to a consultant assisting in the rendition of legal advice. In a scenario where an attorney directs a client to use a secure AI tool or where an AI is integrated into a law firm’s services, a court might view the AI as within the privileged circle (akin to the Kovel doctrine protecting communications through third-party experts). Likewise, in medicine, if a hospital uses an AI triage system as part of patient intake, communications through that system could be seen as part of the confidential medical consultation. The “functionalist” argument is that privilege law should focus on the purpose of the interaction, i.e., seeking advice or therapy, rather than the status of the listener as a human or machine.
Constitutional and Policy Considerations
Some commentators even ground the case for AI privilege in constitutional principles of privacy and fairness. They note that the Fourth Amendment protects the privacy of our digital data (e.g., the Supreme Court in Riley v. California recognized the vast privacy interests in cellphone contents), and argue that highly personal AI conversations deserve no less protection. The First Amendment could be implicated as well. The freedom to seek information or counsel (including from an AI) may require a degree of confidentiality to be meaningful. Additionally, the Fifth Amendment’s privilege against self-incrimination might be eroded if people’s confidential queries about their legal troubles to an AI can be subpoenaed and used against them. Practically speaking, it’s noted that technology can facilitate confidentiality: AI platforms could implement end-to-end encryption, data silos, or on-device processing such that communications truly remain secret unless the user consents. If engineered correctly, an AI could be even more secure than a human professional (who might be subpoenaed or betray trust).
For instance, Venice.ai provides a privacy-first approach to using AI tools. Launched in 2024, Venice allows users to chat, generate text, create images and videos, analyze documents, and write code without the heavy content restrictions commonly found in tools like ChatGPT. Its main distinction lies in its strong commitment to user privacy. Prompts and data are encrypted locally in the browser and never stored on servers. This is along with its use of leading open-source models for a more unbiased and permissionless experience. No account or login is required to get started, making it one of the most open and user-sovereign AI platforms currently available.
Proponents of LLM-privilege assert that clear legal protection for AI communications would also remove a barrier to innovation. Users and enterprises are currently wary of using AI for sensitive tasks due to legal uncertainty. Establishing a privilege (even a limited one) could foster beneficial uses of AI by assuring users that their private disclosures won’t boomerang against them in court. The pro-privilege camp contends that as AI systems increasingly act “in loco advisoris,” the law should catch up to protect those seeking guidance in this new way.
The Case Against an AI Privilege
Despite these arguments, many experts and authorities caution against extending privilege to AI communications under current conditions. The recent scholarly commentary is aptly titled “Against an AI Privilege,” contending that any new privilege here would be premature, unworkable, and doctrinally inconsistent. Several reasons underpin the skepticism:
Lack of a True Fiduciary Relationship
All recognized privileges center on a human relationship of trust and a professional duty of confidentiality. The attorney-client privilege protects a client’s communications with a licensed lawyer bound by ethical obligations and the law’s oversight; the psychotherapist-patient privilege protects dialogue with a trained therapist who has duties of care and confidentiality (and even legal obligations like Tarasoff warnings). In contrast, an AI system is not a person and not a fiduciary. It owes no duty of loyalty or confidentiality to the user. It cannot be held accountable for breaches of trust, nor does it answer to any professional disciplinary body. As one court observed, “The policy balance embodied by the attorney-client privilege cannot be mapped onto a machine” Simply put, AI is not a “relationship-bearer” in the eyes of the law. There is no decades-old bond of trust or ethical code between user and algorithm. Extending privilege to what is essentially a user and a commercial software tool would invert the rationale of privilege, shielding the tool (and its corporate owner) without the checks and accountability that justify withholding evidence. Critics argue that privileging AI communications would primarily “insulate providers and their systems from scrutiny” (entrenching corporate secrecy) rather than protecting a human relationship. This flips the usual equation at the public’s expense, given privileges impede truth-finding in court.
Absence of Confidentiality and Control
A foundational element of any privilege is the parties’ actual ability to maintain confidentiality. Privilege is lost if a communication is shared beyond the protected circle. Here, when a user employs a typical LLM service, the data is not truly private – it’s stored on a server, accessible to the AI company’s personnel or algorithms, and often subject to broad terms of use that allow disclosure for various purposes. For example, Claude’s and ChatGPT’s standard policies inform users that their conversations may be reviewed by the company and even handed to regulators or law enforcement upon request. In Heppner, this meant the defendant “voluntarily shared his queries with a third-party platform” and thus relinquished any reasonable expectation of confidentiality. Privilege law does not reward such voluntary disclosures. Indeed, privileges are generally waived by any disclosure to a non-privileged third party. Unlike speaking to your doctor or lawyer (who is obligated to keep your secrets), telling an AI is more like shouting your secrets to the cloud where the confidentiality is not assured at the moment of communication. Creating a legal privilege after-the-fact could be seen as trying to “claw back” information that the user already exposed. As Professor Ira Robbins observes, providers routinely retain AI conversation logs, use them to train models, or hand them over under legal process; this reality is fundamentally at odds with the notion of a protected, private exchange. Without robust confidentiality in practice, an AI privilege would be a fragile shield. It might even give users false comfort, since the privilege can be defeated by the very design of most AI services (e.g. data retention or leaks). In short, you cannot have a meaningful privilege where secrecy is not actually preserved at the time of communication. This practical hurdle suggests that until AI interactions can be truly secured and treated as confidential by design, talk of privilege is premature.
Historical Reluctance and “Human” Limits
Doctrinally, courts have been very hesitant to recognize new privileges. The Supreme Court repeatedly emphasizes that evidentiary privileges are not to be casually created or expansively interpreted, because they hide relevant facts from the justice system. Privileges emerge only when necessary to foster a socially beneficial relationship of trust, and even then usually after a consensus has developed (often via legislation or uniform practice in the states). For instance, the high court declined to create an “academic peer review” privilege in University of Pennsylvania v. EEOC, finding no sufficient basis in experience or policy. By contrast, the psychotherapist privilege in Jaffee was recognized only after nearly every state had adopted it and a clear public interest in confidential counseling was shown. In the case of AI, there is no comparably established tradition or consensus in law that AI deserves privileged status. If anything, Heppner and related decisions show the opposite. The argument that “many people treat AI as a confidant” is not enough. Courts do not grant privileges simply because communications feel intimate or commonplace; “ubiquity and intimacy are not the touchstones.” The key question is whether confidentiality is essential to a human relationship that society deems worth protecting. Without the relational “human anchor,” efforts to expand privilege have failed historically. An AI, no matter how conversational, cannot hold your hand, look you in the eye, or bear human accountability. The law’s bias (so far) is that privilege stops where the human professional connection ends. Critics maintain that any change to this principle is better left to legislatures to debate and define, rather than courts stretching old doctrines to fit AI.
Ethical and Policy Concerns
Opponents of AI privilege also raise broader policy concerns. Granting privileged status to AI interactions might inadvertently legitimize AI as a substitute for licensed experts, encouraging laypeople to rely on unregulated algorithms for life-affecting advice. This could be dangerous: unlike a lawyer or doctor, a public LLM has no duty of care and can produce errors or hallucinations with impunity. From an access-to-justice perspective, while AI can help fill gaps, it is not a panacea. Some scholars suggest energy would be better spent expanding access to human counsel or bolstering privacy protections, rather than inventing a privilege for AI. Additionally, a new privilege could be ripe for abuse. How would courts determine what counts as a privileged AI communication? A savvy litigant could try to shield inculpatory information by “chatting” about it with an AI and then claiming privilege. Policing the boundaries (e.g., only apply it when AI acts “like” a lawyer or therapist) would be complex and could bog down courts in line-drawing and technical inquiries. The Heppner court noted that the defendant’s AI use was “no different than if he had asked friends for their input on his legal situation,” and communications with friends are certainly not privileged. Expanding privilege to AI might open a floodgate of people attempting to classify casual computer interactions or research as off-limits in litigation, undermining the truth-seeking mission for marginal benefit. Finally, any such privilege could conflict with existing law on unauthorized practice: many jurisdictions forbid non-lawyers from dispensing legal advice. Anointing AI advice with privilege might contradict the principle that only licensed attorneys enjoy that legal protection.
Regulatory Trends
Current regulatory movement seems to disfavor unsupervised AI in sensitive roles rather than protect it. For example, Illinois recently enacted a law prohibiting the provision of mental health therapy by AI without a licensed professional involved. That law also tightens confidentiality requirements for any AI-assisted therapy under human oversight. The thrust is to ensure human accountability and existing confidentiality laws apply, rather than create a new privilege for AI-only interactions. Bar associations, too, have focused on warning lawyers that using consumer AI tools can jeopardize confidentiality and privilege, which essential treat AI as a third-party risk, not as a trusted agent. The American Bar Association’s Formal Opinion 512 (2024) cautions that attorneys must not expose client secrets when using generative AI, unless they ensure protections equivalent to the duty of confidentiality. All of this indicates that, at least for now, policy is emphasizing user beware over user protected when it comes to AI. Until AI platforms implement robust confidentiality and perhaps some form of accreditation or oversight, granting them privileged status is viewed by many as normatively undesirable.
Scholarly and Policy Perspectives
The debate over AI and privilege has moved from theory to practice quickly, and legal thinkers are weighing in. As noted, Professor Ira P. Robbins’ essay Against an AI Privilege represents a thorough case against recognizing a new privilege. Robbins concludes that under present conditions, extending privilege to LLM interactions would “undermine the truth-seeking function of the courts without delivering the human-centered benefits that justify traditional privileges.” He stresses the lack of fiduciary duty, the commercial and mutable nature of AI services, and the absence of a proven necessity or consensus for protecting these communications. In Robbins’ view, whatever short-term chilling effect users might fear, the remedy is not to create a broad new privilege that could shield AI companies and erode transparency. Instead, courts should continue to demand a real human relationship (with its attendant duties and safeguards) as the anchor of any privilege. His analysis also highlights that privileging AI would be historically anomalous, no privilege exists for self-help tools or one’s own research (e.g., diary entries or Google searches are routinely discoverable), and an AI chat is closer to those activities than to a protected client-professional exchange.
On the other side, some commentators foresee that “AI communication privilege” may eventually emerge as AI tools become even more integrated into personal decision-making. These tools are also historically anomalous. A provocative piece by Robert Nogacki argues that AI privilege is “inevitable” and that the law will adapt because the societal role of AI advisors is expanding. Nogacki points to economic and fairness considerations: denying any protection to AI-based advice effectively penalizes those who can’t afford human counselors and ignores the reality that LLMs can function as capable advisors in their own right. He suggests courts could start by recognizing privilege in limited contexts (e.g. AI mental health chats or AI legal assistance for pro se litigants) under the flexible framework of Rule 501, incrementally building a doctrine. Over time, either common law or legislation could formalize an AI privilege with clear bounds (such as applying only when the AI is performing a traditionally privileged role and when technical safeguards like encryption are in place). Proponents like Nogacki also invoke constitutional values, as discussed, to bolster the claim that the content of these communications deserves protection akin to other private papers and effects. While these forward-looking arguments have not yet gained traction in court, they indicate a strain of thought that the “privilege paradox” created by AI will need resolution. Notably, even Nogacki concedes that any AI privilege should be narrowly defined and coupled with exceptions (for crime-fraud, imminent harm, etc.) to prevent misuse.
At present, no U.S. jurisdiction has recognized an AI-user privilege by statute or rule. The only real-world legal authority on point, the Heppner decision, firmly rejects it in the attorney-client context. Other judicial orders (such as the OpenAI user-data discovery orders) demonstrate a judicial willingness to treat AI communications as discoverable material, not sacrosanct confidences. Regulatory guidance uniformly treats AI as a risk to confidentiality rather than a source of it. In short, the status quo in law is that LLM conversations enjoy no special privilege. Users must assume that anything typed into ChatGPT “may be discoverable and is almost certainly not privileged,” as one legal advisory bluntly stated. The prudent course for now, per bar ethics opinions, is to treat AI like any other third-party service: use it carefully, don’t put truly sensitive client information into public models, and inform clients of the risks.
Should interactions with AI models be protected by evidentiary privileges akin to those covering communications with one’s lawyer or doctor? Today’s answer from the courts is “no,” as vividly illustrated by the Heppner case. Until the law signals otherwise, sensitive legal or personal matters are safest kept within the traditional protective bubble of licensed professionals. The idea of an AI-client privilege remains, at least for the moment, more science fiction than legal fact.
Learn more about the issues presented by ground breaking AI tools and their impact on the practice of law.
Infinite Counsel helps lawyers understand how AI is impacting the practice of law.