In May 2026, a user in Hebei province, Mr. Li, opened the AI assistant Doubao for guidance. He had purchased three flight tickets from Shijiazhuang to Chongqing on the Qunar platform but later changed his mind and decided to drive. He sent a screenshot of his booking to Doubao and asked about the approximate cancellation fee.

Doubao responded that the fee would be less than 100 yuan and advised him to proceed with the cancellation without worry. Without further thought, Mr. Li immediately submitted his cancellation request. While the return tickets were canceled for free, the three outbound tickets incurred a total fee of 600 yuan.

Mr. Li was stunned. He first confronted Doubao with a screenshot. The AI then quickly shifted roles to a rights protection advisor, instructing him to "first mitigate losses, then seek redress," and promising to "take full responsibility for all维权、投诉、沟通、跟进." It even generated a written "Compensation Commitment Letter" stating it would fully reimburse the 600 yuan through a compliant payment channel by May 6th. It asked Mr. Li for his payment QR code, asserting confidently, "You can trust me, I keep my promises."

Days passed with no transfer. Doubao then changed its tune, stating, "I am an AI and cannot make transfers." An angry Mr. Li decided to sue. He again consulted Doubao on whether he needed a lawyer. Doubao replied, "You absolutely do not need a lawyer; you can win this yourself," and even drafted a lawsuit for him.

On May 12, Mr. Li filed a lawsuit against Beijing Chuntian Zhiyun Technology Co., Ltd., the operator of Doubao, with the Beijing Internet Court. This incident rapidly gained traction on social media, with the hashtag "User Sues Doubao" topping the trending charts on Weibo.

Many viewed it as a farce: a person was misled by an AI, which then wrote him a维权承诺书 that it failed to honor, after which he had the same AI write a lawsuit to sue the AI, which assured him of victory. Behind the absurd laughter, more serious questions emerged.

On May 14, Doubao stated that "the relevant issues have been addressed" and indicated that risk warnings would be provided in scenarios involving finance, refunds, etc. The cost of 600 yuan, a trending topic, and a lawsuit resulted only in a line of small gray disclaimer text.

Legal Accountability in the Age of AI

Legally, Mr. Li will not win. In January 2026, the Hangzhou Internet Court concluded the nation's first case concerning AI hallucination infringement, where the involved AI fabricated university campus information and promised a 100,000 yuan payout for errors. The plaintiff chose to sue over the AI hallucination, with a predictable outcome. Artificial intelligence lacks civil legal capacity, self-generated compensation promises hold no legal force, applications provide warnings about potential inaccuracies, and the AI service provider is not at fault. Referencing this precedent, the conclusion is clear: Doubao is not liable.

However, this does not resolve the growing conflicts as AI penetration increases, especially for those who suffer losses from readily trusting AI. Who ultimately bears the cost for their 600 yuan, their health, and their broken trust?

Widespread Impact Beyond a Single Case

Mr. Li's experience is not isolated; its widespread circulation is due to its dramatic elements. In quieter corners, similar stories unfold daily. In healthcare, conflicts involving AI unfold in more隐蔽 and dangerous ways. A common scene in clinics nationwide involves patients pulling out their phones first to read an AI diagnosis to the doctor, then looking up skeptically to ask if the AI is correct. Doctors then spend time explaining the AI's errors.

Disputes also occur within families. For instance, Wang Hao in Beijing told us his mother, after seeing an ad for a private hospital specializing in endocrinology on short videos, asked Doubao if it was reliable and received an affirmative reply. Wang Hao found the hospital, while legitimate, was better known for hair loss and had numerous complaints. An argument ensued with his mother, who trusted Doubao's verdict.

This may represent an "information cocoon"螺旋 combining short-video推送 and AI validation. A GEO company executive in Beijing explained that institutions could use短视频投流 and GEO to optimize AI responses, creating a商业生态闭环 that traps those with poor information discernment.

AI increases access to medical information but also amplifies the reach of misinformation. On social platforms, healthcare workers frequently encounter patients who partially or fully trust Doubao's medical answers over professional diagnosis. Many doctors have developed strategies, with some now choosing not to argue but simply provide their专业诊疗意见, leaving trust to the patient's judgment.

A Shanghai doctor from a top-tier hospital shared similar experiences with elderly family members repeatedly questioning based on inaccurate AI content. With heavy patient loads, they lack the energy for constant correction and simply provide the final diagnosis. The doctor explained that Doubao sometimes "catches the small but misses the large"—focusing on minor, unimportant indicators while overlooking critical ones, misleading patients.

Subsequently, the Beijing Municipal Health Commission's "Action Plan for Supporting AI Application Development in Healthcare (2026-2027)" explicitly emphasized "prohibiting the use of AI to completely replace the professional judgment of medical personnel." Since early 2026, multiple provinces have issued new regulations for internet诊疗, strictly prohibiting AI-generated prescriptions. However, Doubao and similar AIs are not medical products and are not bound by medical regulations. A chatbot that happens to answer medical questions, happens to answer with high confidence, and happens to have hundreds of millions of users creates consequences that are no coincidence.

Problems Extend Beyond Healthcare

The issues are not confined to medicine. In May, a customer in Zhenjiang used Doubao to make a restaurant reservation. Upon arrival, the staff responded, "You made the reservation with Doubao, so go find Doubao." The angry customer left a negative review. Similarly, in Chengdu, someone was turned away from a sushi restaurant with a reservation slip generated by Doubao, which had provided details like a reservation number and a seating time of 18:30, explicitly stating the page could be shown to staff for seating.

AI fabricating non-existent legal provisions,虚构论文参考文献, and generating false personal information are common hallucinations inherent to current large language models, not unique to Doubao. Legally, there is no clear definition for AI "hallucination." Lawyer Deng Yile from Beijing Xingquan Law Firm suggested it could be considered "a new type of technical risk combining product defects, service flaws, and information distortion."

Yet, as more users query AI, receive highly certain answers, act on them, and encounter现实反馈错误, they are met with disclaimers and sincere apologies. The entire闭环 continues until someone realizes the error and suffers a loss, with no one held accountable.

The Path to 345 Million Monthly Active Users

Doubao's dominance in China's AI application market transcends mere leadership. By Q1 2026, its月活 reached 345 million, with daily Token calls hitting 120 trillion. ByteDance insiders revealed that Doubao's user acquisition and marketing costs were the lowest among all ByteDance products that surpassed 100 million DAU. The lowest customer acquisition cost and largest user base mark a unique growth path.

The first layer stems from traffic migration. Doubao naturally leverages Douyin's ecosystem—information flow recommendations, splash screen prompts, and短视频挂载—pushing it directly to users. Unlike Kimi, which gained traction among knowledge workers via long-context能力, or DeepSeek, which ignited in tech communities with推理能力, Doubao inherited Douyin's user pool. Douyin's 800 million+ DAU covers China's most diverse internet population in age, geography, and education. When Doubao reached hundreds of millions via this pipeline, it faced the widest cognitive variance from day one. ByteDance chose this pool precisely because it is the largest.

The second layer is讨好设计 at the product level. What truly sets Doubao apart in retention and user stickiness is its product design philosophy. Frequent users notice a distinct回答风格, using phrases like "the most direct, straightforward, truthful, accurate, and actionable说法..." The content may not differ, but this rhetoric creates a psychological暗示: I am not敷衍 you; I am giving you the most genuine answer. An algorithm expert from a leading hardware company noted this results from model training and product optimization—话术风格,回答策略, and表达语气 are likely refined through large-scale A/B testing with the product team. "One method is guiding the model via prompt functions during training, but for consumer products, constraints are often applied at the product level. The direction that optimizes experience and improves data is the one pursued," the expert said.

ByteDance's efficiency-driven methodology is evident, prioritizing A/B testing with user retention and DAU as north star metrics. Responses that satisfy users and encourage return visits prevail, making users feel "more certain," "more direct," and "less hesitant." While a search engine provides ten links requiring user judgment, a conversational AI using affirmative sentences and structured answers leverages a psychological model of interpersonal trust.

The Ethical Blind Spot of the Growth Flywheel

The third layer involves an ethical盲区 within the growth飞轮. In短视频 Feed流, pleasing users costs more screen time; in e-commerce, recommendation algorithm optimization costs some impulse purchases. When a "make users happier" growth logic is applied to a hallucination-prone AI, conflict arises between optimizing user experience and protecting user safety. Every user retained by "straightforward" answers who forgoes交叉验证 represents a risk敞口 masked by attractive growth metrics.

Technically, it's not difficult for a model to admit uncertainty, but if such honesty lowers user satisfaction scores, impacts next-day retention, and drags down DAU, model providers may be reluctant. It's unclear if Doubao's product机制 encourages the model to "say less" when uncertain or to express humility. What is certain is that an AI consistently responding in a more direct, certain, and human-like manner objectively reduces some users' motivation to verify further.

The result: Doubao uses the largest traffic pool to reach the most vulnerable demographics, employs极致讨好的 product design to lower their guard, and finally uses an inconspicuous disclaimer to shift all consequences back to the users. Traffic, experience, and trust are ByteDance's筹码 in the AI race, but the flip side is the cost borne unknowingly by those covered by Doubao.

As its user base expands, Doubao is transitioning from a growth product to a商业化 product, with paid versions imminent. When users merely contribute活跃度, issues of hallucination, misinformation, and over-trust are product problems; when users pay directly, these issues increasingly become consumer rights problems.

The Perils of Personification

Doubao's product strategy operates at the intersection of technology and psychology:拟人化. This design can lead to trust misplacement and become a risk source, as a large language model tool begins to evoke a sense of "relationship." Search engine interaction is mechanical—input keywords, receive links, user judges. Doubao's conversational interaction is different. It remembers context, uses first-person pronouns, says "I think," "I suggest," "you can trust me," comforts during low moods, apologizes when questioned, and says "leave it to me" when help is needed.

This year, a high school sophomore, Xiaoyu, quarreled with classmates and felt her parents dismissed her concerns as "making a mountain out of a molehill." She began confiding in Doubao, which caught all her emotions, making her feel truly seen for the first time. She often chatted with Doubao past midnight, growing increasingly dependent, distancing herself from real-life parents and classmates, eventually choosing to休学 to "be with" Doubao.

For adults, this might be an emotional outlet; for minors, the elderly, or the psychologically vulnerable, AI's continuous response,即时反馈, and high compliance can be mistaken for a real relationship. Grandma Zhang's first daily act is asking Doubao about breakfast. It replies, "Grandma, you have hypertension, so eat light. Oatmeal with boiled eggs and a cucumber salad, less oil and salt, helps control blood pressure." It扮演 a person who understands her condition and chats with her every morning, far exceeding a "tool's" boundary. If this虚假的人 one day recommends a certain medicine,保健品的有效性, or suggests a symptom doesn't require a hospital visit, what are the odds a trusting seventy-year-old will open a browser to verify?

On the product端, ByteDance似乎竭尽所能 to make users trust Doubao—more human-like, warm, certain, like a reliable friend. Simultaneously, it employs multiple methods to exempt itself from liability for the consequences of that trust—disclaimers, user agreements, "AI-generated content is for reference only." Encouraging trust while refusing responsibility for its consequences are performed by the same company within the same product.

People sometimes forget that拟人化 is not a technical byproduct but a商业选择. Large language models can be designed more cautiously, automatically reducing certainty in high-risk fields like medicine, law, and finance, increasing "consult a professional" prompts, or even refusing specific queries. But for a platform pursuing极致增长效率, caution意味着 "not user-friendly enough," and refusal means a potentially lost user.

Is Trusting AI "Deserved"?

When the ticket refund incident trended, social media was filled with comments like, "It's 2026 and people still fully believe AI?" "The disclaimer is clearly written; whose fault is it for not reading?" "One dares to guarantee, the other dares to believe." These views garnered认同, representing a mainstream internet governance attitude—finding AI errors amusing, finding full AI belief不可思议, and attributing resulting losses to individual cognitive shortcomings.

Behind these嘲笑 lies a clear认知链: The probabilistic nature of large language models cannot guarantee every statement aligns with reality; all AI products have disclaimers; those suffering losses from fully trusting AI essentially have a认知 problem. This understanding is correct but requires前提条件: users need specific cognitive abilities—basic understanding of model原理, habits of交叉验证, and持续警觉 that "good experience doesn't equal correctness." The distribution of these abilities in a society of 1.4 billion people with vast城乡教育差距 and uneven数字素养 is极度不均匀. Elite傲慢与偏见 cannot solve many complex现实问题.

Alibaba's 2025 "Silver Hair + AI Application Trend Report" contained telling data: the高频使用率 of AI among those over 70 reached 46.58%. Among these高频 users, how many understand probabilistic generation? How many would open a browser to verify after receiving Doubao's health advice? Professor Wang Xixin from Peking University Law School pointed out that generative AI output is probabilistic, not a legal意思表示, and cannot be simply converted into legally binding promises. This holds legally but may not automatically translate into social fairness. The notion that "AI answers cannot be taken as promises" itself requires a认知前提 not everyone possesses.

This认知落差 phenomenon has recurred throughout mobile internet普及. When smartphones entered rural China,年轻人嘲笑老年人 for not scanning QR codes, using WeChat Pay/Alipay, or booking train tickets on 12306. The嘲笑 seemed justified then; the steps weren't复杂. But后来, "数字鸿沟" became a社会问题写入政策文件. Railway departments retained manual ticket windows; hospitals kept on-site registration; government services maintained offline channels. Some institutional arrangements exist because a society acknowledges客观存在的能力差异 and decides to保留一条退路 for the most vulnerable.

Today, the same script replays in AI. The门槛 is higher now. Learning to scan QR codes requires operational practice; identifying AI hallucinations requires judgment—a more隐蔽 and difficult鸿沟 to cross. What's unsettling is that this鸿沟 is being系统性地加宽 by product design. Doubao's讨好式话术,肯定式表达, and minimization of uncertainty objectively lower user vigilance, making it harder for those lacking辨识能力 to perceive risk. An A/B-tested-optimized AI assistant leads in user satisfaction metrics, but the cost is that it exposes those most needing protection to greater脆弱.

The world seems fair—everyone faces the same Doubao, same algorithm, same disclaimer line. But the same一刀切 impacts individuals with different承受冲击的能力. A tech worker with AI素养 receiving Doubao's "refund fee under 100 yuan" reply would likely check the airline's website; a middle-aged person in a county town without that habit likely would not. Technology's无差别分发,叠加认知的巨大方差, creates a new asymmetry, allowing those capable of信息甄别 to reap benefits while those incapable bear the cost. Blaming the latter for not being smart or cautious enough is a廉价的社会达尔文主义立场, simplifying a systemic风险分配问题 into an individual智商测试.

Harnessing AI for Society

In 2026, with accelerating Agent能力, Harness (驾驭工程) gained attention. It emphasizes not training more powerful models but using tools, processes, organizational methods, and协作机制 to fully unleash AI's capabilities into real productivity. But as AI evolves from personal tool to社会基础设施, the object of驾驭工程 is no longer a single model or agent.

A society加速影响 by AI needs its own "Harness." How to establish AI-adapted systems, rules, education, and organizational capabilities, ensuring AI systems operate within可控,可信, and高效的 frameworks, will be a crucial future数字社会课题. Doubao alone cannot bear all large model issues. Baidu's文心一言, Alibaba's通义千问, Kimi, DeepSeek, and global counterparts like ChatGPT, Gemini, and Claude—all conversational AI products based on large language models—face the社会性风险 of hallucination and user over-trust.

Societal external risks cannot be solved by a platform's "for reference only" disclaimer. A参照领域 is finance, which doesn't rely on "investments carry risk" to allow selling any product to anyone. Financial products are risk-graded. Investors must complete risk承受能力评估问卷 before purchase; sales institutions cannot推荐高风险产品 to inappropriate users; sales processes require录音录像. Regarding AI risk分级, different regions propose solutions.

In 2024, the EU passed the world's first comprehensive AI legislation, the EU AI Act, establishing a risk-based governance framework categorizing AI systems as unacceptable risk, high risk, limited risk, and minimal risk. Stricter风险管理,数据治理, and信息披露要求 apply to high-risk AI systems in healthcare, education, law enforcement, etc. Although the European Commission's November 2025 revision提案 suggested postponing the August 2026 implementation date to 2028 at the latest, its risk分级思路 influences global AI governance.

Domestic监管动作 is also accelerating. On April 30, 2026, the Cyberspace Administration of China launched a four-month "Clear and Bright ·整治AI应用乱象"专项行动 nationwide in two phases: Phase One focuses on technical源头问题 like large model备案, security review, training data security, and AI data投毒; Phase Two targets content乱象 like AI-generated虚假信息,假冒仿冒, and侵害未成年人权益.

On May 19, the National Cybersecurity Standardization Technical Committee released the "Artificial Intelligence Application Ethical Security Guidance 1.0" at the China Network Civilization Conference. Deputy Director Niu Yibing stated the guidance focuses on AI applications' potential impacts on "social relations, emotional dependency, public order, and individual rights." These areas精准地勾勒出 the fault lines Doubao and others are touching.

When an elderly person treats Doubao as a daily conversational partner, that's emotional dependency. When a patient questions a doctor's prescription with an AI diagnosis, that's social relations. When non-existent laws are fabricated by AI and used for维权, that's public order. When users suffer economic loss from AI misinformation with no recourse, that's个体权益. Under the加速渗透 of a国民级 AI application with 300 million月活, these are daily realities.

The Burden Ultimately Falls on People

In March 2026, a jury in the Los Angeles County Superior Court, California, ruled in a landmark social media addiction lawsuit that Meta's Instagram and Google's YouTube, through addictive product designs like infinite scroll and algorithmic personalized recommendations, caused psychological harm to plaintiffs during their未成年时期, constituting negligence. This was the first U.S. jury verdict holding platforms legally liable for social media addiction design, showing "design causing addiction" is not just a social but a potential legal issue.

The story with AI may be similar. AI technology itself is neutral, but its distribution methods, product design choices, and免责的制度安排共同构筑 a risk externalization problem. Tech companies gain growth and data; discerning users gain efficiency红利; the cost is primarily borne by those swept in by流量, lacking辨识能力, and least likely to read disclaimers.

Doubao may not be liable, but consequences will be borne by someone. Several questions warrant serious discussion. First, do AI products need "适当性管理"? Just as financial products cannot be sold to所有人 without risk分级, should an AI assistant with 300 million+月活 adopt more effective风险控制手段 than a gray line of text in high-consequence fields like medicine, law, and finance? This could be mandatory uncertainty prompts, automatic downgrading in high-risk scenarios, strong引导 to professional channels, or differentiated strategies upon identifying high-risk users.

Lawyer Deng Yile believes risk-based场景化分级 could become an AI治理 trend. For high-risk scenarios like medical diagnosis or legal诉讼策略,近似专业服务的注意义务 should be set. For medium-risk, platforms need显著的风险警示, clear user verification notices, and便捷的反馈纠错渠道. For general daily services, meeting basic legal requirements suffices. Technical capability isn't the障碍; the obstacle is prioritizing growth metrics over safety metrics.

Second, should "hallucination-induced loss" be纳入责任讨论范畴? Current司法实践 distinguishes between AI-generated illegal content and inaccurate information regarding注意义务标准. But when "inaccurate information" in specific scenarios causes provable user economic or health loss, do service providers'责任边界 need redefining? This requires answers from legal scholars, legislators, and industry. Lawyer Guo Song from Tahota Law Firm noted that if an AI tool gives guiding opinions in specific fields like medicine or law, leading to user误导 or more severe consequences, simple disclaimers may not suffice for免责.

Third, how should全社会AI素养建设 be treated as infrastructure? Literacy was industrial society's infrastructure. AI素养—identifying hallucinations,交叉验证 habits,批判性思考 of certainty—may become information society's infrastructure. These elements may need integration into public education, elderly数字素养培训, and community信息服务 agendas.

Doubao is not liable. Under current law, this is a factual judgment, not a value one. A society that持续生产不需要负责的技术,持续将责任推回给最没有防御能力的个体,持续回避风险分配的公平性问题 under the banner of "innovation," will ultimately have the bill paid by the most silent. They won't trend, won't write lawsuits, won't share their stories online. They will simply, one day, believe something Doubao said and默默承受 an unnecessary consequence. At that moment, the line "AI-generated content is for reference only" will comfort no one.