Master the Art of Verifying AI : Automated Design for Managing Generative AI Risks
This article explores professional prompt engineering and automated "verification processes" to mitigate AI hallucinations. Learn how to leverage multiple AI models to fact-check outputs, manage risks, and safely maximize the potential of generative AI in a business environment.
Is your company being deceived by the "plausible lies" of AI?
"The sales of Competitor A last fiscal year were XX billion yen." Have you ever experienced a generative AI presenting data during market research with absolute confidence, yet without any factual basis? These plausible lies generated by AI, known as "hallucinations," represent one of the greatest risks in AI utilization.
Severe Business Risks Triggered by AI Lies
- Flawed Decision-Making: Making strategic errors or poor investment choices based on incorrect data or analysis generated by AI.
- Loss of Credibility: Inaccuracies finding their way into client proposals or press releases, shaking the very foundation of corporate trust.
- Inefficient Rework: Proceeding with projects based on AI-generated plans only to discover fatal errors later, leading to massive amounts of wasted labor.
The essence of this challenge lies in whether an organization has a "verification process" to avoid blind faith in AI outputs, recognizing that AI is not omnipotent. However, it is unrealistic for humans to fact-check every single response manually.
By reading this article, you will learn how to master AI as a powerful tool and automatically design a self-contained "verification process" where one AI manages the risks of another.
Use AI to Detect the Lies of Other AI
mitsumonoAI addresses the deep-rooted issue of hallucinations by coordinating multiple AI models with advanced reasoning capabilities. This new approach to risk management allows the AI to verify its own outputs from multiple perspectives.
- GPT-5.1 Thinking / Claude Opus 4.6 Thinking (Logic and Reasoning Verifiers) Equipped with advanced multi-step reasoning capabilities that surpass standard AI. These models identify "logical contradictions" and "weak evidence" in generated text from a critical standpoint.
- Sensei AI (Industry Standard Verifier) Trained on specialized industry knowledge and practical know-how. It evaluates the feasibility and validity of academic theories generated by general AI against "real-world field conditions" and "industry norms."
By cross-checking AI responses through three filters—Logic, Expertise, and Alternative Perspectives—errors and risks that humans might miss are automatically detected, drastically improving the reliability of the output.
Step-by-Step Guide: Automated Design of an AI Verification Process
Step 1: [Prevention] Enforce "Defensive Prompts" to Suppress Hallucinations
First, at the initial stage of generating a response, follow strict "best practices" to minimize the probability of hallucinations occurring.
Preparation & Execution
Establish a rule to include the following four elements when giving instructions to the AI:
1.Restrict Information Sources: Direct the AI's focus to specific data.
Example:
"Answer based only on the information in the attached PDF document."
2.Command "I don't know": Prohibit "pretending to know," which is the leading cause of AI lies.
Example:
"If you do not know the answer, do not guess; simply reply 'Unknown'."
3.Require Evidence: Ensure traceability so humans can verify the correctness later.
Example:
"For each part of your response, clearly cite which section of the source material you referenced."
Step 2: [Self-Verification] Let the AI Self-Report "Logical Contradictions"
Next, have the AI critically review its own generated response. For this example, we request market research from GPT-5.1 Thinking.
Preparation & Execution
Launch GPT-5.1 Thinking and instruct it to perform a market analysis with the following prompt:
Prompt Example: GPT-5.1 Thinking (Market Research Request)
"(Paste text extracted from PDF)
[Analysis Request]
Based strictly on the information above, create a report to resolve the decrease in young tourists (20s) for City XX:
1.Current Status (SWOT): Organize City XX’s Strengths, Weaknesses, Opportunities, and Threats compared to City A.
2.Target Insight: Deep-dive into the positive/negative impressions of tourists in their 20s using provided keywords and feedback.
3.Strategic Proposal: Propose three specific tourism ideas to address 20s' frustrations and differentiate from City A. Include a catchy slogan for each.
[Strict Constraints]
・Use ONLY the information provided in the material. Do not use personal knowledge.
・If information is insufficient for a logical answer, do not guess; state clearly that it is "Unknown."
・For every analysis result, cite the specific part of the document referenced."
Next, have the AI review the response it just generated from a critical perspective.
Prompt Example: GPT-5.1 Thinking (Self-Verification)
"Now, regarding the response you just generated, please identify three points from the most critical perspective—including potential logical contradictions, weak evidence, or areas that are too general and lack specificity—along with the reasons why."
Through this step, the AI self-reports its "low-confidence areas" or "leaps in logic," making it clear where humans need to focus their checks.
Step 3: [Expert Verification] Review via "Industry Norms" with Specialist AI
Next, have the plan generated by the general AI reviewed by the industry-specific Sensei AI to verify its feasibility.
Preparation & Execution
Present the response obtained in Step 2 to Sensei AI and request feedback from an expert's perspective.
Prompt Example: Sensei AI - 佐竹 正範 (Masanori Satake)
"The following text is an 'Off-Peak Season Strategy Plan' proposed by another AI.
(Paste the plan generated and self-verified in Step 2)
Please rigorously point out three areas where feasibility is low or improvements are needed from a professional perspective, in light of tourism industry field operations and recent traveler trends."
This step allows for the preemptive identification of real operational risks, such as "consideration for transportation and changing facilities" or "the fact that extending restaurant night hours directly conflicts with labor shortages."
Step 4: [Multi-Perspective Verification] Cross-Check with Different AI to Eliminate Cognitive Bias
Finally, have an AI from a different developer (with different reasoning logic) review the same theme to eliminate the risk of relying on the biases of a single AI.
Preparation & Execution
Present the proposal completed in GPT-5.1 Thinking to Claude Opus 4.6 Thinking and request a review from a different perspective.
Prompt Example: Claude Opus 4.6 Thinking (Cross-Check)
"Regarding the following business proposal (created by GPT), please identify risks or concerns from qualitative perspectives that GPT tends to overlook, such as emotional value in the customer experience or impact on long-term brand image.
(Paste the proposal created based on Step 2)"
Application and Expansion: Establishing a Company-Wide "AI Quality Assurance Process"
This verification process can be rolled out across the entire company as a quality assurance mechanism for AI utilization in every department.
- Legal Department: Cross-check contract reviews with multiple AI models to prevent overlooking risks.
- PR Department: Have AI review press releases or official announcements to detect potentially controversial expressions or misleading statements in advance.
- HR Department: Let a different AI verify from an ethical standpoint whether interview question lists created by AI contain biases against specific demographics.
Summary
This article introduced specific verification process design techniques to manage and suppress the inevitable risk of "hallucinations" in generative AI using the power of AI itself.
- AI lies are unavoidable. The key is to build a "verification system" based on that premise.
- AI responses significantly gain reliability when verified through multi-dimensional filters: ① Self-contradiction, ② Expert common sense, and ③ Perspectives from different AI models.
- By automating this verification process with AI, humans can focus on more sophisticated final decision-making.
If used incorrectly, AI becomes a risk, but if managed properly, it becomes the ultimate partner in expanding human capabilities.
Move from the stage of fearing AI "lies" to the stage of intelligently detecting and mastering them. Why not achieve more advanced and safer decision-making together with AI?
mitsumonoAI is a business-specialized AI platform designed to simultaneously enhance "work quality" and "speed" through the use of AI.
Our suite of advanced reasoning AI models manages the risks of AI utilization in your business and maximizes its effectiveness.
