AI Validation
AI Validation is a type of AI Response Rule in Coheso’s platform that adds a human review step to certain AI-generated answers. When an AI response involves specified sensitive documents or topics, the platform will hold the answer for approval by a designated Power User before delivering it to the end user. This ensures that complex or high-stakes queries get proper oversight. In practice, routine questions are answered instantly by the AI, while sensitive requests are automatically routed to the appropriate team members for validation. The AI will draft an answer with relevant citations, but the final answer is only sent to the user after a Power User reviews and approves it.
Accessing AI Response Rules
In the Coheso chat interface, navigate to Docs on the left sidebar to manage AI response rules. The Docs section is where you can access your organization’s documents and AI Response Rules settings.
On the Documents page, switch to the AI Response Rules view by clicking the AI Response Rules button (top right). This will open the Response Rules interface where all AI response rules are maintained.
The Response Rules page displays a table of all existing AI response rules. Any previously created rules (of any type) would be listed here along with their details.
To add a new rule, click the Create New Rule button at the top right of the Response Rules page. This will open a dialog box for creating a new AI response rule.
Creating a New AI Validation Rule
When the Create New Rule dialog appears, fill in the required details for the Validation Flow rule:
Rule Name: Enter a clear, descriptive name for the rule (for example, the policy or topic it covers).
Type: Select Validation from the “Type” dropdown to designate this as a Validation Flow rule.
Document Triggers: Click Select files and choose one or more documents that will trigger the rule. Whenever an end user’s question involves any of these selected documents, the Validation Flow will be activated (the AI will withhold a direct answer and initiate the request creation flow).
Save the Rule: Once all fields are set, click Create Rule to save the new rule. The rule will be added to the list of AI Response Rules immediately.
Example of a configured Validation rule. In this example, the rule “Require Validation for Anti-Bribery Questions” is being created. The Slipstream Anti-Bribery Policy document is selected as the trigger. This means whenever the AI’s answer draws from the Anti-Bribery Policy, the response will require approval before it’s sent out. After saving, the new rule appears in the AI Response Rules list.
Managing Validation Rules
All AI Response Rules, including validation rules, can be modified or removed as needed. On the AI Response Rules page, you’ll see options to edit or delete each rule in the list:
To edit a rule, click the pencil icon next to that rule. This opens the rule’s configuration dialog where you can change the rule name, switch the rule type, or update the associated document(s). After making changes, click Save to apply the updates.
To delete a rule, click the trash bin icon next to the rule. You will be asked to confirm the deletion in a pop-up dialog. Confirming will permanently remove the rule from the list.
AI Responses with Validation Flow
With the AI Validation rule configured - an end user asks a question that triggers the rule. Whenever an end user’s query would result in an AI answer involving one of the specified documents, the validation rule intercepts the process. Instead of delivering the AI’s answer immediately, the assistant will respond with a confirmation link that the answer requires validation. In this way, the user is guided to initiate a WMS request rather than receiving an AI answer in chat.
Once the user clicks the link to confirm, an intake request is automatically created in Coheso’s Work Management System. Power Users are immediately notified in WMS that a new request has been created.
Behind the scenes, Coheso’s Work Management module now contains a new validation task. This task includes the full context of the user’s question and the AI’s first-pass drafted answer (complete with any source citations the AI used). The AI draft and context are provided to help the Power User evaluate the response.
Upon opening the request, a draft answer is already generated internally by Coheso AI. The AI automatically posts a first-pass internal comment within the “Internal Legal Dept. Comments” section, providing a structured draft response based on the relevant document. This draft remains visible only to Power Users, ensuring internal review and refinement before anything is shared with the end user.
The Power User can review and refine the AI-generated draft response. After making any necessary edits or confirming the content, the Power User can send the answer to the requester by posting it in the Requestor Comments section. This action notifies the requester that a response is available.
The requester is notified in the chat interface once their question has been answered via the WMS ticket. A notification will indicate the update, and the requester can click it to view the response instantly.
The requester can view the final response on the Request page. By using Validation Flow rules, the sensitive query was handled through the proper workflow - the AI’s draft was vetted by a human, and the final answer was delivered via a WMS request, rather than the AI answering immediately in the chat.
Last updated