Fault-Shifting or Fact? Under the Hood of AI-Driven Claims Investigation (Part II)

As announced in my previous post, Fault-Shifting or Fact? Insurance Claims Investigation with AI Agents (Part I), this article dives into the technical details behind the insurance use case. In this post, I want to explore:

• Which agents are involved in this scenario?
• How do they interact with the underlying data?
• What technologies are powering them behind the scenes?

The personas John (the policyholder) and Elena (the claims adjuster) interact through distinct channels. John uses a mobile app for his claim submission, while Elena works out of a dashboard. Through these channels, two separate agents, along with their sub-agents and tools, are triggered: a Pre-Screening Agent and a Claims Adjuster Agent. Both were developed and tested using IBM Bob and deployed as native agents on IBM watsonx Orchestrate, which acts as the Agentic AI platform. The data foundation consists of CRM and telemetry data, which are provided as data products out of IBM watsonx.data and Confluent.

The data products: From structured to unstructured information

First, let’s take a look at the data products. watsonx.data ensures the data quality of the CRM database.

With Confluent, telemetry data is read from the OBD-2 (On-Board-Diagnostics) dongle in real-time.

Taking a closer look at the data reveals a broad spectrum: ranging from structured and semi-structured to unstructured data. The CRM data for customers and policies, as well as the telemetry data, are fully structured. On the other hand, the policyholder’s description captured during claim intake and the textual findings from the image analysis constitute unstructured text content.

The claims themselves sit right in the middle as semi-structured data, since they contain both structured elements (e.g., customer ID, policy ID) and unstructured components (e.g., conversation history, image analysis, policyholder description, claim amount assessment, and fraud assessment). This also highlights how certain information is enriched and generated ad-hoc – such as the image analysis, claim amount assessment, or fraud assessment. These processes either build on top of existing data products or rely on information enriched dynamically during the workflow (such as a user uploading a photo of the damage).

Beyond the data structure itself, the system must also handle a wide variety of information origins. In this specific use case, the spectrum ranges from completely objective, fact-based data (such as sensor telemetry) to AI-generated insights (such as automated fraud assessments), all the way to purely human-generated content (such as the initial description of how the accident occurred).

Following John‘s claim: A behind-the-scenes look at the Pre-Screening Agent

The Pre-Screening Agent is an integral part of the claim submission process within the mobile app. It is built as a collaborative system consisting of two specialized sub-agents:

• Amount Calculator Agent: This agent analyzes the image description to generate a detailed repair cost estimate, complete with a structured justification for the projected expenses.
• Fraud Classification Agent: Operating on both the image description and the policyholder’s description of the incident. This agent delivers a comprehensive fraud assessment with a clear rationale, ultimately assigning a specific fraud risk flag (Low, Medium, or High).

Elena’s investigation: A behind-the-scenes look at the Claims Adjuster Agent

The Claims Adjuster Agent powers the detail view of a claim and operates behind the scenes of the interactive chat, named as AI Investigation Agent. Depending on the specific query, the main Orchestrator Agent dynamically triggers the appropriate sub-agents:

• CRM Data Agent: This agent has direct access to the CRM system containing customer profiles, policies, and claim histories. It is used, for example, to quickly verify details like “telematics coverage” for a specific claim.
• Telemetry Analysis Agent: Triggered by a Vehicle Identification Number (VIN), this agent pulls vehicle telemetry data. It retrieves heartbeat data for a specific window – typically 10 seconds before and after the impact – as well as the core incident-impact data packet captured at the exact moment of the accident.
• Knowledge Base Agent: This agent connects to a built-in Milvus vector database containing internal guidelines and standard operating procedures. It utilizes the default embedding model ibm/slate-125m-english-rtrvr-v configured with a chunk size of 256 tokens, a chunk overlap of 128 tokens, and 768 embedding dimensions.

Next, let’s take a deeper look into how the Claims Adjuster Agent operates under the hood. The following video demonstrates an example of how the watsonx Orchestrate control plane supports to:

• Track reasoning in detail: Monitor the agent’s step-by-step decision-making process, including exactly which tool calls are being triggered at any given moment.
• Utilize debug mode: Gain deep insights into execution tracking with comprehensive node logs, providing full visibility into backend operations such as model usage and token consumption needed to generate a response.
• Conduct evaluation and testing: Perform live evaluation and testing directly within the integrated user interface.

Wrap-up

Looking behind the scenes highlights a fundamental shift in the AI landscape: Today, the primary challenge for most companies is no longer how to build agents, but rather maintaining oversight and control. We are no longer operating in a greenfield environment. Instead, enterprises must navigate a complex, fragmented reality:

• No more clean slates: Most companies already have their preferred platforms, open-source frameworks, and selected foundation models firmly in place.
• A highly fragmented ecosystem: Organizations rarely rely on a single platform. Instead, they manage an intricate mix of cloud providers, on-premises infrastructure, open-source tools, models running locally or in the cloud, and specialized tools connected via protocols like MCP or agents via A2A.
• Diverse regulatory demands: Due to strict data privacy laws and legal requirements, different use cases often demand entirely different technological implementations and compliance guardrails.

As a result, the strategic focus is rapidly shifting away from just spinning up new agents, and moving toward a much bigger question: How do I maintain comprehensive visibility, governance, and control across it all?

For those interested in diving deeper into this topic, you can find two of my previous blog posts on this [here] and [here].