How to Reconstruct an Enterprise-Level AI Data Platform When Data Moves Towards Understanding?

This article is based on the speech given by the author at the MIP product launch event on October 20. On October 20, Jingzhou Technology held the MIP product launch conference, announcing a strategic shift from real-time analysis to a unified data analysis and intelligence platform, and officially launched the ultra-fast unified enterprise-level AI data platform MIP. MIP is based on StarRocks technology, connecting a unified data foundation, enterprise-level semantic layer, and trusted Data Agent, supported by governance capabilities throughout the platform, helping enterprises use data for AI according to business standards, with results that are verifiable and traceable.

Author | Dong Yingtong, Product Head of Jingzhou Technology

Produced by | Industry Home

Three Questions for Implementing AI Agents in Enterprises

When Agents enter real business scenarios, they inevitably encounter issues related to data, standards, and knowledge. Based on our own practice, we have summarized three common questions and provided our thoughts on each.

Question 1: Data is multi-modal and dispersed.

When an Agent processes tasks, it may simultaneously access 2D business tables, unstructured documents, images, and vector data. These data often exist in different storage media and systems, requiring further correlation and analysis after retrieval. This poses challenges for building AI Agent systems.

Our thought is: various types of data should be able to correlate in SQL. Data from business tables, data lakes, and unstructured data in object storage should all be searchable and queryable, and the results can be further analyzed. It is normal for data to exist in different places, but the cost of migration is high, so let the computation find the data, and the dispersion issue can be resolved at the query level.

Question 2: Standards need repeated explanations.

Taking data analysis as an example, "How is sales calculated?" and "Which orders are included in the calculation?" must be clearly defined first. However, this knowledge is often scattered across historical SQL, documents, and even the personal experiences of analysts. As a result, the same question can yield different answers.

Our thought is: define business standards in the database. Such definitions can be directly converted into executable calculations. We use Semantic View to map metrics to clear expressions, allowing query results to trace back to the corresponding formulas, statistical ranges, and the final executed SQL, facilitating unified explanations of standards.

Question 3: The building process involves a lot of repetition.

When building multiple Agents, different data access requirements arise, necessitating the reorganization of related knowledge and context. When corrections are needed, it is also difficult to synchronize between different Agents, leading to a lot of redundant work during the Agent construction process.

Our thought is: use the database to carry the management capabilities of reusable enterprise context, with MirrorMind organizing knowledge and its associations, versions, and access permissions for multiple Agents to use as needed; session and task running states are managed by MirrorPilot.

The three questions point to the same root cause: data, standards, and context, which lack a unified carrier.

On October 20, at the Jingzhou Technology product launch conference, we officially launched MIP—an ultra-fast unified enterprise-level AI data platform. With the database at its core, its architectural design unfolds in two directions: downward, adding AI capabilities to the database and upgrading the data foundation; upward, database for AI, allowing enterprise knowledge and AI Agent capabilities to serve more business needs.

Wherever the data is, understanding and computation should be there. Putting data, standards, and context into the database is because the capabilities lacking in these three areas need to grow out of the database.

Three-Layer Architecture: MIP Organizes Data and Knowledge into Shared Capabilities

MIP organizes enterprise data and knowledge into three layers, forming a set of shared capabilities.

The bottom layer is MirrorData, an enterprise-level data query engine centered on the open-source StarRocks, providing ultra-fast query retrieval and multi-modal analysis capabilities.

The middle layer is MirrorMind, centered on Semantic View, linking relevant business knowledge and the personal experiences of analysts for Agents to use as needed.

The upper layer is MirrorPilot, the operating environment for business Agents.

Above the three layers are various business applications. Commonly used business analyses, after-sales diagnostics, and product selections in enterprise operations can all be built using this unified foundation.

The following chapters will unfold these three layers step by step, demonstrating the usage of the entire foundation.

• Using "After-Sales Images" to showcase MirrorData's multi-modal retrieval and business association

• Using "Activity Budget Comparison" to explain the role of semantic definitions in MirrorMind

• Using "Retail Business Analysis" to connect configuration, querying, analysis, and reporting into a complete usage process.

MirrorData: A Unified Data Foundation

Standard SQL Access, A Set of Computational Capabilities Serving All Applications

MirrorData is the data foundation of MIP, with the core being a query engine.

Upper-level BI reports, interactive analysis, data applications, and Agents can all access through standard SQL. The same set of computational capabilities serves different applications, with MirrorData as the unified query entry.

The query engine inherits the high-performance query capabilities of StarRocks. On this basis, we have made enterprise-level enhancements: permission management, resource control, and resource isolation.

Supports Unified Access to Internal Tables, Lake Tables, and Multi-Modal Content

There are three ways to access data on the data side.

1. Traditional Internal Tables. Internal tables carry queryable data such as business fields, JSON, text, and vectors; for images, documents, and other content, associations can be established through file references and business object IDs, linking them to their derived representations and business records.

2. Lake Data. Many customers' data has already settled in data lakes. MirrorData unifies access through external Catalogs, supporting mainstream lake table formats such as Paimon, Iceberg, and Hive, providing efficient lake table access capabilities.

3. Underutilized Historical Files. Existing images, documents, and audio-visual materials in enterprises often find it difficult to participate directly in business analysis. Original files can be retained in object storage, and after parsing and vector generation as needed for tasks, file references, extracted text, and vectors can be written into tables, associating with business fields. MirrorData unifies retrieval, filtering, association, and aggregation.

One SQL to Complete Mixed Retrieval

Based on ultra-fast analysis and data lake queries, MirrorData further expands multi-modal retrieval and association analysis. The same business object can simultaneously have structured fields, JSON attributes, text, embeddings, and file references, connecting content and business facts through business object IDs, allowing retrieval results to continue participating in analysis.

The underlying capabilities are three types of queries:

• Structured Data: Completing ultra-fast analysis through partition pruning and columnar scanning.

• Full-text Retrieval: Performing term matching based on inverted indexes, for example, matching records in descriptions that contain "unboxing" or "packing" through MATCH_ANY.

• Vector Retrieval: Finding related content based on similarity or distance.

One SQL can link the three together, with the specific access path depending on the query method and indexing method.

Demonstration: After-Sales Order Query

To intuitively showcase SQL and query results, we created an after-sales retrieval operation page. Users upload a photo of a damaged screen, looking for recent after-sales records with similar damage patterns, and then associate order, batch, carrier, and refund information to provide a basis for subsequent analysis.

First, the reference photo is converted into a query vector, serving as a parameter for the retrieval SQL. Then, the SQL calculates image similarity, supplementing batch, carrier, and refund information through the association between after-sales orders and order details, and adding time, category, and "unboxing/packing" full-text conditions.

These conditions ultimately form a complete SQL. In other words, one SQL can achieve unified retrieval of structured and unstructured data.

Order Image Retrieval

MirrorMind: Enabling Agents to Use the Right Standards

Once data can be queried, the next step is for Agents to understand the business definitions and background knowledge of the enterprise. MirrorMind, centered on Semantic View, addresses this issue.

For further details, you can read the Semantic View series articles:

• Semantic View Technical Analysis (Part One)

• Semantic View Technical Analysis (Part Two)

• Semantic View Technical Analysis (Part Three)

What is Defined in Semantic View

Semantic View records the structured definitions needed for Agents to understand the business, including:

• Tables and Associations: Defining the scope of data tables and the relationships between tables.

• Dimensions: Which dimensions can metrics be rolled up or drilled down on. For example, activity budgets can be broken down by channel, activity type, or summarized by date.

• Metrics: The calculation expressions and explanations for each metric. In addition to atomic metrics, derived metrics and complex calculation logic such as window functions can also be defined.

• Synonyms: Maintaining business names, aliases, and explanations for metrics, such as "total market budget" and "total promotional subsidy." Agents can use these definitions to identify the metrics referred to by users, reducing misunderstandings caused by different expression habits.

When an Agent receives a question, it first finds the applicable definition in Semantic View, then converts it into an executable statement according to this standard, and submits it to the database for execution.

Comparison: Is There a Semantic View?

Using the same question for comparison: "What is the company's activity budget?"

Without a Semantic View, the Agent directly queries the base table. The table contains four amount fields, and the Agent does not understand which one is the activity budget, so it sums all amount fields, resulting in approximately 29,161,800 yuan. This result is incorrect, as unrelated fields were also included in the calculation.

In the campaign_analytics used for demonstration, "Total Campaign Budget" corresponds to total_bbbb_amount, with the calculation expression being SUM(bbbb_amount). After reading this definition, the Agent sums the corresponding fields, resulting in 1,980,000 yuan; it did not include other monetary fields. The final execution in the video still uses the base SQL, while the Semantic View provides the business basis for selecting and calculating metrics.

The Semantic View organizes business names, metric formulas, and analysis dimensions into reusable definitions, allowing different Agents to use a common calculation basis, reducing deviations caused by individual interpretations of fields and interface paths. Query results can be verified against metric definitions and executed SQL.

Example: Order Domain for Retail Customers

Taking the order analysis of a retail customer as an example. In the order domain, it is necessary to distinguish between orders and order details. An order may contain multiple detailed items, and the statistical granularity of the two is not the same. The Semantic View establishes business themes based on orders and item details:

• The order theme defines "Number of Orders" and "Cancellation Rate," analyzed by store and month;

• The sales detail theme defines "Net Amount Received After Discount," analyzed by channel, category, and product.

The statistical boundaries of the two themes are also clearly defined: sales statistics count paid orders using the net amount received after discount from the sales detail theme; the cancellation rate is the number of canceled orders divided by the total number of orders.

The Semantic View defines how metrics are calculated and which dimensions can be analyzed; business rules supplement statistical boundaries and exceptions; business materials provide policies, events, and original text basis; analysis methods specify decomposition and verification steps; task memory retains confirmed conditions and conclusions. These contents can be independently maintained, managed by MirrorMind for associations, sources, versions, and access permissions, and then used by Agents according to tasks. This illustrates that Context is not only the Semantic View but also supports continuous knowledge iteration in business.

Managing Knowledge Like Code

The context needed by Agents involves more than one Semantic View. For example, analyzing a metric anomaly often requires associating business knowledge and standard analysis processes. We collectively refer to this as enterprise context. MirrorMind manages these contexts uniformly: the associations, sources, versions, and access permissions of the content.

The management method is similar to code; we draw on Git's ideas: knowledge can create branches, each branch having different versions; it can be traced back by version, allowing snapshots of any version at a certain time; after business confirmation, the correct version goes live. As business iterates, knowledge also iterates, but each Agent receives a version that has been confirmed by the business and is live. User feedback leaves a complete trace record, becoming input for the next round of knowledge iteration.

Multiple tasks reuse confirmed business definitions and background knowledge, reducing discrepancies; knowledge updates are adopted after verification, retaining versions and revision bases for subsequent task reuse and tracing.

MirrorPilot: Agent's Operation and Delivery Environment

Data can be queried, and the metrics are reliable; the last step is to organize them into Agents and deliver them to the business. This is the responsibility of MirrorPilot. It is the operational and delivery environment for enterprise Agents, linking MirrorData and MirrorMind downwards and delivering results externally.

Completing a Task in Four Steps

On the user side, there are two paths to issue tasks. One is direct dialogue, completing analysis through the product workspace; the other is through API integration with existing business systems, reusing Agent capabilities.

The execution of a task is divided into four steps. First, understand the objective; then, combine context and existing Skills to organize an execution plan; next, call tools, such as querying specific definitions in the Semantic View; finally, process the results. Simple tasks are delivered directly at this point, while complex tasks will involve repeated retrievals and multiple executions, returning the final results to the business.

MirrorMind provides context as needed for tasks, while MirrorData is responsible for querying and verifying results. Surrounding the operation itself is a complete set of peripheral management: session management, task status, resource scope, budget, timeout, and operation records, all built into MirrorPilot.

Delivering More Than Just Answers

The deliverables of MirrorPilot can be an answer, a report, or an API. The execution process will strictly leave traces: which SQL was used for this calculation, what the metric rules are, all clearly traceable. Users can continue to ask questions or trace the entire task.

Open Access, Capability Accumulation

Access is open. MirrorPilot provides Agent APIs, allowing enterprises to initiate tasks from existing BI, operational portals, or business systems, obtaining task results or session links after server authentication, and completing integration according to existing application interaction methods. Custom Skills are used to extend analysis steps, and MCP is used to access external tools.

Capabilities can accumulate. Existing analysis logic can be condensed into SQL templates, allowing similar problems to reuse query logic based on time, dimensions, and other parameters, returning results after execution, reducing the work of generating queries repeatedly. Business knowledge, SQL templates, and Skills can all be continuously reused and iterated, and support the access and management of custom Skills.

Thus, MirrorPilot can operate independently or integrate with existing enterprise applications, gradually expanding analytical capabilities.

MirrorControl: Supporting Management Control Backend

Accompanying MirrorPilot is the MirrorControl management control backend.

When configuring Agents, here you select the data scope, knowledge base, and the Semantic Views and Skills to be used. It also provides knowledge retrieval capabilities, such as "How is the cancellation rate defined internally in the enterprise?" Inputting a question will yield definitions and prompts.

Example: A Business Analysis Agent for a Retail Customer

Finally, using a desensitized case, we connect the three layers: a certain customer configured their own business analysis Agent using MIP.

Configuration: First Define the Scope

The first step in configuring the Agent is to define the business scope: what is the data range required by the Agent, which Semantic Views are related to the business domain, and where the relevant knowledge base is. The purpose of defining the scope is to narrow the Agent's retrieval range, avoiding knowledge confusion that could lead to incorrect answers.

Next is the configuration of roles, response rules, and Skills. For example, what standard processes to follow during follow-up questions and drilling down, and what style to use for output reports, all defined here. The querying and retrieval tools are built into the Agent, allowing direct retrieval of internal enterprise knowledge and Semantic Views.

Demonstration: From a Question to a Report

After configuration, let’s look at a complete usage process.

Enter the platform, first select the Agent, whose analysis scope has already been defined during configuration. Directly ask questions in natural language, such as asking the Agent to break down sales by channel. The Agent will provide analysis results for each channel, along with charts and the original SQL.

For answers involving data queries, users can view the relevant query statements, metrics, and time ranges, and verify results against calculation standards. Retaining query and knowledge sources makes the analysis process easier to trace.

Then follow up with, "Is this metric normal? Is it reasonable to break it down by department?" The Agent will invoke standard processes and SOPs from the enterprise context, such as identifying anomalies in the cancellation rate. The rules for anomaly determination and analysis methods are uniformly managed by MirrorMind as enterprise context, while MirrorPilot reads relevant knowledge, organizes steps, and calls MirrorData to execute queries.

If you ask again, "Why did it spike?" the Agent will continue to check categories, brands, and products according to the analysis methods, and retrieve warehouse events and inventory records. It combines this information with changes in store cancellation rates to propose possible reasons, related evidence, and suggested actions for business personnel to further verify.

After the analysis is complete, users can continue to request the generation of a management report through dialogue. The Agent will organize the data, charts, business materials, and analysis conclusions from the current task into an HTML report that can be opened and downloaded for discussion and verification by the business department.

Demonstration Video

Returning to the Three Initial Questions

MirrorData allows retrieval results to continue participating in SQL associations and analysis; MirrorMind centrally maintains business metrics and enterprise knowledge, enabling multiple Agents to reuse common definitions and bases; MirrorPilot organizes data, Context, and analysis methods into business tasks, delivering verifiable and traceable answers and reports.

The three questions rest on the same foundation, and that foundation is MIP.

This content is provided for general informational purposes only and doesn't constitute financial, investment, legal, or tax advice. Any events, rewards, online promotions, or related information mentioned herein should not be considered a recommendation, solicitation, or invitation to purchase, sell, trade, or otherwise deal in any crypto assets. Crypto assets are highly volatile and may result in loss. The availability of WEEX services, products, and related events may vary by region. You are responsible for ensuring that your participation is in accordance with applicable local laws and regulations.