Developer Guide Overview

This section provides technical documentation for developers working on or integrating the Data Valuation Engine.

The Data Valuation Engine is a service-oriented system that supports:

  • Feature Correlation

  • Feature Importance

  • Spatial Bias Audit

  • Spatial Bias Mitigation

It is designed to run both as a standalone tool and as part of data pipeline.

Architecture

This section provides technical documentation for developers working on or integrating the Data Valuation Engine within the AI-DAPT platform. Specifically, it is delivered as a micro-frontend plus a backend service, and it can run standalone or integrated into AI-DAPT. The system is composed of the following main components:

  • Backend API – Python 3.10, FastAPI, Uvicorn

  • (Micro-) Frontend UI – Vue 3, TypeScript, Vite, Tailwind CSS

  • Machine Learning Modules – PyCaret for model training, SHAP for explainability

  • Communication with data pipelines and data storage – MongoDB

  • Authentication – Keycloak

The following image highlights the previously-mentioned modules of the system and how they interact with AI-DAPT services.

Feature Importance UI results
Figure 1. Data Valuation Engine Overview and Integration with AI-DAPT.

The backend exposes REST endpoints that the micro-frontend or other components can call for:

  • correlation analysis

  • feature importance analysis

  • spatial bias audit

  • spatial bias mitigation (relabeling and threshold adjustment)

  • task-based (HITL) workflows

The frontend consumes these APIs and visualizes results using tables, plots, and interactive maps.

Core Concepts (Developer Perspective)

  • Standalone vs data pipeline workflows Some endpoints operate on fully provided request payloads (standalone). Others operate on persisted configuration documents (Dave tasks), enabling Human-in-the-Loop workflows.

Repository Structure

.
├── backend/        # FastAPI backend
├── frontend/       # Vue 3 frontend
├── docker-compose.yml
├── .env-example
└── docs/           # Antora documentation (this section)

Getting Started (Local)

Prerequisites

  • Docker & Docker Compose

  • Git

Quick start

# Create Docker network (first time only)
docker network create experiment-network

# Build and run
docker compose up --build

The application will be available at:

Development Notes

  • MongoDB and Keycloak are controlled via environment variables.

  • Task-based (HITL) endpoints rely on persisted configuration documents and are typically used in interactive workflows.

Next Steps