Tutorial: Standalone Usage

This tutorial demonstrates how to use the Data Cleaning Engine as a standalone Python package, without the AI-DAPT platform.

You will learn how to: - Install the package locally - Load a time series dataset - Perform data imputation - Detect and handle anomalous values

Installation

Clone the repository and install the package in editable mode:

pip install -e .

This allows you to modify the source code locally while using the package.

Quick Start

In this example, we work with a time series dataset and apply both data imputation and anomaly detection.

Load the Dataset

First, import the package and load a CSV file containing time series data. The name of the datetime column must be explicitly provided.

import datacleaner as dc

path = 'datasets/ETTw2.csv'
df = dc.timeseries.read_csv(path, date_col='date')

Data Imputation

To demonstrate imputation, we first create an artificial missing-value mask and apply it to the dataset. We then impute the missing values using a time series–specific method.

# Create missing values
mask = dc.utils.create_missing_mask(df, percentage=0.1)
miss_df = df.where(mask)

# Impute missing values
imp_df = dc.timeseries.imputation.impute(
    miss_df,
    method='saits'
)

The resulting DataFrame has the same shape as the input, with missing values replaced according to the selected method.

Anomaly (Error) Detection

Next, we simulate anomalous values and detect them using an anomaly detection method.

# Create artificial anomalies
mask = dc.utils.create_outlier_mask(df, percentage=0.1)
err_df = mask.combine_first(df)

# Detect anomalies
clean_df = dc.tabular.anomaly.detect(
    err_df,
    method='k_nearest_neighbors_detector'
)

Detected anomalous values are replaced with NaN, allowing them to be handled explicitly in subsequent processing steps.

What’s Next

This tutorial demonstrated standalone usage of the Data Cleaning Engine through its Python API.

To continue:

  • Follow the platform-based tutorial to see how the engine is used within the AI-DAPT ecosystem

  • Explore task-specific sections for available methods and their parameters