Blocks Library

The Read Dataset block allows you to read a dataset from the available data sources (csv or parquet files). You can select the dataset you want to use from a dropdown menu or search for it by name. Once selected, the block will read the dataset and make it available for subsequent transformations in the pipeline.

The Remove Constant block identifies and removes constant or duplicate columns from the dataset. Constant columns are those that have the same value across all rows, which do not provide any useful information for modeling and can be safely removed to reduce dimensionality.

It also provides an option to set a threshold for quasi-constant columns, which are columns that have a single value in a high percentage of rows (e.g., 95%). If the percentage of the most frequent value exceeds the specified threshold, the column will be considered quasi-constant and removed from the dataset.

This helps to further reduce dimensionality by eliminating columns that have very low variance.

The Imputation block allows you to handle missing values in your dataset by applying various imputation strategies. You can select the column(s) with missing values and choose an imputation method, (mean, median, most frequent, or a constant value).

This block helps to ensure that your dataset is complete and ready for analysis or modeling by filling in the gaps left by missing data.

The Outlier Handling block provides methods for detecting and removing outliers in your dataset. You can select the column(s) you want to analyze for outliers and choose a detection mode (e.g., Trimmer, User defined cap) alongside capping methods (e.g., IQR, Quantiles).

The Resample block allows you to change the frequency of your time series data. You can select whether to upsample or downsample the data. You need to select the Index (timestamp) column and the desired frequency (e.g., 5 seconds, 1 day).

When downsampling, you can select the method for aggregating the data (e.g., mean, sum, min, max).

The block will then resample the data accordingly, which can be useful for handling irregular time intervals or reducing the granularity of the data for analysis.

The Flatten block allows you to flatten nested data structures in your dataset, namely lists or dictionaries, into a tabular format.

The Math Functions block allows you to apply mathematical functions to columns in your dataset. You can select the column(s) you want to transform and choose a function to apply (e.g., logarithm, exponential, square root).

This can be useful for normalizing data, reducing skewness, or creating new features based on mathematical transformations.

The Quantile block calculates the specified quantiles for selected columns in your dataset. You can select the column(s) you want to analyze and specify the quantiles you want to compute (e.g., 0.25, 0.5, 0.75).

The Stats Summary block provides a summary of basic statistics for selected columns in your dataset. You can select the column(s) you want to analyze, and the block will compute statistics such as count, mean, standard deviation, minimum, and maximum values.

The Correlation block calculates the correlation between the selected columns of your dataset. There are 3 types of correlation that can be calculated: Pearson, Spearman, and Kendall

The block will output a correlation matrix that shows the pairwise correlation coefficients between the selected columns.

The Rename Columns block allows you to rename columns in your dataset. You can select the column you want to rename and provide a new name for it.

The Select Columns block allows you to select and keep a subset of columns from your dataset.

The Drop Columns block allows you to select and remove a subset of columns from your dataset.

The Pivot Table block creates a pivot table from your dataset. You can select the index column, the column(s) to pivot, and the values column(s). You can also choose an aggregation function (e.g., mean, sum, count) to apply to the values.

For example:

The Sorting block allows you to sort your dataset based on the values of one or more columns. You can select the column(s) to sort by and specify the sorting order (ascending or descending).

The Transpose block allows you to transpose your dataset, swapping rows and columns. This can be useful for certain types of analysis or visualization where you want to change the orientation of the data.

For example:

The Split Column block allows you to split a column in your dataset into multiple columns based on a specified delimiter. You can select the column you want to split and provide the delimiter (e.g., comma, space, hyphen).

The Concatenate Columns block allows you to concatenate 2 columns in your dataset or a column plus a string into a single column. You can select the columns/values and provide a separator (e.g., comma, space, hyphen) to use between the concatenated values.

The Filter Rows block allows you to filter and keep rows in your dataset based on specified conditions. The conditions are built between one column and another column or a value.

For example:

The Aggregation block allows you to create new features by applying aggregation functions to groups of data. You can select the column(s) to group by, the column(s) to aggregate, and the aggregation function (e.g., mean, sum, count).

The Math Features block allows you to create new features by applying mathematical operations to existing columns. You can select the columns to use and choose an operation (e.g., addition, subtraction, multiplication, division).

The Cyclical Features block allows you to create new features that capture the cyclical nature of certain data, such as time-based features (e.g., hour of day, day of week). You can select the column(s) to transform and specify the period of the cycle (e.g., 24 for hours in a day).

The output is a pair of sine and cosine transformed features that can be used to capture the cyclical patterns in the data.

The Create Column block allows you to create a new column in your dataset. The column may be a date range, a constant value, or a sequence of numbers.

The Relative Features block allows you to create new features by performing operations between columns. You can select the column(s), the reference column(s) and operation(s) (e.g., addition, subtraction, multiplication, division).

The Decision Tree Features block allows you to create new features based on the splits of a decision tree model. You can select the column(s) to use as input for the decision tree, alongside the target column. The block will then train a decision tree model and create new features based on the splits in the tree.

The Variance Threshold block allows you to remove features that show a low variance (based on a specified threshold) across the dataset.

Low variance features may not provide useful information for modeling and can be safely removed to reduce dimensionality.

The Univariate Feature Selection block allows you to select and keep the top k features based on how good they can predict the target variable, based on a scoring function (e.g., mutual information).

The Recursive Feature Elimination block allows you to select and keep the top k features by recursively considering smaller and smaller sets of features. It trains a model (Random Forest) on the dataset and removes the least important feature(s) based on feature importance until the specified number of features is reached.

The Select From Model block allows you to select and keep features based on a model’s performance. Each column is passed as input to a model (Gradient Boosting or Random Forest) and the performance of the model is used to determine the importance of the feature. The top k features are then selected based on their importance scores.

The Sequential Feature Selection block allows you to select and keep the top k features by sequentially adding or removing features based on model performance. It trains a model (Gradient Boosting or Random Forest) on the dataset and adds (forward selection) or removes (backward selection) features one at a time based on their importance until the specified number of features is reached.

The Statistical Tests block allows you to select and keep features based on statistical tests. You can select the column(s) to analyze and choose a statistical test (e.g., chi-squared, ANOVA) to apply. The block will then compute and display the test statistic for each feature.

The statistical tests supported are:

The Smart Correlated Selection block allows you to select and keep features based on their correlation with the target variable and with each other. You can select the column(s) to analyze, the correlation method (e.g., Pearson, Spearman). The block will then compute the correlation matrix and select features that are highly correlated with the target variable while being less correlated with each other. If two features are highly correlated with each other, the block will select which to keep based on different criteria (e.g., correlation with target, variance, missing values).

The MRMR block allows you to select and keep features based on the Minimum Redundancy Maximum Relevance (MRMR) algorithm. You can select the column(s) to analyze and the block will compute the relevance of each feature with the target variable while minimizing redundancy between features, selecting the top k features based on these criteria.

The Transformer block allows you to transform the data based on the desired transformer process. The processes are:

The PCA block allows you to perform Principal Component Analysis (PCA) on your dataset. You select the column(s), the SVD solver and the desired number of components.

The block aims to transform many correlated variables into a smaller set of uncorrelated variables (principal components) while preserving as much variation in the data as possible.

The String Transformer block offers simple transformations to string columns (e.g., conversion to upper case, or finding a substring).

The Convert DataType block allows you to modify the data type of a selected column(s). Particularly useful for converting string dates to datetime objects.

The Datetime eatures block allows you to extract features from a datetime column like month, day of week, hour etc.

The Lag features block creates time-shifted columns from the selected ones. The user inputs multiple shift values (as a list), with negative numbers denoting leads (future values).

This block is particularly useful for easily creating features for forecasting.

The Windows (rollng) block allows you to perform aggregations (e.g., sum) over a sliding partition of the values.

For example:

The Seasonal Decomposition block allows you to decompose a time series into three components: trend, seasonality and residuals.

This block is particularly useful for time series analysis.

The Remove Trend/Seasonality block allows you to remove the trend and/or seasonality components from a time series.

The Acf block generates the correlation of a given time series with itself at different lags (past values).

This block allows one to easily understand how the past influences the future.

The Pacf block generates the partial correlation of a given time series with itself at different lags (past values). Partial autocorrelation removes the influence of intermediate lags, providing a clearer picture of the direct relationship between a variable and its past values. Unlike autocorrelation, partial autocorrelation focuses on the direct correlation at each lag.

The Filter block applies a filter to a signal column which removes low and high frequencies. It offers a variety of methods:

The Decompose block decomposes a signal into different sources using different methods, such as Empirical Mode Decomposition (EMD) or Singular spectrum analysis (SSA)-based signal separation method.

The Smooth block smooths a signal using convolution or LOESS methods.

The Derivative block calculates up to the 3rd order derivative of a signal.

The Find Peaks block detects the peaks of a signal based on attributes such as minimum or maximum height.

These blocks offer specific signal-processing tools for Photoplethysmography (PPG) signals.

These blocks offer specific signal-processing tools for Electrodermal Activity (EDA) signals.