Try to process Titanic data with preprocessing library DataLiner (Append)

Introduction

This is the 4th article that introduces each process of Python's preprocessing library DataLiner. This time I would like to introduce the Append system. This completes all the pre-processing currently implemented.
We are planning to release Ver1.2 with some pre-processing added after GW, so I would like to write an introductory article again at that time.

Release article: https://qiita.com/shallowdf20/items/36727c9a18f5be365b37 document: https://shallowdf20.github.io/dataliner/preprocessing.html

Installation

! pip install -U dataliner

Data preparation

Prepare Titanic data as usual.

import pandas as pd
import dataliner as dl

df = pd.read_csv('train.csv')
target_col = 'Survived'

X = df.drop(target_col, axis=1)
y = df[target_col]

AppendAnomalyScore The Isolation Forest is trained based on the data, and the outlier score is added as a new feature. Missing value completion and categorical variable processing are required before use.

trans = dl.AppendAnomalyScore()
process = make_pipeline(
    dl.ImputeNaN(),
    dl.RankedTargetMeanEncoding(),
    trans
)
process.fit_transform(X, y)

AppendCluster The data is clustered in KMeans ++, and as a result, the number of the cluster to which each data belongs is added as a new feature. Missing value completion and categorical variable processing are required before use. Scaling is also recommended.

trans = dl.AppendCluster()
process = make_pipeline(
    dl.ImputeNaN(),
    dl.RankedTargetMeanEncoding(),
    dl.StandardScaling(),
    trans
)
process.fit_transform(X, y)

AppendClusterDistance The data is clustered in KMeans ++, and as a result, the distance from each data to each cluster is added as a new feature. Missing value completion and categorical variable processing are required before use. Scaling is also recommended.

trans = dl.AppendClusterDistance()
process = make_pipeline(
    dl.ImputeNaN(),
    dl.RankedTargetMeanEncoding(),
    dl.StandardScaling(),
    trans
)
process.fit_transform(X, y)

AppendPrincipalComponent Principal component analysis is performed on the data, and the principal component is added as a new feature. Missing value completion and categorical variable processing are required before use. Scaling is also recommended.

trans = dl.AppendPrincipalComponent()
process = make_pipeline(
    dl.ImputeNaN(),
    dl.RankedTargetMeanEncoding(),
    dl.StandardScaling(),
    trans
)
process.fit_transform(X, y)

in conclusion

Introduced Append items of DataLiner. In the future, I would like to write an introductory article about the function when updating DataLiner.

Dataliner release article: https://qiita.com/shallowdf20/items/36727c9a18f5be365b37 Documentation: https://shallowdf20.github.io/dataliner/preprocessing.html GitHub: https://github.com/shallowdf20/dataliner PyPI: https://pypi.org/project/dataliner/