Principal component analysis using python from nim with nimpy

Overview

I have summarized the procedure for performing principal component analysis using sklearn from Nim. Principal component analysis can be performed in a few lines by using sklearn, so I will try using a library called nimpy to execute it from Nim. The part that actually performs principal component analysis using sklearn is written in Python.

I've done it before about how to call Python with nimpy, so it's an application. Call python from nim with Nimpy

Preparation on the Python side

As in the example (refer to the above article), add CONFIGURE_OPTS ="-enable-shared " when inserting arbitrary Python with pyenv etc. so that libpython is generated. Then add sklearn using poetry etc.

pyproject.toml


[tool.poetry]
name = "nimpy_pca"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]

[tool.poetry.dependencies]
python = "^3.7"
numpy = "^1.18"
scikit-learn = "^0.22.2"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry>=1.0"]
build-backend = "poetry.masonry.api"

Since it is difficult to call PCA in sklearn from nim, write the process in python. (In nimpy, the method is called, so you can't use the style like initializer or syntax without return value.) I try to exchange data and results using JSON to simplify the passing of types.

pca.py


from sklearn.decomposition import PCA
import numpy as np
import json

def pca(json_text):
    data = json.loads(json_text)
    pca = PCA(n_components=2)
    A = np.array(data)
    pca.fit(A)
    return {
        "components": pca.components_.tolist(),
        "varianceRatio": pca.explained_variance_ratio_.tolist()
    }

Call Python from Nim with nimpy

First, specify the libpython path so that the sklearn entered in poetry can be read.

import nimpy
import nimpy/py_lib as pyLib

pyLib.pyInitLibPath("/root/.pyenv/versions/3.7.7/lib/libpython3.7m.so")

Also, add the path where the file is located so that you can load the Python you just created.

discard pyImport("sys").path.append("/workspace/src")

Then use the JSON data and pass the data to the previous method. I'm using toJson to convert the return value PyObject to a JsonNode type.

let pcaResult = pyImport("pca.py").callMethod("pca", json).toJson

The rest is completed by expanding the value contained in pcaResult.

import sugar

let projectedValues = datas.map(data =>
    pcaResult["components"].getElems.map(c => c.getElems.zip(data).map(n => n[0].getFloat * n[1]).foldl(a + b))
)

Summary

I got stuck in the way to specify the path of pyImport and the name collision of the python file, but I was able to read python this way. If you apply this, it seems that you can easily execute other statistical processing and linear processing by leaving it to Python.

Recommended Posts

Principal component analysis using python from nim with nimpy
Call python from nim with Nimpy
Principal component analysis with Power BI + Python
Using Rstan from Python with PypeR
Principal component analysis with Spark ML
Python: Unsupervised Learning: Principal Component Analysis
Principal component analysis
Principal Component Analysis with Livedoor News Corpus-Practice-
From preparation for morphological analysis with python using polyglot to part-of-speech tagging
Principal component analysis with Livedoor News Corpus --Preparation--
Dimensional compression with self-encoder and principal component analysis
I tried principal component analysis with Titanic data!
Robot grip position (Python PCA principal component analysis)
Data analysis with python 2
Data analysis using Python 0
Voice analysis with python
Principal component analysis (Principal component analysis: PCA)
Voice analysis with python
Data analysis with Python
2. Multivariate analysis spelled out in Python 3-2. Principal component analysis (algorithm)
Collaborative filtering with principal component analysis and K-means clustering
Learn Nim with Python (from the beginning of the year).
Collecting information from Twitter with Python (morphological analysis with MeCab)
Mathematical understanding of principal component analysis from the beginning
Principal component analysis Analyze handwritten numbers using PCA. Part 2
Principal component analysis (PCA) and independent component analysis (ICA) in python
Principal component analysis Analyze handwritten numbers using PCA. Part 1
2. Multivariate analysis spelled out in Python 3-1. Principal component analysis (scikit-learn)
Flatten using Python yield from
[Python] Morphological analysis with MeCab
[Co-occurrence analysis] Easy co-occurrence analysis with Python! [Python]
Using Quaternion with Python ~ numpy-quaternion ~
[Python] Using OpenCV with Python (Basic)
Sentiment analysis with Python (word2vec)
Unsupervised learning 3 Principal component analysis
Planar skeleton analysis with Python
Data analysis using python pandas
With skype, notify with skype from python!
Muscle jerk analysis with Python
Using OpenCV with Python @Mac
Send using Python with Gmail
I tried using the Python library from Ruby with PyCall
Coursera Machine Learning Challenges in Python: ex7-2 (Principal Component Analysis)
Visualize the correlation matrix by principal component analysis in Python
Principal component analysis hands-on with PyCaret [normalization + visualization (plotly)] memo
Morphological analysis using Igo + mecab-ipadic-neologd in Python (with Ruby bonus)
Complement python with emacs using company-jedi
Harmonic mean with Python Harmonic mean (using SciPy)
Call C from Python with DragonFFI
3D skeleton structure analysis with Python
Trial of voice recognition using Azure with Python (input from microphone)
[Python] Using OpenCV with Python (Image transformation)
Impedance analysis (EIS) with python [impedance.py]
Create folders from '01' to '12' with python
Introduction to Python Basics of Machine Learning (Unsupervised Learning / Principal Component Analysis)
Notes on using MeCab from Python
[Python] Using OpenCV with Python (Edge Detection)
Text mining with Python ① Morphological analysis
Introduction to Data Analysis with Python P17-P26 [ch02 1.usa.gov data from bit.ly]
[In-Database Python Analysis Tutorial with SQL Server 2017] Step 6: Using the model
Using Cloud Storage from Python3 (Introduction)