Create a pandas Dataframe from a string.

Trigger

Note that I was having trouble converting a string to a pandas Dataframe. The trigger was a scene where I wanted to handle a file of about 300GB with pandas. It seems impossible with read_csv () directly, so it was good to cut it finely with open () or readline (), but by the way, it was the first time I made a Dataframe from a character string. When I looked it up, I couldn't find anything good.

solution

I tried using numpy's reshape ().

pd.DataFrame(np.array(data.split()).reshape(-1,7))

The image looks like this.

Fortunately, the number of columns was fixed. If the number of lines is not fixed, it seems better to put -1 in the argument of reshape (). The number of columns is also the same. Please let me know if there is another good way.

Recommended Posts

Create a pandas Dataframe from a string.

Create a dataframe from excel using pandas

How to create a function object from a string

[Python pandas] Create an empty DataFrame from an existing DataFrame

[python] Create table from pandas DataFrame to postgres

Create a datetime object from a string in Python (Python 3.3)

Django: Import a class from a string

Python-dynamically call a function from a string

Create a random string in Python

Create an instance of a predefined class from a string in Python

DataFrame of pandas From creating a DataFrame from two lists to writing a file

How to create a clone from Github

# 5 [python3] Extract characters from a character string

Create a standings from JFL match results

[Pandas] Expand the character string to DataFrame

[Pandas_flavor] Add a method of Pandas DataFrame

Create a deb file from a python package

Generate a class from a string in Python

How to create a repository from media

Download Pandas DataFrame as a CSV file

[Python / Tkinter] Search for Pandas DataFrame → Create a simple search form to display

Create a game UI from scratch with pygame2!

Create a New Todoist Task from Python Script

Create a record table from JFL match results

Create a phylogenetic tree from Biopyton using ClustalW2

Create a protein sequence mutation library in pandas

Create a score ranking from JFL match results

Extract lines containing a specific "string" in Pandas

Create a decision tree from 0 with Python (1. Overview)

Convert pandas dataframe elements to regular string type

Create a Word Cloud from an academic program

Create a Django schedule

Create a Python module

Create a Bootable LV

Create a Python environment

Python script to create a JSON file from a CSV file

How to create a kubernetes pod from python code

Python application: Pandas # 3: Dataframe

Create a machine learning environment from scratch with Winsows 10

Create a slack bot

Create a decision tree from 0 with Python and understand it (3. Data analysis library Pandas edition)

Create a new csv with pandas based on the local csv

Convert from Pandas DataFrame to System.Data.DataTable using Python for .NET

Create a data frame from the acquired boat race text data

I made a tool to create a word cloud from wikipedia

Put the lists together in pandas to make a DataFrame

A handy function to add a column anywhere in a Pandas DataFrame

Python: Create a dictionary from a list of keys and values

Python> Read from a multi-line string instead of a file> io.StringIO ()

Create a GCE instance from a GCR Docker image using terraform

Create a message corresponding to localization with python translation string

Create a correlation diagram from the conversation history of twitter

[Python] Random data extraction / combination from DataFrame using random and pandas

Create a Wox plugin (Python)

Create a local pypi repository

Create a function in Python

Create a dictionary in Python

[Python] Use a string sequence

Create a (simple) REST server

Create a homepage with django

Is this string a decimal?