Temporarily save a Python object and reuse it in another Python

This article is the 16th day article of Takumi Akashiro Alone Advent Calendar 2020.

Finally, the stock of this ad-care that had been for several days has run out, The deadline was so high that I couldn't post in advance in the morning!

The feeling of being driven is fun!

At the beginning

Have you ever wanted to temporarily save a Python object?

For example, debugging ... No, nowadays, with DCC tools, debugging via ptvsd with VS Code is a major [^ 1] ... For example, you want to use a variable with a complicated structure used in Maya in Python of another process ... This seems to be the last minute. Let's assume this today!

[^ 1]: I haven't faced Python very much this year, so I'm scared to write that even this method may be out of date.

What is serialization?

In object-oriented programming, writing an object as a file is called serialization.

According to Wikipedia, it is written as follows.

Serialization-Wikipedia The second meaning, serialization, is a term used in object-oriented programming to convert an object that exists in an environment into a byte string or XML format. More specifically, it means writing out a variable (field) that represents the state of the object and, in some cases, some identifier that represents the type (class) of the object in the form of a byte string or XML format that can be filed. .. This makes it possible to save the data represented by the object as a file or send it over the network. The byte string and XML format obtained in this way are restored to the original object by serialization restoration or deserialization.

Also, serializing objects and saving them in persistent storage such as files is called persistence. ``

... I knew the word itself, but for the first time I looked it up for today's article and knew it was an object-oriented term.

And the standard module that does serialize in Python is ... pickle! !! Let's use it for the time being!

Try using pickle

Let's save it first!

#! python3.8
import pickle
import tempfile
import pathlib
import re

data = {"nemui": "nero", "ohayou": "oyasumi", "nanka": "kanka", "hoge": pathlib.Path("d:/"), "reg": re.compile("\D{2}\d{16}")}

with open(pathlib.Path(tempfile.gettempdir()) / 'sample.pickle', 'wb') as f:
    pickle.dump(data, f)

It's coming, let's read this next. image.png

#! python3.8
import pickle
import tempfile
import pathlib

with open(pathlib.Path(tempfile.gettempdir()) / 'sample.pickle', 'rb') as f:
    data = pickle.load(f)

for k, v in data.items():
    print(f"\t{k}:\t{v} ({type(v)})")

image.png

You can restore the variables properly, and the order is preserved according to the implementation of dict in Python3! (It is natural because the order is maintained for the convenience of the mounting side ...)

Read the pickle output in Python3 in Python2

Now let's bring this to Python2! Ah ... Python2 isn't on my current personal computer!

It can't be helped, do you move it in the old Houdini ... image.png

I wonder if the position of temp is different, you ... Aside from the farce (?), Let's take a second look and run it in Python2 this time.

#! python2
import pickle
import tempfile

with open(tempfile.gettempdir() + '/sample.pickle', 'rb') as f:
    data = pickle.load(f)

for k, v in data.items():
    print("\t{0}:\t{1} ({2})".format(k, v, type(v)))

image.png

I get an error saying "Protocol Gachigaimasu"!

You did it! (?)

Read the official documentation

If you have a problem, read the official document!

pickle --- Serialization of Python Objects — Python 3.9.1 Documentation Currently, there are 6 protocols available for pickle. The higher the protocol used, the more you will need the latest version of Python needed to read the generated pickle.

  • Protocol version 3 was added in Python 3.0.
    It explicitly supports byte objects and cannot be unpickled in Python 2.x.
  • Protocol version 4 was added in Python 3.4.
    Added support for very large objects, pickling for more types of objects, and some data format optimizations. This is the default protocol starting with Python 3.8. See PEP 3154 for improvements made with Protocol 4.
  • Protocol version 5 was added in Python 3.8.
    Added support for out-of-band data and faster in-band data. See PEP 574 for improvements to Protocol 5.

(Omitted)

pickle.dump(obj, file, protocol=None, *, fix_imports=True, buffer_callback=None)

So, if you replace it with pickle (data, f, protocol = 2) ...

image.png

Ah ... yes ... pathlib wasn't in Python 2 ... It can't be helped, so if you replace the path that was set to pathlib with string

image.png

You can read the variables of python3 safely with Python2 system! (Do you need this crap to stop using Python2 anymore?)

Tightening

I use it very rarely, but I recommend it because it is reasonably convenient.

Well, if it's a variable with a simple structure, or if you can sacrifice speed Personally, I prefer json, which is highly readable.

It's easy to deal with when you traverse.

Recommended Posts

Temporarily save a Python object and reuse it in another Python
Draw a weakness graph in Python and save it in various formats (Raspberry Pi, macOS)
Create a JSON object mapper in Python
Organize python modules and packages in a mess
I wrote a class in Python3 and Java
Let's write a Python program and run it
Create a datetime object from a string in Python (Python 3.3)
POST JSON in Python and receive it in PHP
Insert an object inside a string in Python
[Python] How to save the installed package and install it in a new environment at once Mac environment
Object oriented in python
A Python script that crawls RSS in Azure Status and posts it to Hipchat
The result of making a map album of Italy honeymoon in Python and sharing it
[Selenium] Open the link in a new tab and move it [Python / Chrome Driver]
I want to replace the variables in the python template file and mass-produce it in another file.
Until you get daily data for multiple years of Japanese stocks and save it in a single CSV (Python)
Try to make it using GUI and PyQt in Python
Get the matched string with a regular expression and reuse it when replacing on Python3
I wrote python3.4 in .envrc with direnv and allowed it, but I got a syntax error
Create a function in Python
Create a dictionary in Python
Shuffle the images in any directory with Python and save them in another folder with serial numbers.
String object methods in Python
It is blocked by Proxy, a connection error occurs in Python or pip, and it is retried.
Null object comparison in Python
Associate Python Enum with a function and make it Callable
How to interactively draw a machine learning pipeline with scikit-learn and save it in HTML
Get a Python web page, character encode it, and display it
Recursively get the Excel list in a specific folder with python and write it to Excel.
A standard way to develop and distribute packages in Python
Collect tweets using tweepy in Python and save them in MongoDB
Make a bookmarklet in Python
Create code that outputs "A and pretending B" in python
Stack and Queue in Python
I created a class in Python and tried duck typing
Create a deploy script with fabric and cuisine and reuse it
Python variables and object IDs
Object-oriented in C: Refactored "○ ✕ game" and ported it to Python
Unittest and CI in Python
Draw a heart in Python
Install CaboCha in Ubuntu environment and call it with Python.
Just try to receive a webhook in ngrok and python
AtCoder ABC168 A case expression solved in Ruby and Python
An easy way to view the time taken in Python and a smarter way to improve it
Started Python: Swap an array of values obtained from SQL results to a list type and use it in IN of another query
[Python] The role of the asterisk in front of the variable. Divide the input value and assign it to a variable
processing to use notMNIST data in Python (and tried to classify it)
How to install OpenCV on Cloud9 and run it in Python
Process Splunk execution results using Python and save to a file
Delete a particular character in Python if it is the last
Specify or create a python folder and then save the screenshot.
Read big endian binary in Python and convert it to ndarray
Draw a watercolor illusion with edge detection in Python3 and openCV3
[Python3] Save the mean and covariance matrix in json with pandas
I want to create a pipfile and reflect it in docker
Create a CGH for branching a laser in Python (laser and SLM required)
Maybe in a python (original title: Maybe in Python)
MIDI packages in Python midi and pretty_midi
Difference between list () and [] in Python
[python] Manage functions in a list
Touch a Python object from Elixir