How to access with cache when reading_json in pandas

Purpose

Do not generate communication every time you create a DataFrame from a web resource with read_json

background

I want to get information using web api and do various things, but there is a limit to the number of requests, and if I access it every time, I immediately get caught in the upper limit. I don't need real-time information when writing test code, so I want to keep it locally in the cache.

Library

  1. pandas
  2. requests
  3. requests_cache
  4. StringIO

Sample code

import pandas as pd
import requests
import requests_cache
from io import StringIO

requests_cache.install_cache()
response = requests.get("https://financialmodelingprep.com/api/v3/income-statement/AAPL?apikey=demo")
content = response.content.decode()
df = pd.read_json(StringIO(content))

Commentary

requests_cache If the cache exists when fetching with request.get, communication is not performed and response is returned from the cache. You can change the cache name with requests_cache.install_cache ("test ") and manage each one. Caching is done with sqlite. You can delete the cache by deleting the file. StringIO Since the file name and URL are specified in pd.read_json, the str obtained by requests cannot be specified as it is. By using StringIO (content), you can treat str like a file.

Recommended Posts

How to access with cache when reading_json in pandas
How to write soberly in pandas
How to create dataframes and mess with elements in pandas
How to not escape Japanese when dealing with json in python
How to access environment variables in Python
How to work with BigQuery in Python
How to reassign index in pandas dataframe
How to read CSV files in Pandas
How to deal with memory leaks in matplotlib.pyplot
How to deal with errors when hitting pip ②
[REAPER] How to play with Reascript in Python
[Python] How to read excel file with pandas
How to deal with run-time errors in subprocess.call
How to deal with SessionNotCreatedException when using Selenium
How to use tkinter with python in pyenv
How to use Pandas 2
Convert 202003 to 2020-03 with pandas
How to display in the entire window when setting the background image with tkinter
How to convert / restore a string with [] in python
How to "cache enabled" access to FPGA memory from Linux
How to do hash calculation with salt in Python
Explain in detail how to make sounds with python
How to convert horizontally held data to vertically held data with pandas
How to deal with pyenv initialization failure in fish 3.1.0
How to do zero-padding in one line with OpenCV
How to run tests in bulk with Python unittest
How to extract null values and non-null values with pandas
How to extract non-missing value nan data with pandas
How to output CSV of multi-line header with pandas
How to convert JSON file to CSV file with Python Pandas
[Python] How to deal with pandas read_html read error
How to exit when using Python in Terminal (Mac)
I tried to summarize how to use pandas in python
Convert numeric variables to categorical with thresholds in pandas
How to deal with Executing transaction: failed in Anaconda
How to extract non-missing value nan data with pandas
How to deal with "Type Error: No matching signature found" error when using pandas fillna
How to update with SQLAlchemy?
How to use Pandas Rolling
How to cast with Theano
How to Alter with SQLAlchemy?
How to separate strings with','
How to RDP with Fedora31
How to develop in Python
How to Delete with SQLAlchemy?
[TensorFlow 2 / Keras] How to run learning with CTC Loss in Keras
How to handle static files when deploying to production with Django
How to output a document in pdf format with Sphinx
How to extract any appointment in Google Calendar with Python
How to not load images when using PhantomJS with Selenium
Ingenuity to handle data with Pandas in a memory-saving manner
[Django] How to give input values in advance with ModelForm
How to manipulate the DOM in an iframe with Selenium
How to resolve CSRF Protection when using AngularJS with Django
[Linux] How to deal with garbled characters when viewing files
How to specify command line arguments when debugging in PyCharm
[AWS] How to deal with "Invalid codepoint" error in CloudSearch
For beginners, how to deal with common errors in keras
Things to keep in mind when using Python with AtCoder
Precautions when changing unix time to datetime type in pandas
Things to keep in mind when using cgi with python.