I tried scraping Yahoo weather (Python edition)

Introduction

It's close to my own memo, but I tried it so I wrote an article. Originally I did a lot of research to do rasp pie signage, I didn't have the information I wanted to pinpoint, so I ended up investigating only the scraping method and managed to do it myself. (I wasn't a Python user or a person in the web area, so I struggled for a moment ...) Yahoo has a weather api, but I didn't use it because I only know the amount of precipitation.

C # edition https://qiita.com/MonoShobel/items/3ae0c051d2aadba698a4 There is.

References I referred to the following article when scraping with Python. [Scraping with python and letting Google Home talk about the weather forecast](https://xn--u9jwc776tibf.net/?p=1494) [Introduction to Python Web Scraping Practice](https://qiita.com/Azunyan1111/items/9b3d16428d2bcc7c9406)

There are some good and bad explanations about scraping itself, so you should read it.

By the way, it's Yahoo weather, but maybe scraping isn't prohibited. Yahoo Finance was explicitly banned, but I couldn't find any mention of the ban on the weather. Just keep moderation. Even if you scrape regularly, I think that access every hour or 30 minutes is enough for the weather.

What to do I have a lot of information I want to get, but for the time being, I got the weather today in Tokyo. Get today and tomorrow's weather in the image below. 天気.png

Code You can get the weather in Tokyo today and tomorrow by actually executing the code. (I wrote and debugged various things with Visual Code.) Mostly in the comments you will find what you are doing. It's almost a copy / paste source, but the point of interest is getting the text part with select_one. I tried to get it automatically with Google Chrome, but it didn't work, so I searched by hand and it worked.

tenki.py


import urllib3
from bs4 import BeautifulSoup

#URL to access
url = 'https://weather.yahoo.co.jp/weather/jp/13/4410.html'

#Access the URL In the return value, the instance containing the access result and HTML etc. will be returned.
http = urllib3.PoolManager()
instance = http.request('GET', url)
#Extract HTML from instance and parse it for beautiful Soup
soup = BeautifulSoup(instance.data, 'html.parser')

#Get the weather text with CSS selectors.
#today's weather
tenki_today = soup.select_one('#main > div.forecastCity > table > tr > td > div > p.pict')
print ("How is the weather today"+tenki_today.text)

#Tomorrow's weather
tenki_tomorrow = soup.select_one('#main > div.forecastCity > table > tr > td + td > div > p.pict')
print ("Tomorrow's weather"+tenki_tomorrow.text)

#Wait for input so that the screen does not disappear in an instant
input()

Below, the execution result 実行画面.png

At the end I think you can do various things based on this code. I think that I will add it to this article or write it in the article of Razpa Signage if I get it regularly. Also, I think that there are some things that could not be executed due to environmental problems, so I will add that later.

Recommended Posts

I tried scraping Yahoo weather (Python edition)
I tried scraping Yahoo News with Python
I tried scraping with Python
I tried scraping with python
I tried web scraping with python.
I tried scraping
I tried Python> autopep8
I tried Python> decorator
I tried L-Chika with Raspberry Pi 4 (Python edition)
I tried playing mahjong with Python (single mahjong edition)
I tried fp-growth with python
Scraping weather forecast with python
[Python] I tried using OpenPose
I tried gRPC with Python
I tried [scraping] fashion images and text sentences in Python.
[Python scraping] I tried google search top10 using Beautifulsoup & selenium
I tried to touch Python (installation)
I tried using Thonny (Python / IDE)
I tried Grumpy (Go running Python).
I tried running prolog with python 3.8.2.
I tried Line notification in Python
I tried SMTP communication with Python
Get weather information with Python & scraping
[Python] I tried using YOLO v3
I tried scraping the ranking of Qiita Advent Calendar with Python
I tried to solve the ant book beginner's edition with python
I tried scraping conversation data from Askfm
I tried to summarize Python exception handling
I tried to implement PLSA in Python
I tried to implement permutation in Python
Wrangle x Python book I tried it [2]
Python3 standard input I tried to summarize
I tried sending an email with python.
I tried using Bayesian Optimization in Python
I tried using UnityCloudBuild API from Python
I tried to implement ADALINE in Python
I tried a functional language with Python
I tried recursion with Python ② (Fibonacci sequence)
I tried to implement PPO in Python
Python: I tried the traveling salesman problem
Wrangle x Python book I tried it [1]
Mayungo's Python Learning Episode 8: I tried input
[Python] I tried to calculate TF-IDF steadily
I tried to touch Python (basic syntax)
I tried to summarize Ansible modules-Linux edition
I tried the Python Tornado Testing Framework
#I tried something like Vlookup with Python # 2
I tried to make a traffic light-like with Raspberry Pi 4 (Python edition)
[Python / DynamoDB / boto3] List of operations I tried
Python scraping notes
I tried web scraping to analyze the lyrics.
I tried hundreds of millions of SQLite with python
Python Scraping get_ranker_categories
[Python] I tried substituting the function name for the function name
Scraping with Python
I tried cluster analysis of the weather map
I made a weather forecast bot-like with Python.
vprof --I tried using the profiler for Python
I tried "differentiating" the image with Python + OpenCV
Scraping with Python
I tried to get an image by scraping