Is it Google Colaboratory?

I read the fucking app Advent Calendar along with Pick up the 2020 Advent Calendar I want to read in 2020. So, along with "I made friends because I miss my friends (LineBot)" (a shit title: smiley :), I didn't know much about Google Colaboratory, so I looked it up.

Google Colaboratory is convenient! It's convenient, but I think it's not used much in GSuite, so let's use it! It is an article called.

point

  1. ** Python ** can be used
  2. So it can be used for ** simple spreadsheets, CSV and other formatting **
  3. And ** easy to share **

Overview: First Machine Learning with Google Colaboratory

Colab Notebooks are Jupyter notebooks hosted by Colab. For more information on the Jupyter project, please visit jupyter.org.

Anyway, a tool that can share and execute Python code, which is expected to grow for companies using GSuite. It is a story that it is a dream to use this rather than making a little god Excel rather than machine learning. Don't be afraid of the word machine learning. From the basics of the basics below, I will write what you can do, what you can do, what you cannot do, and what you cannot do, so I hope you find it helpful.

Basic: Hello world

Let's run the test code. Just press the play button. image.png

1. Python can be used

This is a chance because I don't usually touch Python in the main development. You can also send a message to Slack.

Send a message to Slack

Below, Python that sends a message to Slack with python and Webhook.

Send to Slack Try Slack's Incoming Webhook in Python Webhooks can also be set from the workflow builder.

image.png If you send a message with, it will be like this. image.png

import requests, json;
from bs4 import BeautifulSoup;
web_hook_url = 'https://hooks.slack.com/services/xxxxxxxxxx/xxxxxxxxxx/xxxxxxxxxx'

requests.post(web_hook_url, data = json.dumps({
    'text': 'Are you Slack?', #This is what you want to send
    'username': 'Do you have'  #It's the name of the bot
}));

On Colab: image.png that's all.

2. So it can be used for simple spreadsheets, CSV and other formatting

This seems to be useful. Data can be read from a certain file.

First CSV: How to read csv to dataframe in Google Colab As far as I can see, it is convenient to combine it with Create "operation log" CSV formatting tool in 5 days with Python Pandas PyInstaller.

I read the sample CSV below. image.png I also tried Operation Log and it worked. image.png

Please refer to Colaboratory data input / output summary.

CSV aggregation

Technology trend 2020 summary in 30,000 Hatena bookmarks etc. You can try things like inquiry trend analysis.

3. And easy to share

So, I want everyone to use it. Yes, it's the same as how you normally use GSuite. Empower and share.

image.png

By the way: thinking:

When that happens. Wouldn't it be quite convenient to do this Link with GAS? I thought. The following are precautions for use.

[Cannot] Autorun

Current status: Cannot. It seems. It seems that Selenium and Mechanism to force periodic execution are doing something about it. A little tricky.

[Sometimes you can't do it] Scraping

Going back to the scraping that was done at the beginning I made friends because I miss my friends (LineBot), for example, the following is detailed. Try to parse HTML with Python and collect data? "Python second grade" that understands scraping from the beginning

scraiping_sample.py


import requests
from bs4 import BeautifulSoup

#Get the web page and analyze it
load_url = "https://ameblo.jp/masayukimakino/entry-11084331526.html"
html = requests.get(load_url)
soup = BeautifulSoup(html.content, "html.parser")

#It will search for all strong tags and display the string.
for element in soup.find_all("strong"):    #Would you like to search for and display all strong tags?
    print(element.text)

Then, for example, you can do something like this. image.png Is scraping illegal? A lawyer will explain three legal issues and countermeasures in 5 minutes.

Purpose of use Target of scraping Compliance with access restrictions Terms of service

List of precautions for web scraping Let's talk about the law of web scraping! It is necessary to make it a tool well after considering. Therefore, study is required. Obtain Qiita trends (ranking) and send them to Slack Refer to the articles of our predecessors. It seems to be convenient for internally analyzing in-house materials.

[Caution] Save to Github

One more thing, I seemed to be confused, so I added it as a reflection statement. image.png

It also has an easy function to save a copy on Github. However, if you use it internally, it will be published on Github, so be careful.

Conclusion

The Jupyter guy introduced at the beginning, Are you still using Jupyter Notebook? A comfortable Python life with Jupyter life (.py) with VS Code ?! seems to be able to be further developed and used. Especially if you haven't touched Python before, it's easy to give it a try! These are the recommendations for in-house Gsuite tools.

Recommended Posts

Is it Google Colaboratory?
Google colaboratory
Google Colaboratory setup summary
Use music21 on Google Colaboratory
Try StyleGAN on Google Colaboratory
Study Python with Google Colaboratory
Is Vtuber Positive? Is it negative?
What is Google Cloud Dataflow?
Try OpenCV with Google Colaboratory
Tool organization: Google Colaboratory (updated 2020.2.24)
Pandas 100 knocks on Google Colaboratory
How to use Google Colaboratory
What is Python? What is it used for?
Is it a character string operation?
■ [Google Colaboratory] Use morphological analysis (janome)
■ [Google Colaboratory] Use morphological analysis (MeCab)
Snippets (scraping) registered in Google Colaboratory
OpenCV feature detection with Google Colaboratory
100 language processing knock 2020 "for Google Colaboratory"
Run Keras on Google Colaboratory TPU
I took Apple Watch data into Google Colaboratory and analyzed it
Is it deprecated to use pip directly?
Use The Metabolic Disassembler on Google Colaboratory
[Beginner] Python web scraping using Google Colaboratory
Usual processing notes when using Google Colaboratory
How to search Google Drive with Google Colaboratory
Make a cascade classifier with google colaboratory
Use cartopy without bugs in Google Colaboratory
Manage deals with Trello + Google Colaboratory (Part 1)
[Python] Complete preprocessing Memo as it is
First-principles calculations for free using Google Colaboratory
Using Java's Jupyter Kernel with Google Colaboratory
Use TPU and Keras with Google Colaboratory
Bakthat is good for backup, isn't it?
Free Python runtime environment Google Colaboratory memo