I wrote a script to extract a web page link in Python

BeautifulSoup4, great.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# python 2.x
#
# usage: python this_script.py "http://qiita.com/"
#
#Install beautifulsoup4 in advance
# pip install beautifulsoup4

import codecs
import urllib2
import sys
from bs4 import BeautifulSoup

url = sys.argv[1]
html = urllib2.urlopen(url).read().decode('utf-8', 'ignore')
soup = BeautifulSoup(html, "html.parser")
links = [a.get("href") for a in soup.find_all("a")]

for l in links: print l

Recommended Posts

I wrote a script to extract a web page link in Python
I wrote a function to load a Git extension script in Python
I wrote a script to get a popular site in Japan
A memo that I wrote a quicksort in Python
I want to create a window in Python
I wrote a class in Python3 and Java
I wrote "Introduction to Effect Verification" in Python
Steps to develop a web application in Python
I wrote a script to upload a WordPress plugin
Extract data from a web page with Python
I wrote a code to convert quaternions to z-y-x Euler angles in Python
I made a web application in Python that converts Markdown to HTML
I want to embed a variable in a Python string
I want to easily implement a timeout in python
I wrote python in Japanese
I tried to implement a pseudo pachislot in Python
A memorandum to run a python script in a bat file
I want to work with a robot in python.
I searched for the skills needed to become a web engineer in Python
I was soberly addicted to calling awscli from a Python 2.7 script registered in crontab
I tried to implement a one-dimensional cellular automaton in Python
I wrote a program quickly to study DI with Python ①
[Django3] Display a web page in Django3 + WSL + Python virtual environment
I tried "How to get a method decorated in Python"
I wrote an empty directory automatic creation script in Python
I wrote a script to combine the divided ts files
I tried to make a stopwatch using tkinter in python
I want to make input () a nice complement in python
I wrote a script that splits the image in two
I wrote Fizz Buzz in Python
I wrote the queue in Python
I wrote the stack in Python
I made a script in Python to convert a text file for JSON (for vscode user snippet)
I tried to create a Python script to get the value of a cell in Microsoft Excel
I wrote a doctest in "I tried to simulate the probability of a bingo game with Python"
I wrote a script to create a Twitter Bot development environment quickly with AWS Lambda + Python 2.7
How to pass arguments to a Python script in SPSS Modeler Batch
I just wrote a script to build Android on another machine
I tried to implement a misunderstood prisoner's dilemma game in Python
I wrote a script to help goodnotes5 and Anki work together
I tried to implement PLSA in Python
I tried to implement permutation in Python
I want to print in a comprehension
I made a payroll program in Python!
Pin current directory to script directory in Python
Sample script to trap signals in Python
I tried to implement PLSA in Python 2
I want to build a Python environment
Run the Python interpreter in a script
How to get a stacktrace in python
How to extract polygon area in Python
I tried to implement ADALINE in Python
I wanted to solve ABC159 in Python
I tried to implement PPO in Python
I made a script to display emoji
I tried to make a Web API
I created a password tool in Python.
How to run a Maya Python script
Get a capture of the entire web page in Selenium Python VBA
AtCoder writer I wrote a script to aggregate the contests for each writer
I want to convert a table converted to PDF in Python back to CSV