Create a python script to check if the link at the specified URL is valid 2

The following updated version

Create a python script to check if the link at the specified URL is valid

code

Same as in here

Create URL list to be checked

Grep as much as possible the character string that followshttps: // ...under the current directory. Save as input.txt.

grep -r "https://" * > test.txt
cat test.txt | sed -e 's/.*https//' | sed "s/^/http/g"  > test.txt
cat test2.txt | sed 's/>//g' | sed 's/"//g' | sed 's/)//g' | sed 's/;//g' | sed 's/]//g' | cut -d' ' -f 1 > input.txt 

check

Check if you can access the URL of input.txt.

$ python3 check_url.py

#Output result
#Can access → OK
#Inaccessible → Not Found
#* However, the grep result may not be as intended, so check it.

NotFound:http://www.kernel.org/pub/linux/kernel/v5.x/linux-${PV}.tar.xz
OK:http://facebook.github.io/watchman/
...

check_url.py


#-*- using:utf-8 -*-
import urllib.request, urllib.error

with open('out.txt', 'w') as txt:
    txt.write("chdck result\n")

def checkURL(url):
    try:
        f = urllib.request.urlopen(url)
        f.close()
        return True
    except:
        return False

if __name__ == '__main__':

    with open("./input.txt") as f:
        for url in f:
            # print(url, end='')
            ret = checkURL(url)
            if ret == True:
                result = "OK:"
            else:
                result = "NotFound:"

            ret_text = result + url
            #ret_text = ret_text.replace('\n', '')
            print(ret_text)
            if ret != True:
                with open('out.txt', 'a') as txt:
                    txt.write(ret_text)

result

Output the result OK/NG to the following

cat out.txt

reference

Create a python script to check if the link at the specified URL is valid (https://qiita.com/seigot/items/534ca3089d217200a1d6) Extract the character string to the right of the specified character string from the input character string

Recommended Posts

Create a python script to check if the link at the specified URL is valid 2
Create a python script to check if the link at the specified URL is valid
I created a script to check if English is entered in the specified position of the JSON file in Python.
[Blender] Script to check if the selected one is a mesh
Check if the string is a number in python
How to check in Python if one of the elements of a list is in another list
Create a shell script to run the python file multiple times
Check if the URL exists in Python
What is the fastest way to create a reverse dictionary in python?
I want to receive the configuration file and check if the JSON file generated by jinja2 is a valid JSON
python> check NoneType or not> if a == None:> if a is None:
Python script to create a JSON file from a CSV file
[python] How to check if the Key exists in the dictionary
Probably the easiest way to create a pdf with Python3
Create a tool to check scraping rules (robots.txt) in Python
How to create a new file when the specified file does not exist — write if the file exists
I tried to create a Python script to get the value of a cell in Microsoft Excel
A Python script that goes from Google search to saving the Search results page at once
Python> __init__.py> Required to handle the specified directory as a package (empty file is acceptable)
A Python script that allows you to check the status of the server from your browser
Python will fail if there is a space after the backslash
Write a script in Shell and Python to notify you in Slack when the process is finished
How to check the memory size of a variable in Python
Delete a particular character in Python if it is the last
How to check the memory size of a dictionary in Python
I wrote a script to extract a web page link in Python
I made a function to check if the webhook is received in Lambda for the time being
Determine if standard output is piped when running a Python script
A script that returns 0, 1 attached to the first Python prime number
5 Ways to Create a Python Chatbot
Check if you can connect to a TCP port in Python
Have python check if the string can be converted / converted to int
[Golang] Check if a specific character string is included in the character string
What to do if there is a decimal in python json .dumps
[Python] What is a formal argument? How to set the initial value
What is the XX file at the root of a popular Python project?
Create a Python development environment locally at the fastest speed (for beginners)
[Introduction to Python] What is the difference between a list and a tuple?
Test & Debug Tips: Create a file of the specified size in Python
I made a program to check the size of a file in Python
Run the Python interpreter in a script
[python] [meta] Is the type of python a type?
A script that morphologically parses a specified URL
Script to create a Mac dictionary file
How to run a Maya Python script
Even if the development language is changed to python3 in Cloud9, version 2 is displayed in python --version
Python script to get a list of input examples for the AtCoder contest
Convert the cURL API to a Python script (using IBM Cloud object storage)
I made a script to record the active window using win32gui of Python
What to do if (base) is displayed at the beginning of the Mac terminal
[Python] List Comprehension Various ways to create a list
Edit Excel from Python to create a PivotTable
How to create a Python virtual environment (venv)
Check if the characters are similar in Python
I want to create a window in Python
How to create a JSON file in Python
If you want to create a Word Cloud.
Python> Run-time argument> Check if -d is attached
Create a New Todoist Task from Python Script
A convenient function memo to use when you want to enter the debugger if an error occurs when running a Python script.
Steps to create a Twitter bot with python