Python URL and query parameter parsing summary

tl;dr

This section summarizes how to parse URLs and query parameters in Python and convert percent encoding.

In Python3, modules such as URL acquisition and parsing are unified to ʻurllib`. The correspondence table is summarized below.

Python2 Python3
URL parse urlparse.urlparse urllib.parse.urlparse
Query string parsing urlparse.parse_qs urllib.parse.parse_qs
Query parameter creation urllib.urlencode urllib.parse.urlencode
escape urllib.quote urllib.parse.quote
Escape restore urllib.unquote urllib.parse.unquote

Python3

The code below assumes an import of ʻurllib`.

import urllib

URL parsing

urllib.parse.urlparse("http://example.com/example.html?foo=bar&hoge=fuga")
# => ParseResult(scheme='http', netloc='example.com', path='/example.html', params='', query='foo=bar&hoge=fuga', fragment='') 

Parsing query parameters

urllib.parse.parse_qs("foo=bar&hoge=fuga")
# => {'hoge': ['fuga'], 'foo': ['bar']}                                                                                           
urllib.parse.parse_qs("query=%E3%83%86%E3%82%B9%E3%83%88")
# => {'query': ['test']}

Create query parameters from dictionary

urllib.parse.urlencode({"query":"test"})
# => query=%E3%83%86%E3%82%B9%E3%83%88

Percent-encoding string conversion

urllib.parse.quote("test")
# => %E3%83%86%E3%82%B9%E3%83%88

urllib.parse.unquote("%E3%83%86%E3%82%B9%E3%83%88")
# =>test

Python2

The code below assumes the import of ʻurllib and ʻurlparse.

import urllib
import urlparse

URL parsing

urlparse.urlparse("http://example.com/example.html?foo=bar&hoge=fuga")
# => ParseResult(scheme='http', netloc='example.com', path='/example.html', params='', query='foo=bar&hoge=fuga', fragment='') 

Parsing query parameters

urlparse.parse_qs("foo=bar&hoge=fuga")
# => {'hoge': ['fuga'], 'foo': ['bar']}                                                                                           
urlparse.parse_qs("query=%E3%83%86%E3%82%B9%E3%83%88")
# => {'query': ['test']}

http://docs.python.jp/2/library/urlparse.html#urlparse.parse_qs

Create query parameters from dictionary

urllib.urlencode({"query":"test"})
# => query=%E3%83%86%E3%82%B9%E3%83%88

Percent-encoding string conversion

urllib.quote("test")
# => %E3%83%86%E3%82%B9%E3%83%88

urllib.unquote("%E3%83%86%E3%82%B9%E3%83%88")
# =>test

reference

Recommended Posts

Python URL and query parameter parsing summary
Summary of Python indexes and slices
Python Summary
Python summary
Correspondence summary of array operation of ruby and python
Summary of the differences between PHP and Python
Python tutorial summary
Installation of Python3 and Flask [Environment construction summary]
Compare xml parsing speeds with Python and Go
Add query to url parsed url and return to original url
[Django3] Environment construction and various settings summary [Python3]
python related summary
I / O related summary of python and fortran
Python basics summary
Python --Explanation and usage summary of the top 24 packages
[Python] Type Error: Summary of error causes and remedies for'None Type'
Summary of date processing in Python (datetime and dateutil)
[python] Compress and decompress
Batch design and python
Python iterators and generators
Python packages and modules
Vue-Cli and Python integration
Ruby, Python and map
Summary about Python3 + OpenCV3
python input and output
Python directory operation summary
URL shortening with Python
Python AI framework summary
Python iteration related summary
Summary of Python arguments
Python3, venv and Ansible
Python asyncio and ContextVar
[python] Summary of how to retrieve lists and dictionary elements
Summary of Hash (Dictionary) operation support for Ruby and Python
Sample of HTTP GET and JSON parsing with python of pepper
Module summary that automates and assists WebDriver installation with Python