Create an application that just searches using the Google Custom Search API with Python 3.3.1 in Bottle

Leave for later people.

--There are few materials for Python3 -Bottle has few Japanese materials

For the above reasons, I had a hard time in a wasteful place ...

table of contents

  1. Reason for this combination
  2. Custom search example
  3. The part to search using Custom Search
  4. The part that returns the response as a Web application
  5. Completion
  6. Impressions
  7. Source code

Reason for this combination

Reasons for Python 3

--First, I decided to use Python for my research. Python with Numpy and Scipy is suitable for research on information retrieval. --Python3 has utf-8 as the default encoding, so it is easier to handle Japanese than Python2.

Reason for Bottle

――Since it is a research on the Web, I thought that it should be implemented as a Web application. --Do not use DB. So the micro framework is the best. --Flask and Bottle are famous as Python micro-frameworks. But Flask doesn't support Python 3. (As of May 23, 2013. Maybe it will be supported soon) Addendum: As of November 29, 2013, Flask is compatible with Python3! !! Yeah!

Custom Search example

See this article for sample code to search using the Custom Search API. http://qiita.com/items/92febaf8bbea541b1e36

However, the example is Python2 series code, so some modifications are needed.

The part to search using Custom Search

Define the process that actually fetches the json of the search result as a function called simple_search (query) in googlesearch.py.

googlesearch.py


import urllib
import urllib.request
import urllib.parse
import json


def simple_search(query):
    QUERY = query
    API_KEY = 'AIzaSyBdhBWUc5W3Aco3YGPwOlS_rYM0LENl_jo'
    NUM = 1

    url = 'https://www.googleapis.com/customsearch/v1?'
    params = {
        'key': API_KEY,
        'q': QUERY,
        'cx': '013036536707430787589:_pqjad5hr1a',
        'alt': 'json',
        'lr': 'lang_ja', }
    start = 1

    for i in range(0, NUM):
        params['start'] = start
        request_url = url + urllib.parse.urlencode(params)
        try:
            response = urllib.request.urlopen(request_url)
            json_body = json.loads(response.read().decode('utf-8'))
            items = json_body['items']
        except:
            print('Error')

    return items

Put this file in the same directory as app.py. In addition, this code takes only one page, that is, 10 search results. If you want to take more, change the value of params ['start'] and turn the for loop.

important point

Attention in Python3. Some standard libraries have changed from Python 2.

--urllib.urlopen (url) is now urllib.request.urlopen (url). --urllib.urlencode (params) is now urllib.parse.urlencode (params).

Also, use your own API_KEY. You can get the API key for your Google account at https://code.google.com/apis/console/.

The part that returns the response as a Web application

Bottle writes the process corresponding to the URL and http request, that is, the controller code, in app.py.

This time, I used Mako as the template used for the view part. Put the templates used by Mako in the static / templates directory.

app.py


from bottle import Bottle, route, run, static_file, request
from mako.template import Template
import googlesearch
import pdb


template = Template(filename='static/templates/index.tmpl')
app = Bottle()


@route('/static/:path#.+#', name='static')
def static(path):
    return static_file(path, root='static')


@route('/results')
def results_get():
    return template.render(items='')


@route('/results', method='POST')
def results():
    query = request.forms.decode().get('query')
    items = googlesearch.simple_search(query)
    return template.render(items=items)


@route('/')
def greet():
    return template.render(items='')
run(host='localhost', port=1234, debug=True)

The point is the function of @route ('/ results', method ='POST'),

query = request.forms.decode().get('query')

To be. If you want to search in Japanese, you need decode (). This method is found on Stackoverflow, but if there are few Japanese materials It's sad that it takes a lot of time and effort to read the English manual and search for Stackoverflow when doing this kind of processing. English-speaking programmers don't think about multi-byte characters and it's hard.

Display the search results with the @route ('/ results', method ='POST') function. I think there is a way to search with the GET method instead of POST, but I don't know how to do it and I feel that POST is okay. And on the view side.

static/templates/index.tmpl


#coding: utf-8
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Subtask Search</title>
    <meta name="viewport" content="width=device-width, initial-scale=1.0">

    <!-- Loading Bootstrap -->
    <link href="static/css/bootstrap.css" rel="stylesheet">

    <!-- Loading Flat UI -->
    <link href="static/css/flat-ui.css" rel="stylesheet">
    <link rel="shortcut icon" href="static/images/favicon.ico">

    <!-- HTML5 shim, for IE6-8 support of HTML5 elements. All other JS at the end of file. -->
    <!--[if lt IE 9]>
      <script src="static/js/html5shiv.js"></script>
    <![endif]-->
  </head>
  <body>
    <div class="container">
      <div class="demo-headline">
        <a href="/">
        <h1 class="demo-logo">
          Subtask Search
        </h1>
      </a>
      </div> <!-- /demo-headline -->
        <div class="span4 offset4">
				<form action="/results" method="post">
      <input type="text" name="query" value placeholder="Input your task" class="span4 offset4" />
			<input type="submit" value="Search" />
			</form>
      </div>
      <div class="span8 offset2">
        <ul class="unstyled">
        % for item in items:
        <li>
          <a href= ${item['link']}>
            ${item['title']}
          </a>
        </li>
    % endfor
  </ul>
      </div>
    </div> <!-- /container -->



    <!-- Load JS here for greater good =============================-->
    <script src="static/js/jquery-1.8.2.min.js"></script>
    <script src="static/js/jquery-ui-1.10.0.custom.min.js"></script>
    <script src="static/js/jquery.dropkick-1.0.0.js"></script>
    <script src="static/js/custom_checkbox_and_radio.js"></script>
    <script src="static/js/custom_radio.js"></script>
    <script src="static/js/jquery.tagsinput.js"></script>
    <script src="static/js/bootstrap-tooltip.js"></script>
    <script src="static/js/jquery.placeholder.js"></script>
    <script src="http://vjs.zencdn.net/c/video.js"></script>
    <script src="static/js/application.js"></script>
    <!--[if lt IE 8]>
      <script src="static/js/icon-font-ie7.js"></script>
      <script src="static/js/icon-font-ie7-24.js"></script>
    <![endif]-->
  </body>
</html>

I used Flat UI for this index.tmpl to make it look nice. css and js are downloaded from Flat UI and put in the static directory.

Mako is working

       % for item in items:
        <li>
          <a href= ${item['link']}>
            ${item['title']}
          </a>
        </li>
    % endfor

Only the part of. app.py

    return template.render(items=items)

So, I put the items with the result of simple_search in Mako's items and render this view.

Complete

Subtask Search.jpg You can do this.

Impressions

It's hard to handle multi-byte characters. It should have been easier with Python3, but it's still hard. There are few Japanese materials, so it's even harder. I want everyone to share more know-how.

Source code

https://github.com/katryo/google_simple_search I left the entire application in. Please take a look.

Recommended Posts

Create an application that just searches using the Google Custom Search API with Python 3.3.1 in Bottle
Create an application using the Spotify API
Create an app that works well with people's reports using the COTOHA API
To automatically send an email with an attachment using the Gmail API in Python
Using the National Diet Library Search API in Python
I built an application with Lambda that notifies LINE of "likes" using the Qiita API
Create REST API that returns the current time with Python3 + Falcon
Regularly upload files to Google Drive using the Google Drive API in Python
Try using the Wunderlist API in Python
Try using the Kraken API in Python
Tweet using the Twitter API in Python
[Python] Quickly create an API with Flask
Image collection using Google Custom Search API
Build an application with Clean Architecture while using DI + mock in Python
Create a record with attachments in KINTONE using the Python requests module
Create an app that guesses students with python
Try using the BitFlyer Ligntning API in Python
Create an image with characters in python (Japanese)
Create an API server quickly with Python + Falcon
Try using the DropBox Core API in Python
I want to create an API that returns a model with a recursive relationship in the Django REST Framework
Create an application that inputs, displays, and deletes forms by using an array as a DB with Python / Flask.
Upload JPG file using Google Drive API in Python
Initial settings when using the foursquare API in python
Create an exe file that works in a Windows environment without Python with PyInstaller
A story about a Python beginner trying to get Google search results using the API
Create a tweet heatmap with the Google Maps API
How to create an image uploader in Bottle (Python)
[Python] Create an infrastructure diagram in 3 minutes using diagrams
A story that I wanted to display the division result (%) on HTML with an application using django [Beginner learns python with a reference book in one hand]
Creating an API that returns negative-positive inference results using BERT in the Django REST framework
Read English sentences by hitting Google Translate API with Python without using the distributed module
Let's create a script that registers with Ideone.com in Python.
Basic authentication with an encrypted password (.htpasswd) in bottle with python
Create an example game-like stage with just the Blender 2.80 script
Scripts that can be used when using bottle in Python
Create API with Python, lambda, API Gateway quickly using AWS SAM
Google search for the last line of the file in Python
Issue reverse geocoding in Japanese with Python Google Maps API
Play with YouTube Data API v3 using Google API Python Client
Create an API with Django
Call the API with python3.
If you think that the person you put in with pip doesn't work → Maybe you are using python3?
Create a graph that displays an image with a mouse hover using the data visualization library Dash
An easy way to hit the Amazon Product API in Python
Create an API that returns data from a model using turicreate
Create an alias for Route53 to CloudFront with the AWS API
[Python / Django] Create a web API that responds in JSON format
How to create a heatmap with an arbitrary domain in Python
[Ev3dev] Create a program that captures the LCD (screen) using python
[LINE Messaging API] Create a BOT that connects with someone with Python
I tried to make a todo application using bottle with python
Solve the subset sum problem with a full search in Python
[SEO] Flow / sample code when using Google Analytics API in Python
Getting the arXiv API in Python
Create Awaitable with Python / C API
Hit the Sesami API in Python
[Python] Hit the Google Translation API
[Python3] Google translate google translate without using api
Create an Excel file with Python3
Hit the Etherpad-lite API with Python