I want to do a full text search with elasticsearch + python

  1. download & install
bin/elasticsearch

A must-have plugin for elasticsearch

bin/plugin install mobz/elasticsearch-head

Plugin that analyzes morphological elements

bin/plugin install elasticsearch/elasticsearch-analysis-kuromoji/2.5.0
  1. setting kuromoji

If it is troublesome (restart elasticsearch)

config/elasticsearch.yml


index.analysis.analyzer.default.type: custom
index.analysis.analyzer.default.tokenizer: kuromoji_tokenizer

If you set by index

curl -XPUT http://localhost:9200/index1/ -d '
{
  "index": {
    "analysis": {
      "tokenizer": {
        "kuromoji": {
          "type": "kuromoji_tokenizer"
        }
      },
      "analyzer": {
        "analyzer": {
          "type": "custom",
          "tokenizer":"kuromoji"
        }
      }
    }
  }
}'
  1. check

Check analyzer

curl -XPOST http://localhost:9200/index1/_analyze?analyzer=analyzer&petty -d 'this is a pen'

{
  "tokens": [
    {
      "token": "this",
      "start_offset": 0,
      "end_offset": 2,
      "type": "word",
      "position": 1
    },
    {
      "token": "Is",
      "start_offset": 2,
      "end_offset": 3,
      "type": "word",
      "position": 2
    },
    {
      "token": "pen",
      "start_offset": 3,
      "end_offset": 5,
      "type": "word",
      "position": 3
    },
    {
      "token": "is",
      "start_offset": 5,
      "end_offset": 7,
      "type": "word",
      "position": 4
    }
  ]
}

Umm. It looks like it's working properly.

Sample registration 1

curl -XPUT http://localhost:9200/index1/type1/1 -d '{"text":"This is bread"}'

Sample registration 2

curl -XPUT http://localhost:9200/index1/type1/2 -d '{"text":"this is a pen"}'

Search!

curl -XGET http://localhost:9200/index1/type1/_search -d '{"query": {"match": {"text": "pen"}}}'
{
  "took": 5,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 0.15342641,
    "hits": [
      {
        "_index": "index1",
        "_type": "type1",
        "_id": "2",
        "_score": 0.15342641,
        "_source": {
          "text": "this is a pen"
        }
      }
    ]
  }
}

  1. python client
$ pip install elasticsearch

Recommended Posts

I want to do a full text search with elasticsearch + python
I want to make a game with Python
I want to write to a file with Python
I want to work with a robot in python.
[ML Ops] I want to do multi-project with Python
I want to run a quantum computer with Python
I want to do ○○ with Pandas
I want to debug with Python
I want to build a Python environment
I want to analyze logs with Python
I want to play with aws with python
I want to use a wildcard that I want to shell with Python remove
[Introduction] I want to make a Mastodon Bot with Python! 【Beginners】
I want to do Dunnett's test in Python
I want to use MATLAB feval with python
I want to create a window in Python
I want to use Temporary Directory with Python2
#Unresolved I want to compile gobject-introspection with Python3
I want to solve APG4b with Python (Chapter 2)
I want to do it with Python lambda Django, but I will stop
I want to do a monkey patch only partially safely in Python
[Python] Although it is a humanities, I will do my best to understand bit full search
I want to easily implement a timeout in python
I want to iterate a Python generator many times
I want to display only different lines of a text file with diff
I want to transition with a button in flask
I want to handle optimization with python and cplex
I want to climb a mountain with reinforcement learning
I tried to draw a route map with Python
I want to write in Python! (2) Let's write a test
I tried to make a castle search API with Elasticsearch + Sudachi + Go + echo
I want to randomly sample a file in Python
Links to do what you want with Sublime Text
I want to inherit to the back with python dataclass
Full bit search with Python
I want to split a character string with hiragana
[Python] I want to make a nested list a tuple
I tried to automatically generate a password with Python3
I want to AWS Lambda with Python on Mac!
I want to manually create a legend with matplotlib
I made a python text
I made a python library to do rolling rank
I want to do something in Python when I finish
I want to bind a local variable with lambda
[Python] I want to add a static directory with Flask [I want to use something other than static]
[Python] I want to use only index when looping a list with a for statement
I want to tell people who want to import from a higher directory with Python direnv
[Mac] I want to make a simple HTTP server that runs CGI with Python
I want to save a file with "Do not compress images in file" set in OpenPyXL
I want to be able to analyze data with Python (Part 3)
I want to specify another version of Python with pyvenv
I made a package to filter time series with python
I want to do something like sort uniq in Python
I wrote a program quickly to study DI with Python ①
I want to make a blog editor with django admin
I want to start a jupyter environment with one command
[Python] I want to get a common set between numpy
I want to start a lot of processes from python
[NetworkX] I want to search for nodes with specific attributes
I want to make a click macro with pyautogui (desire)
I want to be able to analyze data with Python (Part 4)