Analyzing and visualizing csv logs with Excel Elastic Stack (docker-compose) --Two ways to deal with Logstash OutOfMemoryError

Introduction

Thanks! An engineer in charge of the product inspection process in the production engineering department. Analyzing and visualizing csv logs with Excel Elastic Stack (docker-compose) --Elastic Stack is a continuation.

Target audience

This article is intended for those who are new to Elastic Stack and who are thinking about trying it out.

Content of this article

When transferring a large number of files from filebeat to logstash, logstash may crash with the following error. I will show you how to deal with it.

java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid1.hprof ...
Heap dump file created [3178685347 bytes in 34.188 secs]
warning: thread "[main]>worker11" terminated with exception (report_on_exception is true):
warning: thread "[main]>worker4" terminated with exception (report_on_exception is true):
java.lang.OutOfMemoryError: Java heap space

I have put a set of configuration files in GitLab, so please refer to it. Click here for repository-> elastic-stack

Cause of error

As the error content shows, the cause is insufficient heap size.

What to do if there is enough physical memory size

Create a jvm.options file and change the following heap size. Follow the instructions in JVM Setting to find out how much you should set. Please consult.

logstash/config/jvm.options


## JVM configuration

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms1g
-Xmx1g

What to do if there is not enough physical memory size

You can limit the amount of processing by changing pipeline.batch.size or pipeline.workers. The default value for batch size is 125, and workers is the number of CPUs on the host PC. The official documentation is here.

logstash/config/pipelines.yml


- pipeline.id: filebeat-processing
  pipeline.batch.size: 50

Finally

Regarding Elasticsearch, according to the Official Blog, the heap area usage of Elasticsearch from 7.7 Seems to go down significantly. If you are having trouble with Elasticsearch memory, please review the version.

Recommended Posts

Analyzing and visualizing csv logs with Excel Elastic Stack (docker-compose) --Two ways to deal with Logstash OutOfMemoryError
Analyzing and visualizing csv logs with Excel Elastic Stack (docker-compose) --What is Elastic Stack?
Analyzing and visualizing csv logs with Excel Elastic Stack (docker-compose) --How to deal with data duplication errors in Elasticsearch
Analyze and visualize csv logs with Excel Elastic Stack (docker-compose) --Set up with docker-compose
Analyze and visualize csv logs with Excel Elastic Stack (docker-compose) --Receive input from multiple beats with Pipeline-to-Pipeline of Logstash
Build Elastic Stack with Docker and analyze IIS logs
Analyze and visualize csv logs with Excel Elastic Stack (docker-compose) --Parse "year / month / day, hour: minute: second" in multiline with grok filter and treat it as Japan time
Analyze and visualize csv logs with Excel Elastic Stack (docker-compose)-(1st line: date, 2nd and subsequent lines: csv data) date is added to each line after the 2nd line as a timestamp field.
Run logstash with Docker and try uploading data to Elastic Cloud