I tried Facebook's Java concurrency bug analysis tool "Racer D"

I tried the Java concurrency bug analysis tool "RacerD" released by Facebook as open source. In this article, we will share the following two points.

--RacerD installation procedure (using Docker) --Detection results of typical Java concurrency bugs listed in JCIP (Java Concurrency in Practice) and CC / CERT

What is "Racer D"?

"RacerD" is a static code analysis tool for Java concurrency bugs released by Facebook as open source. It is used as a part of ** the function of the static code analysis tool "Infer" released as open source in 2015 **. (BSD license as of February 2018)

As of March 2018, there is also Awesome Java, which lists only carefully selected Java tools. , Infer is introduced.

Facebook has already incorporated RacerD as a CI (Continuous Integration), and its Android app has detected more than 1000 concurrency bugs.

Install Infer

First, install the static code analysis tool "Infer" (the version at the time of writing this article is 0.13.1).

Clone git repository

Enter the following command in any directory.

$ git clone https://github.com/facebook/infer.git

Rewrite Dockerfile

Infer supports not only Java but also C, C ++, Objective-C (however, RacerD is Java only). In this article, we will rewrite the Dockerfile because it is enough to have the environment required for Java static code analysis.

Navigate to the directory that contains the files for installing the Docker image.

$ cd infer/docker

The directory structure is as follows.

$ tree
.
├── Dockerfile
├── README.md
└── run.sh

Rewrite the Dockerfile as follows.

** Difference between Dockerfile.origin (before modification) and Dockerfile (after modification) **

$ diff -u Dockerfile.origin Dockerfile
--- Dockerfile.origin	2018-02-22 09:47:10.504984404 +0000
+++ Dockerfile	2018-02-22 09:47:46.533118963 +0000
@@ -31,17 +31,8 @@

 # Compile Infer
 RUN OCAML_VERSION=4.05.0+flambda; \
-    cd /infer && ./build-infer.sh --opam-switch $OCAML_VERSION && rm -rf /root/.opam
+    cd /infer && ./build-infer.sh java --yes --opam-switch $OCAML_VERSION && rm -rf /root/.opam

 # Install Infer
 ENV INFER_HOME /infer/infer
 ENV PATH ${INFER_HOME}/bin:${PATH}
-
-ENV ANDROID_HOME /opt/android-sdk-linux
-WORKDIR $ANDROID_HOME
-RUN curl -o sdk-tools-linux.zip \
-      https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip && \
-    unzip sdk-tools-linux.zip && \
-    rm sdk-tools-linux.zip
-ENV PATH ${ANDROID_HOME}/tools/bin:${PATH}
-RUN echo "sdk.dir=${ANDROID_HOME}" > /infer/examples/android_hello/local.properties

Install the Docker image

Run the following shell to install the Docker image.

./run.sh

In the environment used in this article ** It took about 20 minutes to install. ** **

Check the operation of Infer

Create a container from the Infer Docker image by entering the following command:

$ docker run -v /home/user/project:/mnt --rm -it infer

Running Infer in Docker · facebook/infer Wiki · GitHub

Change to the following mounted directory.

# cd mnt

Create Hello.java in the mnt directory on the Infer web page (Hello, World! | Infer) ..

You can analyze the static code of Hello.java by entering the following command.

# infer run -- javac Hello.java
Capturing in javac mode...
Found 1 source file to analyze in /mnt/infer-out
Starting analysis...

legend:
  "F" analyzing a file
  "." analyzing a procedure

F..

Found 1 issue

Hello.java:5: error: NULL_DEREFERENCE
  object `s` last assigned on line 4 could be null and is dereferenced at line 5.
  3.       int test() {
  4.           String s = null;
  5. >         return s.length();
  6.       }
  7.   }


Summary of the reports

  NULL_DEREFERENCE: 1

Try RacerD with Concurrency Bug Code

Now I would like to start analyzing the concurrency bug, which is the main subject.

RacerD performs static analysis on Java code using ** "lock" or "@ThreadSafe annotation" **. You can download the @ThreadSafe annotation jar file from the JCIP website (Java Concurrency in Practice).

In this article, I created the following sample code in the mnt directory.

.
|-- Hello.java
|-- LongContainer.java
|-- UnsafeSequence.java
|-- jcip-annotations.jar

Example 1. An ordinal generation method that is not thread-safe

First, try RacerD using the violation code posted on JCIP.

Violation code

Java Concurrency in Practice - Code Listings

package net.jcip.examples;

import net.jcip.annotations.*;

/**
 * UnsafeSequence
 *
 * @author Brian Goetz and Tim Peierls
 */

@ThreadSafe
public class UnsafeSequence {
    private int value;

    /**
     * Returns a unique value.
     */
    public int getNext() {
        return value++;
    }
}

The problem with the above code is that at the wrong time, the two threads calling getNext will receive ** the same value **. Although the description of increment such as nextValue ++ looks like one operation, it actually performs the following three operations.

  1. Read the value
  2. Add 1
  3. Write out the new value

Therefore, depending on the timing of multiple threads If two threads read the same value at the same time and add 1 in the same way, it is possible for the two threads to return the same value.

Analysis result: "THREAD_SAFETY_VIOLATION" detected

infer --racerd-only -- javac -classpath jcip-annotations.jar UnsafeSequence.java
Capturing in javac mode...
Found 1 source file to analyze in /mnt/infer-out
Starting analysis...

legend:
  "F" analyzing a file
  "." analyzing a procedure

F..

Found 1 issue

UnsafeSequence.java:19: error: THREAD_SAFETY_VIOLATION
  Unprotected write. Non-private method `net.jcip.examples.UnsafeSequence.getNext` writes to field `&this.net.jcip.examples.UnsafeSequence.value` outside of synchronization.
 Reporting because the current class is annotated `@ThreadSafe`, so we assume that this method can run in parallel with other non-private methods in the class (incuding itself).
  17.        */
  18.       public int getNext() {
  19. >         return value++;
  20.       }
  21.   }


Summary of the reports

  THREAD_SAFETY_VIOLATION: 1

Example 2. Reading and writing 64-bit values

VNA05-J. Read and write 64-bit values atomically

In the Java memory model of the programming language, ** a single write to a non-volatile long or double value is treated as two writes, 32 bits each. ** As a result, the thread can see the combination of the first 32 bits of a 64-bit value write and the next 32 bits of another write.

This behavior may lead to indeterminate values being read in code that is required to be thread safe. Therefore, multithreaded programs must ensure that 64-bit values are read and written atomically.

Violation code

In the following violation code, if one thread repeatedly calls the assignValue () method and another thread repeatedly calls the printLong () method, the ** printLong () method will use a value of i that is neither 0 nor the value of the argument j. May be output. ** **

import net.jcip.annotations.*;

@ThreadSafe
class LongContainer {
  private long i = 0;

  void assignValue(long j) {
    i = j;
  }

  void printLong() {
    System.out.println("i = " + i);
  }
}

Analysis result: "THREAD_SAFETY_VIOLATION" detected

infer --racerd-only -- javac -classpath jcip-annotations.jar LongContainer.java
Capturing in javac mode...
Found 1 source file to analyze in /mnt/infer-out
Starting analysis...

legend:
  "F" analyzing a file
  "." analyzing a procedure

F...

Found 2 issues

LongContainer.java:9: error: THREAD_SAFETY_VIOLATION
  Unprotected write. Non-private method `LongContainer.assignValue` writes to field `&this.LongContainer.i` outside of synchronization.
 Reporting because the current class is annotated `@ThreadSafe`, so we assume that this method can run in parallel with other non-private methods in the class (incuding itself).
  7.   
  8.     void assignValue(long j) {
  9. >     i = j;
  10.     }
  11.   

LongContainer.java:13: error: THREAD_SAFETY_VIOLATION
  Read/Write race. Non-private method `LongContainer.printLong` reads without synchronization from `&this.LongContainer.i`. Potentially races with writes in method `void LongContainer.assignValue(long)`.
 Reporting because the current class is annotated `@ThreadSafe`, so we assume that this method can run in parallel with other non-private methods in the class (incuding itself).
  11.   
  12.     void printLong() {
  13. >     System.out.println("i = " + i);
  14.     }
  15.   }


Summary of the reports

  THREAD_SAFETY_VIOLATION: 2

Summary

Classes that I want to use for thread safety need to be explicitly annotated with @ ThreadSafe, but I found RacerD's analysis of concurrency bugs to be quite effective.

Recommended Posts

I tried Facebook's Java concurrency bug analysis tool "Racer D"
I tried Drools (Java, InputStream)
I tried using Java REPL
I tried source code analysis
Java concurrency I don't understand
I tried metaprogramming in Java
I tried morphological analysis with MeCab
I tried to interact with Java
I tried UDP communication with Java
I tried using Java8 Stream API
I tried Java's micro-benchmark tool JMH
I tried using JWT in Java
I tried to summarize Java 8 now
I tried using Java memo LocalDate
I tried using GoogleHttpClient of Java
I tried using Elasticsearch API in Java
I tried using Java's diagnostic tool Arthas
I tried Cassandra's Object Mapper for Java
I tried to summarize Java lambda expressions
I tried the new era in Java
I tried using OpenCV with Java + Tomcat
I tried Google's entrance exam (unofficial) [java]