I made a JAVA framework "numatrix" that easily generates unique numbers in a distributed environment & multithreading.

Overview

numatrix is a framework that generates unique 64-bit numbers in a distributed environment that runs in parallel.

Feature

Having a unique ID for each distributed JVM allows each to generate a unique number across the board. Since each is independent, it runs very fast. It is thread-safe and runs in parallel for each thread.

specification

numatrix decomposes 64-bit into three fields and assigns a value to each.

  1. Timestamp The default number of bits in this field is 32 bits. The value is the number of seconds that have elapsed since the ** reference time **. The standard time is 0:00 on November 3, 2017 by default. The value is determined when you start publishing the number and is updated when the next ** consecutive number ** returns to 0.
  2. Consecutive numbers The number of bits in this field is the remainder of the number of bits consumed by the previous ** timestamp ** and the unique ID ** in the next ** environment. The value starts at 0 and is incremented each time you issue a number. When the number of bits in this field is exceeded, it returns to 0 and updates the value of the previous ** timestamp **.
  3. Unique ID in the environment The number of bits in this field is determined by the maximum number of JVMs and threads in your environment. The value is a number issued to ensure that it is unique in your environment.

The first bit represents a sign, so if you do not output a negative value, it is fixed at 0 and is not used.

How to use

Download the source from numatrix and build with maven.

Implemented NumatrixNumberGeneratorFactory

Create a class that inherits from the NumatrixNumberGeneratorFactory class.

package numatrix;

import java.io.FileInputStream;
import java.io.IOException;
import java.util.Properties;

public class NumatrixNumberGeneratorFactoryImple extends NumatrixNumberGeneratorFactory {

  @Override
  protected int getTypeId() {
    return 0;
  }

  @Override
  protected int getJvmId() {
    Properties properties = new Properties();
    try {
      properties.load(new FileInputStream("Property file path"));
  } catch (IOException e) {
      throw new RuntimeException(e);
  }
    return Integer.parseInt(properties.getProperty("jvmid"));
  }

  @Override
  protected int getMaxJvmCount() {
    return 100;
  }

  @Override
  protected int getMaxGeneratorCount() {
    return 300;
  }

}

Implementation of getTypeId ()

A unique number can be generated for each return value of this method. When operating multiple unique numbers, create multiple classes that implement this method and use them properly.

Implementation of getJvmId ()

Please return a unique value for each JVM that uses this implementation class in the environment where you want to make the numerical value unique. Please use the return values in order from 0.

Implementation of getMaxJvmCount ()

Please return the maximum number of JVMs that use this implementation class in the environment where you want the numbers to be unique.

Implementation of getMaxGeneratorCount ()

Please return the maximum number of NumatlixNumberGenerator instances generated for each JVM. A NumatrixNumberGenerator instance is assigned to each thread that generates a unique number.

Generate numbers from NumatrixNumberGenerator instance

Get a NumatrixNumberGenerator instance from an instance of the implementation class of NumatrixNumberGeneratorFactory and generate a unique number.

public class Sample {

  public static void main(String[] args) throws NumatrixNumberGenerateException {
    NumatrixNumberGeneratorFactoryImple factory = new NumatrixNumberGeneratorFactoryImple();
    NumatrixNumberGenerator generator = factory.getGenerator();
    long uniqueueNumber = generator.generate();

  }

}

Get NumatrixNumberGenerator instance

The NumatlixNumberGenerator instance of the return value of getGenerator () is different for each thread that called it. If the thread that called getGenerator () is no longer referenced and can be destroyed by garbage collection, that thread's NumatlixNumberGenerator instance will be reused by another thread.

Generation of unique numbers

You can generate a unique number with NumatrixNumberGenerator # generate ().

Notes

** numatrix will be unavailable after about 136 years from 2017. ** ** The workaround is explained in Customization.

Customize

You can customize it by creating a class that inherits from the NumatrixNumberGenerator class and overriding each method. Override the getGenerator () method in the implementation class of NumatrixNumberGeneratorFactory and implement it to return an instance of the inherited class.

public class NumatrixNumberGeneratorFactoryImple extends NumatrixNumberGeneratorFactory {

  class NumatrixNumberGeneratorEx extends NumatrixNumberGenerator {
    //Customize TODO Numatrix Number Generator
    public NumatrixNumberGeneratorEx(int generatorId, int generatorIdBitLength) {
      super(generatorId, generatorIdBitLength);
    }

  }

  @Override
  protected NumatrixNumberGenerator makeGenerator(int generatorId, int generatorIdBitLength) {
    return new NumatrixNumberGeneratorEx(generatorId, generatorIdBitLength);
  }

Increase in maximum number of outputs per second with unique numerical value (performance improvement)

If one ** consecutive number ** of a 64-bit trisection field exceeds the maximum value, the value of the timestamp field must be updated, but the timestamp field is updated every second. For, ** Maximum number of consecutive numbers = Maximum number of outputs per second that is unique for each thread ** It will be. Since the number of bits of consecutive numbers is the remainder of the number of bits consumed by other fields, ** you can increase the maximum value by reducing the number of bits of other fields. ** **

Since the number of bits of the ID field unique to the environment is determined by the implementation of getMaxJvmCount () and getMaxGeneratorCount () of NumatlixNumberGeneratorFactory, the number of bits can be reduced by reducing their return values.

The other fields change the number of bits as follows.

Negative value output

By allowing the output of negative values, the first 1 bit of 64 bits becomes available, and the number of bits in the consecutive number field can be increased.

public class NumatrixNumberGeneratorEx extends NumatrixNumberGenerator {

  public NumatrixNumberGeneratorEx(int generatorId, int generatorIdBitLength) {
    super(generatorId, generatorIdBitLength);
  }

  @Override
  public boolean isOutMinus() {
    return true;
  }

}

Change the number of bits in the timestamp field

You can increase the number of bits in consecutive number fields by reducing the number of bits in the timestamp field. The initial value of the time stamp field is 32 bits, which can represent a maximum of about 136 seconds, but each time the number of bits is reduced by one, the maximum value of the time stamp is halved and the expiration date is available. Will be shorter.

public class NumatrixNumberGeneratorEx extends NumatrixNumberGenerator {

  public NumatrixNumberGeneratorEx(int generatorId, int generatorIdBitLength) {
    super(generatorId, generatorIdBitLength);
  }

  @Override
  public int getTimestampBitLength() {
    //Reduce from 32 bits
    return 31;
  }

}

Extension of usable period (2017 to 136 years)

One ** timestamp ** of a 64-bit divided into three fields cannot exceed the number that can be represented by the number of bits in the field (32 bits by default). Since the time stamp is the number of seconds that have passed since the base time (default setting is 00:00 on November 3, 2017), you can extend the available period by changing the base time. ** **

Change of reference time

Change the reference time. For the reference time, define the time before the start of using numatrix.

public class NumatrixNumberGeneratorEx extends NumatrixNumberGenerator {

  public NumatrixNumberGeneratorEx(int generatorId, int generatorIdBitLength) {
    super(generatorId, generatorIdBitLength);
  }

  @Override
  public Date getBaseTime() {
    try {
      //Change of reference time
      return new SimpleDateFormat("yyyyMMdd").parse("20171111");
    } catch (ParseException e) {
      throw new RuntimeException();
    }
  }

}

Recommended Posts

I made a JAVA framework "numatrix" that easily generates unique numbers in a distributed environment & multithreading.
I made a primality test program in Java
I made a rock-paper-scissors game in Java (CLI)
I made a simple calculation problem game in Java
I made a Wrapper that calls KNP from Java
I made roulette in Java.
[Beginner] I made a program to sell cakes in Java
I made a program in Java that solves the traveling salesman problem with a genetic algorithm
I wrote a Stalin sort that feels like a mess in Java
I made a source that automatically generates JPA Entity class files
I created a PDF in Java.
I made a shopify app @java
I made an annotation in Java.
I made a class that can use JUMAN and KNP from Java
I made a class that automatically generates Json that specifies Minecraft texture etc. [1.12.2]
Play Framework 2.6 (Java) environment construction in Eclipse
I made a new Java deployment tool
I tried to easily put CentOS-7 in a PC that I no longer need
I made a Diff tool for Java files
I wanted to make (a == 1 && a == 2 && a == 3) true in Java
I wrote a primality test program in Java
Let's create a super-simple web framework in Java
I made a viewer app that displays a PDF
I made a Ruby extension library in C
A bat file that uses Java in windows
I wrote a prime factorization program in Java
I made a function to register images with API in Spring Framework. Part 1 (API edition)