I investigated the behavior when file mapping is done in Java, so I will summarize it.
Environment CentOS7.3.1611, openjdk-1.8.0.131
You can use the open + mmap system call in C to map files to memory, but I've always been concerned about using it in Java. I will actually try it.
Evaluate with the following sources.
mmap01.java
import java.lang.Thread;
import java.io.File;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
public class mmap01
{
public static void main(String[] args) throws Exception
{
for (int argc = 0; argc < args.length; argc++ ) {
File file = new File(args[argc]);
long length = file.length();
System.out.println("file:" + args[0] + " size:" + length);
MappedByteBuffer in = new RandomAccessFile(file, "r")
.getChannel().map(FileChannel.MapMode.READ_ONLY, 0, length);
for (long i = 0; i < length; i++) {
in.get();
}
System.out.println("Finished reading");
}
System.out.println("wait");
Thread.sleep(1000000);
}
}
By the way, in the case of a file that exceeds Integer.MAX_VALUE (2 ^ 31), the following error will occur at the timing of map. I was able to map with a size of 2 ^ 31-1.
$ dd if=/dev/zero of=2gb bs=1 seek=2147483647 count=1
$ dd if=/dev/zero of=2gb-1 bs=1 seek=2147483646 count=1
$ dd if=/dev/zero of=2gb-2 bs=1 seek=2147483645 count=1
$ java mmap01 2gb
file:2gb size:2147483648
Exception in thread "main" java.lang.IllegalArgumentException: Size exceeds Integer.MAX_VALUE
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:869)
at mmap01.main(mmap01.java:17)
$ java mmap01 2gb-1
file:2gb-1 size:2147483647
Finished reading
wait
Checking the memory map status and Java status of the java process at this time is as follows. It seems to be mapped with mmap. From the jstat results, it seems that the mapped area is out of the heap.
$ cat /proc/$(pidof java)/maps
(Abbreviation)
7f97c8000000-7f9848000000 r--s 00000000 00:26 5 /mnt/hgfs/COMMON/Java_Mmap/2gb-1
(Abbreviation)
$ jstat -gc $(pidof java)
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
1024.0 1024.0 0.0 0.0 8192.0 671.9 20480.0 0.0 4480.0 866.3 384.0 73.9 0 0.000 0 0.000 0.000
$ java mmap01 2gb-1 2gb-2
file:2gb-1 size:2147483647
Finished reading
file:2gb-1 size:2147483646
Finished reading
wait
Both files are properly mapped. The heap has not increased as much as the additionally read area (2 ^ 31-2).
$ cat /proc/$(pidof java)/maps
(Abbreviation)
7fd08c000000-7fd10c000000 r--s 00000000 00:26 6 /mnt/hgfs/COMMON/Java_Mmap/2gb-2
7fd10c000000-7fd18c000000 r--s 00000000 00:26 5 /mnt/hgfs/COMMON/Java_Mmap/2gb-1
(Abbreviation)
$ jstat -gc $(pidof java)
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
1024.0 1024.0 0.0 0.0 8192.0 835.7 20480.0 0.0 4480.0 866.3 384.0 73.9 0 0.000 0 0.000 0.000
But why did we set a 2GB limit on the mapable area? It is also difficult to handle because there is no explicit unmapping method. Notes on DirectByteBuffer and performance comparison also has a description of release.
Recommended Posts