Introduction

When implementing processing with Java List, you may say "I want to process 100 lists at a time". Isn't it time to bulk insert in DB? Without it, I would be crazy. It is a memo that you should not forget because it seems that you will make the same mistake by implementing so-called Bulk-like processing with the one implemented in Java.

reference

There was a post that I referred to before, and I mostly use it, but even if I looked it up, it didn't come out easily. .. I didn't like it. ,. ,. ,. , If you say "I've seen this code" or "I mean this post", I'd appreciate it if you could let me know.

Implemented

public interface BulkHelper<T> extends Iterable<T> {
	
	/**
	 *Iterate a List of the size set by the first argument.
	 * @param size The size of the list to be processed in a batch
	 * @param list List to process
	 */
	default void doMethod(int size, Consumer<? super List<T>> list) {
		final List<T> _list = new ArrayList<T>(size);
		for (T t : this) {
	        if(_list.size() == size) {
	        	list.accept(_list);
	        	_list.clear();
	        }
	        _list.add(t);
	    }
		if(_list.size() > 0) list.accept(_list);
	}
	
	/**
	 *Set the list to iterate.
	 *This method is mandatory to call.
	 * @param itr
	 * @return
	 */
	public static <T> BulkHelper<T> setList(Iterable<T> itr){
		return () -> itr.iterator();
	}
}

** When using **

public static void main(String[] args) {
		
　　List<String> list = Arrays.asList("AAA", "BBB", "CCC", "DDD", "EEE", "FFF","GGG");
		
　　BulkHelper.setList(list).doMethod(2, _list -> {
	　　System.out.println(_list);
　　});		
}

result

[AAA, BBB]
[CCC, DDD]
[EEE, FFF]
[GGG]

I was addicted to

I did this when I wanted to repack the batch processing into a List and process it in parallel.

public static void main(String[] args) {
	
	List<String> list = Arrays.asList("AAA", "BBB", "CCC", "DDD", "EEE", "FFF","GGG");
	
	List<List<String>> copyList = new ArrayList<>();
	BulkHelper.setList(list).doMethod(2, _list -> {
		copyList.add(_list);
	});
	
	copyList.parallelStream().forEach(System.out::println);
}

result

[[GGG], [GGG], [GGG], [GGG]]

No. ,

Cause

The processing here in the BulkHelper interface.

if(_list.size() == size) {
     list.accept(_list);
     _list.clear();
}
_list.add(t);

The reference destination is always _list, and because clear () and add () are repeated, all the elements have referred to the [GGG] that was last processed in the List. It's just a mistake.

Solution

BulkHelper.setList(list).doMethod(2, _list -> {
	ArrayList<String> clone = new ArrayList<>(_list);
	copyList.add(clone);
});

Actually, I want to do something on the interface side, but what I suddenly came up with was to do something new inside the process.

Postscript

Thanks to @Kilisame and @swordone's suggestions, we were able to solve the interface neatly as follows! Thank you!

public interface BulkHelper<T> extends Iterable<T> {
	
	/**
	 *The List of the first argument is processed for each size of the second argument.
	 * @param list List to process
	 * @param size Bulk processing size
	 * @param bulkList Processing content
	 */
	public static <T> void extract(List<T> list, int size, Consumer<? super List<T>> bulkList) {
	    for (int i = 0; i < list.size(); i += size) {
	        List<T> _list = new ArrayList<>(list.subList(i, Integer.min(i + size, list.size())));
	        bulkList.accept(_list);
	    }
	}
}