[JAVA] [Spring Batch] Realization verification of chunk step and tasklet step of Spring Batch

Introduction

We will verify the implementation method for two types of Spring Batch. This time, it will be realized by Java annotation.

Step type

There are two main types of steps. "Tasklet-only steps" and "Chunk steps". Let's take a brief look at each one below.

1. Tasklet only steps

   

2. Chunk steps

   

environment

Windows 10 JDK 1.8.0_251 STS 4.6.2

Creating a project

Create a "Spring Starter Project". When creating a project, when it is a dependency, select only "Spring Batch".

• After creating the project, manually add the dependency of "hsqldb" to pom.xml. Add the following source to the dependencies block.

pom.xml

        <dependency>
			<groupId>org.hsqldb</groupId>
			<artifactId>hsqldb</artifactId>
			<scope>runtime</scope>
        </dependency>

chunk step Sample source description

■ Start Main method

BatchTestAppApplication.java

@SpringBootApplication
public class BatchTestAppApplication {

	public static void main(String[] args) {
		SpringApplication.run(BatchTestAppApplication.class, args);
	}

}

■ SQL for table creation

schema-all.sql

DROP TABLE people IF EXISTS;

CREATE TABLE people  (
    person_id BIGINT IDENTITY NOT NULL PRIMARY KEY,
    first_name VARCHAR(20),
    last_name VARCHAR(20)
);

■ Input CSV file

sample-data.csv

Jill,Doe
Joe,Doe
Justin,Doe
Jane,Doe
John,Doe

■ DTO class

Person.java

public class Person {

	private String lastName;
	private String firstName;

	public Person() {
	}

	public Person(String firstName, String lastName) {
		this.firstName = firstName;
		this.lastName = lastName;
	}

	public void setFirstName(String firstName) {
		this.firstName = firstName;
	}

	public String getFirstName() {
		return firstName;
	}

	public String getLastName() {
		return lastName;
	}

	public void setLastName(String lastName) {
		this.lastName = lastName;
	}

	@Override
	public String toString() {
		return "firstName: " + firstName + ", lastName: " + lastName;
	}

}

■ Implementation class of ItemProcessor interface

PersonItemProcessor.java

public class PersonItemProcessor implements ItemProcessor<Person, Person> {

	private static final Logger log = LoggerFactory.getLogger(PersonItemProcessor.class);

	@Override
	public Person process(final Person person) throws Exception {
		final String firstName = person.getFirstName().toUpperCase();
		final String lastName = person.getLastName().toUpperCase();

		final Person transformedPerson = new Person(firstName, lastName);

		log.info("Converting (" + person + ") into (" + transformedPerson + ")");

		return transformedPerson;
	}

}

Description:

Implement the process method. Change the simple name to uppercase. An info log is output to check the execution history.

■ Listener class

Implement the afterJob method of JobExecutionListenerSupport to check the job execution result.

JobCompletionNotificationListener.java

@Component
public class JobCompletionNotificationListener extends JobExecutionListenerSupport {

	private static final Logger log = LoggerFactory.getLogger(JobCompletionNotificationListener.class);

	private final JdbcTemplate jdbcTemplate;

	@Autowired
	public JobCompletionNotificationListener(JdbcTemplate jdbcTemplate) {
		this.jdbcTemplate = jdbcTemplate;
	}

	@Override
	public void afterJob(JobExecution jobExecution) {
		if (jobExecution.getStatus() == BatchStatus.COMPLETED) {
			log.info("!!! JOB FINISHED! Time to verify the results");

			jdbcTemplate
					.query("SELECT first_name, last_name FROM people",
							(rs, row) -> new Person(rs.getString(1), rs.getString(2)))
					.forEach(person -> log.info("Found <" + person + "> in the database."));
		}
	}
}

■ Each Bean definition class in Batch

BatchConfiguration.java

@Configuration
@EnableBatchProcessing
public class BatchConfiguration {

	private static final Logger log = LoggerFactory.getLogger(BatchConfiguration.class);

	@Autowired
	public JobBuilderFactory jobBuilderFactory;

	@Autowired
	public StepBuilderFactory stepBuilderFactory;
	
	@Bean
	public FlatFileItemReader<Person> reader() {
		return new FlatFileItemReaderBuilder<Person>().name("personItemReader")
				.resource(new ClassPathResource("sample-data.csv")).delimited()
				.names(new String[] { "firstName", "lastName" })
				.fieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {
					{
						setTargetType(Person.class);
					}
				}).build();
	}

	@Bean
	public PersonItemProcessor processor() {
		return new PersonItemProcessor();
	}

	@Bean
	public JdbcBatchItemWriter<Person> writer(DataSource dataSource) {
		return new JdbcBatchItemWriterBuilder<Person>()
				.itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>())
				.sql("INSERT INTO people (first_name, last_name) VALUES (:firstName, :lastName)").dataSource(dataSource)
				.build();
	}

	@Bean
	public Job importUserJob(JobCompletionNotificationListener listener, Step step1) {
		return jobBuilderFactory.get("importUserJob").incrementer(new RunIdIncrementer()).listener(listener).flow(step1)
				.end()
				.build();
	}
	
	@Bean
	public Step step1(JdbcBatchItemWriter<Person> writer) {
		return stepBuilderFactory.get("step1").<Person, Person>chunk(10).reader(reader()).processor(processor())
				.writer(writer).build();
	}
}

Description:

1. reader

Returns FlatFileItemReader as ItemReader. Reads the data in the input CSV file and stores it in the Person DTO Bean. 2. processor

Return PersonItemProcessor as ItemProcessor. 3. writer

Returns JdbcBatchItemWriter as ItemWriter. Store the processing result of ItemProcessor in DB. 4. step1

Link FlatFileItemReader, PersonItemProcessor and JdbcBatchItemWriter in StepBuilderFactory to generate chunk step bean of Spring Batch. 5. importUserJob

Create a Spring Batch Job using the customized JobCompletionNotificationListener Bean and the chunk step Bean generated in step1. When you start the corresponding batch, you can execute the Job generated by this method.

tasklet step Sample source description

Leverage the above project to modify BatchConfiguration to implement the tasklet step sample. The corresponding tasklet step executes the OS command.

■ Repair of Batch Configuration

--Generate tasklet bean

python

	@Bean
	public Tasklet copyFile() {
		log.info("Start FileCopy");
		SystemCommandTasklet tasklet = new SystemCommandTasklet();
		tasklet.setCommand("cmd /C copy C:\\data\\test.txt C:\\data\\test2.txt");
		tasklet.setTimeout(3000);
		return tasklet;
	}

• Check the Javadoc for the explanation of each attribute of SystemCommandTasklet.

--step Bean generation

python

	@Bean
	public Step step2() {
		return stepBuilderFactory.get("step2").tasklet(copyFile()).build();
	}

--Job generation

python

	@Bean
	public Job cmdJob(JobCompletionNotificationListener listener, Step step1, Step step2, Step step3) {
		return jobBuilderFactory.get("cmdJob").incrementer(new RunIdIncrementer()).listener(listener).
				flow(step2)
				.end()
				.build();
	}

reference

Creating a Batch Service [Java] [Spring Boot] Try using Tasklet with Spring Batch