MultiResourceItemReader in Spring Batch Example

Class MultiResourceItemReader

Reads items from multiple resources sequentially – resource list is given by setResources(Resource[]), the actual reading is delegated to setDelegate(ResourceAwareItemReaderItemStream). Input resources are ordered using setComparator(Comparator) to make sure resource ordering is preserved between job runs in restart scenario.

MultiResourceItemReader class reads items from multiple resources sequentially.

Suppose one of the question in mind of user-
 
I have a Spring batch operation where I have sources file1, file2 and file3 to read from, and want to write to fileA, fileB and fileC like the following:

file1->fileA
file2->fileB
file3->fileC

In some scenario use want to put these three resources into a single file. In this example, we will show you how to read items from multiple resources (multiple xml files), and write the items into a single xml file.

emp-111.xml + emp-222.xml + emp-333.xml = employees.xml

Tools and libraries used

• Spring Tool Suite (STS)
• JDK 1.6
• Spring Core 3.2.2.RELEASE
• Spring OXM 3.2.2.RELEASE
• Spring Batch 2.2.0.RELEASE

1. Project Directory Structure

2. Input XML Files

There are 3 xml files, later we will use MultiResourceItemReader to read it one by one.

emp-111.xml

<?xml version="1.0" encoding="UTF-8"?>
<employees>
 <employee>
  <address>Mumbai</address>
  <age>17</age>
  <empid>111</empid>
  <name>ATUL KUMAR</name>
  <salary>700000.0</salary>
 </employee>
</employees>

emp-222.xml

<?xml version="1.0" encoding="UTF-8"?>
<employees>
 <employee>
  <address>Noida</address>
  <age>21</age>
  <empid>222</empid>
  <name>ASHUTOSH RAJPUT</name>
  <salary>400000.0</salary>
 </employee>
</employees>

emp-333.xml

<?xml version="1.0" encoding="UTF-8"?>
<employees>
 <employee>
  <address>Delhi</address>
  <age>17</age>
  <empid>333</empid>
  <name>Dinesh Rajput</name>
  <salary>300000.0</salary>
 </employee>
</employees>

3. ItemReader for XML File

In this example, we use Jaxb2Marshaller to map XML values and attributes to an object.

<bean class="org.springframework.batch.item.xml.StaxEventItemReader" id="xmlItemReader">
   <property name="unmarshaller" ref="empUnMarshaller">
  <property name="fragmentRootElementName" value="employee">
 </property></property></bean>

Employee.java

package com.doj.batch.bean;

import javax.xml.bind.annotation.XmlAccessOrder;
import javax.xml.bind.annotation.XmlAccessorOrder;
import javax.xml.bind.annotation.XmlRootElement;

/**
 * @author Dinesh Rajput
 *
 */
@XmlRootElement(name="employee")
@XmlAccessorOrder(XmlAccessOrder.UNDEFINED)
public class Employee {
 private int empid;
 private String name;
 private int age;
 private float salary;
 private String address;
 /**
  * @return the empid
  */
 public int getEmpid() {
  return empid;
 }
 /**
  * @param empid the empid to set
  */
 public void setEmpid(int empid) {
  this.empid = empid;
 }
 /**
  * @return the name
  */
 public String getName() {
  return name;
 }
 /**
  * @param name the name to set
  */
 public void setName(String name) {
  this.name = name;
 }
 /**
  * @return the age
  */
 public int getAge() {
  return age;
 }
 /**
  * @param age the age to set
  */
 public void setAge(int age) {
  this.age = age;
 }
 /**
  * @return the salary
  */
 public float getSalary() {
  return salary;
 }
 /**
  * @param salary the salary to set
  */
 public void setSalary(float salary) {
  this.salary = salary;
 }
 /**
  * @return the address
  */
 public String getAddress() {
  return address;
 }
 /**
  * @param address the address to set
  */
 public void setAddress(String address) {
  this.address = address;
 }
 
}

4. Spring Batch Core configuration

Define jobRepository and jobLauncher.
applicationContext.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:context="http://www.springframework.org/schema/context"
 xmlns:p="http://www.springframework.org/schema/p" 
 xmlns:mvc="http://www.springframework.org/schema/mvc" 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-4.0.xsd
http://www.springframework.org/schema/mvc
http://www.springframework.org/schema/mvc/spring-mvc-4.0.xsd">

 <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager"/>
 
    <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
        <property name="jobRepository" ref="jobRepository"/>
    </bean>
 
    <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
        <property name="transactionManager" ref="transactionManager"/>
    </bean>
 
    <bean id="simpleJob" class="org.springframework.batch.core.job.SimpleJob" abstract="true">
        <property name="jobRepository" ref="jobRepository" />
    </bean>
 
</beans>

5. Spring Batch Jobs Configuration

simple-job.xml
A job to read resources that matches this pattern xml/inputs/emp-*.xml, and write it into a single xml file employees.xml.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:context="http://www.springframework.org/schema/context"
 xmlns:p="http://www.springframework.org/schema/p" 
 xmlns:batch="http://www.springframework.org/schema/batch"
 xmlns:mvc="http://www.springframework.org/schema/mvc" 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-4.0.xsd
http://www.springframework.org/schema/mvc
http://www.springframework.org/schema/mvc/spring-mvc-4.0.xsd
http://www.springframework.org/schema/batch 
http://www.springframework.org/schema/batch/spring-batch-2.0.xsd">

 <import resource="applicationContext.xml"/>
 
 <bean id="xmlItemReader" class="org.springframework.batch.item.xml.StaxEventItemReader">
  <property name="unmarshaller" ref="empUnMarshaller" />
  <property name="fragmentRootElementName" value="employee" />
   </bean>
   
 <bean id="empUnMarshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
  <property name="classesToBeBound">
   <value>com.doj.batch.bean.Employee</value>
  </property>
   </bean>
   
   <bean id="multiResourceReader" class=" org.springframework.batch.item.file.MultiResourceItemReader">
  <property name="resources" value="file:xml/inputs/emp-*.xml" />
  <property name="delegate" ref="xmlItemReader" />
  </bean>
  
 <bean id="xmlItemWriter" class="org.springframework.batch.item.xml.StaxEventItemWriter">
  <property name="resource" value="file:xml/outputs/employees.xml" />
  <property name="marshaller" ref="empMarshaller" />
  <property name="rootTagName" value="employees" />
  </bean> 
  
  <bean id="empMarshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
  <property name="classesToBeBound">
   <value>com.doj.batch.bean.Employee</value>
  </property>
   </bean> 
    
    <batch:job id="simpleDojJob" parent="simpleJob">
     <batch:step id="step1">
      <batch:tasklet>
       <batch:chunk reader="multiResourceReader" writer="xmlItemWriter" commit-interval="1"/>
      </batch:tasklet>
     </batch:step>
    </batch:job>   
</beans>

6. Launching Batch Job-

Spring Batch comes with a simple utility class called CommandLineJobRunner which has a main() method which accepts two arguments. First argument is the spring application context file containing job definition and the second is the name of the job to be executed.

Now run as a java application with both two arguments.
org.springframework.batch.core.launch.support.CommandLineJobRunner
simple-job.xml simpleDojJob

Output. The Spring Batch are created the content of employees.xml.

<?xml version="1.0" encoding="UTF-8"?>
<employees>
 <employee>
  <address>Mumbai</address>
  <age>17</age>
  <empid>111</empid>
  <name>ATUL KUMAR</name>
  <salary>700000.0</salary>
 </employee>
 <employee>
  <address>Noida</address>
  <age>21</age>
  <empid>222</empid>
  <name>ASHUTOSH RAJPUT</name>
  <salary>400000.0</salary>
 </employee>
 <employee>
  <address>Delhi</address>
  <age>17</age>
  <empid>333</empid>
  <name>Dinesh Rajput</name>
  <salary>300000.0</salary>
 </employee>
</employees>