<div dir="ltr" style="text-align: justify;">
<h2 dir="ltr" style="text-align: justify;">Spring Batch and MongoDB-</h2>
<div dir="ltr" style="text-align: justify;">In Spring Batch version introduce NoSQL database support with introducing <i>org.springframework.batch.item.data</i> package which contain following classes.</p>
<ul style="text-align: left;">
<li>AbstractPaginatedDataItemReader.class</li>
<li>GemfireItemWriter.class</li>
<li>MongoItemReader.class</li>
<li>MongoItemWriter.class</li>
<li>Neo4jItemReader.class</li>
<li>Neo4jItemWriter.class</li>
<li>RepositoryItemReader.class</li>
<li>RepositoryItemWriter.class</li>
<li>SpELMappingGemfireItemWriter.class</li>
</ul>
<h2><b>Spring Batch</b></h2>
<p>Spring Batch is a Spring-based framework for enterprise Java batch processing. An important aspect of Spring Batch is the separation between reading from and writing to resources and the processing of a single record, called item in the Spring Batch lingo. There are a lot of existing item readers and writers for a wide range of resources like JDBC databases, JMS messaging systems, flat file etc. If the resource of your choice is not supported of of the box, it is easy to implement your own reader and writer as we will see in a minute.</p>
<h2><b>MongoDB</b></h2>
<p>MongoDB is a popular NoSQL datastore. It stores so called documents (basically an ordered set of key/value pairs where a value can be a simple data type like String or integer but also an array of values or a sub document). MongoDB is optimized for heavy write throughput and horizontal scaling.</p>
<p>Since I am a big fan of MongoDB on the one hand and introducing the Spring Batch framework at one of my customers on the other hand, why not implement a Spring Batch item reader(xml reader) and writer for MongoDB(MongoItemWriter).</p>
<h2><b>MongoDB Item Writer-<i>MongoItemWriter</i></b></h2>
<p>My first approach to the item writer was very naive. I just took the DBObject item list and inserted them into the target collection. This can be done with the following configuration:</p>
<pre class="highlight"><;!-- write it to MongoDB, 'employee' collection (table) -->; 
 <;bean id="mongodbItemWriter" class="org.springframework.batch.item.data.MongoItemWriter">; 
 <;property name="template" ref="mongoTemplate" />; 
 <;property name="collection" value="employee" />; 
 <;/bean>; 
</pre>
<div id="ads-id" align="center"></div>
<p><b>These are possible parameters:</b><br />
<i><b>template </b></i>and <b><i>collection </i></b>determine the MongoDB template and what collection to write to. These parameters are required, all other are optional.</p>
<h2><b>MongoDB Item Reader-<i>MongoItemReader</i></b></h2>
<p>Implementing the item reader was straightforward. It was merely a matter of passing parameters to the underlying MongoDB driver API.</p>
<pre class="highlight"><;!-- reader it from MongoDB, 'employee' collection (table) -->; 
 <;bean id="mongodbItemReader" class="org.springframework.batch.item.data.MongoItemReader">; 
 <;property name="template" ref="mongoTemplate" />; 
 <;property name="query" value="{age: {$gt: 22}" />; 
 <;/bean>; 
</pre>
<p><b>We have three kinds of parameters:</b></p>
<p><b><i>template </i></b>and <i><b>collection </b></i>determine the MongoDB template and what collection to read from. These parameters are required, all other are optional.<br />
<i><b>query </b></i>and <i><b>keys </b></i>are making up the MongoDB query. The first one is the query itself, the second one selects the field to read. If you don’t set a query string, all documents from the collection are read.</p>
<p>By default, the item reader emits DBObject instances that come from the MongoDB driver API. These objects are basically ordered hashmaps. If you want to use another representation of your data in the item processor, you can write a custom converter.</p>
<pre class="highlight">public class DocumentEmployeeConverter implements Converter<;DBObject Employee>; { 
 
 @Override 
 public Employee convert(DBObject document) { 
 Employee emp = new Employee(); 
 emp.setEmpid((String)document.get("_id")); 
 emp.setName((String)document.get("name")); 
 emp.setAge((Integer)document.get("age")); 
 emp.setSalary((Integer)document.get("salary")); 
 emp.setAddress((String)document.get("address")); 
 return emp; 
 } 
} 
</pre>
<h2><b>Example XML File To MongoDB Database-</b></h2>
<p>Now we will discuss how to configure a Spring Batch job to read data from an XML file (XStream library) into a no SQL database (MongoDB). In additional, create a unit test case to launch and test the batch jobs.</p>
<h3><b>Tools and libraries used</b></h3>
<ul style="text-align: left;">
<li>Spring Tool Suite (STS)</li>
<li>JDK 1.6</li>
<li>Spring Core 3.2.2.RELEASE</li>
<li>Spring OXM 3.2.2.RELEASE</li>
<li>Spring Batch 2.2.0.RELEASE</li>
<li>MongoDB Java Driver 2.7.3</li>
<li>MongoDB 2.10.1</li>
</ul>
<h3><b>1. Project Directory Structure</b></h3>
<div class="separator" style="clear: both; text-align: center;"><img src="https://dineshonjava.com/wp-content/uploads/2014/03/111-3.png" border="0" /></div>
<h3><b>2. Input XML File</b><br />
<b>employees.xml</b></h3>
<pre class="highlight"><;?xml version="1.0" encoding="UTF-8"?>; 
<;employees>; 
 <;employee>; 
 <;address>;delhi<;/address>; 
 <;age>;17<;/age>; 
 <;empid>;1111<;/empid>; 
 <;name>;ATUL KUMAR<;/name>; 
 <;salary>;300000.0<;/salary>; 
 <;/employee>; 
 <;employee>; 
 <;address>;delhi<;/address>; 
 <;age>;27<;/age>; 
 <;empid>;2222<;/empid>; 
 <;name>;Dinesh Rajput<;/name>; 
 <;salary>;60000.0<;/salary>; 
 <;/employee>; 
 <;employee>; 
 <;address>;delhi<;/address>; 
 <;age>;21<;/age>; 
 <;empid>;3333<;/empid>; 
 <;name>;ASHUTOSH RAJPUT<;/name>; 
 <;salary>;400000.0<;/salary>; 
 <;/employee>; 
 <;employee>; 
 <;address>;Kanpur<;/address>; 
 <;age>;27<;/age>; 
 <;empid>;4444<;/empid>; 
 <;name>;Adesh Verma<;/name>; 
 <;salary>;80000.0<;/salary>; 
 <;/employee>; 
 <;employee>; 
 <;address>;Noida<;/address>; 
 <;age>;37<;/age>; 
 <;empid>;5555<;/empid>; 
 <;name>;Dinesh Rajput<;/name>; 
 <;salary>;300000.0<;/salary>; 
 <;/employee>; 
<;/employees>; 
</pre>
<h3><b>3. ItemReader for XML File</b></h3>
<p>In this example, we use Jaxb2Marshaller to map XML values and attributes to an object.</p>
<pre class="highlight"><;bean id="xmlItemReader" class="org.springframework.batch.item.xml.StaxEventItemReader">; 
 <;property name="resource" value="classpath:xml/employees.xml" />; 
 <;property name="unmarshaller" ref="empUnMarshaller" />; 
 <;property name="fragmentRootElementName" value="employee" />; 
 <;/bean>; 
</pre>
<p><b>Employee.java</b></p>
<pre class="highlight">package com.doj.batch.bean; 
 
import javax.xml.bind.annotation.XmlAccessOrder; 
import javax.xml.bind.annotation.XmlAccessorOrder; 
import javax.xml.bind.annotation.XmlRootElement; 
 
/** 
 * @author Dinesh Rajput 
 * 
 */ 
@XmlRootElement(name="employee") 
@XmlAccessorOrder(XmlAccessOrder.UNDEFINED) 
public class Employee { 
 private int empid; 
 private String name; 
 private int age; 
 private float salary; 
 private String address; 
 /** 
 * @return the empid 
 */ 
 public int getEmpid() { 
 return empid; 
 } 
 /** 
 * @param empid the empid to set 
 */ 
 public void setEmpid(int empid) { 
 this.empid = empid; 
 } 
 /** 
 * @return the name 
 */ 
 public String getName() { 
 return name; 
 } 
 /** 
 * @param name the name to set 
 */ 
 public void setName(String name) { 
 this.name = name; 
 } 
 /** 
 * @return the age 
 */ 
 public int getAge() { 
 return age; 
 } 
 /** 
 * @param age the age to set 
 */ 
 public void setAge(int age) { 
 this.age = age; 
 } 
 /** 
 * @return the salary 
 */ 
 public float getSalary() { 
 return salary; 
 } 
 /** 
 * @param salary the salary to set 
 */ 
 public void setSalary(float salary) { 
 this.salary = salary; 
 } 
 /** 
 * @return the address 
 */ 
 public String getAddress() { 
 return address; 
 } 
 /** 
 * @param address the address to set 
 */ 
 public void setAddress(String address) { 
 this.address = address; 
 } 
 
} 
</pre>
<h3><b>4. MongoDB configuration</b><br />
<b>mongodbConfig.xml</b></h3>
<pre class="highlight"><;?xml version="1.0" encoding="UTF-8"?>; 
<;beans xmlns="http://www.springframework.org/schema/beans" 
 xmlns:mongo="http://www.springframework.org/schema/data/mongo" 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 xsi:schemaLocation="http://www.springframework.org/schema/beans 
 http://www.springframework.org/schema/beans/spring-beans-4.0.xsd 
 http://www.springframework.org/schema/data/mongo 
 http://www.springframework.org/schema/data/mongo/spring-mongo-1.0.xsd">; 
 
 <;mongo:mongo host="127.0.0.1" port="27017" />; 
 <;mongo:db-factory dbname="davdb" id="mongoDbFactory"/>; 
 
 <;bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate">; 
 <;constructor-arg name="mongoDbFactory" ref="mongoDbFactory" />; 
 <;/bean>; 
 
<;/beans>; 
</pre>
<h3><b>5. Spring Batch Core configuration</b><br />
<b>Define <i>jobRepository </i>and <i>jobLauncher</i>.</b><br />
<b>applicationContext.xml</b></h3>
<pre class="highlight"><;?xml version="1.0" encoding="UTF-8"?>; 
<;beans xmlns="http://www.springframework.org/schema/beans" 
 xmlns:context="http://www.springframework.org/schema/context" 
 xmlns:p="http://www.springframework.org/schema/p" 
 xmlns:mvc="http://www.springframework.org/schema/mvc" 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 xsi:schemaLocation="http://www.springframework.org/schema/beans 
http://www.springframework.org/schema/beans/spring-beans-4.0.xsd 
http://www.springframework.org/schema/context 
http://www.springframework.org/schema/context/spring-context-4.0.xsd 
http://www.springframework.org/schema/mvc 
http://www.springframework.org/schema/mvc/spring-mvc-4.0.xsd">; 
 
 <;bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager"/>; 
 
 <;bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">; 
 <;property name="jobRepository" ref="jobRepository"/>; 
 <;/bean>; 
 
 <;bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">; 
 <;property name="transactionManager" ref="transactionManager"/>; 
 <;/bean>; 
 
 <;bean id="simpleJob" class="org.springframework.batch.core.job.SimpleJob" abstract="true">; 
 <;property name="jobRepository" ref="jobRepository" />; 
 <;/bean>; 
 
<;/beans>; 
</pre>
<h3><b>6. Spring Batch Jobs Configuration file</b></h3>
<p>First, I define the <i><b>simple-job.xml</b></i> and <i><b>mongodbConfig.xml</b></i> files for configuration. In these file, I specify the <i>org.springframework.batch.item.xml.StaxEventItemReader,</i> which is a class from the Spring Batch framework. I specify the resource to the <i>org.springframework.batch.item.xml.StaxEventItemReader</i> as the path of the input xml file. Here I say the resource value is <b><i>classpath:xmlemployees.xml</i></b>, i.e., the location of input file <i><b>employees.xml</b></i>. I also define the <i>unmarshaller </i>object for converting xml data to java object of Employee class. Then I define <i>fragmentRootElementName</i>, which have value employee . I can cater that through my defined <i><b>EmployeeFilterProcessor </b></i>class which implements the ItemProcessor class of the Spring Batch framework.</p>
<p>After this, I specify the MongoDB details by mentioning the hostname where the database is installed and also the port number. I access the database through the MongoTemplate, which takes the reference of the database details mentioned through the id (i.e., Mongo as the argument). In the MongoTemplate I also pass the other argument (i.e., the name of the database I will work with inside the MongoDB), and in this case it is &#8220;new.&#8221; Now I define my own class, MongoDBItemWriter, which is the extension of the ItemWriter class in Spring Batch. This class now reads the MongoTemplate to get the details of the database.</p>
<p>Next, I specify the DynamicJobParameters class, which implements the JobParametersIncrementer from the Spring Batch. This works as the incrementer for the job.</p>
<p>Finally, I specify my batch job where I give the batch:step and batch:tasklet details. The batch job here is simpleDojJob, which contains a single step that holds the tasklet where the task mentioned is to read the batch:chunk from the xmlItemReader. I also mention the process and the itemwriter details.</p>
<pre class="highlight"><;?xml version="1.0" encoding="UTF-8"?>; 
<;beans xmlns="http://www.springframework.org/schema/beans" 
 xmlns:context="http://www.springframework.org/schema/context" 
 xmlns:p="http://www.springframework.org/schema/p" 
 xmlns:batch="http://www.springframework.org/schema/batch" 
 xmlns:mvc="http://www.springframework.org/schema/mvc" 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 xsi:schemaLocation="http://www.springframework.org/schema/beans 
http://www.springframework.org/schema/beans/spring-beans-4.0.xsd 
http://www.springframework.org/schema/context 
http://www.springframework.org/schema/context/spring-context-4.0.xsd 
http://www.springframework.org/schema/mvc 
http://www.springframework.org/schema/mvc/spring-mvc-4.0.xsd 
http://www.springframework.org/schema/batch 
http://www.springframework.org/schema/batch/spring-batch-2.0.xsd">; 
 
 <;import resource="applicationContext.xml"/>; 
 <;import resource="mongodbConfig.xml"/>; 
 <;bean id="employeeFilterProcessor" class="com.doj.batch.processor.EmployeeFilterProcessor">; 
 <;bean id="xmlItemReader" class="org.springframework.batch.item.xml.StaxEventItemReader">; 
 <;property name="resource" value="classpath:xml/employees.xml" />; 
 <;property name="unmarshaller" ref="empUnMarshaller" />; 
 <;property name="fragmentRootElementName" value="employee" />; 
 <;/bean>; 
 <;bean id="empUnMarshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">; 
 <;property name="classesToBeBound">; 
 <;value>;com.doj.batch.bean.Employee<;/value>; 
 <;/property>; 
 <;/bean>; 
 <;!-- write it to MongoDB, 'employee' collection (table) -->; 
 <;bean id="mongodbItemWriter" class="org.springframework.batch.item.data.MongoItemWriter">; 
 <;property name="template" ref="mongoTemplate" />; 
 <;property name="collection" value="employee" />; 
 <;/bean>; 
 
 <;batch:job id="simpleDojJob" parent="simpleJob">; 
 <;batch:step id="step1">; 
 <;batch:tasklet>; 
 <;batch:chunk reader="xmlItemReader" processor="employeeFilterProcessor" writer="mongodbItemWriter" commit-interval="2"/>; 
 <;/batch:tasklet>; 
 <;/batch:step>; 
 <;/batch:job>; 
<;/beans>; 
</pre>
<p><b>8. EmployeeFilterProcessor.java</b></p>
<pre class="highlight">package com.doj.batch.processor; 
 
import org.springframework.batch.item.ItemProcessor; 
 
import com.doj.batch.bean.Employee; 
 
/** 
 * @author Dinesh Rajput 
 * 
 */ 
public class EmployeeFilterProcessor implements ItemProcessor<;Employee, Employee>; { 
 
 @Override 
 public Employee process(Employee emp) throws Exception { 
 if(emp.getSalary() >; 70000.0){ 
 return emp; 
 }else{ 
 return null; 
 } 
 } 
}</pre>
<h2><b>Launching Batch Job-</b></h2>
<p>Spring Batch comes with a simple utility class called CommandLineJobRunner which has a main() method which accepts two arguments. First argument is the spring application context file containing job definition and the second is the name of the job to be executed.</p>
<p><b>Now run as a java application with both two arguments.</b><br />
<i><b>org.springframework.batch.core.launch.support.CommandLineJobRunner</b></i><br />
<i><b>simple-job.xml simpleDojJob</b></i></p>
<p>Output. The Spring Batch metadata tables are created, and the content of employees.xml is inserted into mongodb database &#8220;davdb&#8221; collection &#8220;EMPLOYEE&#8221;.</p>
<div class="separator" style="clear: both; text-align: center;"><img src="https://dineshonjava.com/wp-content/uploads/2014/03/111-4.png" border="0" /></div>
</div>
<h2><b>Download Source Code with Jars</b><br />
<b><a href="https://sites.google.com/a/dineshonjava.com/dineshonjava/dineshonjava/SpringBatchXMLtoMongoDB.zip?attredirects=0&;d=1" target="_blank" rel="noopener">SpringBatchXMLtoMongoDB.zip</a></b></h2>
<div style="text-align: center;"><b><;<;<a href="https://dineshonjava.com/spring-batch-csv-to-mysql-database/">Previuos</a><;<; <a href="https://dineshonjava.com/spring-tutorial/">Index </a>>;>; <a href="https://dineshonjava.com/multiresourceitemreader-in-spring-batch/">Next</a>>;>;</b></div>
<p> ;</p>
</div>
<div class="wp-post-navigation"> 
									 <div class="wp-post-navigation-pre"> 
									 <a href="https://dineshonjava.com/spring-batch-csv-to-mysql-database/">Previous</a> 
									 </div> 
									 <div class="wp-post-navigation-next"> 
									 <a href="https://dineshonjava.com/multiresourceitemreader-in-spring-batch/">Next</a> 
									 </div> 
									</div>
<script type="text/javascript">
jQuery(document).ready(function($) {
 $.post('https://dineshonjava.com/wp-admin/admin-ajax.php', {action: 'mts_view_count', id: '222'});
});
</script>
Strategy Design Patterns We can easily create a strategy design pattern using lambda. To implement…
Decorator Pattern A decorator pattern allows a user to add new functionality to an existing…
Delegating pattern In software engineering, the delegation pattern is an object-oriented design pattern that allows…
Technology has emerged a lot in the last decade, and now we have artificial intelligence;…
Managing a database is becoming increasingly complex now due to the vast amount of data…
Overview In this article, we will explore Spring Scheduler how we could use it by…