Connect to OceanBase Database by using Spring Batch

2024-06-28 05:30:30  Updated

This topic introduces how to build an application by using the Spring Batch framework and OceanBase Database. It also covers the use of the application for fundamental database operations, including table creation, data insertion, and data query.

Prerequisites

  • You have installed OceanBase Database.
  • You have installed JDK 1.8 and Maven.
  • You have installed IntelliJ IDEA.

Note

The tool used to run the sample code in this topic is IntelliJ IDEA 2021.3.2 (Community Edition), but you can also choose a tool that suits your personal preference to run the code.

Procedure

Note

The steps outlined in this topic are for the Windows environment. If you are using a different operating system or compiler, the steps may vary slightly.

  1. Obtain the OceanBase Database connection string.
  2. Import the java-oceanbase-springbatch project into IDEA.
  3. Modify the database connection information in the java-oceanbase-springbatch project.
  4. Run the java-oceanbase-springbatch project.

Step 1: Obtain the OceanBase Database connection string

  1. Contact the deployment personnel or administrator of OceanBase Database to obtain the database connection string.

    obclient -hxx.xx.xx.xx -P2883 -uroot@sys#cluster -p**** -A

  2. Fill in the URL below based on the deployed OceanBase database.

    Note

    The URL here is required in the application.properties file.

    jdbc:oceanbase://host:port/schema_name?user=$user_name&password=$password&characterEncoding=utf-8

    Parameters in the URL are described as follows:

    • host: the IP address for connecting to OceanBase Database. For connection through OceanBase Database Proxy (ODP), this parameter is the IP address of an ODP. For direct connection, this parameter is the IP address of an OBServer node.
    • port: the port for connecting to OceanBase Database. For connection through ODP, the default value is 2883, which can be customized when ODP is deployed. For direct connection, the default value is 2881, which can be customized when OceanBase Database is deployed.
    • schema_name: the name of the schema to access.
    • user_name: the tenant account. For connection through ODP, the tenant account can be in the username@tenant name#cluster name or cluster name:tenant name:username format. For direct connection, the tenant account is in the username@tenant name format.
    • password: the account password.
    • characterEncoding: the character encoding method.

    For more information about URL parameters, see Database URL.

Step 2: Import the java-oceanbase-springbatch project into IDEA

  1. Start IntelliJ IDEA and choose File > Open….

    file

  2. In the Open File or Project window that appears, select the corresponding project file and click OK to import the project file.

  3. IntelliJ IDEA automatically identifies all types of files in the project. In the Project window, you can view the directory structure, list of files, list of modules, and dependencies of the project. The Project window is usually on the far left side in IntelliJ IDEA and is displayed by default. If the Project window is closed, you can choose View > Tool Windows > Project from the menu or use the Alt + 1 shortcut to open the window.

    Note

    When you use IntelliJ IDEA to import a project, IntelliJ IDEA automatically detects the pom.xml file in the project, downloads the required libraries based on the dependencies defined in the file, and adds the libraries to the project.

  4. View the project.

    springbatch

Step 3: Modify the database connection information in the java-oceanbase-springbatch project

Modify the database connection information in the application.properties file based on the information obtained in Step 1: Obtain the OceanBase Database connection string.

Here is an example:

  • The name of the database driver is com.oceanbase.jdbc.Driver.
  • The IP address of the OBServer node is 10.10.10.1.
  • The port is 2881.
  • The name of the schema to access is sys.
  • The tenant account is sys@xyoracle, where xyoracle is a user tenant created in the Oracle mode of OceanBase Database, and sys is a username in the xyoracle tenant.
  • The password is ******.

The sample code is as follows:

spring.datasource.driver-class-name=com.oceanbase.jdbc.Driver
spring.datasource.url=jdbc:oceanbase://10.10.10.1:2881/sys?characterEncoding=utf-8
spring.datasource.username=sys@xyoracle
spring.datasource.password=******

spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=update

spring.batch.job.enabled=false

logging.level.org.springframework=INFO
logging.level.com.example=DEBUG

Step 4: Run the java-oceanbase-springbatch project

  • Run the AddDescPeopleWriterTest.java file.

    1. Find the AddDescPeopleWriterTest.java file under src > test > java in the project package.
    2. Choose Run > Run… > AddDescPeopleWriterTest.testWrite in the menu bar or click the green triangle in the upper-right corner to run the file.
    3. View the logs and output of the project in the IDEA console.
    Data in the people_desc table:
PeopleDESC [name=John, age=25, desc=This is John with age 25]
PeopleDESC [name=Alice, age=30, desc=This is Alice with age 30]
Batch Job execution completed.

  • Run the AddPeopleWriterTest.java file.

    1. Find the AddDescPeopleWriterTest.java file under src > test > java in the project package.
    2. Choose Run > Run… > AddPeopleWriterTest.testWrite in the menu bar or click the green triangle in the upper-right corner to run the file.
    3. View the logs and output of the project in the IDEA console.
    Data in the people table:
People [name=zhangsan, age=27]
People [name=lisi, age=35]
Batch Job execution completed.

Project code introduction

Click java-oceanbase-springbatch to download the project code, which is a compressed file named java-oceanbase-springbatch.

Decompress the package to obtain a folder named java-oceanbase-springbatch. The directory is structured as follows:

│  pom.xml
│
├─.idea
│
├─src
│  ├─main
│  │  ├─java
│  │  │  └─com
│  │  │      └─oceanbase
│  │  │          └─example
│  │  │              └─batch
│  │  │                  │──BatchApplication.java
│  │  │                  │
│  │  │                  ├─config
│  │  │                  │   └─BatchConfig.java
│  │  │                  │
│  │  │                  ├─model
│  │  │                  │   ├─People.java
│  │  │                  │   └─PeopleDESC.java
│  │  │                  │
│  │  │                  ├─processor
│  │  │                  │   └─AddPeopleDescProcessor.java
│  │  │                  │
│  │  │                  └─writer
│  │  │                      ├─AddDescPeopleWriter.java
│  │  │                      └─AddPeopleWriter.java
│  │  │
│  │  └─resources
│  │      └─application.properties
│  │
│  └─test
│      └─java
│          └─com
│              └─oceanbase
│                  └─example
│                      └─batch
│                          ├─config
│                          │   └─BatchConfigTest.java
│                          │
│                          ├─processor
│                          │   └─AddPeopleDescProcessorTest.java
│                          │
│                          └─writer
│                              ├─AddDescPeopleWriterTest.java
│                              └─AddPeopleWriterTest.java
│
└─target

Here is a breakdown of the files and directories:

  • pom.xml: the configuration file of the Maven project, which contains the dependencies, plug-ins, and build information of the project.
  • .idea: the directory used in the Integrated Development Environment (IDE) for storing project-related configurations.
  • src: the directory for storing source code of the project.
  • main: the directory for storing main source code and resource files.
  • java: the directory for storing Java source code.
  • com: the root directory for storing the Java package.
  • oceanbase: the root directory for storing the project.
  • example: the root directory for storing the project.
  • batch: the main package of the project.
  • BatchApplication.java: the entry class to the application, which contains the main method of the application.
  • config: the configuration class folder that contains the configuration class of the application.
  • BatchConfig.java: the configuration class of the application, which is used to configure some properties and behavior of the application.
  • model: the model class folder that contains the data model classes of the application.
  • People.java: the personnel data model class.
  • PeopleDESC.java: the personnel DESC data model class.
  • processor: the processor class folder that contains the processor class of the application.
  • AddPeopleDescProcessor.java: the processor class that adds personnel DESC information.
  • writer: the writer class folder that contains the writer classes of the application.
  • AddDescPeopleWriter.java: the writer class that writes personnel DESC information.
  • AddPeopleWriter.java: the writer class that writes personnel information.
  • resources: the resource folder that contains the configuration file and other static resource files of the application.
  • application.properties: the configuration file of the application, which is used to configure the properties of the application.
  • test: the directory for storing the test code and resource files.
  • BatchConfigTest.java: the test class for the configuration class of the application.
  • AddPeopleDescProcessorTest.java: the test class for the processor class that adds personnel DESC information.
  • AddDescPeopleWriterTest.java: the test class for the writer class that writes personnel DESC information.
  • AddPeopleWriterTest.java: the test class for the writer class that writes personnel information.
  • target: the directory for storing compiled class files and JAR packages.

Code in pom.xml

Note

If you just want to verify the sample project, use the default code without modification. You can also modify the pom.xml file as required based on the following instructions.

To configure the pom.xml file, perform the following steps:

  1. Declare the file.

    Declare the file to be an XML file that uses XML standard 1.0 and character encoding UTF-8.

    The sample code is as follows:

    <?xml version="1.0" encoding="UTF-8"?>

  2. Configure the namespaces and the POM model version.

    1. Use xmlns to specify http://maven.apache.org/POM/4.0.0 as the default XML namespace for the POM.
    2. Use xmlns:xsi to specify http://www.w3.org/2001/XMLSchema-instance as the XML namespace for xsi-prefixed elements.
    3. Use xsi:schemaLocation to provide a mapping from the default XML namespace for the POM (http://maven.apache.org/POM/4.0.0) to the location of the POM’s XML schema definition (XSD) file (https://maven.apache.org/xsd/maven-4.0.0.xsd).
    4. Use <modelVersion> to specify 4.0.0 as the model version used by the POM.

    The sample code is as follows:

     <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
     <modelVersion>4.0.0</modelVersion>
</project>

  3. Configure parent project information.

    1. Use <groupId> to specify org.springframework.boot as the ID of the parent project group.
    2. Use <artifactId> to specify spring-boot-starter-parent as the ID of the parent project.
    3. Use <version> to specify 2.7.11 as the version of the parent project.
    4. Use <relativePath> to specify an empty path for the parent project.

    The sample code is as follows:

     <parent>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-parent</artifactId>
     <version>2.7.11</version>
     <relativePath/>
 </parent>

  4. Configure basic project information.

    1. Use <groupId> to specify com.oceanbase as the ID of the project group.
    2. Use <artifactId> to specify java-oceanbase-springboot as the ID of the project.
    3. Use <version> to specify 0.0.1-SNAPSHOT as the version of the project.
    4. Use <description> to describe the project as Demo project for Spring Batch.

    The sample code is as follows:

     <groupId>com.oceanbase</groupId>
 <artifactId>java-oceanbase-springboot</artifactId>
 <version>0.0.1-SNAPSHOT</version>
 <name>java-oceanbase-springbatch</name>
 <description>Demo project for Spring Batch</description>

  5. Configure the Java version.

    Specify to use Java 1.8 for the project.

    The sample code is as follows:

      <properties>
      <java.version>1.8</java.version>
  </properties>

  6. Configure core dependencies.

    1. Specify org.springframework.boot as the ID of the group that the dependency belongs to, and spring-boot-starter as the dependency ID. This dependency contains default features provided by Spring Boot, such as web, data processing, security, and test.

      The sample code is as follows:

      <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter</artifactId>
</dependency>

    2. Specify org.springframework.boot as the ID of the group that the dependency belongs to, and spring-boot-starter-jdbc as the dependency ID. This dependency contains JDBC features provided by Spring Boot, such as the connection pool and data source configuration.

      The sample code is as follows:

      <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>

    3. Specify org.springframework.boot as the ID of the group that the dependency belongs to, and spring-boot-starter-test as the dependency ID. This dependency takes effect on test and provides test frameworks and tools of Spring Boot, such as JUnit, Mockito, and Hamcrest.

      The sample code is as follows:

      <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>

    4. Specify com.oceanbase as the ID of the group that the dependency belongs to, oceanbase-client as the dependency ID, and 2.4.3 as the dependency version. This dependency allows the project to use the client features, such as connection, query, and transaction, provided by OceanBase.

      The sample code is as follows:

          <dependency>
        <groupId>com.oceanbase</groupId>
        <artifactId>oceanbase-client</artifactId>
        <version>2.4.3</version>
    </dependency>

    5. Specify org.springframework.boot as the ID of the group that the dependency belongs to, and spring-boot-starter-batch as the dependency ID. This dependency contains the batch processing feature provided by Spring Boot.

      The sample code is as follows:

      <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch</artifactId>
</dependency>

    6. Specify org.springframework.boot as the ID of the group that the dependency belongs to, and spring-boot-starter-data-jpa as the dependency ID. This dependency contains necessary dependencies and configurations for JPA-based database accesses, and is a Spring Boot starter.

      The sample code is as follows:

      <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>

    7. Specify org.apache.tomcat as the ID of the group that the dependency belongs to, and tomcat-jdbc as the dependency ID. This dependency allows the application to use JDBC connection pool features provided by Tomcat, including connection pool configuration, connection acquisition and release, and connection management.

      The sample code is as follows:

      <dependency>
    <groupId>org.apache.tomcat</groupId>
    <artifactId>tomcat-jdbc</artifactId>
</dependency>

    8. Specify junit as the ID of the group that the dependency belongs to, junit as the dependency ID, 4.10 as the dependency version, and test as the effective scope. This dependency allows the application to use JUnit.

      The sample code is as follows:

      <dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.10</version>
    <scope>test</scope>
</dependency>

    9. Specify javax.activation as the ID of the group that the dependency belongs to, javax.activation-api as the dependency ID, and 1.2.0 as the dependency version. This dependency provides the Java Activation Framework (JAF) API.

      The sample code is as follows:

      <dependency>
    <groupId>javax.activation</groupId>
    <artifactId>javax.activation-api</artifactId>
    <version>1.2.0</version>
</dependency>

    10. Specify jakarta.persistence as the ID of the group that the dependency belongs to, jakarta.persistence-api as the dependency ID, and 2.2.3 as the dependency version. This dependency provides the Jakarta Persistence API.

      The sample code is as follows:

      <dependency>
    <groupId>jakarta.persistence</groupId>
    <artifactId>jakarta.persistence-api</artifactId>
    <version>2.2.3</version>
</dependency>

  7. Configure the Maven plug-in.

    Specify org.springframework.boot as the ID of the group that the plug-in belongs to, and spring-boot-maven-plugin as the plug-in ID. This plug-in can be used to package Spring Boot applications as executable JAR packages or WAR packages, or directly run Spring Boot applications.

    The sample code is as follows:

     <build>
     <plugins>
         <plugin>
             <groupId>org.springframework.boot</groupId>
             <artifactId>spring-boot-maven-plugin</artifactId>
         </plugin>
     </plugins>
 </build>

Code in application.properties

The application.properties file contains database connection configurations, such as the database driver, URL, username, and password. It also contains configurations related to the Java Persistence API (JPA), Spring Batch, and log level.

  1. Configure the database connection.

    • Use spring.datasource.driver to specify com.oceanbase.jdbc.Driver as the database driver for establishing a connection with OceanBase Database.
    • Use spring.datasource.url to specify the URL of the database.
    • Use spring.datasource.username to specify the username for connecting to the database.
    • Use spring.datasource.password to specify the password for connecting to the database.

    The sample code is as follows:

    spring.datasource.driver-class-name=com.oceanbase.jdbc.Driver
spring.datasource.url=jdbc:oceanbase://host:port/schema_name?characterEncoding=utf-8
spring.datasource.username=user_name
spring.datasource.password=******

  2. Configure the JPA.

    • Use spring.jpa.show-sql to specify whether to display SQL statements in logs. The value true here indicates that SQL statements are displayed in logs.
    • Use spring.jpa.hibernate.ddl-auto to specify the DDL operation performed by Hibernate. The value update here indicates that Hibernate automatically updates the database schema when the application starts.

    The sample code is as follows:

    spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=update

  3. Configure Spring Batch.

    Use spring.batch.job.enabled to specify whether to enable Spring Batch jobs. The value false here indicates that Spring Batch jobs are disabled.

    The sample code is as follows:

    spring.batch.job.enabled=false

  4. Configure the log level.

    • Use logging.level.org.springframework to specify INFO as the log level of the Spring framework.
    • Use logging.level.com.example to specify DEBUG as the log level for the custom code of the application.

    The sample code is as follows:

    logging.level.org.springframework=INFO
logging.level.com.example=DEBUG

Code in BatchApplication.java

The BatchApplication.java file is the entry file to the Spring Boot application.

To configure the BatchApplication.java file, perform the following steps:

  1. Reference other classes and interfaces.

    Declare that the current file contains the following interfaces and classes:

    • SpringApplication class: launches the Spring Boot application.
    • SpringBootApplication annotation: marks the class as the entry to the Spring Boot application.

    The sample code is as follows:

        import org.springframework.boot.SpringApplication;
    import org.springframework.boot.autoconfigure.SpringBootApplication;

  2. Define the BatchApplication class.

    Use the @SpringBootApplication annotation to mark the BatchApplication class as the entry to the Spring Boot application. In the BatchApplication class, define a static main method as the entry to the program. In the main method, use the SpringApplication.run method to launch the Spring Boot application. Define a method named runBatchJob to run the batch job.

    The sample code is as follows:

    

    @SpringBootApplication
    public class BatchApplication {
        public static void main(String[] args) {
            SpringApplication.run(BatchApplication.class, args);
        }

        public void runBatchJob() {
        }
    }

Code in BatchConfig.java

The BatchConfig.java file configures components such as the step, reader, processor, and writer for batch jobs.

To configure the BatchConfig.java file, perform the following steps:

  1. Reference other classes and interfaces.

    Declare that the current file contains the following interfaces and classes:

    • People class: stores personnel information read from the database.
    • PeopleDESC class: stores the description converted or processed from personnel information.
    • AddPeopleDescProcessor class: converts a People object to a PeopleDESC object. This class implements the ItemProcessor interface.
    • AddDescPeopleWriter class: writes a PeopleDESC object to a specified destination. This class implements the ItemWriter interface.
    • Job interface: indicates a batch job.
    • Step interface: indicates a step in a job.
    • EnableBatchProcessing annotation: enables and configures Spring Batch features.
    • JobBuilderFactory class: creates and configures jobs.
    • StepBuilderFactory class: creates and configures steps.
    • RunIdIncrementer class: the run ID incrementer of Spring Batch, which is used to increment the run ID each time a job is run.
    • ItemProcessor interface: processes or converts the read items.
    • ItemReader interface: reads items from the data source.
    • ItemWriter interface: writes processed or converted items to a specified destination.
    • JdbcCursorItemReader class: reads data from the database and returns a cursor result set.
    • Autowired annotation: injects dependencies.
    • Bean annotation: creates and configures beans.
    • ComponentScan annotation: specifies the package or class to scan for components.
    • Configuration annotation: marks a class as a configuration class.
    • EnableAutoConfiguration annotation: enables automatic configuration of Spring Boot.
    • SpringBootApplication annotation: marks the class as the entry to the Spring Boot application.
    • DataSource interface: obtains database connections.

    The sample code is as follows:

    import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import com.oceanbase.example.batch.processor.AddPeopleDescProcessor;
import com.oceanbase.example.batch.writer.AddDescPeopleWriter;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.database.JdbcCursorItemReader;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.BeanPropertyRowMapper;

import javax.sql.DataSource;

  2. Define the BatchConfig class.

    In the class, define how data is read, processed, and written, and encapsulate these steps as a simple Spring Batch job. Use the annotations and automatic configuration feature of Spring Batch to create corresponding component instances by calling various @Bean methods, and use these components to read, process, and write data in step1.

    • Use @Configuration to mark this class as a configuration class.
    • Use @EnableBatchProcessing to enable Spring Batch. The annotation automatically creates necessary beans, such as JobRepository and JobLauncher.
    • Use @SpringBootApplication to mark the main class of the Spring Boot application. The Spring Boot application is launched from the main class.
    • Use @ComponentScan to specify the package to scan for components. Spring Boot scans and registers all components in the package and its sub-packages.
    • Use @EnableAutoConfiguration to automatically configure the infrastructure of the Spring Boot application.

    The sample code is as follows:

     @Configuration
 @EnableBatchProcessing
 @SpringBootApplication
 @ComponentScan("com.oceanbase.example.batch.writer")
 @EnableAutoConfiguration
 public class BatchConfig {
 }

    1. Use the @Autowired annotation.

      Use the @Autowired annotation to inject JobBuilderFactory, StepBuilderFactory, and DataSource as member variables in the BatchConfig class. JobBuilderFactory is the factory class used to create and configure jobs. StepBuilderFactory is the factory class used to create and configure steps. DataSource is the interface used to obtain database connections.

      The sample code is as follows:

      @Autowired
private JobBuilderFactory jobBuilderFactory;

@Autowired
private StepBuilderFactory stepBuilderFactory;

@Autowired
private DataSource dataSource;

    2. Use the @Bean annotation.

      Use the @Bean annotation to define methods for creating readers, processors, writers, steps, and jobs.

      • Call the peopleReader method to create an instance of the ItemReader component. The component uses JdbcCursorItemReader to read People objects from the database. Set the data source in dataSource, set RowMapper to map database rows to People objects, and set the SQL query statement to SELECT * FROM people.

      • Call the addPeopleDescProcessor method to create an instance of the ItemProcessor component. The component uses AddPeopleDescProcessor to process People objects and return PeopleDESC objects.

      • Call the addDescPeopleWriter method to create an instance of the ItemWriter component. The component uses the AddDescPeopleWriter to write PeopleDESC objects to the destination.

      • Call the step1 method to create an instance of the Step component. Name the instance as step1. Call stepBuilderFactory.get to get the step builder. Set the reader to the ItemReader component, the processor to the ItemProcessor component, the writer to the ItemWriter component, and the chunk size to 10. Call build to build and return the configured Step instance.

      • Call the importJob method to create an instance of the Job component. Name the job as importJob. Call jobBuilderFactory.get to get the job builder. Set the incrementer to RunIdIncrementer and the initial step in flow to Step. Call build to build and return the configured Job instance.

        The sample code is as follows:

        @Bean
public ItemReader<People> peopleReader() {
    JdbcCursorItemReader<People> reader = new JdbcCursorItemReader<>();
    reader.setDataSource((javax.sql.DataSource) dataSource);
    reader.setRowMapper(new BeanPropertyRowMapper<>(People.class));
    reader.setSql("SELECT * FROM people");
    return reader;
}

@Bean
public ItemProcessor<People, PeopleDESC> addPeopleDescProcessor() {
    return new AddPeopleDescProcessor();
}

@Bean
public ItemWriter<PeopleDESC> addDescPeopleWriter() {
    return new AddDescPeopleWriter();
}

@Bean
public Step step1(ItemReader<People> reader, ItemProcessor<People, PeopleDESC> processor,
                ItemWriter<PeopleDESC> writer) {
    return stepBuilderFactory.get("step1")
            .<People, PeopleDESC>chunk(10)
            .reader(reader)
            .processor(processor)
            .writer(writer)
            .build();
}

@Bean
public Job importJob(Step step1) {
    return jobBuilderFactory.get("importJob")
            .incrementer(new RunIdIncrementer())
            .flow(step1)
            .end()
            .build();
}

Code in People.java

The People.java file defines a data model class named People to represent personnel information. The class contains two private member variables: name and age, and corresponding getter and setter methods. The toString method is overridden to print the object information. name indicates the name of a person, and age indicates the age of a person. The getter and setter methods get and set the values of these attributes.

The class provides data storage and transfer means for the input and output of a batch program. In batch reads and writes, People objects store data, setter methods set data, and getter methods get data.

The sample code is as follows:

    public class People {
        private String name;
        private int age;

            // getters and setters

        public String getName() {
            return name;
        }

        public void setName(String name) {
            this.name = name;
        }

        public int getAge() {
            return age;
        }

        public void setAge(int age) {
            this.age = age;
        }
        @Override
        public String toString() {
            return "People [name=" + name + ", age=" + age + "]";
        }
        // Getters and setters
    }

Code in PeopleDESC.java

The PeopleDESC.java file defines a data model class named PeopleDESC to represent the description of personnel information. The PeopleDESC class contains four attributes: name, age, desc, and id, which represent the name, age, description, and identifier of a person. The class also contains corresponding getter and setter methods for getting and setting the attribute values. The toString method is overridden to return a string representation of the class, including the name, age, and description.

Similar to the People class, the PeopleDESC class provides data storage and transfer means for the input and output of a batch program.

The sample code is as follows:

    public class PeopleDESC {
        private String name;
        private int age;
        private String desc;
        private int id;

        public String getName() {
            return name;
        }

        public void setName(String name) {
            this.name = name;
        }

        public int getAge() {
            return age;
        }

        public void setAge(int age) {
            this.age = age;
        }

        public String getDesc() {
            return desc;
        }

        public void setDesc(String desc) {
            this.desc = desc;
        }

        public int getId() {
            return id;
        }

        public void setId(int id) {
            this.id = id;
        }

        @Override
        public String toString() {
            return "PeopleDESC [name=" + name + ", age=" + age + ", desc=" + desc + "]";
        }
    }

Code in AddPeopleDescProcessor.java

The AddPeopleDescProcessor.java file defines a class named AddPeopleDescProcessor that implements the ItemProcessor interface for converting People objects to PeopleDESC objects.

To configure the AddPeopleDescProcessor.java file, perform the following steps:

  1. Reference other classes and interfaces.

    Declare that the current file contains the following interfaces and classes:

    • People class: stores personnel information read from the database.
    • PeopleDESC class: stores the description converted or processed from personnel information.
    • ItemProcessor interface: processes or converts the read items.

    The sample code is as follows:

    import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import org.springframework.batch.item.ItemProcessor;

  2. Define the AddPeopleDescProcessor class.

    The AddPeopleDescProcessor class of the ItemProcessor interface converts People objects to PeopleDESC objects, thus implementing the processing logic for the input data during batch processing.

    In the process method of this class, create a PeopleDESC object named desc, and then use the item parameter to obtain the attributes (name and age) of the People object and set these attributes to the desc object. At the same time, assign a value to the desc attribute of the desc object. The value assignment logic is to generate the description of the People object based on its attributes. Finally, return the processed PeopleDESC object.

    The sample code is as follows:

    public class AddPeopleDescProcessor implements ItemProcessor<People, PeopleDESC> {
    @Override
    public PeopleDESC process(People item) throws Exception {
        PeopleDESC desc = new PeopleDESC();
        desc.setName(item.getName());
        desc.setAge(item.getAge());
        desc.setDesc("This is " + item.getName() + " with age " + item.getAge());
        return desc;
    }
}

Code in AddDescPeopleWriter.java

The AddDescPeopleWriter.java file implements the AddDescPeopleWriter class of the ItemWriter interface to write PeopleDESC objects to the database.

To configure the AddDescPeopleWriter.java file, perform the following steps:

  1. Reference other classes and interfaces.

    Declare that the current file contains the following interfaces and classes:

    • PeopleDESC class: stores the description converted or processed from personnel information.
    • ItemWriter interface: writes processed or converted items to a specified destination.
    • Autowired annotation: injects dependencies.
    • JdbcTemplate class: provides methods for executing SQL statements.
    • List interface: handles query result sets.

    The sample code is as follows:

    import com.oceanbase.example.batch.model.PeopleDESC;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;

import java.util.List;

  2. Define the AddDescPeopleWriter class.

    1. Use the @Autowired annotation to automatically inject the JdbcTemplate instance, which will be used to perform database operations for data writes.

      The sample code is as follows:

          @Autowired
    private JdbcTemplate jdbcTemplate;

    2. In the write method, traverse List<? extends PeopleDESC> that is passed in to fetch all PeopleDESC objects in sequence.

      1. Execute the SQL statement DROP TABLE people_desc to drop the table named people_desc that may already exist.
      2. Execute the SQL statement CREATE TABLE people_desc (id INT PRIMARY KEY, name VARCHAR2(255), age INT, description VARCHAR2(255)) to create a table named people_desc with four columns: id, name, age, and description.
      3. Execute the SQL statement INSERT INTO people_desc (id, name, age, description) VALUES (?, ?, ?, ?) to insert attribute values of each PeopleDESC object into the people_desc table.

      The sample code is as follows:

          @Override
    public void write(List<? extends PeopleDESC> items) throws Exception {
        // Drop the table that may already exist.
        String dropTableSql = "BEGIN EXECUTE IMMEDIATE 'DROP TABLE people_desc'; EXCEPTION WHEN OTHERS THEN IF SQLCODE != -942 THEN RAISE; END IF; END;";
  jdbcTemplate.execute(dropTableSql);
        // Create the table.
        String createTableSql = "CREATE TABLE people_desc (id INT PRIMARY KEY, name VARCHAR2(255), age INT, description VARCHAR2(255))";
        jdbcTemplate.execute(createTableSql);
        for (PeopleDESC item : items) {
            String sql = "INSERT INTO people_desc (id, name, age, description) VALUES (?, ?, ?, ?)";
            jdbcTemplate.update(sql, item.getId(), item.getName(), item.getAge(), item.getDesc());
        }
    }

Code in AddPeopleWriter.java

The AddPeopleWriter.java file implements the AddPeopleWriter class of the ItemWriter interface to write People objects to the database.

To configure the AddPeopleWriter.java file, perform the following steps:

  1. Reference other classes and interfaces.

    Declare that the current file contains the following interfaces and classes:

    • People class: stores personnel information read from the database.
    • ItemWriter interface: writes processed or converted items to a specified destination.
    • Autowired annotation: injects dependencies.
    • JdbcTemplate class: provides methods for executing SQL statements.
    • Component annotation: marks the class as a Spring component.
    • List interface: handles query result sets.

    The sample code is as follows:

    import com.oceanbase.example.batch.model.People;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Component;

import java.util.List;

  2. Define the AddPeopleWriter class.

    1. Use the @Autowired annotation to automatically inject the JdbcTemplate instance, which will be used to perform database operations for data writes.

      The sample code is as follows:

          @Autowired
    private JdbcTemplate jdbcTemplate;

    2. In the write method, traverse List<? extends People> that is passed in to fetch each all People objects in sequence.

      1. Execute the SQL statement DROP TABLE people to drop the people table that may already exist.
      2. Execute the SQL statement CREATE TABLE people (name VARCHAR2(255), age INT) to create a table named people with two columns: name and age.
      3. Execute the SQL statement INSERT INTO people (name, age) VALUES (?, ?) to insert the attribute values of each People object into the people table.

      The sample code is as follows:

      @Override
public void write(List<? extends People> items) throws Exception {
    // Drop the table that may already exist.
    String dropTableSql = "BEGIN EXECUTE IMMEDIATE 'DROP TABLE people'; EXCEPTION WHEN OTHERS THEN IF SQLCODE != -942 THEN RAISE; END IF; END;";
  jdbcTemplate.execute(dropTableSql);
    // Create the table.
    String createTableSql = "CREATE TABLE people (name VARCHAR2(255), age INT)";
    jdbcTemplate.execute(createTableSql);
    for (People item : items) {
        String sql = "INSERT INTO people (name, age) VALUES (?, ?)";
        jdbcTemplate.update(sql, item.getName(), item.getAge());
    }
}

Code in BatchConfigTest.java

The BatchConfigTest.java file defines a class for testing the job configuration of Spring Batch by using JUnit.

To configure the BatchConfigTest.java file, perform the following steps:

  1. Reference other classes and interfaces.

    Declare that the current file contains the following interfaces and classes:

    • Assert class: asserts test results.
    • Test annotation: marks a test method.
    • RunWith annotation: specifies the test runner.
    • Job interface: indicates a batch job.
    • JobExecution class: indicates the execution of a batch job.
    • JobParameters class: indicates the parameters of a batch job.
    • JobParametersBuilder class: builds parameters of a batch job.
    • JobLauncher interface: launches a batch job.
    • Autowired annotation: injects dependencies.
    • SpringBootTest annotation: marks the test class as a Spring Boot test.
    • SpringRunner class: specifies SpringRunner as the test runner.

    The sample code is as follows:

    import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import javax.batch.runtime.BatchStatus;

  2. Define the BatchConfigTest class.

    With the SpringBootTest annotation and the SpringRunner runner, this class can perform Spring Boot integration tests. In the testJob method, use the JobLauncherTestUtils helper class to launch a batch job and use an assertion to verify the execution status of the job.

    1. Use the @Autowired annotation to automatically inject the JobLauncherTestUtils instance.

      The sample code is as follows:

      @Autowired
private JobLauncherTestUtils jobLauncherTestUtils;

    2. Use the @Test annotation to mark the testJob method as a test method. In this method, create a JobParameters object, call the jobLauncherTestUtils.launchJob method to launch the batch job, and then call the Assert.assertEquals method to assert the execution status of the job as COMPLETED.

      The sample code is as follows:

      @Test
public void testJob() throws Exception {
    JobParameters jobParameters = new JobParametersBuilder()
            .addString("jobParam", "paramValue")
            .toJobParameters();

    JobExecution jobExecution = jobLauncherTestUtils.launchJob(jobParameters);

    Assert.assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus());
}

    3. Use the @Autowired annotation to automatically inject the JobLauncher instance.

      The sample code is as follows:

      @Autowired
private JobLauncher jobLauncher;

    4. Use the @Autowired annotation to automatically inject the Job instance.

      The sample code is as follows:

      @Autowired
private Job job;

    5. Define a private class named JobLauncherTestUtils to assist in launching a batch job. In the class, define the launchJob method for launching a batch job. In this method, call the jobLauncher.run method to launch a job and return the execution result of the job.

      The sample code is as follows:

      private class JobLauncherTestUtils {
    public JobExecution launchJob(JobParameters jobParameters) throws Exception {
        return jobLauncher.run(job, jobParameters);
    }
}

Code in AddPeopleDescProcessorTest.java

The AddPeopleDescProcessorTest.java file defines a class for testing the job configuration of Spring Batch by using JUnit.

To configure the AddPeopleDescProcessorTest.java file, perform the following steps:

  1. Reference other classes and interfaces.

    Declare that the current file contains the following interfaces and classes:

    • People class: stores personnel information read from the database.
    • PeopleDESC class: stores the description converted or processed from personnel information.
    • Test annotation: marks a test method.
    • RunWith annotation: specifies the test runner.
    • Autowired annotation: injects dependencies.
    • SpringBootTest annotation: marks the test class as a Spring Boot test.
    • SpringRunner class: specifies SpringRunner as the test runner.

    The sample code is as follows:

    import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

  2. Define the AddPeopleDescProcessorTest class.

    Use the SpringBootTest annotation and the SpringRunner runner for Spring Boot integration testing.

    1. Use the @Autowired annotation to automatically inject the AddPeopleDescProcessor instance.

      The sample code is as follows:

      @Autowired
private AddPeopleDescProcessor processor;

    2. Use the @Test annotation to mark the testProcess method as a test method. In this method, create a People object, call the processor.process method to process the object, and then assign the result to a PeopleDESC object.

      The sample code is as follows:

      @Test
public void testProcess() throws Exception {
    People people = new People();
    PeopleDESC desc = processor.process(people);
}

Code in AddDescPeopleWriterTest.java

The AddDescPeopleWriterTest.java file defines a class for testing the write logic of AddDescPeopleWriter by using JUnit.

To configure the AddDescPeopleWriterTest.java file, perform the following steps:

  1. Reference other classes and interfaces.

    Declare that the current file contains the following interfaces and classes:

    • PeopleDESC class: stores the description converted or processed from personnel information.
    • Assert class: asserts test results.
    • Test annotation: marks a test method.
    • RunWith annotation: specifies the test runner.
    • Autowired annotation: injects dependencies.
    • SpringBootTest annotation: marks the test class as a Spring Boot test.
    • JdbcTemplate class: provides methods for executing SQL statements.
    • SpringRunner class: specifies SpringRunner as the test runner.
    • ArrayList class: creates an empty list.
    • List interface: handles query result sets.

    The sample code is as follows:

    import com.oceanbase.example.batch.model.PeopleDESC;
import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.ArrayList;
import java.util.List;

  2. Define the AddDescPeopleWriterTest class.

    Use the SpringBootTest annotation and the SpringRunner runner for Spring Boot integration testing.

    1. Use @Autowired to inject instances. Use the @Autowired annotation to automatically inject AddPeopleDescProcessor and JdbcTemplate instances.

      The sample code is as follows:

      @Autowired
private AddDescPeopleWriter writer;
@Autowired
private JdbcTemplate jdbcTemplate;

    2. Use the method marked with @Test to test data insertion and output. Use the @Test annotation to mark the testWrite method as a test method. In this method, create an empty peopleDescList list and then add two PeopleDESC objects to the list. Call the writer.write method to write data in the list to the database. Use jdbcTemplate to execute the query statement that obtains data from the people_desc table. Execute the assertion statement to verify the correctness of the data. Output the query results to the console and output a message to indicate that job execution is complete.

      1. Insert data into the people_desc table. Create an empty PeopleDESC object list named peopleDescList. Create two PeopleDESC objects named desc1 and desc2 and set their attribute values. Add desc1 and desc2 to the peopleDescList list. Call the write method of writer to write the objects in the peopleDescList list to the people_desc table in the database. Call JdbcTemplate to execute the query statement SELECT COUNT(*) FROM people_desc that obtains the number of records in the people_desc table. Assign the result to the count variable. Call the Assert.assertEquals method to assert whether the value of count is 2.

        The sample code is as follows:

           List<PeopleDESC> peopleDescList = new ArrayList<>();
   PeopleDESC desc1 = new PeopleDESC();
   desc1.setId(1);
   desc1.setName("John");
   desc1.setAge(25);
   desc1.setDesc("This is John with age 25");
   peopleDescList.add(desc1);
   PeopleDESC desc2 = new PeopleDESC();
   desc2.setId(2);
   desc2.setName("Alice");
   desc2.setAge(30);
   desc2.setDesc("This is Alice with age 30");
   peopleDescList.add(desc2);
   writer.write(peopleDescList);

   String selectSql = "SELECT COUNT(*) FROM people_desc";
   int count = jdbcTemplate.queryForObject(selectSql, Integer.class);
   Assert.assertEquals(2, count);

      2. Output data in the people_desc table. Use JdbcTemplate to execute the query statement SELECT * FROM people_desc, and use the lambda expression to process the query results. In the lambda expression, use methods such as rs.getInt and rs.getString to obtain field values in the query result set and set the field values to the newly created PeopleDESC objects. Add all the newly created PeopleDESC objects to the result list resultDesc. Print the prompt line Data in the people_desc table:. Then, use a for loop to traverse the resultDesc list and use System.out.println to print the PeopleDESC objects in the list one by one. Finally, print a message to indicate that job execution is complete.

        The sample code is as follows:

        List<PeopleDESC> resultDesc = jdbcTemplate.query("SELECT * FROM people_desc", (rs, rowNum) -> {
   PeopleDESC desc = new PeopleDESC();
   desc.setId(rs.getInt("id"));
   desc.setName(rs.getString("name"));
   desc.setAge(rs.getInt("age"));
   desc.setDesc(rs.getString("description"));
   return desc;
});

System.out.println("Data in the people_desc table:");
for (PeopleDESC desc : resultDesc) {
   System.out.println(desc);
}

// Output a message to indicate that job execution is complete.
System.out.println("Batch Job execution completed.");

Code in AddPeopleWriterTest.java

The AddPeopleWriterTest.java file defines a class for testing the write logic of AddPeopleWriterTest by using JUnit.

To configure the AddPeopleWriterTest.java file, perform the following steps:

  1. Reference other classes and interfaces.

    Declare that the current file contains the following interfaces and classes:

    • People class: stores personnel information read from the database.
    • Test annotation: marks a test method.
    • RunWith annotation: specifies the test runner.
    • Autowired annotation: injects dependencies.
    • SpringBootApplication annotation: marks the class as the entry to the Spring Boot application.
    • SpringBootTest annotation: marks the test class as a Spring Boot test.
    • ComponentScan annotation: specifies the package or class to scan for components.
    • JdbcTemplate class: provides methods for executing SQL statements.
    • SpringRunner class: specifies SpringRunner as the test runner.
    • ArrayList class: creates an empty list.
    • List interface: handles query result sets.

    The sample code is as follows:

    import com.oceanbase.example.batch.model.People;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.ArrayList;
import java.util.List;

  2. Define the AddPeopleWriterTest class.

    Use the SpringBootTest annotation and the SpringRunner runner for Spring Boot integration testing, and use the @ComponentScan annotation to specify the package to scan.

    1. Use @Autowired to inject instances. Use the @Autowired annotation to automatically inject addPeopleWriter and JdbcTemplate instances.

      The sample code is as follows:

      @Autowired
private AddPeopleWriter addPeopleWriter;
@Autowired
private JdbcTemplate jdbcTemplate;

    2. Use the method marked with @Test to test data insertion and output.

      1. Insert data into the people table. First, create an empty People object list named peopleList. Then, create two People objects: person1 and person2, and set their name and age attributes. Add the two People objects to the peopleList list. Call the write method of addPeopleWriter and pass peopleList to the method as an argument, so as to write the People objects to the database.

        The sample code is as follows:

           List<People> peopleList = new ArrayList<>();
   People person1 = new People();
   person1.setName("zhangsan");
   person1.setAge(27);
   peopleList.add(person1);
   People person2 = new People();
   person2.setName("lisi");
   person2.setAge(35);
   peopleList.add(person2);
   addPeopleWriter.write(peopleList);

      2. Output data in the people table. First, call JdbcTemplate to execute the query statement SELECT * FROM people, and use the lambda expression to process the query results. In the lambda expression, call the rs.getString and rs.getInt methods to obtain the field values in the query result set and set the field values to newly created People objects. Add all the newly created People objects to the result list result. Print the prompt line Data in the people table:. Then, use a for loop to traverse the result list and use System.out.println to print the People objects in the list one by one. Finally, print a message to indicate that job execution is complete.

        The sample code is as follows:

           List<People> result = jdbcTemplate.query("SELECT * FROM people", (rs, rowNum) -> {
       People person = new People();
       person.setName(rs.getString("name"));
       person.setAge(rs.getInt("age"));
       return person;
   });

   System.out.println("Data in the people table:");
   for (People person : result) {
       System.out.println(person);
   }

   // Output a message to indicate that job execution is complete.
   System.out.println("Batch Job execution completed.");

Complete code examples

pom.xml
application.properties
BatchApplication.java
BatchConfig.java
People.java
PeopleDESC.java
AddPeopleDescProcessor.java
AddDescPeopleWriter.java
AddPeopleWriter.java
BatchConfigTest.java
AddPeopleDescProcessorTest.java
AddDescPeopleWriterTest.java
AddPeopleWriterTest.java 
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.11</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.oceanbase</groupId>
    <artifactId>java-oceanbase-springboot</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>java-oceanbase-springbatch</name>
    <description>Demo project for Spring Batch</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>com.oceanbase</groupId>
            <artifactId>oceanbase-client</artifactId>
            <version>2.4.3</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-batch</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.tomcat</groupId>
            <artifactId>tomcat-jdbc</artifactId>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>javax.activation</groupId>
            <artifactId>javax.activation-api</artifactId>
            <version>1.2.0</version>
        </dependency>
        <dependency>
            <groupId>jakarta.persistence</groupId>
            <artifactId>jakarta.persistence-api</artifactId>
            <version>2.2.3</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>


#configuration database

spring.datasource.driver-class-name=com.oceanbase.jdbc.Driver
spring.datasource.url=jdbc:oceanbase://host:port/schema_name?characterEncoding=utf-8
spring.datasource.username=user_name
spring.datasource.password=

# JPA
spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=update

# Spring Batch
spring.batch.job.enabled=false

#
logging.level.org.springframework=INFO
logging.level.com.example=DEBUG

package com.oceanbase.example.batch;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class BatchApplication {
    public static void main(String[] args) {
        SpringApplication.run(BatchApplication.class, args);
    }

    public void runBatchJob() {
    }
}


package com.oceanbase.example.batch.config;

import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import com.oceanbase.example.batch.processor.AddPeopleDescProcessor;
import com.oceanbase.example.batch.writer.AddDescPeopleWriter;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.database.JdbcCursorItemReader;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.BeanPropertyRowMapper;

import javax.sql.DataSource;
//import javax.activation.DataSource;

@Configuration
@EnableBatchProcessing
@SpringBootApplication
@ComponentScan("com.oceanbase.example.batch.writer")
@EnableAutoConfiguration
public class BatchConfig {
    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    @Autowired
    private DataSource dataSource;// Use the default dataSource provided by automatic Spring Boot configuration



    @Bean
    public ItemReader<People> peopleReader() {
        JdbcCursorItemReader<People> reader = new JdbcCursorItemReader<>();
        reader.setDataSource((javax.sql.DataSource) dataSource);
        reader.setRowMapper(new BeanPropertyRowMapper<>(People.class));
        reader.setSql("SELECT * FROM people");
        return reader;
    }

    @Bean
    public ItemProcessor<People, PeopleDESC> addPeopleDescProcessor() {
        return new AddPeopleDescProcessor();
    }

    @Bean
    public ItemWriter<PeopleDESC> addDescPeopleWriter() {
        return new AddDescPeopleWriter();
    }

    @Bean
    public Step step1(ItemReader<People> reader, ItemProcessor<People, PeopleDESC> processor,
                      ItemWriter<PeopleDESC> writer) {
        return stepBuilderFactory.get("step1")
                .<People, PeopleDESC>chunk(10)
                .reader(reader)
                .processor(processor)
                .writer(writer)
                .build();
    }

    @Bean
    public Job importJob(Step step1) {
        return jobBuilderFactory.get("importJob")
                .incrementer(new RunIdIncrementer())
                .flow(step1)
                .end()
                .build();
    }
}

package com.oceanbase.example.batch.model;

public class People {
    private String name;
    private int age;

        // getters and setters

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }
    @Override
    public String toString() {
        return "People [name=" + name + ", age=" + age + "]";
    }
    // Getters and setters
}

package com.oceanbase.example.batch.model;

public class PeopleDESC {
    private String name;
    private int age;
    private String desc;
    private int id;

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }

    public String getDesc() {
        return desc;
    }

    public void setDesc(String desc) {
        this.desc = desc;
    }

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    @Override
    public String toString() {
        return "PeopleDESC [name=" + name + ", age=" + age + ", desc=" + desc + "]";
    }
}

package com.oceanbase.example.batch.processor;

import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import org.springframework.batch.item.ItemProcessor;


public class AddPeopleDescProcessor implements ItemProcessor<People, PeopleDESC> {
    @Override
    public PeopleDESC process(People item) throws Exception {
        PeopleDESC desc = new PeopleDESC();
        desc.setName(item.getName());
        desc.setAge(item.getAge());
        desc.setDesc("This is " + item.getName() + " with age " + item.getAge());
        return desc;
    }
}

package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.PeopleDESC;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;

import java.util.List;

@Component
public class AddDescPeopleWriter implements ItemWriter<PeopleDESC> {
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Override
    public void write(List<? extends PeopleDESC> items) throws Exception {
        // Drop the table that may already exist.
        String dropTableSql = "BEGIN EXECUTE IMMEDIATE 'DROP TABLE people_desc'; EXCEPTION WHEN OTHERS THEN IF SQLCODE != -942 THEN RAISE; END IF; END;";
        jdbcTemplate.execute(dropTableSql);
        // Create the table.
        String createTableSql = "CREATE TABLE people_desc (id INT PRIMARY KEY, name VARCHAR2(255), age INT, description VARCHAR2(255))";
        jdbcTemplate.execute(createTableSql);
        for (PeopleDESC item : items) {
            String sql = "INSERT INTO people_desc (id, name, age, description) VALUES (?, ?, ?, ?)";
            jdbcTemplate.update(sql, item.getId(), item.getName(), item.getAge(), item.getDesc());
        }
    }
}

package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.People;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Component;

import java.util.List;

@Component
public class AddPeopleWriter implements ItemWriter<People> {
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Override
    public void write(List<? extends People> items) throws Exception {
        // Drop the table that may already exist.
        String dropTableSql = "BEGIN EXECUTE IMMEDIATE 'DROP TABLE people'; EXCEPTION WHEN OTHERS THEN IF SQLCODE != -942 THEN RAISE; END IF; END;";
        jdbcTemplate.execute(dropTableSql);
        // Create the table.
        String createTableSql = "CREATE TABLE people (name VARCHAR2(255), age INT)";
        jdbcTemplate.execute(createTableSql);
        for (People item : items) {
            String sql = "INSERT INTO people (name, age) VALUES (?, ?)";
            jdbcTemplate.update(sql, item.getName(), item.getAge());
        }
    }
}


package com.oceanbase.example.batch.config;

import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import javax.batch.runtime.BatchStatus;

@RunWith(SpringRunner.class)
@SpringBootTest
public class BatchConfigTest {
    @Autowired
    private JobLauncherTestUtils jobLauncherTestUtils;

    @Test
    public void testJob() throws Exception {
        JobParameters jobParameters = new JobParametersBuilder()
                .addString("jobParam", "paramValue")
                .toJobParameters();

        JobExecution jobExecution = jobLauncherTestUtils.launchJob(jobParameters);

        Assert.assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus());
    }

    @Autowired
    private JobLauncher jobLauncher;

    @Autowired
    private Job job;

    private class JobLauncherTestUtils {
        public JobExecution launchJob(JobParameters jobParameters) throws Exception {
            return jobLauncher.run(job, jobParameters);
        }
    }
}

package com.oceanbase.example.batch.processor;

import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest
public class AddPeopleDescProcessorTest {
    @Autowired
    private AddPeopleDescProcessor processor;

    @Test
    public void testProcess() throws Exception {
        People people = new People();
  //      people.setName("John");
  //      people.setAge(25);

        PeopleDESC desc = processor.process(people);

//      Assert.assertEquals("John", desc.getName());
//        Assert.assertEquals(25, desc.getAge());
 //       Assert.assertEquals("This is John with age 25", desc.getDesc());
    }
}


package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.PeopleDESC;
import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.ArrayList;
import java.util.List;

@RunWith(SpringRunner.class)
@SpringBootTest
public class AddDescPeopleWriterTest {
    @Autowired
    private AddDescPeopleWriter writer;
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Test
    public void testWrite() throws Exception {


        // Insert data into the people_desc table.
        List<PeopleDESC> peopleDescList = new ArrayList<>();
        PeopleDESC desc1 = new PeopleDESC();
        desc1.setId(1);
        desc1.setName("John");
        desc1.setAge(25);
        desc1.setDesc("This is John with age 25");
        peopleDescList.add(desc1);
        PeopleDESC desc2 = new PeopleDESC();
        desc2.setId(2);
        desc2.setName("Alice");
        desc2.setAge(30);
        desc2.setDesc("This is Alice with age 30");
        peopleDescList.add(desc2);
        writer.write(peopleDescList);

        String selectSql = "SELECT COUNT(*) FROM people_desc";
        int count = jdbcTemplate.queryForObject(selectSql, Integer.class);
        Assert.assertEquals(2, count);

        // Output data in the people_desc table.
        List<PeopleDESC> resultDesc = jdbcTemplate.query("SELECT * FROM people_desc", (rs, rowNum) -> {
            PeopleDESC desc = new PeopleDESC();
            desc.setId(rs.getInt("id"));
            desc.setName(rs.getString("name"));
            desc.setAge(rs.getInt("age"));
            desc.setDesc(rs.getString("description"));
            return desc;
        });

        System.out.println("Data in the people_desc table:");
        for (PeopleDESC desc : resultDesc) {
            System.out.println(desc);
        }

        // Output a message to indicate that job execution is complete.
        System.out.println("Batch Job execution completed.");
    }
}

package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.People;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.ArrayList;
import java.util.List;

@RunWith(SpringRunner.class)
@SpringBootTest
@SpringBootApplication
@ComponentScan("com.oceanbase.example.batch.writer")
public class AddPeopleWriterTest {

    @Autowired
    private AddPeopleWriter addPeopleWriter;
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Test
    public void testWrite() throws Exception {
        // Insert data into the people table.
        List<People> peopleList = new ArrayList<>();
        People person1 = new People();
        person1.setName("zhangsan");
        person1.setAge(27);
        peopleList.add(person1);
        People person2 = new People();
        person2.setName("lisi");
        person2.setAge(35);
        peopleList.add(person2);
        addPeopleWriter.write(peopleList);

        // Query and output the result.
        List<People> result = jdbcTemplate.query("SELECT * FROM people", (rs, rowNum) -> {
            People person = new People();
            person.setName(rs.getString("name"));
            person.setAge(rs.getInt("age"));
            return person;
        });

        System.out.println("Data in the people table:");
        for (People person : result) {
            System.out.println(person);
        }

        // Output a message to indicate that job execution is complete.
        System.out.println("Batch Job execution completed.");
    }
}

References

For more information about OceanBase Connector/J, see OceanBase Connector/J.

Last
Next

OceanBaseOceanBase

OceanBase, A Highly Scalable Database for Transactional, Analytical, and AI Workloads.

Company

About OceanBaseContact UsPartnerTrust Center

Product

OceanBase CloudOceanBase EnterpriseQuick StartDownloadPricing

Resources

DocsBlogWhite Paper

Follow us

LinkedInLinkedInDiscordDiscordTwitterTwitterYoutubeYoutubeForumForumGitHubGitHubStack OverflowStack Overflow

© OceanBase 2024. All rights reserved | Cloud Service Agreement | Privacy Policy | Legal | Security

Contact Us