SpringBatch sample application for connecting to OceanBase Cloud

2026-01-21 03:00:30  Updated

This topic describes how to use the SpringBatch framework and OceanBase Cloud to build an application that performs basic operations such as creating tables, inserting data, and querying data.

Prerequisites

  • You have registered an Alibaba Cloud account and created an instance and a MySQL compatible tenant. For more information, see Create an instance and Create a tenant.
  • You have installed JDK 1.8 and Maven.
  • You have installed IntelliJ IDEA.

Note

The code examples in this topic are run in IntelliJ IDEA 2021.3.2 (Community Edition). You can also use your preferred tool to run the code examples.

Procedure

Note

The operations described in this topic are performed in the Windows environment. If you are using other operating systems or compilers, the operations may vary slightly.

  1. Obtain the connection string of the OceanBase Cloud database.
  2. Import the java-oceanbase-springbatch project into IntelliJ IDEA.
  3. Modify the database connection information in the java-oceanbase-springbatch project.
  4. Run the java-oceanbase-springbatch project.

Step 1: Obtain the connection string of the OceanBase Cloud database

  1. Log in to the OceanBase Cloud console. In the instance list, find the target instance and click the Connect > Get Connection String option in the target tenant.

    For more information, see Obtain the connection string.

  2. Fill in the following URL based on the information of the created OceanBase Cloud database.

    Note

    The URL information is required in the application.properties file.

    jdbc:oceanbase://host:port/schema_name?user=$user_name&password=$password&characterEncoding=utf-8

    Parameter description:

    • host: the endpoint of the OceanBase Cloud database, for example, t********.********.oceanbase.cloud.
    • port: the port of the OceanBase Cloud database. The default value is 3306.
    • schema_name: the name of the schema to be accessed.
    • user_name: the username for accessing the database.
    • password: the password for the account.
    • characterEncoding: the character encoding.

For more information about the URL parameters, see Database URL.

Step 2: Import the java-oceanbase-springbatch project into IntelliJ IDEA

  1. Open IntelliJ IDEA and choose File > Open....

    file

  2. In the Open File or Project window that appears, select the project file and click OK.

  3. IntelliJ IDEA automatically identifies various files in the project and displays the project structure, file list, module list, and dependency relationships in the Project tool window. The Project tool window is usually located on the left side of the IntelliJ IDEA interface and is open by default. If the Project tool window is closed, you can click View > Tool Windows > Project in the menu bar or press Alt + 1 to reopen it.

    Note

    When you import a project using IntelliJ IDEA, it automatically detects the pom.xml file in the project, downloads the required dependency libraries based on the described dependencies, and adds them to the project.

  4. View the project.

springbatch

Step 3: Modify the database connection information in the java-oceanbase-springbatch project

Modify the database connection information in the application.properties file based on the information obtained in Step 1: Obtain the connection string of the OceanBase Cloud database.

Here is an example:

  • The name of the database driver is com.mysql.cj.jdbc.Driver.
  • The endpoint of the OceanBase Cloud database is t5******.********.oceanbase.cloud.
  • The port is 3306.
  • The name of the schema to be accessed is test.
  • The username of the tenant is mysql001.
  • The password is ******.

Here is the sample code:

spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:oceanbase://t********.********.oceanbase.cloud:3306/test?characterEncoding=utf-8
spring.datasource.username=mysql001
spring.datasource.password=******

spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=update

spring.batch.job.enabled=false

logging.level.org.springframework=INFO
logging.level.com.example=DEBUG

Step 4: Run the java-oceanbase-springbatch project

  • Run the AddDescPeopleWriterTest.java file.

    1. In the project structure, go to src > test > java and find the AddDescPeopleWriterTest.java file.
    2. In the tool menu bar, choose Run > Run... > AddDescPeopleWriterTest.testWrite, or click the green triangle in the upper-right corner to run.
    3. View the log information and output results in the console of IntelliJ IDEA.
    Data in the people_desc table:
PeopleDESC [name=John, age=25, desc=This is John with age 25]
PeopleDESC [name=Alice, age=30, desc=This is Alice with age 30]
Batch Job execution completed.

  • Run the AddPeopleWriterTest.java file.

    1. In the project structure, go to src > test > java and find the AddDescPeopleWriterTest.java file.
    2. In the tool menu bar, choose Run > Run... > AddPeopleWriterTest.testWrite, or click the green triangle in the upper-right corner to run.
    3. View the log information and output results in the console of IntelliJ IDEA.
    Data in the people table:
People [name=zhangsan, age=27]
People [name=lisi, age=35]
Batch Job execution completed.

FAQ

1. Connection timeout

If you encounter a connection timeout issue, you can configure the connection timeout parameter in the JDBC URL:

jdbc:mysql://host:port/database?connectTimeout=30000&socketTimeout=60000

2. Character set

To ensure the correct character encoding, set the appropriate character set parameter in the JDBC URL:

jdbc:mysql://host:port/database?characterEncoding=utf8&useUnicode=true

3. SSL connection

To enable an SSL connection to OceanBase Cloud, add the following parameter to the JDBC URL:

jdbc:mysql://host:port/database?useSSL=true&requireSSL=true

4. Special characters in the account password

If the username or password contains special characters (such as #), you need to URL-encode them:

String encodedPassword = URLEncoder.encode(password, "UTF-8");

Notice

When using MySQL Connector/J 8.x, ensure that the account password does not contain the # character. Otherwise, you may encounter a connection error.

Project code

Click java-oceanbase-springbatch to download the project code, which is a compressed file named java-oceanbase-springbatch.

After decompressing it, you will find a folder named java-oceanbase-springbatch. The directory structure is as follows:

│  pom.xml
│
├─.idea
│
├─src
│  ├─main
│  │  ├─java
│  │  │  └─com
│  │  │      └─oceanbase
│  │  │          └─example
│  │  │              └─batch
│  │  │                  │──BatchApplication.java
│  │  │                  │
│  │  │                  ├─config
│  │  │                  │   └─BatchConfig.java
│  │  │                  │
│  │  │                  ├─model
│  │  │                  │   ├─People.java
│  │  │                  │   └─PeopleDESC.java
│  │  │                  │
│  │  │                  ├─processor
│  │  │                  │   └─AddPeopleDescProcessor.java
│  │  │                  │
│  │  │                  └─writer
│  │  │                      ├─AddDescPeopleWriter.java
│  │  │                      └─AddPeopleWriter.java
│  │  │
│  │  └─resources
│  │      └─application.properties
│  │
│  └─test
│      └─java
│          └─com
│              └─oceanbase
│                  └─example
│                      └─batch
│                          ├─config
│                          │   └─BatchConfigTest.java
│                          │
│                          ├─processor
│                          │   └─AddPeopleDescProcessorTest.java
│                          │
│                          └─writer
│                              ├─AddDescPeopleWriterTest.java
│                              └─AddPeopleWriterTest.java
│
└─target

File description:

  • pom.xml: the configuration file of the Maven project, which contains information about the project's dependencies, plugins, and build process.
  • .idea: a directory used by the IDE (Integrated Development Environment) to store project-related configuration information.
  • src: a directory typically used to store the source code of the project.
  • main: a directory that stores the main source code and resource files.
  • java: a directory that stores the Java source code.
  • com.oceanbase.example.batch: the package name.
  • BatchApplication.java: the entry class of the application, which contains the main method of the application.
  • config: a directory that stores the configuration classes of the application.
  • BatchConfig.java: the configuration class of the application, used to configure some properties and behaviors of the application.
  • model: a directory that stores the data model classes of the application.
  • People.java: a data model class for people.
  • PeopleDESC.java: a data model class for people's DESC information.
  • processor: a directory that stores the processor classes of the application.
  • AddPeopleDescProcessor.java: a processor class for adding people's DESC information.
  • writer: a directory that stores the writer classes of the application.
  • AddDescPeopleWriter.java: a writer class for writing people's DESC information.
  • AddPeopleWriter.java: a writer class for writing people's information.
  • resources: a directory that stores the configuration files and other static resources of the application.
  • application.properties: the configuration file of the application, used to configure the properties of the application.
  • test: a directory that stores the test code and resource files.
  • BatchConfigTest.java: the test class of the application's configuration class.
  • AddPeopleDescProcessorTest.java: the test class of the add-people-DESC processor.
  • AddDescPeopleWriterTest.java: the test class of the writer for writing people's DESC information.
  • AddPeopleWriterTest.java: the test class of the writer for writing people's information.
  • target: a directory that stores the compiled Class files, JAR packages, and other files.

Introduction to the pom.xml file

Note

If you only want to verify the example, you can use the default code without any modifications. You can also modify the pom.xml file according to your needs as described below.

The content of the pom.xml configuration file is as follows:

  1. File declaration statement.

    This statement declares that the file is an XML file using XML version 1.0 and character encoding UTF-8.

    Sample code:

    <?xml version="1.0" encoding="UTF-8"?>

  2. Configure the namespaces and POM model version.

    1. Use xmlns to set the POM namespace to http://maven.apache.org/POM/4.0.0.
    2. Use xmlns:xsi to set the XML namespace to http://www.w3.org/2001/XMLSchema-instance.
    3. Use xsi:schemaLocation to set the POM namespace to http://maven.apache.org/POM/4.0.0 and the location of the POM's XSD file to https://maven.apache.org/xsd/maven-4.0.0.xsd.
    4. Use the <modelVersion> element to set the POM model version used by the POM file to 4.0.0.

    Sample code:

     <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
     <modelVersion>4.0.0</modelVersion>
</project>

  3. Configure the parent project information.

    1. Use <groupId> to set the parent project identifier to org.springframework.boot.
    2. Use <artifactId> to set the parent project dependency to spring-boot-starter-parent.
    3. Use <version> to set the parent project version to 2.7.11.
    4. Use relativePath to indicate that the parent project path is empty.

    Sample code:

     <parent>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-parent</artifactId>
     <version>2.7.11</version>
     <relativePath/>
 </parent>

  4. Configure the basic information.

    1. Use <groupId> to set the project identifier to com.oceanbase.
    2. Use <artifactId> to set the project dependency to java-oceanbase-springboot.
    3. Use <version> to set the project version to 0.0.1-SNAPSHOT.
    4. Use description to describe the project information as Demo project for Spring Batch.

    Sample code:

     <groupId>com.oceanbase</groupId>
 <artifactId>java-oceanbase-springboot</artifactId>
 <version>0.0.1-SNAPSHOT</version>
 <name>java-oceanbase-springbatch</name>
 <description>Demo project for Spring Batch</description>

  5. Configure the Java version.

    Set the Java version used by the project to 1.8.

    Sample code:

      <properties>
      <java.version>1.8</java.version>
  </properties>

  6. Configure the core dependencies.

    1. Set the organization to org.springframework.boot, the name to spring-boot-starter, and use this dependency to access the components supported by Spring Boot, including Web, data processing, security, and Test.

      Sample code:

      <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter</artifactId>
</dependency>

    2. Set the organization to org.springframework.boot, the name to spring-boot-starter-jdbc, and use this dependency to access the JDBC-related features provided by Spring Boot, including connection pools and data source configurations.

      Sample code:

      <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>

    3. Set the organization to org.springframework.boot, the name to spring-boot-starter-test, and the scope to test. Use this dependency to access the testing framework and tools provided by Spring Boot, including JUnit, Mockito, and Hamcrest.

      Sample code:

      <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>

    4. Set the organization to com.oceanbase, the name to oceanbase-client, and the version to 2.4.12. Use this dependency to access the client features provided by OceanBase, including connections, queries, and transactions.

      Sample code:

          <dependency>
        <groupId>com.oceanbase</groupId>
        <artifactId>oceanbase-client</artifactId>
        <version>2.4.12</version>
    </dependency>

    5. Set the organization to org.springframework.boot, the name to spring-boot-starter-batch, and use this dependency to access the batch processing features provided by Spring Boot.

      Sample code:

      <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch</artifactId>
</dependency>

    6. Set the organization to org.springframework.boot, the name to spring-boot-starter-data-jpa, and use this dependency to access the necessary dependencies and configurations for data access using JPA. Spring Boot Starter Data JPA is a Spring Boot starter.

      Sample code:

      <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>

    7. Set the organization to org.apache.tomcat, the name to tomcat-jdbc, and use this dependency to access the JDBC connection pool features provided by Tomcat, including connection pool configurations, connection acquisition and release, and connection management.

      Sample code:

      <dependency>
    <groupId>org.apache.tomcat</groupId>
    <artifactId>tomcat-jdbc</artifactId>
</dependency>

    8. Set the dependency architecture to junit, the name to junit, the version to 4.10, and the scope to test. Use this dependency to add the JUnit unit test dependency configuration.

      Sample code:

      <dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.10</version>
    <scope>test</scope>
</dependency>

    9. Set the organization to javax.activation, the name to javax.activation-api, and the version to 1.2.0. Use this dependency to import the Java Activation Framework (JAF) library.

      Sample code:

      <dependency>
    <groupId>javax.activation</groupId>
    <artifactId>javax.activation-api</artifactId>
    <version>1.2.0</version>
</dependency>

    10. Set the organization to jakarta.persistence, the name to jakarta.persistence-api, and the version to 2.2.3. Use this dependency to add the Jakarta Persistence API dependency configuration. Sample code:

      <dependency>
    <groupId>jakarta.persistence</groupId>
    <artifactId>jakarta.persistence-api</artifactId>
    <version>2.2.3</version>
</dependency>

  7. Configure the Maven plugin.

    Set the organization to org.springframework.boot, the name to spring-boot-maven-plugin, and use this plugin to package Spring Boot applications into executable JAR or WAR packages that can be directly run.

    Sample code:

     <build>
     <plugins>
         <plugin>
             <groupId>org.springframework.boot</groupId>
             <artifactId>spring-boot-maven-plugin</artifactId>
         </plugin>
     </plugins>
 </build>

application.properties file

The application.properties file is used to configure database connections and other related settings. This includes database drivers, connection URLs, usernames, and passwords. It also contains configurations for JPA (Java Persistence API) and Spring Batch, as well as log level settings.

  1. Database connection configuration.

    • Use spring.datasource.driver to specify the database driver as com.mysql.cj.jdbc.Driver, which is used to connect to the OceanBase Cloud database.
    • Use spring.datasource.url to specify the URL for connecting to the database.
    • Use spring.datasource.username to specify the username for connecting to the database.
    • Use spring.datasource.password to specify the password for connecting to the database.

    Sample code:

    spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:oceanbase://host:port/schema_name?characterEncoding=utf-8
spring.datasource.username=user_name
spring.datasource.password=******

  2. JPA configuration.

    • Use spring.jpa.show-sql to specify whether to display SQL statements in the log, set to true to display SQL statements.
    • Use spring.jpa.hibernate.ddl-auto to specify the Hibernate DDL operation behavior, set to update to automatically update the database structure when the application starts.

    Sample code:

    spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=update

  3. Spring Batch configuration:

    Use spring.batch.job.enabled to specify whether to enable Spring Batch jobs, set to false to disable automatic execution of batch jobs.

    Sample code:

    spring.batch.job.enabled=false

    Note

    In Spring Batch, the spring.batch.job.enabled property controls the execution behavior of batch jobs.

    • spring.batch.job.enabled=true (default): Indicates that all defined batch jobs are automatically executed when the Spring Boot application starts. This means that Spring Batch automatically discovers and executes all defined jobs when the application starts.
    • spring.batch.job.enabled=false: Indicates that automatic execution of batch jobs is disabled. This is typically used in development or testing environments, or when you want to manually control job execution. When set to false, jobs will not be automatically executed when the application starts. You can manually trigger jobs using other methods such as REST APIs or command-line interfaces.
    In summary, setting spring.batch.job.enabled=false helps prevent jobs from being automatically executed when the application starts, providing greater flexibility in controlling when batch jobs are executed.

  4. Log configuration:

    • Use logging.level.org.springframework to set the log level for the Spring framework to INFO.
    • Use logging.level.com.example to set the log level for custom application code to DEBUG.

    Sample code:

    logging.level.org.springframework=INFO
logging.level.com.example=DEBUG

BatchApplication.java file

The BatchApplication.java file is the entry point of the Spring Boot application.

The code in the BatchApplication.java file mainly includes the following parts:

  1. Import other classes and interfaces.

    Declare the interfaces and classes included in the current file:

    • SpringApplication class: used to start the Spring Boot application.
    • SpringBootApplication annotation: used to mark the class as the entry point of the Spring Boot application.

    Sample code:

        import org.springframework.boot.SpringApplication;
    import org.springframework.boot.autoconfigure.SpringBootApplication;

  2. Define the BatchApplication class.

    Use the @SpringBootApplication annotation to mark the BatchApplication class as the entry point of the Spring Boot application. In the BatchApplication class, define a static main method as the entry point of the application. In this method, use the SpringApplication.run method to start the Spring Boot application. Also, define a method named runBatchJob to run the batch job.

    Sample code:

    

    @SpringBootApplication
    public class BatchApplication {
        public static void main(String[] args) {
            SpringApplication.run(BatchApplication.class, args);
        }

        public void runBatchJob() {
        }
    }

Introduction to the BatchConfig.java file

The BatchConfig.java file is used to configure components such as steps, readers, processors, and writers for batch processing jobs.

The code in the BatchConfig.java file mainly includes the following parts:

  1. Import other classes and interfaces.

    The following interfaces and classes are declared in this file:

    • People class: used to store personnel information read from the database.
    • PeopleDESC class: used to store description information after the personnel information is converted or processed.
    • AddPeopleDescProcessor class: an implementation class of the ItemProcessor interface. It converts the People object read to the PeopleDESC object.
    • AddDescPeopleWriter class: an implementation class of the ItemWriter interface. It writes the PeopleDESC object to the target location.
    • Job interface: represents a batch processing job.
    • Step interface: represents a step in a job.
    • EnableBatchProcessing annotation: a Spring Batch configuration annotation used to enable and configure the Spring Batch processing feature.
    • JobBuilderFactory class: used to create and configure jobs.
    • StepBuilderFactory class: used to create and configure steps.
    • RunIdIncrementer class: a Spring Batch run ID (Run ID) auto-incrementer. It is used to increase the run ID each time the job is run.
    • ItemProcessor interface: used to process or convert the read items.
    • ItemReader interface: used to read items from the data source.
    • ItemWriter interface: used to write the processed or converted items to the specified target location.
    • JdbcCursorItemReader class: used to read data from the database and return the cursor result set.
    • Autowired annotation: used for dependency injection.
    • Bean annotation: used to create and configure beans.
    • ComponentScan annotation: used to specify the package or class to be scanned for components.
    • Configuration annotation: used to mark a class as a configuration class.
    • EnableAutoConfiguration annotation: used to enable Spring Boot auto-configuration.
    • SpringBootApplication annotation: used to mark the class as the entry point of the Spring Boot application.
    • DataSource interface: used to represent the database connection.

    Code:

    import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import com.oceanbase.example.batch.processor.AddPeopleDescProcessor;
import com.oceanbase.example.batch.writer.AddDescPeopleWriter;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.database.JdbcCursorItemReader;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.BeanPropertyRowMapper;

import javax.sql.DataSource;

  2. Define the BatchConfig class.

    This is a simple Spring Batch batch processing job. It defines the methods for reading, processing, and writing data, and encapsulates these steps into a job. By using Spring Batch annotations and auto-configuration features, you can create corresponding component instances through the @Bean methods in the configuration class and use these components in step1 to complete data reading, processing, and writing.

    • Use @Configuration to indicate that this class is a configuration class.
    • Use @EnableBatchProcessing to enable the Spring Batch processing feature. This annotation automatically creates necessary beans, such as JobRepository and JobLauncher.
    • Use @SpringBootApplication as the main class annotation for Spring Boot applications, which is the starting point of the Spring Boot application.
    • Use @ComponentScan to specify the package to be scanned for components. It tells Spring to scan and register all components in this package and its subpackages.
    • Use @EnableAutoConfiguration to automatically configure the infrastructure of the Spring Boot application.

    Code:

     @Configuration
 @EnableBatchProcessing
 @SpringBootApplication
 @ComponentScan("com.oceanbase.example.batch.writer")
 @EnableAutoConfiguration
 public class BatchConfig {
 }

    1. Define the @Autowired annotation.

      Use the @Autowired annotation to inject JobBuilderFactory, StepBuilderFactory, and DataSource into the member variables of the BatchConfig class. JobBuilderFactory is a factory class used to create and configure jobs (Job), StepBuilderFactory is a factory class used to create and configure steps (Step), and DataSource is an interface used to obtain the database connection.

      Code:

      @Autowired
private JobBuilderFactory jobBuilderFactory;

@Autowired
private StepBuilderFactory stepBuilderFactory;

@Autowired
private DataSource dataSource;

    2. Define the @Bean annotation.

      Use the @Bean annotation to define several methods for creating the reader, processor, writer, step, and job components of the batch processing job.

      • Use the peopleReader method to create an ItemReader component instance. This component uses JdbcCursorItemReader to read People object data from the database. Set the data source dataSource, set the RowMapper to map database rows to People objects, and set the SQL query statement to SELECT * FROM people.

      • Use the addPeopleDescProcessor method to create an ItemProcessor component instance. This component uses AddPeopleDescProcessor to process People objects and returns the converted PeopleDESC objects.

      • Use the addDescPeopleWriter method to create an ItemWriter component instance. This component uses AddDescPeopleWriter to write PeopleDESC objects to the target location.

      • Use the step1 method to create a Step component instance. The step name is step1. Obtain the step builder through stepBuilderFactory.get, set the reader to the ItemReader component, set the processor to the ItemProcessor component, set the writer to the ItemWriter component, set the chunk size to 10, and finally call build to build and return the configured Step.

      • Use the importJob method to create a Job component instance. The job name is importJob. Obtain the job builder through jobBuilderFactory.get, set the incrementer to RunIdIncrementer, set the initial step of the job flow to Step, and finally call build to build and return the configured Job.

        Code:

        @Bean
public ItemReader<People> peopleReader() {
    JdbcCursorItemReader<People> reader = new JdbcCursorItemReader<>();
    reader.setDataSource((javax.sql.DataSource) dataSource);
    reader.setRowMapper(new BeanPropertyRowMapper<>(People.class));
    reader.setSql("SELECT * FROM people");
    return reader;
}

@Bean
public ItemProcessor<People, PeopleDESC> addPeopleDescProcessor() {
    return new AddPeopleDescProcessor();
}

@Bean
public ItemWriter<PeopleDESC> addDescPeopleWriter() {
    return new AddDescPeopleWriter();
}

@Bean
public Step step1(ItemReader<People> reader, ItemProcessor<People, PeopleDESC> processor,
                ItemWriter<PeopleDESC> writer) {
    return stepBuilderFactory.get("step1")
            .<People, PeopleDESC>chunk(10)
            .reader(reader)
            .processor(processor)
            .writer(writer)
            .build();
}

@Bean
public Job importJob(Step step1) {
    return jobBuilderFactory.get("importJob")
            .incrementer(new RunIdIncrementer())
            .flow(step1)
            .end()
            .build();
}

Introduction to the People.java file

The People.java file defines a People class that stores information about a person. The class contains two private member variables, name and age, and corresponding getter and setter methods. It also overrides the toString method to print the information of an object. In this class, name indicates the name of a person, and age indicates the age of a person. You can use the getter and setter methods to obtain and set the values of these attributes.

The People class is used to store and pass data in the input and output of a batch program. In the read and write operations of a batch program, you use the People object to store data, use the setter method to set data, and use the getter method to obtain data.

Sample code:

    public class People {
        private String name;
        private int age;

            // getters and setters

        public String getName() {
            return name;
        }

        public void setName(String name) {
            this.name = name;
        }

        public int getAge() {
            return age;
        }

        public void setAge(int age) {
            this.age = age;
        }
        @Override
        public String toString() {
            return "People [name=" + name + ", age=" + age + "]";
        }
        // Getters and setters
    }

Introduction to the PeopleDESC.java file

The PeopleDESC.java file defines a PeopleDESC class that stores information about a person. The PeopleDESC class has four attributes: name, age, desc, and id, which indicate the name, age, description, and ID of a person, respectively. The class contains corresponding getter and setter methods to access and set the values of these attributes. It also overrides the toString method to return the string representation of the class, which contains the name, age, and description.

Similar to the People class, the PeopleDESC class is used to store and pass data in the input and output of a batch program.

Sample code:

    public class PeopleDESC {
        private String name;
        private int age;
        private String desc;
        private int id;

        public String getName() {
            return name;
        }

        public void setName(String name) {
            this.name = name;
        }

        public int getAge() {
            return age;
        }

        public void setAge(int age) {
            this.age = age;
        }

        public String getDesc() {
            return desc;
        }

        public void setDesc(String desc) {
            this.desc = desc;
        }

        public int getId() {
            return id;
        }

        public void setId(int id) {
            this.id = id;
        }

        @Override
        public String toString() {
            return "PeopleDESC [name=" + name + ", age=" + age + ", desc=" + desc + "]";
        }
    }

Introduction to the AddPeopleDescProcessor.java file

The AddPeopleDescProcessor.java file defines a AddPeopleDescProcessor class that implements the ItemProcessor interface. This class is used to convert a People object to a PeopleDESC object.

The code in the AddPeopleDescProcessor.java file mainly includes the following parts:

  1. Import other classes and interfaces.

    The code declares that the current file contains the following interfaces and classes:

    • People class: stores the information of a person read from a database.
    • PeopleDESC class: stores the description of a person after the information is converted or processed.
    • ItemProcessor interface: processes or converts the read items.

    Sample code:

    import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import org.springframework.batch.item.ItemProcessor;

  2. Define the AddPeopleDescProcessor class.

    The AddPeopleDescProcessor class of the ItemProcessor interface is used to convert a People object to a PeopleDESC object, and implement the logic for processing input data in a batch program.

    In the process method of this class, first, a PeopleDESC object desc is created. Then, the name and age attributes of the People object are obtained from the item parameter, and the values of these attributes are set to the desc object. The desc attribute of the desc object is also set to a description generated based on the attributes of the People object. Finally, the processed PeopleDESC object is returned.

    Sample code:

    public class AddPeopleDescProcessor implements ItemProcessor<People, PeopleDESC> {
    @Override
    public PeopleDESC process(People item) throws Exception {
        PeopleDESC desc = new PeopleDESC();
        desc.setName(item.getName());
        desc.setAge(item.getAge());
        desc.setDesc("This is " + item.getName() + " with age " + item.getAge());
        return desc;
    }
}

Introduction to AddDescPeopleWriter.java

The AddDescPeopleWriter.java file implements the AddDescPeopleWriter class, which implements the ItemWriter interface. This class is used to write People objects to a database.

The code in the AddDescPeopleWriter.java file mainly includes the following parts:

  1. Import other classes and interfaces.

    Declare that the current file contains the following interfaces and classes:

    • PeopleDESC class: used to store description information after conversion or processing of personnel information.
    • ItemWriter interface: used to write processed or converted items to the specified target location.
    • @Autowired annotation: used for dependency injection.
    • JdbcTemplate class: provides methods for executing SQL statements.
    • List interface: used to operate on query result sets.

    Code:

    import com.oceanbase.example.batch.model.PeopleDESC;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;

import java.util.List;

  2. Define the AddDescPeopleWriter class.

    1. Use the @Autowired annotation to automatically inject the JdbcTemplate instance. This instance is used to perform database operations when writing data.

      Code:

          @Autowired
    private JdbcTemplate jdbcTemplate;

    2. In the write method, iterate through the input List<? extends PeopleDESC> and extract each PeopleDESC object. First, execute the SQL statement DROP TABLE people_desc to delete the people_desc table if it exists. Then, execute the SQL statement CREATE TABLE people_desc (id INT PRIMARY KEY, name VARCHAR2(255), age INT, description VARCHAR2(255)) to create a people_desc table with columns id, name, age, and description. Finally, use the SQL statement INSERT INTO people_desc (id, name, age, description) VALUES (?, ?, ?, ?) to insert the attribute values of each PeopleDESC object into the people_desc table.

      Code:

          @Override
    public void write(List<? extends PeopleDESC> items) throws Exception {
        // Drop the table if it exists
        jdbcTemplate.execute("DROP TABLE people_desc");
        // Create the table
        String createTableSql = "CREATE TABLE people_desc (id INT PRIMARY KEY, name VARCHAR2(255), age INT, description VARCHAR2(255))";
        jdbcTemplate.execute(createTableSql);
        for (PeopleDESC item : items) {
            String sql = "INSERT INTO people_desc (id, name, age, description) VALUES (?, ?, ?, ?)";
            jdbcTemplate.update(sql, item.getId(), item.getName(), item.getAge(), item.getDesc());
        }
    }

Introduction to the AddPeopleWriter.java file

The AddPeopleWriter.java file implements the AddDescPeopleWriter class of the ItemWriter interface, which is used to write PeopleDESC objects to a database.

The AddPeopleWriter.java file contains the following code:

  1. Import other classes and interfaces.

    Declare the following interfaces and classes in the current file:

    • People class: used to store personnel information read from a database.
    • ItemWriter interface: used to write processed or converted items to a specified target location.
    • Autowired annotation: used for dependency injection.
    • JdbcTemplate class: provides methods for executing SQL statements.
    • Component annotation: used to mark the class as a Spring component.
    • List interface: used to operate on query result sets.

    Code:

    import com.oceanbase.example.batch.model.People;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Component;

import java.util.List;

  2. Define the AddPeopleWriter class.

    1. Use the @Autowired annotation to automatically inject the JdbcTemplate instance, which is used to execute database operations when writing data.

      Code:

          @Autowired
    private JdbcTemplate jdbcTemplate;

    2. In the write method, traverse the input List<? extends People> and extract each People object. First, execute the SQL statement DROP TABLE people to delete an existing table named people. Then, execute the SQL statement CREATE TABLE people (name VARCHAR2(255), age INT) to create a table named people with two columns: name and age. Finally, use the SQL statement INSERT INTO people (name, age) VALUES (?, ?) to insert the attribute values of each People object into the people table.

      Code:

      @Override
public void write(List<? extends People> items) throws Exception {
    // Drop the table if it exists
    jdbcTemplate.execute("DROP TABLE people");
    // Create the table
    String createTableSql = "CREATE TABLE people (name VARCHAR2(255), age INT)";
    jdbcTemplate.execute(createTableSql);
    for (People item : items) {
        String sql = "INSERT INTO people (name, age) VALUES (?, ?)";
        jdbcTemplate.update(sql, item.getName(), item.getAge());
    }
}

Introduction to the BatchConfigTest.java file

The BatchConfigTest.java file is a class that uses JUnit for testing, used to test the job configuration of Spring Batch.

The BatchConfigTest.java file contains the following code:

  1. Import other classes and interfaces.

    Declare the following interfaces and classes in the current file:

    • Assert class: used to assert test results.
    • Test annotation: used to mark a method as a test method.
    • RunWith annotation: used to specify a test runner.
    • Job interface: represents a batch processing job.
    • JobExecution class: used to represent the execution of a batch processing job.
    • JobParameters class: used to represent the parameters of a batch processing job.
    • JobParametersBuilder class: used to build the parameters of a batch processing job.
    • JobLauncher interface: used to start a batch processing job.
    • Autowired annotation: used for dependency injection.
    • SpringBootTest annotation: used to specify the test class as a Spring Boot test.
    • SpringRunner class: used to specify the test runner as SpringRunner.

    Code:

    import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import javax.batch.runtime.BatchStatus;
import java.util.UUID;

  2. Define the BatchConfigTest class.

    Use the SpringBootTest annotation and SpringRunner runner to perform integration tests for Spring Boot. In the testJob method, use the JobLauncherTestUtils helper class to start a batch processing job and use assertions to verify the job's execution status.

    1. Use the @Autowired annotation to automatically inject the JobLauncherTestUtils instance.

      Code:

      @Autowired
private JobLauncherTestUtils jobLauncherTestUtils;

    2. Use the @Test annotation to mark the testJob method as a test method. In this method, first create a JobParameters object, then use the jobLauncherTestUtils.launchJob method to start the batch processing job, and use the Assert.assertEquals method to assert that the job's execution status is COMPLETED.

      Code:

      @Test
public void testJob() throws Exception {
    JobParameters jobParameters = new JobParametersBuilder()
            .addString("jobParam", "paramValue")
            .toJobParameters();

    JobExecution jobExecution = jobLauncherTestUtils.launchJob(jobParameters);

    Assert.assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus());
}

    3. Use the @Autowired annotation to automatically inject the JobLauncher instance.

      Code:

      @Autowired
private JobLauncher jobLauncher;

    4. Use the @Autowired annotation to automatically inject the Job instance.

      Code:

      @Autowired
private Job job;

    5. Define an internal class named JobLauncherTestUtils to assist in starting the batch processing job. In this class, define a launchJob method to start the batch processing job. In this method, use the jobLauncher.run method to start the job and return the job's execution result.

      Code:

      private class JobLauncherTestUtils {
    public JobExecution launchJob(JobParameters jobParameters) throws Exception {
        return jobLauncher.run(job, jobParameters);
    }
}

AddPeopleDescProcessorTest.java file

The AddPeopleDescProcessorTest.java file is a class that uses JUnit for testing Spring Batch job configurations.

The code in the AddPeopleDescProcessorTest.java file mainly includes the following parts:

  1. Import other classes and interfaces.

    Declare the interfaces and classes included in the current file:

    • People class: stores the personnel information read from the database.
    • PeopleDESC class: stores the description information after the personnel information is converted or processed.
    • Assert class: verifies whether the expected results and actual results in the test are consistent.
    • Test annotation: marks the test method.
    • RunWith annotation: specifies the test runner.
    • Autowired annotation: performs dependency injection.
    • SpringBootTest annotation: specifies the test class as a Spring Boot test.
    • SpringRunner class: specifies the test runner as SpringRunner.

    Code:

    import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

  2. Define the AddPeopleDescProcessorTest class.

    Use the SpringBootTest annotation and SpringRunner runner for Spring Boot integration testing.

    1. Use the @Autowired annotation to automatically inject the AddPeopleDescProcessor instance.

      Code:

      @Autowired
private AddPeopleDescProcessor processor;

    2. Use the @Test annotation to mark the testProcess method as a test method. In this method, first create a People object, then use the processor.process method to process the object, and assign the result to a PeopleDESC object.

      Code:

      @Test
public void testProcess() throws Exception {
People people = new People();
people.setName("John");
people.setAge(25);

PeopleDESC desc = processor.process(people);
}

AddDescPeopleWriterTest.java

AddDescPeopleWriterTest.java is a class that uses JUnit for testing the write logic of AddDescPeopleWriter.

The AddDescPeopleWriterTest.java file contains the following code:

  1. Reference other classes and interfaces.

    This file contains the following interfaces and classes:

    • The PeopleDESC class: used for storing the converted or processed description information for persons.
    • Assert class: Used to assert test results.
    • @Test annotation: marks a method as a test method.
    • RunWith annotation: specifies the test runner.
    • The @Autowired annotation: used for dependency injection.
    • The SpringBootTest annotation: specifies that the test class is for Spring Boot tests.
    • JdbcTemplate class: provides methods for executing SQL statements.
    • SpringRunner class: specifies the test runner as SpringRunner.
    • ArrayList class for creating an empty list.
    • List interface: used to operate on the query result set.

    The following code:

    import com.oceanbase.example.batch.model.PeopleDESC;
import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.ArrayList;
import java.util.List;

  2. Define the AddDescPeopleWriterTest class.

    You can perform integrated testing of Spring Boot applications using the SpringBootTest annotation and the SpringRunner runner.

    1. Inject instances by using the @Autowired annotation. Autowire the AddPeopleDescProcessor and JdbcTemplate instances by using the @Autowired annotation.

      Code example:

      @Autowired
private AddDescPeopleWriter writer;
@Autowired
private JdbcTemplate jdbcTemplate;

    2. Run the @Test method to insert and query the test data. The testWrite method is marked as the test method by using the @Test annotation. In this method, a peopleDescList list is created and two PeopleDESC objects are added to the list. Then, the data in the list is written to the database by using the writer.write method. Next, the jdbcTemplate method is called to execute a query statement, and data is retrieved from the people_desc table. Assertions are used to verify the data. Finally, the query result is output to the console, and a job execution completion message is printed.

      1. Insert data into the people_desc table. First, an empty list of PeopleDESC objects, peopleDescList, is created. Then, two PeopleDESC objects, desc1 and desc2, are created and their properties are set. The desc1 and desc2 objects are added to the peopleDescList list. The write method of writer is then called to write the objects in the peopleDescList list to the people_desc table in the database. The JdbcTemplate is used to execute a query statement SELECT COUNT(*) FROM people_desc to obtain the number of records in the people_desc table and assign the result to the variable count. Finally, the Assert.assertEquals method is used to perform an assertion to verify whether the value of count is equal to 2.

        The code is as follows:

           List<PeopleDESC> peopleDescList = new ArrayList<>();
   PeopleDESC desc1 = new PeopleDESC();
   desc1.setId(1);
   desc1.setName("John");
   desc1.setAge(25);
   desc1.setDesc("This is John with age 25");
   peopleDescList.add(desc1);
   PeopleDESC desc2 = new PeopleDESC();
   desc2.setId(2);
   desc2.setName("Alice");
   desc2.setAge(30);
   desc2.setDesc("This is Alice with age 30");
   peopleDescList.add(desc2);
   writer.write(peopleDescList);

   String selectSql = "SELECT COUNT(*) FROM people_desc";
   int count = jdbcTemplate.queryForObject(selectSql, Integer.class);
   Assert.assertEquals(2, count);

      2. Output the data of the people_desc table. First, the JdbcTemplate executes the statement SELECT * FROM people_desc and uses a lambda expression to handle the query result. In the lambda expression, the methods such as rs.getInt and rs.getString are used to obtain the field values from the result set and set them to a new PeopleDESC object. The new PeopleDESC object is added to a result list resultDesc. Then, a prompt message people_desc table data: is printed, followed by a for loop to traverse each PeopleDESC object in the resultDesc list. The System.out.println method is used to print each object. Finally, an execution completion message is printed.

        The code is as follows:

        List<PeopleDESC> resultDesc = jdbcTemplate.query("SELECT * FROM people_desc", (rs, rowNum) -> {
   PeopleDESC desc = new PeopleDESC();
   desc.setId(rs.getInt("id"));
   desc.setName(rs.getString("name"));
   desc.setAge(rs.getInt("age"));
   desc.setDesc(rs.getString("description"));
   return desc;
});

System.out.println("people_desc table data:");
for (PeopleDESC desc : resultDesc) {
   System.out.println(desc);
}

// Output information after the job finishes.
System.out.println("Batch Job execution completed.");

AddPeopleWriterTest.java File Introduction

The AddPeopleWriterTest.java file is a class that uses JUnit to test the writing logic of the AddPeopleWriterTest class.

The code in the AddPeopleWriterTest.java file is as follows:

  1. References other classes and interfaces.

    This file contains the following interfaces and classes:

    • The People class: Used to store personnel information read from the database.
    • Test annotation: used to mark test methods.
    • RunWith annotation: specifies the test runner.
    • Autowired: A dependency injection annotation.
    • @SpringBootApplication annotation: indicates the class as the entry point of a Spring Boot application.
    • SpringBootTest annotation: used to specify that the test class is a Spring Boot test.
    • ComponentScan Annotation: specifies the package or class for which the component scan is performed.
    • JdbcTemplate class: provides methods to execute SQL statements.
    • SpringRunner class: specifies the test runner as SpringRunner.
    • ArrayList class, which is used to create an empty list.
    • The List interface: used for operating on the query result set.

    Here is the code:

    import com.oceanbase.example.batch.model.People;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.ArrayList;
import java.util.List;

  2. Define the AddPeopleWriterTest class.

    Use the SpringBootTest annotation and the SpringRunner runner to perform integration testing in Spring Boot. Use the @ComponentScan annotation to specify the package paths to be scanned.

    1. Inject instances by using @Autowired. Use the @Autowired annotation to inject addPeopleWriter and JdbcTemplate instances.

      Sample code:

      @Autowired
private AddPeopleWriter addPeopleWriter;
@Autowired
private JdbcTemplate jdbcTemplate;

    2. Insert and output the test data.

      1. Insert data into the people table. First, create an empty peopleList list of People objects. Then, create two People objects named person1 and person2 and set their Name and Age attributes. Next, add these People objects to the peopleList list. Then, call the write method of addPeopleWriter and pass peopleList as the parameter to write these objects to the database.

        Here is the code:

           List<People> peopleList = new ArrayList<>();
   People person1 = new People();
   person1.setName("zhangsan");
   person1.setAge(27);
   peopleList.add(person1);
   People person2 = new People();
   person2.setName("lisi");
   person2.setAge(35);
   peopleList.add(person2);
   addPeopleWriter.write(peopleList);

      2. Output data from the people table. Then, a query statement SELECT * FROM people is executed by using the JdbcTemplate object, and the query result is processed by using a lambda expression. In the lambda expression, the rs.getString and rs.getInt methods are used to obtain field values from the result set. Then, the field values are set to a newly created People object. Each newly created People object is added to a result list result. Then, a prompt message people table data: is printed. Then, each People object in the result list is traversed by using a for loop, and the System.out.println method is used to print the contents of each object. Finally, a message indicating that the operation is completed is printed.

        Code example:

           List<People> result = jdbcTemplate.query("SELECT * FROM people", (rs, rowNum) -> {
       People person = new People();
       person.setName(rs.getString("name"));
       person.setAge(rs.getInt("age"));
       return person;
   });

   System.out.println("people table data:");
   for (People person : result) {
       System.out.println(person);
   }

   // Output information after the job is completed.
   System.out.println("Batch Job execution completed.");

Full code

pom.xml
application.properties
BatchApplication.java
BatchConfig.java
People.java
PeopleDESC.java
AddPeopleDescProcessor.java
AddDescPeopleWriter.java
AddPeopleWriter.java
BatchConfigTest.java
AddPeopleDescProcessorTest.java
AddDescPeopleWriterTest.java
AddPeopleWriterTest.java 
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.11</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.oceanbase</groupId>
    <artifactId>java-oceanbase-springboot</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>java-oceanbase-springbatch</name>
    <description>Demo project for Spring Batch</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>com.oceanbase</groupId>
            <artifactId>oceanbase-client</artifactId>
            <version>2.4.3</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-batch</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.tomcat</groupId>
            <artifactId>tomcat-jdbc</artifactId>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>javax.activation</groupId>
            <artifactId>javax.activation-api</artifactId>
            <version>1.2.0</version>
        </dependency>
        <dependency>
            <groupId>jakarta.persistence</groupId>
            <artifactId>jakarta.persistence-api</artifactId>
            <version>2.2.3</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>


#configuration database

spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:oceanbase://host:port/schema_name?characterEncoding=utf-8
spring.datasource.username=user_name
spring.datasource.password=

# JPA
spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=update

# Spring Batch
spring.batch.job.enabled=false

#
logging.level.org.springframework=INFO
logging.level.com.example=DEBUG

package com.oceanbase.example.batch;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class BatchApplication {
    public static void main(String[] args) {
        SpringApplication.run(BatchApplication.class, args);
    }

    public void runBatchJob() {
    }
}


package com.oceanbase.example.batch.config;

import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import com.oceanbase.example.batch.processor.AddPeopleDescProcessor;
import com.oceanbase.example.batch.writer.AddDescPeopleWriter;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.database.JdbcCursorItemReader;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.BeanPropertyRowMapper;

import javax.sql.DataSource;
//import javax.activation.DataSource;

@Configuration
@EnableBatchProcessing
@SpringBootApplication
@ComponentScan("com.oceanbase.example.batch.writer")
@EnableAutoConfiguration
public class BatchConfig {
    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    @Autowired
    private DataSource dataSource;// Use the default dataSource provided by Spring Boot auto-configuration

    @Bean
    public ItemReader<People> peopleReader() {
        JdbcCursorItemReader<People> reader = new JdbcCursorItemReader<>();
        reader.setDataSource((javax.sql.DataSource) dataSource);
        reader.setRowMapper(new BeanPropertyRowMapper<>(People.class));
        reader.setSql("SELECT * FROM people");
        return reader;
    }

    @Bean
    public ItemProcessor<People, PeopleDESC> addPeopleDescProcessor() {
        return new AddPeopleDescProcessor();
    }

    @Bean
    public ItemWriter<PeopleDESC> addDescPeopleWriter() {
        return new AddDescPeopleWriter();
    }

    @Bean
    public Step step1(ItemReader<People> reader, ItemProcessor<People, PeopleDESC> processor,
                      ItemWriter<PeopleDESC> writer) {
        return stepBuilderFactory.get("step1")
                .<People, PeopleDESC>chunk(10)
                .reader(reader)
                .processor(processor)
                .writer(writer)
                .build();
    }

    @Bean
    public Job importJob(Step step1) {
        return jobBuilderFactory.get("importJob")
                .incrementer(new RunIdIncrementer())
                .flow(step1)
                .end()
                .build();
    }
}

package com.oceanbase.example.batch.model;

public class People {
    private String name;
    private int age;

        // getters and setters

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }
    @Override
    public String toString() {
        return "People [name=" + name + ", age=" + age + "]";
    }
    // Getters and setters
}

package com.oceanbase.example.batch.model;

public class PeopleDESC {
    private String name;
    private int age;
    private String desc;
    private int id;

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }

    public String getDesc() {
        return desc;
    }

    public void setDesc(String desc) {
        this.desc = desc;
    }

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    @Override
    public String toString() {
        return "PeopleDESC [name=" + name + ", age=" + age + ", desc=" + desc + "]";
    }
}

package com.oceanbase.example.batch.processor;

import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import org.springframework.batch.item.ItemProcessor;


public class AddPeopleDescProcessor implements ItemProcessor<People, PeopleDESC> {
    @Override
    public PeopleDESC process(People item) throws Exception {
        PeopleDESC desc = new PeopleDESC();
        desc.setName(item.getName());
        desc.setAge(item.getAge());
        desc.setDesc("This is " + item.getName() + " with age " + item.getAge());
        return desc;
    }
}

package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.PeopleDESC;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;

import java.util.List;

public class AddDescPeopleWriter implements ItemWriter<PeopleDESC> {
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Override
    public void write(List<? extends PeopleDESC> items) throws Exception {
        // Drop the table if it already exists.
        jdbcTemplate.execute("DROP TABLE people_desc");
        // Table creation statement
        String createTableSql = "CREATE TABLE people_desc (id INT PRIMARY KEY, name VARCHAR2(255), age INT, description VARCHAR2(255))";
        jdbcTemplate.execute(createTableSql);
        for (PeopleDESC item : items) {
            String sql = "INSERT INTO people_desc (id, name, age, description) VALUES (?, ?, ?, ?)";
            jdbcTemplate.update(sql, item.getId(), item.getName(), item.getAge(), item.getDesc());
        }
    }
}

package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.People;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Component;

import java.util.List;

@Component
public class AddPeopleWriter implements ItemWriter<People> {
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Override
    public void write(List<? extends People> items) throws Exception {
        // First, delete existing tables.
        jdbcTemplate.execute("DROP TABLE people");
        // The CREATE TABLE statement.
        String createTableSql = "CREATE TABLE people (name VARCHAR2(255), age INT)";
        jdbcTemplate.execute(createTableSql);
        for (People item : items) {
            String sql = "INSERT INTO people (name, age) VALUES (?, ?)";
            jdbcTemplate.update(sql, item.getName(), item.getAge());
        }
    }
}


package com.oceanbase.example.batch.config;

import com.oceanbase.example.batch.writer.AddDescPeopleWriter;
import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

import org.springframework.test.context.junit4.SpringRunner;

import javax.annotation.Resource;
import javax.batch.runtime.BatchStatus;
import java.util.UUID;

@RunWith(SpringRunner.class)
@SpringBootTest
public class BatchConfigTest {

    @Test
    public void testJob() throws Exception {
        JobParameters jobParameters = new JobParametersBuilder()
                .addString("jobParam", UUID.randomUUID().toString())
                .toJobParameters();

        JobLauncherTestUtils jobLauncherTestUtils = new JobLauncherTestUtils();
        JobExecution jobExecution = jobLauncherTestUtils.launchJob(jobParameters);
        Assert.assertEquals(BatchStatus.COMPLETED.toString(), jobExecution.getStatus().toString());
    }

    @Autowired
    private JobLauncher jobLauncher;

    @Autowired
    private Job job;

    private class JobLauncherTestUtils {

        public JobExecution launchJob(JobParameters jobParameters) throws Exception {
            return jobLauncher.run(job, jobParameters);
        }
    }
}


package com.oceanbase.example.batch.processor;

import com.oceanbase.example.batch.model.People;
import com.oceanbase.example.batch.model.PeopleDESC;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest
public class AddPeopleDescProcessorTest {
    @Autowired
    private AddPeopleDescProcessor processor;

    @Test
    public void testProcess() throws Exception {
        People people = new People();
  //      people.setName("John");
  //      people.setAge(25);

        PeopleDESC desc = processor.process(people);

//      Assert.assertEquals("John", desc.getName());
//        Assert.assertEquals(25, desc.getAge());
 //       Assert.assertEquals("This is John with age 25", desc.getDesc());
    }
}


package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.PeopleDESC;
import org.junit.Assert;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.ArrayList;
import java.util.List;

@RunWith(SpringRunner.class)
@SpringBootTest
public class AddDescPeopleWriterTest {
    @Autowired
    private AddDescPeopleWriter writer;
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Test
    public void testWrite() throws Exception {

        // Insert data into the people_desc table.
        List<PeopleDESC> peopleDescList = new ArrayList<>();
        PeopleDESC desc1 = new PeopleDESC();
        desc1.setId(1);
        desc1.setName("John");
        desc1.setAge(25);
        desc1.setDesc("This is John with age 25");
        peopleDescList.add(desc1);
        PeopleDESC desc2 = new PeopleDESC();
        desc2.setId(2);
        desc2.setName("Alice");
        desc2.setAge(30);
        desc2.setDesc("This is Alice with age 30");
        peopleDescList.add(desc2);
        writer.write(peopleDescList);

        String selectSql = "SELECT COUNT(*) FROM people_desc";
        int count = jdbcTemplate.queryForObject(selectSql, Integer.class);
        Assert.assertEquals(2, count);

        // Output the data in the people_desc table.
        List<PeopleDESC> resultDesc = jdbcTemplate.query("SELECT * FROM people_desc", (rs, rowNum) -> {
            PeopleDESC desc = new PeopleDESC();
            desc.setId(rs.getInt("id"));
            desc.setName(rs.getString("name"));
            desc.setAge(rs.getInt("age"));
            desc.setDesc(rs.getString("description"));
            return desc;
        });

        System.out.println("people_desc table data:");
        for (PeopleDESC desc : resultDesc) {
            System.out.println(desc);
        }

        // Output the information after the job is completed.
        System.out.println("Batch Job execution completed.");
    }
}

package com.oceanbase.example.batch.writer;

import com.oceanbase.example.batch.model.People;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.ArrayList;
import java.util.List;

@RunWith(SpringRunner.class)
@SpringBootTest
@SpringBootApplication
@ComponentScan("com.oceanbase.example.batch.writer")
public class AddPeopleWriterTest {

    @Autowired
    private AddPeopleWriter addPeopleWriter;
    @Autowired
    private JdbcTemplate jdbcTemplate;

    @Test
    public void testWrite() throws Exception {
        // Insert data into the people table.
        List<People> peopleList = new ArrayList<>();
        People person1 = new People();
        person1.setName("zhangsan");
        person1.setAge(27);
        peopleList.add(person1);
        People person2 = new People();
        person2.setName("lisi");
        person2.setAge(35);
        peopleList.add(person2);
        addPeopleWriter.write(peopleList);

        // Query and output the result.
        List<People> result = jdbcTemplate.query("SELECT * FROM people", (rs, rowNum) -> {
            People person = new People();
            person.setName(rs.getString("name"));
            person.setAge(rs.getInt("age"));
            return person;
        });

        System.out.println("people table data:");
        for (People person : result) {
            System.out.println(person);
        }

        // Output the information after the job is completed.
        System.out.println("Batch Job execution completed.");
    }
}

References

For more information about OceanBase Connector/J, see OceanBase JDBC driver.

Last
Next

OceanBaseOceanBase

OceanBase, A Highly Scalable Database for Transactional, Analytical, and AI Workloads.

Company

About OceanBaseContact UsPartnerTrust Center

Product

OceanBase CloudOceanBase EnterpriseQuick StartDownloadPricing

Resources

DocsBlogWhite Paper

Follow us

LinkedInLinkedInDiscordDiscordTwitterTwitterYoutubeYoutubeForumForumGitHubGitHubStack OverflowStack Overflow

© OceanBase 2024. All rights reserved | Cloud Service Agreement | Privacy Policy | Legal | Security

Contact Us