iSharkFly-Docs/java-tutorials
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

提交更新的 Readme 文件

Browse Source
This commit is contained in:
YuCheng Hu 2020-02-24 23:09:39 -05:00
parent 13ddc8e390
commit 034b72ea25
2 changed files with 54 additions and 183 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

183
README.adoc
View File

@ -1,183 +0,0 @@
:spring_version: current
:spring_boot_version: 2.1.4.RELEASE
:Component: http://docs.spring.io/spring/docs/{spring_version}/javadoc-api/org/springframework/stereotype/Component.html
:SpringApplication: http://docs.spring.io/spring-boot/docs/{spring_boot_version}/api/org/springframework/boot/SpringApplication.html
:toc:
:icons: font
:source-highlighter: prettify
:project_id: gs-batch-processing
This guide walks you through the process of creating a basic batch-driven solution.
== What you'll build
You'll build a service that imports data from a CSV spreadsheet, transforms it with custom code, and stores the final results in a database.
== What you'll need
:java_version: 1.8
include::https://raw.githubusercontent.com/spring-guides/getting-started-macros/master/prereq_editor_jdk_buildtools.adoc[]
include::https://raw.githubusercontent.com/spring-guides/getting-started-macros/master/how_to_complete_this_guide.adoc[]
include::https://raw.githubusercontent.com/spring-guides/getting-started-macros/master/hide-show-gradle.adoc[]
include::https://raw.githubusercontent.com/spring-guides/getting-started-macros/master/hide-show-maven.adoc[]
include::https://raw.githubusercontent.com/spring-guides/getting-started-macros/master/hide-show-sts.adoc[]
== Business Data
Typically your customer or a business analyst supplies a spreadsheet. In this case, you make it up.
`src/main/resources/sample-data.csv`
[source,csv]
----
include::initial/src/main/resources/sample-data.csv[]
----
This spreadsheet contains a first name and a last name on each row, separated by a comma. This is a fairly common pattern that Spring handles out-of-the-box, as you will see.
Next, you write a SQL script to create a table to store the data.
`src/main/resources/schema-all.sql`
[source,sql]
----
include::initial/src/main/resources/schema-all.sql[]
----
NOTE: Spring Boot runs `schema-@@platform@@.sql` automatically during startup. `-all` is the default for all platforms.
[[initial]]
== Create a business class
Now that you see the format of data inputs and outputs, you write code to represent a row of data.
`src/main/java/hello/Person.java`
[source,java]
----
include::complete/src/main/java/hello/Person.java[]
----
You can instantiate the `Person` class either with first and last name through a constructor, or by setting the properties.
== Create an intermediate processor
A common paradigm in batch processing is to ingest data, transform it, and then pipe it out somewhere else. Here you write a simple transformer that converts the names to uppercase.
`src/main/java/hello/PersonItemProcessor.java`
[source,java]
----
include::complete/src/main/java/hello/PersonItemProcessor.java[]
----
`PersonItemProcessor` implements Spring Batch's `ItemProcessor` interface. This makes it easy to wire the code into a batch job that you define further down in this guide. According to the interface, you receive an incoming `Person` object, after which you transform it to an upper-cased `Person`.
NOTE: There is no requirement that the input and output types be the same. In fact, after one source of data is read, sometimes the application's data flow needs a different data type.
== Put together a batch job
Now you put together the actual batch job. Spring Batch provides many utility classes that reduce the need to write custom code. Instead, you can focus on the business logic.
`src/main/java/hello/BatchConfiguration.java`
[source,java]
----
include::complete/src/main/java/hello/BatchConfiguration.java[]
----
For starters, the `@EnableBatchProcessing` annotation adds many critical beans that support jobs and saves you a lot of leg work. This example uses a memory-based database (provided by `@EnableBatchProcessing`), meaning that when it's done, the data is gone.
Break it down:
`src/main/java/hello/BatchConfiguration.java`
[source,java]
----
include::/complete/src/main/java/hello/BatchConfiguration.java[tag=readerwriterprocessor]
----
The first chunk of code defines the input, processor, and output.
* `reader()` creates an `ItemReader`. It looks for a file called `sample-data.csv` and parses each line item with enough information to turn it into a `Person`.
* `processor()` creates an instance of our `PersonItemProcessor` you defined earlier, meant to uppercase the data.
* `write(DataSource)` creates an `ItemWriter`. This one is aimed at a JDBC destination and automatically gets a copy of the dataSource created by `@EnableBatchProcessing`. It includes the SQL statement needed to insert a single `Person` driven by Java bean properties.
The next chunk focuses on the actual job configuration.
`src/main/java/hello/BatchConfiguration.java`
[source,java]
----
include::/complete/src/main/java/hello/BatchConfiguration.java[tag=jobstep]
----
The first method defines the job and the second one defines a single step. Jobs are built from steps, where each step can involve a reader, a processor, and a writer.
In this job definition, you need an incrementer because jobs use a database to maintain execution state. You then list each step, of which this job has only one step. The job ends, and the Java API produces a perfectly configured job.
In the step definition, you define how much data to write at a time. In this case, it writes up to ten records at a time. Next, you configure the reader, processor, and writer using the injected bits from earlier.
NOTE: chunk() is prefixed `<Person,Person>` because it's a generic method. This represents the input and output types of each "chunk" of processing, and lines up with `ItemReader<Person>` and `ItemWriter<Person>`.
`src/main/java/hello/JobCompletionNotificationListener.java`
[source,java]
----
include::/complete/src/main/java/hello/JobCompletionNotificationListener.java[]
----
This code listens for when a job is `BatchStatus.COMPLETED`, and then uses `JdbcTemplate` to inspect the results.
== Make the application executable
Although batch processing can be embedded in web apps and WAR files, the simpler approach demonstrated below creates a standalone application. You package everything in a single, executable JAR file, driven by a good old Java `main()` method.
`src/main/java/hello/Application.java`
[source,java]
----
include::complete/src/main/java/hello/Application.java[]
----
include::https://raw.githubusercontent.com/spring-guides/getting-started-macros/master/spring-boot-application.adoc[]
For demonstration purposes, there is code to create a `JdbcTemplate`, query the database, and print out the names of people the batch job inserts.
include::https://raw.githubusercontent.com/spring-guides/getting-started-macros/master/build_an_executable_jar_subhead.adoc[]
include::https://raw.githubusercontent.com/spring-guides/getting-started-macros/master/build_an_executable_jar_with_both.adoc[]
The job prints out a line for each person that gets transformed. After the job runs, you can also see the output from querying the database.
....
Converting (firstName: Jill, lastName: Doe) into (firstName: JILL, lastName: DOE)
Converting (firstName: Joe, lastName: Doe) into (firstName: JOE, lastName: DOE)
Converting (firstName: Justin, lastName: Doe) into (firstName: JUSTIN, lastName: DOE)
Converting (firstName: Jane, lastName: Doe) into (firstName: JANE, lastName: DOE)
Converting (firstName: John, lastName: Doe) into (firstName: JOHN, lastName: DOE)
Found <firstName: JILL, lastName: DOE> in the database.
Found <firstName: JOE, lastName: DOE> in the database.
Found <firstName: JUSTIN, lastName: DOE> in the database.
Found <firstName: JANE, lastName: DOE> in the database.
Found <firstName: JOHN, lastName: DOE> in the database.
....
== Summary
Congratulations! You built a batch job that ingested data from a spreadsheet, processed it, and wrote it to a database.
== See also
The following guides may also be helpful:
* https://spring.io/guides/gs/spring-boot/[Building an Application with Spring Boot]
* https://spring.io/guides/gs/accessing-data-gemfire/[Accessing Data with GemFire]
* https://spring.io/guides/gs/accessing-data-jpa/[Accessing Data with JPA]
* https://spring.io/guides/gs/accessing-data-mongodb/[Accessing Data with MongoDB]
* https://spring.io/guides/gs/accessing-data-mysql/[Accessing data with MySQL]
include::https://raw.githubusercontent.com/spring-guides/getting-started-macros/master/footer.adoc[]

54
README.md Normal file
View File

@ -0,0 +1,54 @@
**UPDATE**: The price of "Learn Spring Security OAuth" will permanently change on the 11th of December, along with the upcoming OAuth2 material: http://bit.ly/github-lss
The Courses
==============================
Here's the new "Learn Spring" course: <br/>
**[>> LEARN SPRING - THE MASTER CLASS](https://www.baeldung.com/learn-spring-course?utm_source=github&utm_medium=social&utm_content=tutorials&utm_campaign=ls#master-class)**
Here's the Master Class of "REST With Spring" (along with the new announced Boot 2 material): <br/>
**[>> THE REST WITH SPRING - MASTER CLASS](https://www.baeldung.com/rest-with-spring-course?utm_source=github&utm_medium=social&utm_content=tutorials&utm_campaign=rws#master-class)**
And here's the Master Class of "Learn Spring Security": <br/>
**[>> LEARN SPRING SECURITY - MASTER CLASS](https://www.baeldung.com/learn-spring-security-course?utm_source=github&utm_medium=social&utm_content=tutorials&utm_campaign=lss#master-class)**
Java and Spring Tutorials
================
This project is **a collection of small and focused tutorials** - each covering a single and well defined area of development in the Java ecosystem.
A strong focus of these is, of course, the Spring Framework - Spring, Spring Boot and Spring Security.
In additional to Spring, the modules here are covering a number of aspects in Java.
Building the project
====================
To do the full build, do: `mvn clean install`
Building a single module
====================
To build a specific module run the command: `mvn clean install` in the module directory
Running a Spring Boot module
====================
To run a Spring Boot module run the command: `mvn spring-boot:run` in the module directory
Working with the IDE
====================
This repo contains a large number of modules.
When you're working with an individual module, there's no need to import all of them (or build all of them) - you can simply import that particular module in either Eclipse or IntelliJ.
Running Tests
=============
The command `mvn clean install` will run the unit tests in a module.
To run the integration tests, use the command `mvn clean install -Pintegration-lite-first`