Passing Data to Future Steps in Spring Batch

It is often useful to pass information from one step to another. This can be done through the ExecutionContext. The catch is that there are two ExecutionContexts: one at the Step level and one at the Job level. The Step ExecutionContext remains only as long as the step, while the Job ExecutionContext remains through the whole Job. On the other hand, the Step ExecutionContext is updated every time the Step commits a chunk, while the Job ExecutionContext is updated only at the end of each Step.

The consequence of this separation is that all data must be placed in the Step ExecutionContext while the Step is executing. Doing so ensures that the data is stored properly while the Step runs. If data is stored to the Job ExecutionContext, then it is not persisted during Step execution. If the Step fails, that data is lost.

public class SavingItemWriter implements ItemWriter<Object> {
    private StepExecution stepExecution;

    public void write(List<? extends Object> items) throws Exception {
        // ...

        ExecutionContext stepContext = this.stepExecution.getExecutionContext();
        stepContext.put("someKey", someObject);
    }

    @BeforeStep
    public void saveStepExecution(StepExecution stepExecution) {
        this.stepExecution = stepExecution;
    }
}

To make the data available to future Steps, it must be “promoted” to the Job ExecutionContext after the step has finished. Spring Batch provides the ExecutionContextPromotionListener for this purpose. The listener must be configured with the keys related to the data in the ExecutionContext that must be promoted. It can also, optionally, be configured with a list of exit code patterns for which the promotion should occur (COMPLETED is the default). As with all listeners, it must be registered on the Step as shown in the following example:

Java Configuration

@Bean
public Job job1() {
	return this.jobBuilderFactory.get("job1")
				.start(step1())
				.next(step1())
				.build();
}

@Bean
public Step step1() {
	return this.stepBuilderFactory.get("step1")
				.<String, String>chunk(10)
				.reader(reader())
				.writer(savingWriter())
				.listener(promotionListener())
				.build();
}

@Bean
public ExecutionContextPromotionListener promotionListener() {
	ExecutionContextPromotionListener listener = new ExecutionContextPromotionListener();

	listener.setKeys(new String[] {"someKey" });

	return listener;
}

Finally, the saved values must be retrieved from the Job ExecutionContext, as shown in the following example:

public class RetrievingItemWriter implements ItemWriter<Object> {
    private Object someObject;

    public void write(List<? extends Object> items) throws Exception {
        // ...
    }

    @BeforeStep
    public void retrieveInterstepData(StepExecution stepExecution) {
        JobExecution jobExecution = stepExecution.getJobExecution();
        ExecutionContext jobContext = jobExecution.getExecutionContext();
        this.someObject = jobContext.get("someKey");
    }
}

In similar way, @AfterStep annotated method can be used to read/write the ExecutionContext and can access in future Step.

@AfterStep     
public ExitStatus afterStep(StepExecution execution) {         
         this.stepExecution = stepExecution;
         return execution.getExitStatus();
 }

I hope you have enjoyed this post and it helped you to Passing Data to Future Steps in Spring Batch. Please like and share and feel free to comment if you have any suggestions or feedback.

Tagged : / /

Configure Logging to File in Spring Boot in just one line

Introduction

Logging in spring boot is very flexible and easy to configure. Spring Boot’s default configurations provides a support for the use of Java Util Logging, Log4j2, and Logback. In each case, loggers are pre-configured to use console output with optional file output also available.

A new, freshly-generated Spring Boot application doesn’t come with any explicit logging configuration, and it will use Logback by default, the logging framework for Java, and the successor to the old log4j.
Logging is configured automatically when you add the spring-boot-starter-web dependency to your project.
Spring boot’s internal logging is written with Apache Commons Logging so it is one and only mandatory dependency. Till, boot 1.x – we had to import it manually. Since boot 2.x, it is downloaded transitively. To be more precise, spring-boot-starter-web depends on spring-boot-starter-logging, which pulls in spring-jcl for us.

File Output

By default, Spring Boot logs only to the console and does not write log files. To make Spring Boot write to a log file, you can set the logging.path property in your application.properties file:

logging.path = ./                      #  spring.log in current folder
logging.path = relativepath/logs       #  relativepath/logs/spring.log
logging.path = /fullpath/logs          #  /fullpath/logs/spring.log

With this configuration, Spring Boot will write to the console and also to a log file called spring.log, at the path you specify.
If you want to choose your own log filename instead of spring.log, then you can set the property logging.file, in your application.properties, e.g.:

logging.file = logfile.log                  # in current folder
logging.file = relativepath/logfile.log     # relative path with filename
logging.file = /fullpath/logfile.log        # full path with filename

Log files rotate when they reach 10 MB and, as with console output, ERROR-level, WARN-level, and INFO-level messages are logged by default. Size limits can be changed using the logging.file.max-size property. Previously rotated files are archived indefinitely unless the logging.file.max-history property has been set.

Logback does not have a FATAL level. It is mapped to ERROR.

We can configure separate property files for each environment like application-dev.properties and application-prod.properties. Follow the naming convention of application-{environment}.properties where {environment} is replaced with the environment name. Depending on your VM options or environment variables, one of these can be chosen.

Spring Cloud Config doesn’t support default Logback properties.

Log Format

The default log output from Spring Boot resembles the following example:

2020-03-08 11:57:51.102  INFO 55469 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet Engine: Apache Tomcat/7.0.52
2020-03-08 11:57:51.203  INFO 55469 --- [ost-startStop-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2019-03-08 11:57:51.203  INFO 55469 --- [ost-startStop-1] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 1358 ms
2020-03-08 11:57:51.608  INFO 55469 --- [ost-startStop-1] o.s.b.c.e.ServletRegistrationBean        : Mapping servlet: 'dispatcherServlet' to [/]

The following items are output:

  • Date and Time: Millisecond precision and easily sort-able.
  • Log Level: ERRORWARNINFODEBUG, or TRACE.
  • Process ID.
  • --- separator to distinguish the start of actual log messages.
  • Thread name: Enclosed in square brackets (may be truncated for console output).
  • Logger name: This is usually the source class name (often abbreviated).
  • The log message.

Customization

To help with the customization, some other properties are transferred from the Spring Environment to System properties, as described in the following table:

Spring EnvironmentSystem PropertyComments
logging.exception-conversion-wordLOG_EXCEPTION_CONVERSION_WORDThe conversion word used when logging exceptions.
logging.fileLOG_FILEIf defined, it is used in the default log configuration.
logging.file.max-sizeLOG_FILE_MAX_SIZEMaximum log file size (if LOG_FILE enabled). (Only supported with the default Logback setup.)
logging.file.max-historyLOG_FILE_MAX_HISTORYMaximum number of archive log files to keep (if LOG_FILE enabled). (Only supported with the default Logback setup.)
logging.pathLOG_PATHIf defined, it is used in the default log configuration.
logging.pattern.consoleCONSOLE_LOG_PATTERNThe log pattern to use on the console (stdout). (Only supported with the default Logback setup.)
logging.pattern.dateformatLOG_DATEFORMAT_PATTERNAppender pattern for log date format. (Only supported with the default Logback setup.)
logging.pattern.fileFILE_LOG_PATTERNThe log pattern to use in a file (if LOG_FILE is enabled). (Only supported with the default Logback setup.)
logging.pattern.levelLOG_LEVEL_PATTERNThe format to use when rendering the log level (default %5p). (Only supported with the default Logback setup.)

You can customize the default logging configuration by:

  • Setting some specific Spring Boot properties (these can go in your application.propertiesapplication.yml or as environment variables)
  • Adding a logback.xml onto the classpath, which Spring Boot will detect and use to configure Logback
  • Adding a logback-spring.xml onto the classpath, which Spring Boot will detect and use to configure Logback.
Tagged : / / / / /
%d bloggers like this: