Rename Files

Example is inspired by a question in Spring Batch Forum Dynamically build FlatFileItemWriter file name at runtime

Usecase

• create output file names at runtime in relation to business data from the item reader, e.g. from header lines in input files

Configuration Details

For general Maven, Spring and Database setup see the project setup wiki page.

In the rename-files-tasklet example are actually many different examples:

1. simple example - works with one file only, rather clunky getting work done configuration
2. Partition Step example - works with multiple files, reads header lines to create the output file name
3. Extra Step example - combines and enhances the simple and the partition step example

Pitfalls

Moment of File Resource Creation(read: stream.open)

In Spring Batch file resources (even for writers) are created before the step reads or processes an item. Thus it is not possible to create the file resource for a writer at runtime.

For the implementation of the usecase a simple trick is used - just rename the files after the step.

When is the writer.close() method called ?

In my example projects i noticed that the .close() method of itemwriter is called after Listener.afterStep methods. So it is not good e.g. to try and delete a used file in an afterStep method, works in OSX, but not Windows. I solved this with a call to file.deleteOnExit().

Requests that the file or directory denoted by this abstract pathname be deleted when the virtual machine >terminates. Files (or directories) are deleted in the reverse order that they are registered. Invoking this >method to delete a file or directory that is already registered for deletion has no effect. Deletion will >be attempted only for normal termination of the virtual machine, as defined by the Java Language >Specification.

Examples

Simple Example

Should be taken as proof of and a very clunky concept - read: anti-pattern.

• item reader puts desired file output name in step execution context, uses business data to create it (once)
• ExecutionContextPromotionListener promotes the desired name to job execution context
• item writer works with some dummy output file
• taskletstep pulls both the desired and the dummy name from job execution context and job parameters to fire the rename action

Partition Step Example

• reads 3 file at once with partition step setup
  • each file contains a header line
  • each header line contains a specific business key, which is pushed to step execution context
• custom multi resource partitioner
  • creates distinct output file names for the writer, i created a little example for this too, see the accompanying Multi-Resource-Partitioner wiki page
• each step has a scope="prototype" listener
  • listener gets the output file name for the step with late binding
  • listener pulls the business key from step execution context
  • listener renames the file
    • actually the file is copied and file.deleteOnExit() called, the itemwriter still holds a lock at this point, its .close() method is called after @AfterStep

Extra Step Example

Basically it is a more general variant of the simple example combined with the partition step example.

The Differences are:

• no rename-files-listener, uses extra Tasklet step instead
• no need for file.deleteOnExit() - works the "normal" way
• uses listener to propagate the file names and business key per step

It's more general because:

• output file names and business keys are stored in a ConcurrentHashMap (just a simple singleton spring bean)
• the RenameFilesTaskletStep iterates through the map and renames the files