Monday, April 05, 2010

How to pass filename to FlatFileItemReader with JobParameters in Spring Batch?



 The follow config was tested and succeeded on Spring Batch 2.0.4.
<bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
  <property name="resource"  value="file:#{jobParameters['input.file.name']}" />
  <property name="lineMapper" ref="lineMapper"/>
</bean>



Java code:

try {
//launcher.run(job, new JobParameters());
JobParametersBuilder jpBuilder = new   JobParametersBuilder().addString("input.file.name", "files/input/test.txt");

 jpBuilder.addString("output.file.name", "files/output/"+System.currentTimeMillis()+"_test.txt");
launcher.run(job, jpBuilder.toJobParameters());
} catch (Exception e) {
e.printStackTrace();
}


Another config used FileSystemResource for resource likes the follow.

<property name="resource">
   <bean class="org.springframework.core.io.FileSystemResource" scope="step" autowire-candidate="false">
     <constructor-arg value="#{jobParameters['input.file.name']}" />
   </bean>
</property>



For ItemWriter, the configuration almost same.
NOTE:
The 'file:' is mandatory if the resource is a file needed to be load from specified path.
The  scope="step" is very important. It seems the file cannot be found if it is missed.
From the reference, I found 'Any bean that uses late-binding must be declared with scope="step".'.


The follow code is is used by SpringBatch 1.x.
Just like the reference metioned, you need to define a listener for step to the proxy to have the access to the StepExecution.

<property name="listeners">
  <list>
    <ref bean="inputFile" />
  </list>
</property>

<bean id="inputFile" class="org.springframework.batch.core.resource.StepExecutionResourceProxy">

  <property name="filePattern" value="file://%file.name%"/>
</bean>

<bean id="myFileItemReader" class="org.springframework.batch.item.file.FlatFil eItemReader">
  <property name="resource" ref="inputFile" />
</bean>


Java Code:

JobParametersBuilder builder = new JobParametersBuilder();
builder.addString("file.name", "files/input/test.txt");

JobExecution jobExecution = getJobLauncher().run(getJob(), builder.toJobParameters());

1 comment:

Unknown said...

mainDir - FileA
processDir - FileB, FileC, FileD



1. Check file in mainDir. If present, check files in processDir. If not present, copy FileA to processDir.
2. Check file in mainDir. If present, check files in processDir. If present, check for mapping.
- Creating a mapping object
mappingObj1(FileA_Name, FileB_Name)
mappingObj1(FileA_Name, FileC_Name)
mappingObj1(FileB_Name, FileD_Name)






FeedDef
1|LEM| |TradeRef
1|LEM| |TradeDate
2 RAVEN LOH
SystemMapping
1,2
1,3
2,4



----------------------------------------------------------------------------------------------------------------
Hashmap fdMap
for(String fdLine : fdLines)
{
String[] fd = fdLine.split("|");
FeedDef fdObj = new FeedDef();
fdObj.setSourceSystemName(fd[1]);
fdObj.setSourceSystemSiteCode(fd[2]);
fdMap.put(fd[0], fdObj);
}


HashMap
List sysMappingList
for(String smLine : smLines)
{
String[] sm = smLine.split("|");
for(String sm : smList)
{
SystemMapping sysMapping;
sysMapping.setSourceSystemId((sm[0]);
sysMapping.setSourceSystemName((fdMap.get(sm.[0]).getSourceSystemName()));
sysMapping.setSourceSystemSiteCode((fdMap.get(sm.get(0)).getSourceSystemSiteCode()));

sysMapping.setTargetSystemId((sm[1]);
sysMapping.setTargetSystemName((fdMap.get(sm.[1]).getSourceSystemName()));
sysMapping.setTargetSystemSiteCode((fdMap.get(sm.get(1)).getSourceSystemSiteCode()));

sysMappingList.add(sysMapping);
}
}

FileToProcess - sourceFileName, targetFileName
ArrayList fileToProcessList
File[] mainDirFiles
if(null != mainDirFiles)
{
move files to processDir
File[] processDirFiles;
if(null != processDirFiles)
{
for(File mdFile : mainDirFiles)
{
for(File pdFile : processDirFiles)
{
for(SystemMapping sysMaping : sysMappingList)
{
if(mdFile.getName().contains(sysMaping.getSourceSystemName() && mdFile.getName().contains(sysMaping.getSourceSystemSiteCode()))
{
if(pdFile.getName().contains(sysMaping.getTargetSystemName() && pdFile.getName().contains(sysMaping.getTargetSystemSiteCode()))
{
FileToProcess fileToProcess = new FileToProcess();
fileToProcess.setSourceFileName(mdFile.getName());
fileToProcess.setTargetFileName(pdFile.getName());
fileToProcessList.add();
}
}
else if(pdFile.getName().contains(sysMaping.getSourceSystemName() && pdFile.getName().contains(sysMaping.getSourceSystemSiteCode()))
{
if(mdFile.getName().contains(sysMaping.getTargetSystemName() && mdFile.getName().contains(sysMaping.getTargetSystemSiteCode()))
{
FileToProcess fileToProcess = new FileToProcess();
fileToProcess.setSourceFileName(mdFile.getName());
fileToProcess.setTargetFileName(pdFile.getName());
fileToProcessList.add();
}
}
}

}
}
} else
{
don't start job
}
} else
{
don't start job
}


For(FileToProcess fileToProcess : fileToProcessList){

try {
JobParametersBuilder jpBuilder = new JobParametersBuilder().addString("source.file.name", fileToProcess.getSourceFileName());
jpBuilder.addString("target.file.name", fileToProcess.getTargetFileName());
launcher.run(job, jpBuilder.toJobParameters());
} catch (Exception e) {
e.printStackTrace();
}




The follow config was tested and succeeded on Spring Batch 2.0.4.





}