春季批处理：如何使用平面文件读取CSV文件的页脚并进行验证

Question

我正在使用弹簧批处理和扁平文件条目阅读器来读取.CSV 文件。文件具有页眉（第一行）、详细信息和页脚（最后一行）。因此，我想通过页脚行验证详细信息的总数。这是我.csv文件的示例。电影.csv名称|类型|诺丁山|浪漫喜剧|1999年玩具总动员3|动画|2010美国队长3：第一复仇者|动作|20113从示例文件第一行是标题（我忽略它）。第 2-4 行是细节行，最后是页脚。我想读取页脚并获取值（最后一行= 3），然后，获取详细信息的总数（在这种情况下，我们有3行），最后我将验证页脚（3）的总数，详细信息记录的总数（3）是否相等？这是我的代码。@Bean@StepScopepublic FlatFileItemReader<Movie> movieItemReader(String filePath) {        FlatFileItemReader<Movie> reader = new FlatFileItemReader<>();        reader.setLinesToSkip(1);   //skip header line        reader.setResource(new PathResource(filePath));        DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer("|");        DefaultLineMapper<Movie> movieLineMapper = new DefaultLineMapper<>();        FieldSetMapper<Movie> movieMapper = movieFieldSetMapper();        movieLineMapper.setLineTokenizer(tokenizer);        movieLineMapper.setFieldSetMapper(movieFieldSetMapper);        movieLineMapper.afterPropertiesSet();        reader.setLineMapper(movieLineMapper);        return reader;}public FieldSetMapper<Movie> movieFieldSetMapper() {        BeanWrapperFieldSetMapper<Movie> movieMapper = new BeanWrapperFieldSetMapper<>();        movieMapper.setTargetType(Movie.class);        return movieMapper;}

Answer 1

可以使用面向块的步骤作为作业业务逻辑之前的验证步骤。此步骤将使用 a 保存最后一项，使用 a 进行验证。下面是一个快速示例：ItemReadListenerStepExecutionListener

import org.springframework.batch.core.ExitStatus;

import org.springframework.batch.core.ItemReadListener;

import org.springframework.batch.core.Job;

import org.springframework.batch.core.JobParameters;

import org.springframework.batch.core.Step;

import org.springframework.batch.core.StepExecution;

import org.springframework.batch.core.StepExecutionListener;

import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;

import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;

import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;

import org.springframework.batch.core.configuration.annotation.StepScope;

import org.springframework.batch.core.launch.JobLauncher;

import org.springframework.batch.core.listener.StepExecutionListenerSupport;

import org.springframework.batch.item.ItemWriter;

import org.springframework.batch.item.file.FlatFileItemReader;

import org.springframework.batch.item.file.mapping.PassThroughLineMapper;

import org.springframework.batch.repeat.RepeatStatus;

import org.springframework.beans.factory.annotation.Autowired;

import org.springframework.context.ApplicationContext;

import org.springframework.context.annotation.AnnotationConfigApplicationContext;

import org.springframework.context.annotation.Bean;

import org.springframework.context.annotation.Configuration;

import org.springframework.core.io.ByteArrayResource;

@Configuration

@EnableBatchProcessing

public class MyJob {

    @Autowired

    private JobBuilderFactory jobs;

    @Autowired

    private StepBuilderFactory steps;

    @Bean

    @StepScope

    public FlatFileItemReader<String> itemReader() {

        FlatFileItemReader<String> reader = new FlatFileItemReader<>();

        reader.setLinesToSkip(1);   //skip header line

        reader.setResource(new ByteArrayResource("header\nitem1\nitem2\n2".getBytes()));

        reader.setLineMapper(new PassThroughLineMapper());

        return reader;

    }

    @Bean

    public ItemWriter<String> itemWriter() {

        return items -> {

            for (String item : items) {

                System.out.println("item = " + item);

            }

        };

    }

    @Bean

    public Step step1() {

        MyListener myListener = new MyListener();

        return steps.get("step1")

                .<String, String>chunk(5)

                .reader(itemReader())

                .writer(itemWriter())

                .listener((ItemReadListener<String>) myListener)

                .listener((StepExecutionListener) myListener)

                .build();

    }

    @Bean

    public Step step2() {

        return steps.get("step2")

                .tasklet((contribution, chunkContext) -> {

                    System.out.println("Total count is ok as validated by step1");

                    return RepeatStatus.FINISHED;

                })

                .build();

    }

    @Bean

    public Job job() {

        return jobs.get("job")

                .start(step1())

                .next(step2())

                .build();

    }

    static class MyListener extends StepExecutionListenerSupport implements ItemReadListener<String> {

        private String lastItem;

        @Override

        public void beforeRead() {

        }

        @Override

        public void afterRead(String item) {

            this.lastItem = item;

        }

        @Override

        public void onReadError(Exception ex) {

        }

        @Override

        public ExitStatus afterStep(StepExecution stepExecution) {

            int readCount = stepExecution.getReadCount();

            int totalCountInFooter = Integer.valueOf(this.lastItem); // TODO sanity checks (number format, etc)

            System.out.println("readCount = " + (readCount - 1)); // substract footer from the read count

            System.out.println("totalCountInFooter = " + totalCountInFooter);

            // TODO do validation on readCount vs totalCountInFooter

            return ExitStatus.COMPLETED; // return appropriate exit status according to validation result

        }

    }

    public static void main(String[] args) throws Exception {

        ApplicationContext context = new AnnotationConfigApplicationContext(MyJob.class);

        JobLauncher jobLauncher = context.getBean(JobLauncher.class);

        Job job = context.getBean(Job.class);

        jobLauncher.run(job, new JobParameters());

    }

}

此示例打印：

item = item1

item = item2

item = 2

readCount = 2

totalCountInFooter = 2

Total count is ok as validated by step1

希望这有帮助。