Encountered an error executing step masterStep Execption Loading data to mysql Spring BatchSpring batch job throws error like 'Partition handler returned an unsuccessful step'spring boot :Injection of autowired dependencies failed;Adding maven dependency gives a specific error during my spring batchEncountering error during context initialization in springLazy load spring beanCannot load driver class: com.mysql.jdbc.Driver Error on Spring Boot using EclipseJob called twice in SpringBatch when configured with modular=trueUnable to start embedded Tomcat NoClassDefFoundError DispatcherServletPatherror:spring boot database error “org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaConfiguration”spring boot database error datasource "org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaConfiguration”Spring batch pass value of resource from step 1 to the next step

Importance of moon phases for Apollo missions

Can the caster of Time Stop still use their bonus action or reaction?

What should I watch before playing Alien: Isolation?

What does a Nintendo Game Boy do when turned on without a game cartridge inserted?

I have a domain, static IP and many devices I'd like to access outside my house. How to route them?

What do Unicorns want?

Is it better to have a 10 year gap or a bad reference?

Would using carbon dioxide as fuel work to reduce the greenhouse effect?

Did Don Young threaten John Boehner with a 10 inch blade to the throat?

what does the term highest qualification mean?

Why is the UH-60 tail rotor canted?

Why does airflow separate from the wing during stall?

Is there an English word to describe when a sound "protrudes"?

Cargo capacity of a kayak

What is the best word describing the nature of expiring in a short amount of time, connoting "losing public attention"?

Which dice game has a board with 9x9 squares that has different colors on the diagonals and midway on some edges?

Monday's Blocking Donimoes Problem

Why can't a country print its own money to spend it only abroad?

What is the metal bit in the front of this propeller spinner?

Reissue US, UK, Canada visas in stolen passports

What would be the effects of (relatively) widespread precognition on the stock market?

Oriented vector bundle with odd-dimensional fibers

"It is what it is"

Why was Quirrell said to be in the Black Forest if Voldemort was actually in Albania?



Encountered an error executing step masterStep Execption Loading data to mysql Spring Batch


Spring batch job throws error like 'Partition handler returned an unsuccessful step'spring boot :Injection of autowired dependencies failed;Adding maven dependency gives a specific error during my spring batchEncountering error during context initialization in springLazy load spring beanCannot load driver class: com.mysql.jdbc.Driver Error on Spring Boot using EclipseJob called twice in SpringBatch when configured with modular=trueUnable to start embedded Tomcat NoClassDefFoundError DispatcherServletPatherror:spring boot database error “org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaConfiguration”spring boot database error datasource "org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaConfiguration”Spring batch pass value of resource from step 1 to the next step






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;








0















I am trying to load set of csv file into mysql using spring batch.
My code is given below. But when I run with 2 or 3 files the data is loading fine. but when I try with 100 or more Its trowing an execption aftre loading 4 to 8 files data.
Exception:



ERROR 11633 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step masterStep in job importUserJob
org.springframework.batch.core.JobExecutionException: Partition handler returned an unsuccessful step
at org.springframework.batch.core.partition.support.PartitionStep.doExecute(PartitionStep.java:112) ~[spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:66) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:136) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:308) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:141) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.0.5.RELEASE.jar!/:5.0.5.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:134) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:163) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:179) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:134) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:128) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:797) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:781) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:335) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:137) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]


I don't know whats happening behind, Any help appreciated.
Code:



@Configuration
@EnableBatchProcessing
public class ImportJobConfig

@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Autowired
private DataSource dataSource;
@Autowired
private JdbcBatchItemWriter<Person> writer;
@Autowired
private FlatFileItemReader<Person> personItemReader;
@Bean
public DataSource dataSource()
final DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("URL");
dataSource.setUsername("USERNAME");
dataSource.setPassword("PASSWORD");
return dataSource;

@Bean
public ThreadPoolTaskExecutor taskExecutor()
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setMaxPoolSize(4);
taskExecutor.setCorePoolSize(4);
taskExecutor.setQueueCapacity(4);
taskExecutor.afterPropertiesSet();
taskExecutor.setAllowCoreThreadTimeOut(true);
return taskExecutor;

@Bean
@Qualifier("masterStep")
public Step masterStep()
return stepBuilderFactory.get("masterStep").partitioner("step1", partitioner()).step(step1())
.taskExecutor(taskExecutor()).build();

@Bean("partitioner")
@StepScope
public Partitioner partitioner()
System.out.println("In Partitioner");
MultiResourcePartitioner partitioner = new MultiResourcePartitioner();
ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
String filePath = "//data//person//*.csv";
Resource[] resources = null;
try
resources = resolver.getResources("file:" + filePath);
catch (IOException e)
e.printStackTrace();

partitioner.setResources(resources);
partitioner.partition(4);
return partitioner;

@Bean
public APSUploadFileItemProcessor processor()
return new APSUploadFileItemProcessor();

@Bean
public JdbcBatchItemWriter<Person> writer()
JdbcBatchItemWriter<Person> writer = new JdbcBatchItemWriter<>();
try
writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<Person>());
writer.setSql("INSERT INTO person(name,business,phone) VALUES(name, :business, :phone)");
writer.setDataSource(dataSource);
catch (Exception e)
e.printStackTrace();

return writer;

@Bean
public Job importUserJob(JobCompletionNotificationListener listener, Step step1)
return jobBuilderFactory.get("importUserJob").incrementer(new RunIdIncrementer()).listener(listener)
.flow(masterStep()).end().build();

@Bean
public Step step1()
return stepBuilderFactory.get("step1").<Person, Person>chunk(10000).processor(processor())
.writer(writer).reader(personItemReader).build();

@Bean
@StepScope
@Qualifier("personItemReader")
@DependsOn("partitioner")
public FlatFileItemReader<Person> personItemReader(
@Value("#stepExecutionContext['fileName']") String filename) throws MalformedURLException ")
.names(new String[] "name","business","phone" )
.fieldSetMapper(new BeanWrapperFieldSetMapper<Person>()

setTargetType(Person.class);

).resource(new UrlResource(filename)).build();











share|improve this question

















  • 1





    Please post the full stack trace of the exception.

    – Mahmoud Ben Hassine
    Mar 26 at 14:21












  • this is the entire stack trace I'm getting

    – Juhan
    Mar 26 at 15:12






  • 2





    See > stackoverflow.com/questions/32629724/…

    – Mountain-Lion
    Mar 26 at 15:55











  • @Mountain-Lion I went through the question and made changes to the code, but still facing the same issue

    – Juhan
    Mar 27 at 2:23

















0















I am trying to load set of csv file into mysql using spring batch.
My code is given below. But when I run with 2 or 3 files the data is loading fine. but when I try with 100 or more Its trowing an execption aftre loading 4 to 8 files data.
Exception:



ERROR 11633 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step masterStep in job importUserJob
org.springframework.batch.core.JobExecutionException: Partition handler returned an unsuccessful step
at org.springframework.batch.core.partition.support.PartitionStep.doExecute(PartitionStep.java:112) ~[spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:66) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:136) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:308) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:141) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.0.5.RELEASE.jar!/:5.0.5.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:134) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:163) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:179) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:134) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:128) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:797) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:781) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:335) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:137) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]


I don't know whats happening behind, Any help appreciated.
Code:



@Configuration
@EnableBatchProcessing
public class ImportJobConfig

@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Autowired
private DataSource dataSource;
@Autowired
private JdbcBatchItemWriter<Person> writer;
@Autowired
private FlatFileItemReader<Person> personItemReader;
@Bean
public DataSource dataSource()
final DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("URL");
dataSource.setUsername("USERNAME");
dataSource.setPassword("PASSWORD");
return dataSource;

@Bean
public ThreadPoolTaskExecutor taskExecutor()
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setMaxPoolSize(4);
taskExecutor.setCorePoolSize(4);
taskExecutor.setQueueCapacity(4);
taskExecutor.afterPropertiesSet();
taskExecutor.setAllowCoreThreadTimeOut(true);
return taskExecutor;

@Bean
@Qualifier("masterStep")
public Step masterStep()
return stepBuilderFactory.get("masterStep").partitioner("step1", partitioner()).step(step1())
.taskExecutor(taskExecutor()).build();

@Bean("partitioner")
@StepScope
public Partitioner partitioner()
System.out.println("In Partitioner");
MultiResourcePartitioner partitioner = new MultiResourcePartitioner();
ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
String filePath = "//data//person//*.csv";
Resource[] resources = null;
try
resources = resolver.getResources("file:" + filePath);
catch (IOException e)
e.printStackTrace();

partitioner.setResources(resources);
partitioner.partition(4);
return partitioner;

@Bean
public APSUploadFileItemProcessor processor()
return new APSUploadFileItemProcessor();

@Bean
public JdbcBatchItemWriter<Person> writer()
JdbcBatchItemWriter<Person> writer = new JdbcBatchItemWriter<>();
try
writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<Person>());
writer.setSql("INSERT INTO person(name,business,phone) VALUES(name, :business, :phone)");
writer.setDataSource(dataSource);
catch (Exception e)
e.printStackTrace();

return writer;

@Bean
public Job importUserJob(JobCompletionNotificationListener listener, Step step1)
return jobBuilderFactory.get("importUserJob").incrementer(new RunIdIncrementer()).listener(listener)
.flow(masterStep()).end().build();

@Bean
public Step step1()
return stepBuilderFactory.get("step1").<Person, Person>chunk(10000).processor(processor())
.writer(writer).reader(personItemReader).build();

@Bean
@StepScope
@Qualifier("personItemReader")
@DependsOn("partitioner")
public FlatFileItemReader<Person> personItemReader(
@Value("#stepExecutionContext['fileName']") String filename) throws MalformedURLException ")
.names(new String[] "name","business","phone" )
.fieldSetMapper(new BeanWrapperFieldSetMapper<Person>()

setTargetType(Person.class);

).resource(new UrlResource(filename)).build();











share|improve this question

















  • 1





    Please post the full stack trace of the exception.

    – Mahmoud Ben Hassine
    Mar 26 at 14:21












  • this is the entire stack trace I'm getting

    – Juhan
    Mar 26 at 15:12






  • 2





    See > stackoverflow.com/questions/32629724/…

    – Mountain-Lion
    Mar 26 at 15:55











  • @Mountain-Lion I went through the question and made changes to the code, but still facing the same issue

    – Juhan
    Mar 27 at 2:23













0












0








0


2






I am trying to load set of csv file into mysql using spring batch.
My code is given below. But when I run with 2 or 3 files the data is loading fine. but when I try with 100 or more Its trowing an execption aftre loading 4 to 8 files data.
Exception:



ERROR 11633 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step masterStep in job importUserJob
org.springframework.batch.core.JobExecutionException: Partition handler returned an unsuccessful step
at org.springframework.batch.core.partition.support.PartitionStep.doExecute(PartitionStep.java:112) ~[spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:66) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:136) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:308) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:141) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.0.5.RELEASE.jar!/:5.0.5.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:134) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:163) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:179) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:134) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:128) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:797) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:781) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:335) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:137) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]


I don't know whats happening behind, Any help appreciated.
Code:



@Configuration
@EnableBatchProcessing
public class ImportJobConfig

@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Autowired
private DataSource dataSource;
@Autowired
private JdbcBatchItemWriter<Person> writer;
@Autowired
private FlatFileItemReader<Person> personItemReader;
@Bean
public DataSource dataSource()
final DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("URL");
dataSource.setUsername("USERNAME");
dataSource.setPassword("PASSWORD");
return dataSource;

@Bean
public ThreadPoolTaskExecutor taskExecutor()
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setMaxPoolSize(4);
taskExecutor.setCorePoolSize(4);
taskExecutor.setQueueCapacity(4);
taskExecutor.afterPropertiesSet();
taskExecutor.setAllowCoreThreadTimeOut(true);
return taskExecutor;

@Bean
@Qualifier("masterStep")
public Step masterStep()
return stepBuilderFactory.get("masterStep").partitioner("step1", partitioner()).step(step1())
.taskExecutor(taskExecutor()).build();

@Bean("partitioner")
@StepScope
public Partitioner partitioner()
System.out.println("In Partitioner");
MultiResourcePartitioner partitioner = new MultiResourcePartitioner();
ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
String filePath = "//data//person//*.csv";
Resource[] resources = null;
try
resources = resolver.getResources("file:" + filePath);
catch (IOException e)
e.printStackTrace();

partitioner.setResources(resources);
partitioner.partition(4);
return partitioner;

@Bean
public APSUploadFileItemProcessor processor()
return new APSUploadFileItemProcessor();

@Bean
public JdbcBatchItemWriter<Person> writer()
JdbcBatchItemWriter<Person> writer = new JdbcBatchItemWriter<>();
try
writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<Person>());
writer.setSql("INSERT INTO person(name,business,phone) VALUES(name, :business, :phone)");
writer.setDataSource(dataSource);
catch (Exception e)
e.printStackTrace();

return writer;

@Bean
public Job importUserJob(JobCompletionNotificationListener listener, Step step1)
return jobBuilderFactory.get("importUserJob").incrementer(new RunIdIncrementer()).listener(listener)
.flow(masterStep()).end().build();

@Bean
public Step step1()
return stepBuilderFactory.get("step1").<Person, Person>chunk(10000).processor(processor())
.writer(writer).reader(personItemReader).build();

@Bean
@StepScope
@Qualifier("personItemReader")
@DependsOn("partitioner")
public FlatFileItemReader<Person> personItemReader(
@Value("#stepExecutionContext['fileName']") String filename) throws MalformedURLException ")
.names(new String[] "name","business","phone" )
.fieldSetMapper(new BeanWrapperFieldSetMapper<Person>()

setTargetType(Person.class);

).resource(new UrlResource(filename)).build();











share|improve this question














I am trying to load set of csv file into mysql using spring batch.
My code is given below. But when I run with 2 or 3 files the data is loading fine. but when I try with 100 or more Its trowing an execption aftre loading 4 to 8 files data.
Exception:



ERROR 11633 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step masterStep in job importUserJob
org.springframework.batch.core.JobExecutionException: Partition handler returned an unsuccessful step
at org.springframework.batch.core.partition.support.PartitionStep.doExecute(PartitionStep.java:112) ~[spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:66) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:136) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:308) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:141) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.0.5.RELEASE.jar!/:5.0.5.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:134) [spring-batch-core-4.0.1.RELEASE.jar!/:4.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:163) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:179) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:134) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:128) [spring-boot-autoconfigure-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:797) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:781) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:335) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]
at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:137) [spring-boot-2.0.1.RELEASE.jar!/:2.0.1.RELEASE]


I don't know whats happening behind, Any help appreciated.
Code:



@Configuration
@EnableBatchProcessing
public class ImportJobConfig

@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
private StepBuilderFactory stepBuilderFactory;
@Autowired
private DataSource dataSource;
@Autowired
private JdbcBatchItemWriter<Person> writer;
@Autowired
private FlatFileItemReader<Person> personItemReader;
@Bean
public DataSource dataSource()
final DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("URL");
dataSource.setUsername("USERNAME");
dataSource.setPassword("PASSWORD");
return dataSource;

@Bean
public ThreadPoolTaskExecutor taskExecutor()
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setMaxPoolSize(4);
taskExecutor.setCorePoolSize(4);
taskExecutor.setQueueCapacity(4);
taskExecutor.afterPropertiesSet();
taskExecutor.setAllowCoreThreadTimeOut(true);
return taskExecutor;

@Bean
@Qualifier("masterStep")
public Step masterStep()
return stepBuilderFactory.get("masterStep").partitioner("step1", partitioner()).step(step1())
.taskExecutor(taskExecutor()).build();

@Bean("partitioner")
@StepScope
public Partitioner partitioner()
System.out.println("In Partitioner");
MultiResourcePartitioner partitioner = new MultiResourcePartitioner();
ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
String filePath = "//data//person//*.csv";
Resource[] resources = null;
try
resources = resolver.getResources("file:" + filePath);
catch (IOException e)
e.printStackTrace();

partitioner.setResources(resources);
partitioner.partition(4);
return partitioner;

@Bean
public APSUploadFileItemProcessor processor()
return new APSUploadFileItemProcessor();

@Bean
public JdbcBatchItemWriter<Person> writer()
JdbcBatchItemWriter<Person> writer = new JdbcBatchItemWriter<>();
try
writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<Person>());
writer.setSql("INSERT INTO person(name,business,phone) VALUES(name, :business, :phone)");
writer.setDataSource(dataSource);
catch (Exception e)
e.printStackTrace();

return writer;

@Bean
public Job importUserJob(JobCompletionNotificationListener listener, Step step1)
return jobBuilderFactory.get("importUserJob").incrementer(new RunIdIncrementer()).listener(listener)
.flow(masterStep()).end().build();

@Bean
public Step step1()
return stepBuilderFactory.get("step1").<Person, Person>chunk(10000).processor(processor())
.writer(writer).reader(personItemReader).build();

@Bean
@StepScope
@Qualifier("personItemReader")
@DependsOn("partitioner")
public FlatFileItemReader<Person> personItemReader(
@Value("#stepExecutionContext['fileName']") String filename) throws MalformedURLException ")
.names(new String[] "name","business","phone" )
.fieldSetMapper(new BeanWrapperFieldSetMapper<Person>()

setTargetType(Person.class);

).resource(new UrlResource(filename)).build();








java spring spring-boot spring-batch batch-processing






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Mar 26 at 13:24









JuhanJuhan

7601 gold badge5 silver badges21 bronze badges




7601 gold badge5 silver badges21 bronze badges







  • 1





    Please post the full stack trace of the exception.

    – Mahmoud Ben Hassine
    Mar 26 at 14:21












  • this is the entire stack trace I'm getting

    – Juhan
    Mar 26 at 15:12






  • 2





    See > stackoverflow.com/questions/32629724/…

    – Mountain-Lion
    Mar 26 at 15:55











  • @Mountain-Lion I went through the question and made changes to the code, but still facing the same issue

    – Juhan
    Mar 27 at 2:23












  • 1





    Please post the full stack trace of the exception.

    – Mahmoud Ben Hassine
    Mar 26 at 14:21












  • this is the entire stack trace I'm getting

    – Juhan
    Mar 26 at 15:12






  • 2





    See > stackoverflow.com/questions/32629724/…

    – Mountain-Lion
    Mar 26 at 15:55











  • @Mountain-Lion I went through the question and made changes to the code, but still facing the same issue

    – Juhan
    Mar 27 at 2:23







1




1





Please post the full stack trace of the exception.

– Mahmoud Ben Hassine
Mar 26 at 14:21






Please post the full stack trace of the exception.

– Mahmoud Ben Hassine
Mar 26 at 14:21














this is the entire stack trace I'm getting

– Juhan
Mar 26 at 15:12





this is the entire stack trace I'm getting

– Juhan
Mar 26 at 15:12




2




2





See > stackoverflow.com/questions/32629724/…

– Mountain-Lion
Mar 26 at 15:55





See > stackoverflow.com/questions/32629724/…

– Mountain-Lion
Mar 26 at 15:55













@Mountain-Lion I went through the question and made changes to the code, but still facing the same issue

– Juhan
Mar 27 at 2:23





@Mountain-Lion I went through the question and made changes to the code, but still facing the same issue

– Juhan
Mar 27 at 2:23












0






active

oldest

votes










Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55358288%2fencountered-an-error-executing-step-masterstep-execption-loading-data-to-mysql-s%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes




Is this question similar to what you get asked at work? Learn more about asking and sharing private information with your coworkers using Stack Overflow for Teams.







Is this question similar to what you get asked at work? Learn more about asking and sharing private information with your coworkers using Stack Overflow for Teams.



















draft saved

draft discarded
















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55358288%2fencountered-an-error-executing-step-masterstep-execption-loading-data-to-mysql-s%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Kamusi Yaliyomo Aina za kamusi | Muundo wa kamusi | Faida za kamusi | Dhima ya picha katika kamusi | Marejeo | Tazama pia | Viungo vya nje | UrambazajiKuhusu kamusiGo-SwahiliWiki-KamusiKamusi ya Kiswahili na Kiingerezakuihariri na kuongeza habari

Swift 4 - func physicsWorld not invoked on collision? The Next CEO of Stack OverflowHow to call Objective-C code from Swift#ifdef replacement in the Swift language@selector() in Swift?#pragma mark in Swift?Swift for loop: for index, element in array?dispatch_after - GCD in Swift?Swift Beta performance: sorting arraysSplit a String into an array in Swift?The use of Swift 3 @objc inference in Swift 4 mode is deprecated?How to optimize UITableViewCell, because my UITableView lags

Access current req object everywhere in Node.js ExpressWhy are global variables considered bad practice? (node.js)Using req & res across functionsHow do I get the path to the current script with Node.js?What is Node.js' Connect, Express and “middleware”?Node.js w/ express error handling in callbackHow to access the GET parameters after “?” in Express?Modify Node.js req object parametersAccess “app” variable inside of ExpressJS/ConnectJS middleware?Node.js Express app - request objectAngular Http Module considered middleware?Session variables in ExpressJSAdd properties to the req object in expressjs with Typescript