java - Spring Batch asynchronous processor configuration for best performance -


i have problem creating asynchronous processor in spring batch. processor getting id reader , creating object based on response soap call. 1 input (id) there must e.g. 60-100 soap calls , 1. tried make multithreaded step processing e.g 50 inputs @ time useless because 49 threads done job in 1 second , blocked, waiting 1 doing 60-100 soap calls. use asyncitemprocessor+asyncitemwriter solution works me. input (ids) large, around 25k items read db start ~50-100 inputs @ time.

here configuration:

@configuration public class batchconfig {      @autowired     public jobbuilderfactory jobbuilderfactory;     @autowired     public stepbuilderfactory stepbuilderfactory;     @autowired     private databaseconfig databaseconfig;     @value(value = "classpath:categories.txt")     private resource categories;      @bean     public job processjob() throws exception {         return jobbuilderfactory.get("processjob").incrementer(new runidincrementer()).listener(listener()).flow(orderstep1()).end().build();     }      @bean     public step orderstep1() throws exception {         return stepbuilderfactory.get("orderstep1").<category, categorydailyresult>chunk(1).reader(reader()).processor(asyncitemprocessor()).writer(asyncitemwriter()).taskexecutor(taskexecutor()).build();     }      @bean     public jobexecutionlistener listener() {         return new jobcompletionlistener();     }       @bean     public itemwriter asyncitemwriter() {         asyncitemwriter<categorydailyresult> asyncitemwriter = new asyncitemwriter<>();         asyncitemwriter.setdelegate(itemwriter());         return asyncitemwriter;     }      @bean     public itemwriter<categorydailyresult> itemwriter(){         return new writer();     }      @bean     public itemprocessor asyncitemprocessor() {         asyncitemprocessor<category, categorydailyresult> asyncitemprocessor = new asyncitemprocessor<>();         asyncitemprocessor.setdelegate(itemprocessor());         asyncitemprocessor.settaskexecutor(taskexecutor());         return asyncitemprocessor;     }      @bean     public itemprocessor<category, categorydailyresult> itemprocessor(){         return new processor();     }      @bean     public taskexecutor taskexecutor(){         simpleasynctaskexecutor taskexecutor = new simpleasynctaskexecutor();         taskexecutor.setconcurrencylimit(50);         return taskexecutor;     }      @bean(destroymethod = "")     public itemreader<category> reader() throws exception {         string query = "select c category c not exists elements(c.children)";          jpapagingitemreader<category> reader = new jpapagingitemreader<>();         reader.setsavestate(false);         reader.setquerystring(query);         reader.setentitymanagerfactory(databaseconfig.entitymanagerfactory().getobject());         reader.setpagesize(1);          return reader;     } } 

how can boost application? maybe doing wrong? feedback welcome ;)

@edit: input of ids: 1 100 want e.g 50 threads executing processor. want them not block each other: thread1 process input "1" 2 minutes , @ time want thread2 process input "2", "8", "64" small , execute in few seconds.

@edit2: my goal: have 25k ids in database, read them jpapagingitemreader , every id processed processor. each item independent of each other. each id make soap call 0-100 times in loop , create object pass writer , save in database. how can obtain best performance such task?


Comments

Popular posts from this blog

Is there a better way to structure post methods in Class Based Views -

performance - Why is XCHG reg, reg a 3 micro-op instruction on modern Intel architectures? -

jquery - Responsive Navbar with Sub Navbar -