python - How to insert an already created json-format string to Elasticsearch Bulk -


in python script,

i'm trying elasticsearch.helpers.bulk store multiple records.

i json-format string software, , want attach in source part

i got helpers.bulk format answer

part of code:

def savees(output,name):     es = elasticsearch([{'host':'localhost','port':9200}])      output = output.split('\n')     i=0     datas=[]     while i<len(output):             data = {                     "_index":"name",                     "_type":"typed",                     "_id":savees.counter,                     "_source":[[problem]]             }             i+=1             savees.counter+=1             datas.append(data)      helpers.bulk(es, datas)

i attach json-format string in [[problem]]

how can attach in? have tried hard, not output in correct..

if use: 

"_source":{ "image_name":'"'+name+'",'+output[i] }

and print data result is:

{'_type': 'typed', '_id': 0, '_source': {'image_name': '"nginx","features": "os,disk,package", "emit_shortname": "f0b03efe94ec", "timestamp": "2017-08-18t17:25:46+0900", "docker_image_tag": "latest"'}, '_index': 'name'}

this result show combined single string.

but expect:

{'_type': 'typed', '_id': 0, '_source': {'image_name': 'nginx','features': 'os,disk,package', 'emit_shortname': 'f0b03efe94ec', 'timestamp': '2017-08-18t17:25:46+0900', 'docker_image_tag': 'latest'}, '_index': 'name'}

there many problems in code.

  1. you override value of data in loop
  2. you don't respect norms (pesp8 , stuff)
  3. you while instead of comprehension list
  4. you created 2 useless variable
  5. you instantiate es in function

here improved code

es = elasticsearch([{'host':'localhost','port':9200}]) # don't have initialise variable every time calling function once.   def save_es(output,es):  # peps8 convention     output = output.split('\n') # don't need while loop. comprehension loop avoid lot of trouble     data = [    # please without s in data        {           "_index": "name",           "_type": "typed",           "_id": index,           "_source": {               "image_name":"name" + name}         }         index, name in enumerate(output)     ]         helpers.bulk(es, data)  save_es(output, es)

hope help.


Comments

Post a Comment

Popular posts from this blog

Is there a better way to structure post methods in Class Based Views -

i'm using class based views in django , never specify url in ajax calls , omit action parameter in form because know these requests go through post method of class based view. notice view - profile page make many requests depending on user updating on page - post method become convoluted , if/else mess example: def post(self, request, *args, **kwargs): # handle user changing available date if request.post.get('availabledate') != none: self.updateavailabledate(request) return jsonresponse({'result':'success'}) if request.post.get('newprojectname') != none: creative_user = creativeuserprofile.objects.get(id = request.user.id) project = project.create_project(creative_user, request.post['newprojectname']) project.save() return jsonresponse({'projectid': project.id }) if request.post.get('projectimage') != none , request.files['file']: proj...
Read more

performance - Why is XCHG reg, reg a 3 micro-op instruction on modern Intel architectures? -

i'm doing micro-optimization on performance critical part of code , came across sequence of instructions (in at&t syntax): add %rax, %rbx mov %rdx, %rax mov %rbx, %rdx i thought had use case xchg allow me shave instruction , write: add %rbx, %rax xchg %rax, %rdx however, dimay found agner fog's instruction tables , xchg 3 micro-op instruction 2 cycle latency on sandy bridge, ivy bridge, broadwell, haswell , skylake. 3 whole micro-ops , 2 cycles of latency! 3 micro-ops throws off 4-1-1-1 cadence , 2 cycle latency makes worse original in best case since last 2 instructions in original might execute in parallel. now... cpu might breaking instruction micro-ops equivalent to: mov %rax, %tmp mov %rdx, %rax mov %tmp, %rdx where tmp anonymous internal register , suppose last 2 micro-ops run in parallel latency 2 cycles. given register renaming occurs on these micro-architectures, though, doesn't make sense me done way. why wouldn't register renamer ...
Read more

c# - Asp.net web api : redirect unauthorized requst to forbidden page -

im trying redirect unauthorized request forbidden page instead i'm getting forbidden page in response body , how can fix ? here's startup class : app.createperowincontext(storecontext.create); app.createperowincontext<applicationusermanager>(applicationusermanager.create); app.createperowincontext<applicationsigninmanager>(applicationsigninmanager.create); app.usecookieauthentication(new cookieauthenticationoptions { authenticationtype = defaultauthenticationtypes.applicationcookie, expiretimespan = timespan.fromdays(30), }); app.useexternalsignincookie(defaultauthenticationtypes.externalcookie); app.usetwofactorsignincookie(defaultauthenticationtypes.twofactorcookie, timespan.fromminutes(5)); app.usetwofactorrememberbrowsercookie(defaultauthenticationtypes.twofactorrememberbrowsercookie); the method im trying reach : [httpget] [authorize(roles = "admin")] public string getcurrentusername() { return userman...
Read more