Can someone tell me the code for creating a dataframe in pyspark in python 2.6 and spark 2.0 -


sc = sparkcontext(appname="pythonsparkstreamingkafka_rm_01")
ssc = streamingcontext(sc, 120)

kafkastream = kafkautils.createstream(ssc, '112.10.1.1:2181', 'kafka-spark-streaming', {'kafkatopic':1})

after executing above code , need have code creating dataframe in pyspark


Comments

Post a Comment

Popular posts from this blog

angular - DownloadURL return null in below code -

when change file updated in firebase storage when try downloadurl show null. filechange(event: any) { const imagefolder: string = this.employee.id; const filelist: filelist = event.target.files; const file: file = filelist[0]; const storageref = firebase.storage().ref().child(`${imagefolder}/profile.jpg`).put(file); console.log(storageref.snapshot.downloadurl); } the put() asynchronous. if want url after file uploaded have this: firebase.storage().ref().child(`${imagefolder}/profile.jpg`).put(file).then((snapshot) => { storageref = snapshot.downloadurl: console.log(snapshot.downloadurl); });
Read more

python 2.7 - Given three nested dictionaries, sort the top two nested dictionaries from a value in the innermost dictionary? -

i trying sort both outermost dictionary , "middle" dictionary value of "cal" (highest first) in innermost dictionary. accomplish using ordereddict. given example below, trying accomplish: for foods, order food subtypes highest calorie content (i.e - pizza, pesto should appear first because has higher calories when compared cheese. order foods highest calorie content (i.e - pizza should appear first has subtype more calories other food) i have following data: foods = { "apple": { "red": { "cal": "1", "taste": "4" }, "green": { "cal": "2", "taste": "6" } }, "pizza": { "pesto": { "cal": "200", "taste": "9" }, "cheese": { ...
Read more