data type conversion in R -


a variable rows defined tprocrows <<- 0 in main.r file, default 'double' in function on separate r file trying do- 

tprocrows <<- as.double(row.names(rawdata)[nrow(rawdata)]) + tprocrows

which suprisingly resulting in -

> tprocrows  numeric(0)

i reading rawdata csv file , goes through 2-3 filters after being read. want keep count of processed rows in way can skip many lines of rows when rawdata reading next time. , seems i'm not able that...

this - main.r

rm(list=ls()) cat("\014")    # ffname    <<- "btpdata004.csv"  # data file full name ffname    <<- "bund009.csv"  # data file full name tprocrows <<- 0        # total processed rows far form file  cprocrows <<- 0             # total processed rows form loaded chunk ] chsize    <<- 100000            # chunk size lastflag  <<- 0              # flag indicating last chunk file  opdata    <<- data.frame() timeframe <<- 5 filename  <<- "r_ohcl_lite_dd.csv" # output file  maxlen <<- 0 minlen <<- 1000000  if (file.exists(filename)) file.remove(filename) # delete old 1  starttimeg <<- sys.time()   open   <<- vector(mode = "numeric",length = 0) close  <<- vector(mode = "numeric",length = 0) high   <<- vector(mode = "numeric",length = 0) low    <<- vector(mode = "numeric",length = 0) volume <<- vector(mode = "numeric",length = 0) time   <<-   vector(mode = "character",length = 0) date   <<-   vector(mode = "character",length = 0)  source("loaddata.r") source("processdata.r") source("savedata.r")     ############################### repeat utill complete data processed ###########################  # while(lastflag != 1) { # load data  #print("#####   main: let's load data") # loaddata()  # process data   print("#####   main: let's process data") processdata()  # append processed data frame storage file  # }  if(length(open) < 100){   opdata <<- cbind(date,time,open,high,low,close,volume)   savedata()  # savetofile }    print("#####   end main function, total time taken-->") time.taken = sys.time() - starttimeg print(time.taken)  ############################### repeat utill complete data processed ###########################

loaddata.r

# loads file in chunks  library("iotools") library("chron") library("lubridate")   loaddata <- function(){       # if(tprocrows != 0)  tprocrows <<- as.numeric(row.names(rawdata)[nrow(rawdata)],length=1) + tprocrows         cprocrows <<- 0        nskip     <<- tprocrows      # rawdata   <<- null           rawdata <<- read.csv.raw(file = ffname,sep=",",skip=nskip, nrows = chsize,nrowsclasses = 5000)             if(nrow(rawdata) < chsize){         lastflag <<- 1  # chunk last file        }          rawdata <<- subset.data.frame(rawdata,rawdata$type=="trade")       rawdata$date <<- as.date(rawdata$'date[g]',format = "%d-%b-%y")       rawdata$time <<- lubridate::hms(rawdata$"time[g]")        if(lastflag!=1){       lastday <<- rawdata$date[nrow(rawdata)]  # last complete day       rawdata <<- subset.data.frame(rawdata,rawdata$date < lastday)       }        ############################## line #########         tprocrows <<- tprocrows + as.numeric(row.names(rawdata)[nrow(rawdata)])         print(tprocrows)        ###########################################################        rawdata$`#ric`        <<-   null       rawdata$type          <<-   null       rawdata$`gmt offset`  <<-   null       rawdata$`bid price`   <<-   null       rawdata$`bid size`    <<-   null       rawdata$`ask price`   <<-   null       rawdata$`ask size`    <<-   null       rawdata$qualifiers    <<-   null       rawdata$'date[g]'     <<-   null    }#function

output

[1] "#####   main: let's process data" [1] 88230 [1] "##### file saved-------> " time difference of 2.2081 secs numeric(0) numeric(0) [1] "##### file saved-------> " time difference of 5.0582 secs numeric(0) [1] "##### file saved-------> " time difference of 7.1483 secs numeric(0) [1] "##### file saved-------> " time difference of 9.4814 secs numeric(0) [1] "##### file saved-------> " time difference of 11.5785 secs

so working first time after iteration nothing....

ps. there files has nothing with...this varible

seems code needs cleaned bit:

  if(lastflag!=1){   lastday <<- rawdata$date[nrow(rawdata)]  # last complete day   rawdata <<- subset.data.frame(rawdata,rawdata$date < lastday)   }

i think "rawdata" empty dataframe here not checked;

let's suppose:

rawdata <- data.frame(x=c(), y=c()) tprocrows <- 100

so: 

tprocrows <- tprocrows + as.numeric(row.names(rawdata)[nrow(rawdata)])  print(tprocrows)

output:

numeric(0)

Comments

Post a Comment

Popular posts from this blog

Is there a better way to structure post methods in Class Based Views -

i'm using class based views in django , never specify url in ajax calls , omit action parameter in form because know these requests go through post method of class based view. notice view - profile page make many requests depending on user updating on page - post method become convoluted , if/else mess example: def post(self, request, *args, **kwargs): # handle user changing available date if request.post.get('availabledate') != none: self.updateavailabledate(request) return jsonresponse({'result':'success'}) if request.post.get('newprojectname') != none: creative_user = creativeuserprofile.objects.get(id = request.user.id) project = project.create_project(creative_user, request.post['newprojectname']) project.save() return jsonresponse({'projectid': project.id }) if request.post.get('projectimage') != none , request.files['file']: proj...
Read more

performance - Why is XCHG reg, reg a 3 micro-op instruction on modern Intel architectures? -

i'm doing micro-optimization on performance critical part of code , came across sequence of instructions (in at&t syntax): add %rax, %rbx mov %rdx, %rax mov %rbx, %rdx i thought had use case xchg allow me shave instruction , write: add %rbx, %rax xchg %rax, %rdx however, dimay found agner fog's instruction tables , xchg 3 micro-op instruction 2 cycle latency on sandy bridge, ivy bridge, broadwell, haswell , skylake. 3 whole micro-ops , 2 cycles of latency! 3 micro-ops throws off 4-1-1-1 cadence , 2 cycle latency makes worse original in best case since last 2 instructions in original might execute in parallel. now... cpu might breaking instruction micro-ops equivalent to: mov %rax, %tmp mov %rdx, %rax mov %tmp, %rdx where tmp anonymous internal register , suppose last 2 micro-ops run in parallel latency 2 cycles. given register renaming occurs on these micro-architectures, though, doesn't make sense me done way. why wouldn't register renamer ...
Read more

c# - Asp.net web api : redirect unauthorized requst to forbidden page -

im trying redirect unauthorized request forbidden page instead i'm getting forbidden page in response body , how can fix ? here's startup class : app.createperowincontext(storecontext.create); app.createperowincontext<applicationusermanager>(applicationusermanager.create); app.createperowincontext<applicationsigninmanager>(applicationsigninmanager.create); app.usecookieauthentication(new cookieauthenticationoptions { authenticationtype = defaultauthenticationtypes.applicationcookie, expiretimespan = timespan.fromdays(30), }); app.useexternalsignincookie(defaultauthenticationtypes.externalcookie); app.usetwofactorsignincookie(defaultauthenticationtypes.twofactorcookie, timespan.fromminutes(5)); app.usetwofactorrememberbrowsercookie(defaultauthenticationtypes.twofactorrememberbrowsercookie); the method im trying reach : [httpget] [authorize(roles = "admin")] public string getcurrentusername() { return userman...
Read more