python - machine learning-how to use the past 20 rows as an input for X for each Y value -


i have simple machine learning code here:

# load dataset dataframe = pandas.read_csv("usdjpy,5.csv", header=none) dataset = dataframe.values x = dataset[:,0:59] y = dataset[:,59] #fit dense keras model model.fit(x, y, validation_data=(x,y_test), epochs=150, batch_size=10)

my x values 59 features 60th column being y value, simple 1 or 0 classification label.

considering using financial data, lookback past 20 x values in order predict y value.

so how make algorithm use past 20 rows input x each y value?

i'm relatively new machine learning , spent time looking online solution problem yet not find simple case.

any ideas?

this typically done recurrent neural networks (rnn), retain memory of previous input, when next input received. thats breif explanation of goes on, there plenty of sources on internet better wrap understanding of how work.

lets break down in simple example. lets 5 samples , 5 features of data, , want 2 stagger data 2 rows instead of 20. here data (assuming 1 stock , oldest price value first). , can think of each row day of week

ar = np.random.randint(10,100,(5,5))  [[43, 79, 67, 20, 13],    #<---monday---  [80, 86, 78, 76, 71],    #<---tuesday---  [35, 23, 62, 31, 59],    #<---wednesday---  [67, 53, 92, 80, 15],    #<---thursday---  [60, 20, 10, 45, 47]]    #<---firday---

to use lstm in keras, data needs 3-d, vs current 2-d structure now, , notation each diminsion (samples,timesteps,features). have (samples,features) need augment data.

a2 = np.concatenate([ar[x:x+2,:] x in range(ar.shape[0]-1)]) a2 = a2.reshape(4,2,5)  [[[43, 79, 67, 20, 13],    #see monday first   [80, 86, 78, 76, 71]],   #see tuesday second ---> predict value set tuesday  [[80, 86, 78, 76, 71],    #see tuesday first   [35, 23, 62, 31, 59]],   #see wednesday second ---> predict value set wednesday  [[35, 23, 62, 31, 59],    #see wednesday value first   [67, 53, 92, 80, 15]],   #see thursday values second ---> predict value set thursday  [[67, 53, 92, 80, 15],    #and on   [60, 20, 10, 45, 47]]])

notice how data staggered , 3 dimensional. make lstm network. y remains 2-d since many-to-one structure, need clip first value. 

model = sequential() model.add(lstm(hidden_dims,input_shape=(a2.shape[1],a2.shape[2])) model.add(dense(1))

this brief example moving. there many different setups work (including not using rnn), need find correct 1 data.


Comments

Post a Comment

Popular posts from this blog

What is happening when Matlab is starting a "parallel pool"? -

running parallel cpu processes in matlab starts command parpool() according documentation , function: [creates] special job on pool of workers, , [connects] matlab client parallel pool. this function takes bit of time execute, on order of 30 seconds. in other multi-cpu paradigms openmp, parallel execution seems totally transparent -- i've never noticed behavior analogous matlab (granted i'm not experienced parallel programming). so, happening between time parpool() called , when finishes executing? takes long? parallel computing toolbox enables run matlab code in parallel using several different paradigms (e.g. jobs , tasks, parfor , spmd , parfeval , batch processing), , run either locally (parallelised across cores in local machine) or remotely (parallelised across machines in cluster - either 1 own, or 1 in cloud). in of these cases, code run on matlab workers , copies of matlab without interactive desktop. if you're intending run on remo...
Read more

angular - DownloadURL return null in below code -

when change file updated in firebase storage when try downloadurl show null. filechange(event: any) { const imagefolder: string = this.employee.id; const filelist: filelist = event.target.files; const file: file = filelist[0]; const storageref = firebase.storage().ref().child(`${imagefolder}/profile.jpg`).put(file); console.log(storageref.snapshot.downloadurl); } the put() asynchronous. if want url after file uploaded have this: firebase.storage().ref().child(`${imagefolder}/profile.jpg`).put(file).then((snapshot) => { storageref = snapshot.downloadurl: console.log(snapshot.downloadurl); });
Read more

php - Cannot override Laravel Spark authentication with own implementation -

during login process of laravel spark need request token of remote api. api stores exact same users , passwords laravel spark application. therefore need username , non hashed password of user during authentication process. i thought overriding authenticated method solution problem. in routes/web.php override post /login endpoint , pointing endpoint own logincontroller: <?php namespace app\http\controllers\auth; use illuminate\support\facades\log; use illuminate\http\request; use laravel\spark\http\controllers\auth\logincontroller sparklogincontroller; class logincontroller extends sparklogincontroller { /** * create new login controller instance. * * @return void */ public function __construct() { parent::__construct(); } /** * handle successful authentication attempt. * * @param request $request * @param \illuminate\contracts\auth\authenticatable $user * @return response */ public fu...
Read more