neural network - How to have a variable number of hidden layers in Tensorflow? -

suppose want try sort of hidden layer numbers , size. how can in tensorflow?

consider following example make clear:

# create  neural network layer  def fc_layer(input, size_in, size_out):         w = tf.variable(tf.truncated_normal([none, size_in, size_out]), name="w")         b = tf.variable(tf.constant(0.1, shape=[size_out]))         act = tf.matmul(input, w) + b         return act n_hiddenlayers=3 #number of hidden layers hidden_layer=tf.placeholder(tf.float32,[n_hiddenlayers, none, none]) #considering 4 size of inputs , outputs of layers sizeinpout=4 in range(n_hiddenlayers):     hidden_layer(i,:,:)= tf.nn.sigmoid(fc_layer(x, sizeinpout, sizeinpout)) 

it results in error hidden_layer(i,:,:)= ... in other word, need tensor of tensors.

not direct answer, consider using tensorflow-slim. it's 1 of many apis distributed part of tensorflow. lightweight , compatible defining variables hand doing. if @ webpage linked, slim.repeat , slim.stack allow create multiple layers of different widths in 1 line. make things more complicated: think part of slim module called layers in tensorflow.

but maybe want play directly tf variables understand how works , not use higher level api until later.

in code posted, since want create 3 layers, should call fc_layer 3 times, call once. way implies w , b created 3 different times, different variables different internal tf names. , want.

you should have for-loop or while-loop iterates 3 times. note output tensor @ end of loop become input tensor in next iteration. initial input true input , last output true output.

another issue code non-linearity (the sigmoid) should @ end of fc_layer. want non-linear operation between layers.

---

edit: code of done:

import tensorflow tf  input_size = 10 output_size = 4 layer_sizes  = [7, 6, 5]  def fc_layer(input, size, layer_name):     in_size = input.shape.as_list()[1]     w = tf.variable(tf.truncated_normal([in_size, size]),                     name="w" + layer_name)     b = tf.variable(tf.constant(0.1, shape=[size]),                     name="b" + layer_name)     act = tf.nn.sigmoid(tf.matmul(input, w) + b)     return act  input = tf.placeholder(tf.float32, [none, input_size]) # output intermediate activations successively , in end # final activations (output). output = input i, size in enumerate(layer_sizes + [output_size]):     output = fc_layer(output , size, layer_name=str(i + 1))  print("final output var: " + str(output)) print("all vars in tensorflow graph:") var in tf.global_variables():     print(var) 

with output:

final output: tensor("sigmoid_3:0", shape=(?, 4), dtype=float32)  <tf.variable 'w1:0' shape=(10, 7) dtype=float32_ref> <tf.variable 'b1:0' shape=(7,) dtype=float32_ref> <tf.variable 'w2:0' shape=(7, 6) dtype=float32_ref> <tf.variable 'b2:0' shape=(6,) dtype=float32_ref> <tf.variable 'w3:0' shape=(6, 5) dtype=float32_ref> <tf.variable 'b3:0' shape=(5,) dtype=float32_ref> <tf.variable 'w4:0' shape=(5, 4) dtype=float32_ref> <tf.variable 'b4:0' shape=(4,) dtype=float32_ref> 

in code using same name w, creates conflicts since different variables same name created. fixed in code, if use same name tensorflow intelligent enough , rename each variable unique name adding underscore , number.

---

edit: here think wanted do:

import tensorflow tf  hidden_size = 4 input_size = hidden_size  # equality required! output_size = hidden_size # equality required! n_hidden = 3  meta_tensor = tf.variable(tf.truncated_normal([n_hidden, hidden_size, hidden_size]),                     name="meta")  def fc_layer(input, i_layer):     w = meta_tensor[i_layer]     # more verbose: w = tf.slice(meta_tensor, begin=[i_layer, 0, 0], size=[1, hidden_size, hidden_size])[0]      b = tf.variable(tf.constant(0.1, shape=[hidden_size]),                     name="b" + str(i_layer))     act = tf.nn.sigmoid(tf.matmul(input, w) + b)     return act  input = tf.placeholder(tf.float32, [none, input_size]) # output intermediate activations successively , in end # final activations (output). output = input i_layer in range(0, n_hidden):     output = fc_layer(output, i_layer)  print("final output var: " + str(output)) print("all vars in tensorflow graph:") var in tf.global_variables():     print(var) 

with output:

final output var: tensor("sigmoid_2:0", shape=(?, 4), dtype=float32)  vars in tensorflow graph: <tf.variable 'meta:0' shape=(3, 4, 4) dtype=float32_ref> <tf.variable 'b0:0' shape=(4,) dtype=float32_ref> <tf.variable 'b1:0' shape=(4,) dtype=float32_ref> <tf.variable 'b2:0' shape=(4,) dtype=float32_ref> 

as said not standard. while coding realized quite limiting since hidden layers must have same size. meta-tensor can used store many matrices, must have same dimensions. not did in example above hidden first layer has size 7 , next 1 size 6 , final 1 size 5, before output of size 4.