python - Loss is equal to 0 from the beginning -


i'm trying attempt titanic kaggle competition using tensorflow.

my pre processed train data looks this:

data_x:  passengerid  pclass  sex   age  sibsp  parch  ticket fare  cabin  \ embarked 1              2       1    1  38.0      1      0     500   71.2833    104 2              3       3    1  26.0      0      0     334    7.9250      0 3              4       1    1  35.0      1      0     650   53.1000    130 4              5       3    0  35.0      0      0     638    8.0500      0  data_y:  survived 0 1 1 1 0

a softmax function should work predict if passenger survived or not since it's binary, right?

so here how build model:

x = tf.placeholder(tf.float32, [none, data_x.shape[1]]) y_ = tf.placeholder(tf.float32, [none, 1])  w = tf.variable(tf.truncated_normal([10, 1])) b = tf.variable(tf.zeros([1]))  # parameters learning_rate = 0.001  #the model y = tf.matmul(x,w) + b  # loss function entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y) loss = tf.reduce_mean(entropy) # computes mean on examples in batch  optimizer = tf.train.gradientdescentoptimizer(learning_rate).minimize(loss)  acc = tf.equal(tf.argmax(y_, 1), tf.argmax(y, 1)) acc = tf.reduce_mean(tf.cast(acc, tf.float32))  tf.summary.scalar('loss', loss) tf.summary.scalar('accuracy', acc) merged_summary = tf.summary.merge_all()  init = tf.global_variables_initializer()

and finallyn, training part:

with tf.session() sess:     sess.run(init)     writer = tf.summary.filewriter("./graphs", sess.graph)     in range(1000):         _, l, summary = sess.run([optimizer, loss, merged_summary], feed_dict={x: data_x, y_: data_y})         writer.add_summary(summary, i)         if i%100 == 0:             print (i)             print ("loss = ", l)

but loss equals 0 since first step...

here tensorboard visualization:

enter image description here

any idea what's going on here?

actually, think got idea of softmax wrong. transformes outputs such probability distribution. however, output 1 neuron, softmax transforms 1.

if want softmax + cross entropy logits, need ouput 2 neurons, 1 probability of prediction being 1 (positive), 1 probability of being 0 (negative). need change labels, such positive example has label [1, 0], negative [0, 1]. then, can use cross entropy , should work.

edit: option might use tf.nn.sigmoid_cross_entropy_with_logits loss function. sigmoid transformation [0, 1] interval need cross-entropy, , doesn't worry (possible) other outputs. way, work current labels , architecture.


Comments

Post a Comment

Popular posts from this blog

Is there a better way to structure post methods in Class Based Views -

i'm using class based views in django , never specify url in ajax calls , omit action parameter in form because know these requests go through post method of class based view. notice view - profile page make many requests depending on user updating on page - post method become convoluted , if/else mess example: def post(self, request, *args, **kwargs): # handle user changing available date if request.post.get('availabledate') != none: self.updateavailabledate(request) return jsonresponse({'result':'success'}) if request.post.get('newprojectname') != none: creative_user = creativeuserprofile.objects.get(id = request.user.id) project = project.create_project(creative_user, request.post['newprojectname']) project.save() return jsonresponse({'projectid': project.id }) if request.post.get('projectimage') != none , request.files['file']: proj
Read more

performance - Why is XCHG reg, reg a 3 micro-op instruction on modern Intel architectures? -

i'm doing micro-optimization on performance critical part of code , came across sequence of instructions (in at&t syntax): add %rax, %rbx mov %rdx, %rax mov %rbx, %rdx i thought had use case xchg allow me shave instruction , write: add %rbx, %rax xchg %rax, %rdx however, dimay found agner fog's instruction tables , xchg 3 micro-op instruction 2 cycle latency on sandy bridge, ivy bridge, broadwell, haswell , skylake. 3 whole micro-ops , 2 cycles of latency! 3 micro-ops throws off 4-1-1-1 cadence , 2 cycle latency makes worse original in best case since last 2 instructions in original might execute in parallel. now... cpu might breaking instruction micro-ops equivalent to: mov %rax, %tmp mov %rdx, %rax mov %tmp, %rdx where tmp anonymous internal register , suppose last 2 micro-ops run in parallel latency 2 cycles. given register renaming occurs on these micro-architectures, though, doesn't make sense me done way. why wouldn't register renamer
Read more

jquery - Responsive Navbar with Sub Navbar -

good day, forgive me nav vocabulary not best. if question confusing apologize. i'm working on navigation bar needs centered on page @ 768px , above. items in nav open (or perhaps toggle) subnav lives directly below. these nav should appear centered on page. i have been able work through responsive portion of nav bar, having main items align vertically , sub nav menu of each item roll out below, , gave them indent separation. working well. however, issue not responsive side, behavior @ width greater 768px. happens toggling menu's sub navigation pushes other main items down page below sub nav. here's code: $(document).ready(function () { //stick in fixed 100% height behind navbar don't wrap $('#slide-nav.navbar-inverse').after($('<div class="inverse" id="navbar-height-col"></div>')); $('#slide-nav.navbar-default').after($('<div id="navbar-height-col"></d
Read more