Skip to content
This repository has been archived by the owner on Feb 11, 2019. It is now read-only.

Contradiction of weight sampling procedure between the code and paper #9

Open
MlWoo opened this issue Sep 5, 2018 · 0 comments
Open

Comments

@MlWoo
Copy link

MlWoo commented Sep 5, 2018

            sess_list1 = self.apply_ + self.sd_asn
            sess_list2 = [self.train_step, self.cross_entropy, self.accuracy]
            result1 = self.sess.run(sess_list1, feed_dict = feed_dict)
            result2 = self.sess.run(sess_list2, feed_dict = feed_dict)

As above, the code will execute as the following sequence,
apply_ : sample a new weight with mean (the original weight) and the standard deviation
sd_asn: update the standard deviation with the gradients
train_step: update the the weight with the gradients
cross_entropy: compute the loss after a forward pass

However, the whole procedure which I got from the paper is:
sd_asn: update the standard deviation with the gradients
train_step: update the the weight with the gradients
apply_ : sample a new weight with mean (the original weight) and the standard deviation
cross_entropy: compute the loss after a forward pass

Do I misunderstand something?

@MlWoo MlWoo changed the title some questions about the code and paper the contradiction of weight sampling procedure between the code and paper Sep 5, 2018
@MlWoo MlWoo changed the title the contradiction of weight sampling procedure between the code and paper Contradiction of weight sampling procedure between the code and paper Sep 5, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant