18 دیدگاه برای “Neural networks [7.7] : Deep learning – deep belief network”

  3. In the first step the first two layers(input- h1) are trained as a RBM, in the next step we train h1- h2 as a RBM, while input-h1 now behaves as a SBN, and so on. When you say you are generating samples of observations (in the first half of the video, when you talk about Gibbs sampling) , do you mean that the input is being reconstructed as a part of this network training? Thanks.

  4. Hello, Mr. Hugo.

    I have two quick questions.
    1. After initializing the weights of the 1st two layers with RBM, we update the weights of 2nd two layers with RBM as well.
    In this case, when we train the weights of the 2nd two layers, the weights of the 1st two layers are remained or changed?
    2. When we do fine-tuning (supervised learning between the last hidden layer and the output layer), all the weights of previous layers are remained or updated?

    Thank you for the high quality lectures!

  5. could u please tell me why we should use DBN?
    Is it better than CNN in term of supervise learning??
    If we complete training a DBN network, what can we do further with trained DBN? auto-encoders and auto-reconstruct?

  6. Hi Hugo,
     I am confused between deep autoencoders and deep belief networks. If you go to this link: http://www.cs.toronto.edu/~rsalakhu/code.html where Geoffrey Hinton and Ruslan Salakhutdinov have provided their Matlab implementations, and if you go to the "Deep Belief Networks" link, you see the implementation of deep autoencoders. Also, in the slides provided by Prof. Tom Mitchell of CMU here: http://www.cs.cmu.edu/~tom/10701_sp11/slides/DimensionalityReduction_03_31_2011_ann.pdf , you will find that he shows the same diagram of deep autoencoders that you have, but under the heading "Deep Belief Networks". I am confused how to go about implementing deep belief network( my aim is to train it for speech, for learning purposes). Your suggestion here will be of significant importance to me.

    Thanks.

  7. Hi Hugo! Very good lectures!

    I got I little bit confused. You say at 3:38 that sampling from the model requires a lot of gibbs sampling at the top two layers. But you got h^3 only after stacking the previous two RBMs, right? Thus, does this gibbs sampling performed in training phase, only after training the two previous RBMs?

    Also, if I understood, you are defining a DBN as a model to reconstruct a sample, right? Given that, how the reconstruction is performed? Sigmoids from x to h^1, then from h^1 to h^2, then reconstruction using top RBM, then sigmoid back from h^2 to h^1, and from h^1 to x?

    Thanks,
    Ricardo

  8. Hey. Thanks for a great series. Just one question. Where do i find a good/simple explanation of Up Down algorithm (i've seen Hinton's faster learning algorithm paper) or could you do a related video?

  11. Hi Hugo,
    One question about the joint distribution of DBN. In two hidden layer DBN case, why is the joint distribution  p(x, h^1, h^2) = p(h^1, h^2) p(x|h^1)?

    I mean that p(x, h^1, h^2) = p(h^1, h^2) p(x| h^1, h^2) without considering the conditional independence. But why x and h^2 are independent conditional on h^1 such that p(x|h^1, h^2) = p(x|h^1) in DBN case? In fully directed model, I know I can utilize D-separate technique to find the conditional independence. However, in a mixture of directed and indirected model (DBN), how to I expoit the conditional independence? Thank you.

    Paul

  13. Hey there,

    I just wanted to ask how the higher layers are trained. These directed edges are confusing me. Do the RBMs in higher layers still train on top of the data representations  of the RBMs in previous layers and then thereafter 
    the fine tuning is done with the Up-Down algorithm.

    So the Sigmoid Belief Network in conjunction with the RBM is something like the data generation mode of the DBN?

    Thx,

  14. Hi Hugo,

    I am confused about the essence of Directed Network? Why did DBN follow the specific approach (Top Level RBM, remaining Layers are Directed)? Can you explain this?

  15. Hi Hugo, it would be great if you can incorporate some coding examples in you next version of lecture! I think you can simply use the code from the tutorials on deeplearning.net, walk through the parts that covered by the particular lecture and run a demo. It will help us understand the materials better, and pick up theano.

  16. Thanks Hugo
    A question. Why Sigmoid? I have found answers that talk about the Sigmoid being analytical and allowing non-linear boundaries. Are there other more specific properties for a sigmoid to be chosen?

  17. Hi Hugo,
    If you are taking questions still in this video, I have a doubt.
    If we are training DBNs via an RBM, wouldn't it be Deep Boltzmann machine? What is the exact difference between DBNs and DBMs? Can I know a usecase for DBN so that I could distinguish it from Deep Boltzmann Machine

دیدگاه‌ها بسته شده‌اند.