lecture 12 visualization
Q1 why does visualizing the weights of the filters tell you what the filter is looking for?
template match. (filter = template)
the input which maximizes that activation under a norm constraint on the input is exactly when those two vectors (input + filter) matches up.
visualization of second last layers:
L2 nearest neighbors in feature space can be same types ( tree on the left or tree on the right)
visualization of middle layers
select images with maximal activations in exact channels of exact layers (the selected images are usually same)
mask part of the image before feeding into CNN
saliency maps
- which pixels matter?
- computer gradient of output/input directly
deep dream
set feature maps as the (backpropagation) gradient to maximize the features found by DNN
neural texture synthesis
generate a big texture from a small texture(image)
- how to compute Gram Matrix(also for style transfer)?:
- get a feature map through DNN with size ($C\times H\times W$)
- generate $cchw$ matrix for each pair of c-dimensional vectors measuring co-occurrence ($new_{ij} = old1_iold2_j$)
- average from cchw to cc
- using random image to generate gram matrix and compare it with input’s gram matrix( loss function to compare)
style transfer
- gram matrix is also used
- but the iteration may be too large (random image as initial image), solution: train a DNN(feedforward net in the image) ,input: raw image output: style image and the weight is updated by throwing the output into the gram matrix calculation procedure.
