Odtnhj

I have four sets of TensorFlow Variables.

I am using these variables in an autoencoder based on the archtiecture found in this paper: http://users.cecs.anu.edu.au/~u5098633/papers/www15.pdf

Briefly, only the weights that are connected to certain input nodes are important for training. That is, if we have 10 input nodes, I only want to focus on, say, 5 of these nodes, and the weights that are connected to these nodes. My variable that models these connections is:

self.V = tf.Variable(tf.truncated_normal([input_size, hidden_size]), name='V') 

I am able to get these particular weights by using:

weights_to_update = tf.gather(self.V, [indices_of_weights]).

Now, to get the particular gradients of these weights, I have tried using

grads_and_vars = tf.Optimizer.compute_gradients(my_loss_function, weights_to_update)

which I thought would calculate the gradients of these particular weights, and ignore all other weights in the self.V variable. I would then use tf.Optimizer.apply_gradients(graqds_and_vars) to perform the update step.

However, the output I get from using compute_gradients with the var_list parameter as the weights I want is:

[(None, <tf.Tensor 'GatherV2:0' shape=(20, 200) dtype=float32>)] 

which shows that the shape is correct for what I want, but that this is None values. Subsequently, calling apply_gradients with what this returns leads to an error.

I have searched all over the internet to find out how to compute/apply gradients to a subset of a tf.Variable, but I cannot find out how to do this anywhere.

So, to sum up, for each training step, I would like to know how to train only a subset of parameters from each of my tf.Variable objects in my graph.