Mixing
How to pass a intermediate tensor to a neural network layer inside the body of a while-loop?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
2
I want to pass an intermediate tensor result into a fully connected neural network inside the body of a while-loop.
My issues are with the instantiating of said neural network.
My first try involved making a placeholder for the NN, then feed it using a Session instance. It does not work, because the feed_dict doesn't accept tensors. Fair enough, given the data-flow nature of the graph.
On my second attempt, I instantiated my NN inside the loop body and pass the intermediate tensor directly.
However, when I do so, the following stacktrace shows up:
Caused by op u'net/fc1_W/read', defined at:
File "./main.py", line 176, in <module>
offline_indexing(sys.argv[1])
File "./main.py", line 128, in offline_indexing
test.run(a, f)
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 86, in run
print sess.run(graph_embed(), feed_dict={adj: x, features: y})
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 73, in graph_embed
final_mus = tf.while_loop(cond, body, [mus, features, adj, 0])[0]
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3291, in while_loop
return_same_structure)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3004, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2939, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 63, in body
h2 = fc_NN(mu_neigh_sum)
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 53, in fc_NN
fc1 = fc_layer(ac2, 64, "fc1")
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 43, in fc_layer
W = tf.get_variable(name+'_W', dtype=tf.float32, shape=[embedding_size, embedding_size], initializer=initer)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1487, in get_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1237, in get_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 540, in get_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 492, in _true_getter
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 922, in _get_single_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 183, in __call__
return cls._variable_v1_call(*args, **kwargs)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 146, in _variable_v1_call
aggregation=aggregation)
FailedPreconditionError (see above for traceback): Attempting to use uninitialized value net/fc1_W
[[node net/fc1_W/read (defined at /home/lsv/Desktop/gitlab/Gencoding/test.py:43) = Identity[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](net/fc1_W)]]
Note that the use of Dense layers result in the same error as well.
Here is my code:
def fc_layer(bottom, n_weight, name):
assert len(bottom.get_shape()) == 2
n_prev_weight = bottom.get_shape()[1]
initer = tf.truncated_normal_initializer(stddev=0.01)
W = tf.get_variable(name+'_W', dtype=tf.float32, shape=[embedding_size, embedding_size], initializer=initer)
b = tf.get_variable(name+'_b', dtype=tf.float32, shape=[embedding_size], initializer=tf.zeros_initializer)
fc = tf.nn.bias_add(tf.matmul(bottom, W), b)
return fc
def fc_NN(x):
fc2 = fc_layer(x, 64, "fc2")
ac2 = tf.nn.relu(fc2)
fc1 = fc_layer(ac2, 64, "fc1")
return fc1
fcnn = fc_NN()
def cond(m, f, a, i):
return tf.less(i, T)
def body(m, f, a, i):
mu_neigh_sum = tf.tensordot(a, m, 1)
h1 = tf.matmul(f, W1)
# First incriminating try - with x as a placeholder
h2 = tf.Session().run(fcnn, {x: mu_neigh_sum})
# Second incriminating try - with x as a tensor
h2bis = fc_NN(mu_neigh_sum)
return tf.tanh(h1 + h2), f, a, i+1
final_mus = tf.while_loop(cond, body, [mus, features, adj, 0])[0]
python tensorflow
asked Jan 3 at 10:12
scavlscavl
111
add a comment |
2
I want to pass an intermediate tensor result into a fully connected neural network inside the body of a while-loop.
My issues are with the instantiating of said neural network.
My first try involved making a placeholder for the NN, then feed it using a Session instance. It does not work, because the feed_dict doesn't accept tensors. Fair enough, given the data-flow nature of the graph.
On my second attempt, I instantiated my NN inside the loop body and pass the intermediate tensor directly.
However, when I do so, the following stacktrace shows up:
Caused by op u'net/fc1_W/read', defined at:
File "./main.py", line 176, in <module>
offline_indexing(sys.argv[1])
File "./main.py", line 128, in offline_indexing
test.run(a, f)
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 86, in run
print sess.run(graph_embed(), feed_dict={adj: x, features: y})
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 73, in graph_embed
final_mus = tf.while_loop(cond, body, [mus, features, adj, 0])[0]
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3291, in while_loop
return_same_structure)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3004, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2939, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 63, in body
h2 = fc_NN(mu_neigh_sum)
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 53, in fc_NN
fc1 = fc_layer(ac2, 64, "fc1")
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 43, in fc_layer
W = tf.get_variable(name+'_W', dtype=tf.float32, shape=[embedding_size, embedding_size], initializer=initer)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1487, in get_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1237, in get_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 540, in get_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 492, in _true_getter
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 922, in _get_single_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 183, in __call__
return cls._variable_v1_call(*args, **kwargs)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 146, in _variable_v1_call
aggregation=aggregation)
FailedPreconditionError (see above for traceback): Attempting to use uninitialized value net/fc1_W
[[node net/fc1_W/read (defined at /home/lsv/Desktop/gitlab/Gencoding/test.py:43) = Identity[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](net/fc1_W)]]
Note that the use of Dense layers result in the same error as well.
Here is my code:
def fc_layer(bottom, n_weight, name):
assert len(bottom.get_shape()) == 2
n_prev_weight = bottom.get_shape()[1]
initer = tf.truncated_normal_initializer(stddev=0.01)
W = tf.get_variable(name+'_W', dtype=tf.float32, shape=[embedding_size, embedding_size], initializer=initer)
b = tf.get_variable(name+'_b', dtype=tf.float32, shape=[embedding_size], initializer=tf.zeros_initializer)
fc = tf.nn.bias_add(tf.matmul(bottom, W), b)
return fc
def fc_NN(x):
fc2 = fc_layer(x, 64, "fc2")
ac2 = tf.nn.relu(fc2)
fc1 = fc_layer(ac2, 64, "fc1")
return fc1
fcnn = fc_NN()
def cond(m, f, a, i):
return tf.less(i, T)
def body(m, f, a, i):
mu_neigh_sum = tf.tensordot(a, m, 1)
h1 = tf.matmul(f, W1)
# First incriminating try - with x as a placeholder
h2 = tf.Session().run(fcnn, {x: mu_neigh_sum})
# Second incriminating try - with x as a tensor
h2bis = fc_NN(mu_neigh_sum)
return tf.tanh(h1 + h2), f, a, i+1
final_mus = tf.while_loop(cond, body, [mus, features, adj, 0])[0]
python tensorflow
asked Jan 3 at 10:12
scavlscavl
111
add a comment |
2
2
2
I want to pass an intermediate tensor result into a fully connected neural network inside the body of a while-loop.
My issues are with the instantiating of said neural network.
My first try involved making a placeholder for the NN, then feed it using a Session instance. It does not work, because the feed_dict doesn't accept tensors. Fair enough, given the data-flow nature of the graph.
On my second attempt, I instantiated my NN inside the loop body and pass the intermediate tensor directly.
However, when I do so, the following stacktrace shows up:
Caused by op u'net/fc1_W/read', defined at:
File "./main.py", line 176, in <module>
offline_indexing(sys.argv[1])
File "./main.py", line 128, in offline_indexing
test.run(a, f)
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 86, in run
print sess.run(graph_embed(), feed_dict={adj: x, features: y})
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 73, in graph_embed
final_mus = tf.while_loop(cond, body, [mus, features, adj, 0])[0]
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3291, in while_loop
return_same_structure)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3004, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2939, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 63, in body
h2 = fc_NN(mu_neigh_sum)
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 53, in fc_NN
fc1 = fc_layer(ac2, 64, "fc1")
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 43, in fc_layer
W = tf.get_variable(name+'_W', dtype=tf.float32, shape=[embedding_size, embedding_size], initializer=initer)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1487, in get_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1237, in get_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 540, in get_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 492, in _true_getter
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 922, in _get_single_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 183, in __call__
return cls._variable_v1_call(*args, **kwargs)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 146, in _variable_v1_call
aggregation=aggregation)
FailedPreconditionError (see above for traceback): Attempting to use uninitialized value net/fc1_W
[[node net/fc1_W/read (defined at /home/lsv/Desktop/gitlab/Gencoding/test.py:43) = Identity[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](net/fc1_W)]]
Note that the use of Dense layers result in the same error as well.
Here is my code:
def fc_layer(bottom, n_weight, name):
assert len(bottom.get_shape()) == 2
n_prev_weight = bottom.get_shape()[1]
initer = tf.truncated_normal_initializer(stddev=0.01)
W = tf.get_variable(name+'_W', dtype=tf.float32, shape=[embedding_size, embedding_size], initializer=initer)
b = tf.get_variable(name+'_b', dtype=tf.float32, shape=[embedding_size], initializer=tf.zeros_initializer)
fc = tf.nn.bias_add(tf.matmul(bottom, W), b)
return fc
def fc_NN(x):
fc2 = fc_layer(x, 64, "fc2")
ac2 = tf.nn.relu(fc2)
fc1 = fc_layer(ac2, 64, "fc1")
return fc1
fcnn = fc_NN()
def cond(m, f, a, i):
return tf.less(i, T)
def body(m, f, a, i):
mu_neigh_sum = tf.tensordot(a, m, 1)
h1 = tf.matmul(f, W1)
# First incriminating try - with x as a placeholder
h2 = tf.Session().run(fcnn, {x: mu_neigh_sum})
# Second incriminating try - with x as a tensor
h2bis = fc_NN(mu_neigh_sum)
return tf.tanh(h1 + h2), f, a, i+1
final_mus = tf.while_loop(cond, body, [mus, features, adj, 0])[0]
python tensorflow
asked Jan 3 at 10:12
scavlscavl
111
I want to pass an intermediate tensor result into a fully connected neural network inside the body of a while-loop.
My issues are with the instantiating of said neural network.
My first try involved making a placeholder for the NN, then feed it using a Session instance. It does not work, because the feed_dict doesn't accept tensors. Fair enough, given the data-flow nature of the graph.
On my second attempt, I instantiated my NN inside the loop body and pass the intermediate tensor directly.
However, when I do so, the following stacktrace shows up:
Caused by op u'net/fc1_W/read', defined at:
File "./main.py", line 176, in <module>
offline_indexing(sys.argv[1])
File "./main.py", line 128, in offline_indexing
test.run(a, f)
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 86, in run
print sess.run(graph_embed(), feed_dict={adj: x, features: y})
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 73, in graph_embed
final_mus = tf.while_loop(cond, body, [mus, features, adj, 0])[0]
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3291, in while_loop
return_same_structure)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3004, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2939, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 63, in body
h2 = fc_NN(mu_neigh_sum)
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 53, in fc_NN
fc1 = fc_layer(ac2, 64, "fc1")
File "/home/lsv/Desktop/gitlab/Gencoding/test.py", line 43, in fc_layer
W = tf.get_variable(name+'_W', dtype=tf.float32, shape=[embedding_size, embedding_size], initializer=initer)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1487, in get_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1237, in get_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 540, in get_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 492, in _true_getter
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 922, in _get_single_variable
aggregation=aggregation)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 183, in __call__
return cls._variable_v1_call(*args, **kwargs)
File "/home/lsv/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 146, in _variable_v1_call
aggregation=aggregation)
FailedPreconditionError (see above for traceback): Attempting to use uninitialized value net/fc1_W
[[node net/fc1_W/read (defined at /home/lsv/Desktop/gitlab/Gencoding/test.py:43) = Identity[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](net/fc1_W)]]
Note that the use of Dense layers result in the same error as well.
Here is my code:
def fc_layer(bottom, n_weight, name):
assert len(bottom.get_shape()) == 2
n_prev_weight = bottom.get_shape()[1]
initer = tf.truncated_normal_initializer(stddev=0.01)
W = tf.get_variable(name+'_W', dtype=tf.float32, shape=[embedding_size, embedding_size], initializer=initer)
b = tf.get_variable(name+'_b', dtype=tf.float32, shape=[embedding_size], initializer=tf.zeros_initializer)
fc = tf.nn.bias_add(tf.matmul(bottom, W), b)
return fc
def fc_NN(x):
fc2 = fc_layer(x, 64, "fc2")
ac2 = tf.nn.relu(fc2)
fc1 = fc_layer(ac2, 64, "fc1")
return fc1
fcnn = fc_NN()
def cond(m, f, a, i):
return tf.less(i, T)
def body(m, f, a, i):
mu_neigh_sum = tf.tensordot(a, m, 1)
h1 = tf.matmul(f, W1)
# First incriminating try - with x as a placeholder
h2 = tf.Session().run(fcnn, {x: mu_neigh_sum})
# Second incriminating try - with x as a tensor
h2bis = fc_NN(mu_neigh_sum)
return tf.tanh(h1 + h2), f, a, i+1
final_mus = tf.while_loop(cond, body, [mus, features, adj, 0])[0]
python tensorflow
python tensorflow
asked Jan 3 at 10:12
scavlscavl
111
asked Jan 3 at 10:12
scavlscavl
111
asked Jan 3 at 10:12
scavlscavl
111
asked Jan 3 at 10:12
scavlscavl
111
asked Jan 3 at 10:12
scavlscavl
111
111
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54020164%2fhow-to-pass-a-intermediate-tensor-to-a-neural-network-layer-inside-the-body-of-a%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54020164%2fhow-to-pass-a-intermediate-tensor-to-a-neural-network-layer-inside-the-body-of-a%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
