how to create sparseVector with index and value in Tensorflow, python?










0














i have two Tensors, like this:



>>> xx_idx
<tf.Tensor 'Placeholder:0' shape=(100, ?) dtype=int64>
>>> xx_val
<tf.Tensor 'Placeholder_1:0' shape=(100, ?) dtype=float64>


How to create a SparseTensor from them? xx_idx are the indexes, xx_val are the values.
There are 100 samples.
The dimension of the vector is unknown, maybe 22000.



I tried this:



xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)


but here comes the error:



Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'user_idx' is not defined
>>> xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 671, in merge_with
self.assert_same_rank(other)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 716, in assert_same_rank
other))
ValueError: Shapes (100, ?) and (?,) must have the same rank

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 746, in with_rank
return self.merge_with(unknown_shape(ndims=rank))
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 677, in merge_with
raise ValueError("Shapes %s and %s are not compatible" % (self, other))
ValueError: Shapes (100, ?) and (?,) are not compatible

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/sparse_tensor.py", line 133, in __init__
values_shape = values.get_shape().with_rank(1)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 748, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (100, ?) must have rank 1









share|improve this question





















  • Possible duplicate of Sparse Matrix from a dense one Tensorflow
    – jdehesa
    Nov 12 at 14:07










  • thx, problem solved, I post an answer
    – hvchys
    Nov 13 at 7:40















0














i have two Tensors, like this:



>>> xx_idx
<tf.Tensor 'Placeholder:0' shape=(100, ?) dtype=int64>
>>> xx_val
<tf.Tensor 'Placeholder_1:0' shape=(100, ?) dtype=float64>


How to create a SparseTensor from them? xx_idx are the indexes, xx_val are the values.
There are 100 samples.
The dimension of the vector is unknown, maybe 22000.



I tried this:



xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)


but here comes the error:



Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'user_idx' is not defined
>>> xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 671, in merge_with
self.assert_same_rank(other)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 716, in assert_same_rank
other))
ValueError: Shapes (100, ?) and (?,) must have the same rank

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 746, in with_rank
return self.merge_with(unknown_shape(ndims=rank))
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 677, in merge_with
raise ValueError("Shapes %s and %s are not compatible" % (self, other))
ValueError: Shapes (100, ?) and (?,) are not compatible

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/sparse_tensor.py", line 133, in __init__
values_shape = values.get_shape().with_rank(1)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 748, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (100, ?) must have rank 1









share|improve this question





















  • Possible duplicate of Sparse Matrix from a dense one Tensorflow
    – jdehesa
    Nov 12 at 14:07










  • thx, problem solved, I post an answer
    – hvchys
    Nov 13 at 7:40













0












0








0







i have two Tensors, like this:



>>> xx_idx
<tf.Tensor 'Placeholder:0' shape=(100, ?) dtype=int64>
>>> xx_val
<tf.Tensor 'Placeholder_1:0' shape=(100, ?) dtype=float64>


How to create a SparseTensor from them? xx_idx are the indexes, xx_val are the values.
There are 100 samples.
The dimension of the vector is unknown, maybe 22000.



I tried this:



xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)


but here comes the error:



Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'user_idx' is not defined
>>> xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 671, in merge_with
self.assert_same_rank(other)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 716, in assert_same_rank
other))
ValueError: Shapes (100, ?) and (?,) must have the same rank

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 746, in with_rank
return self.merge_with(unknown_shape(ndims=rank))
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 677, in merge_with
raise ValueError("Shapes %s and %s are not compatible" % (self, other))
ValueError: Shapes (100, ?) and (?,) are not compatible

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/sparse_tensor.py", line 133, in __init__
values_shape = values.get_shape().with_rank(1)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 748, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (100, ?) must have rank 1









share|improve this question













i have two Tensors, like this:



>>> xx_idx
<tf.Tensor 'Placeholder:0' shape=(100, ?) dtype=int64>
>>> xx_val
<tf.Tensor 'Placeholder_1:0' shape=(100, ?) dtype=float64>


How to create a SparseTensor from them? xx_idx are the indexes, xx_val are the values.
There are 100 samples.
The dimension of the vector is unknown, maybe 22000.



I tried this:



xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)


but here comes the error:



Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'user_idx' is not defined
>>> xx_vec = tf.SparseTensor(xx_idx, xx_val, 25000)
Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 671, in merge_with
self.assert_same_rank(other)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 716, in assert_same_rank
other))
ValueError: Shapes (100, ?) and (?,) must have the same rank

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 746, in with_rank
return self.merge_with(unknown_shape(ndims=rank))
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 677, in merge_with
raise ValueError("Shapes %s and %s are not compatible" % (self, other))
ValueError: Shapes (100, ?) and (?,) are not compatible

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/sparse_tensor.py", line 133, in __init__
values_shape = values.get_shape().with_rank(1)
File "/home/work/tf/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 748, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (100, ?) must have rank 1






python tensorflow vector






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 12 at 13:01









hvchys

2127




2127











  • Possible duplicate of Sparse Matrix from a dense one Tensorflow
    – jdehesa
    Nov 12 at 14:07










  • thx, problem solved, I post an answer
    – hvchys
    Nov 13 at 7:40
















  • Possible duplicate of Sparse Matrix from a dense one Tensorflow
    – jdehesa
    Nov 12 at 14:07










  • thx, problem solved, I post an answer
    – hvchys
    Nov 13 at 7:40















Possible duplicate of Sparse Matrix from a dense one Tensorflow
– jdehesa
Nov 12 at 14:07




Possible duplicate of Sparse Matrix from a dense one Tensorflow
– jdehesa
Nov 12 at 14:07












thx, problem solved, I post an answer
– hvchys
Nov 13 at 7:40




thx, problem solved, I post an answer
– hvchys
Nov 13 at 7:40












1 Answer
1






active

oldest

votes


















0














problem solved



import tensorflow as tf
from tensorflow import TensorShape, Dimension


class GetVector:

@classmethod
def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
batch_size = idx_all_0.shape[0].value
'''
cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
cur_val = tf.placeholder(tf.float64, [batch_size, None])
cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
'''
idx_all = cls.idx_reform(idx_all_0)

ans =
for i in range(batch_size):
cur_idx = idx_all[i]
cur_val = val_all[i]
cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
ans = tf.concat([ans, cur_vec_tensor], 0)
ans = tf.reshape(ans, [batch_size, dim_num])
return ans

@classmethod
def idx_reform(cls, idx_all):
batch_size = idx_all.shape[0].value
return tf.reshape(idx_all, [batch_size, -1, 1])





share|improve this answer




















    Your Answer






    StackExchange.ifUsing("editor", function ()
    StackExchange.using("externalEditor", function ()
    StackExchange.using("snippets", function ()
    StackExchange.snippets.init();
    );
    );
    , "code-snippets");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "1"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53262744%2fhow-to-create-sparsevector-with-index-and-value-in-tensorflow-python%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    problem solved



    import tensorflow as tf
    from tensorflow import TensorShape, Dimension


    class GetVector:

    @classmethod
    def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
    batch_size = idx_all_0.shape[0].value
    '''
    cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
    cur_val = tf.placeholder(tf.float64, [batch_size, None])
    cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
    '''
    idx_all = cls.idx_reform(idx_all_0)

    ans =
    for i in range(batch_size):
    cur_idx = idx_all[i]
    cur_val = val_all[i]
    cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
    cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
    ans = tf.concat([ans, cur_vec_tensor], 0)
    ans = tf.reshape(ans, [batch_size, dim_num])
    return ans

    @classmethod
    def idx_reform(cls, idx_all):
    batch_size = idx_all.shape[0].value
    return tf.reshape(idx_all, [batch_size, -1, 1])





    share|improve this answer

























      0














      problem solved



      import tensorflow as tf
      from tensorflow import TensorShape, Dimension


      class GetVector:

      @classmethod
      def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
      batch_size = idx_all_0.shape[0].value
      '''
      cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
      cur_val = tf.placeholder(tf.float64, [batch_size, None])
      cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
      '''
      idx_all = cls.idx_reform(idx_all_0)

      ans =
      for i in range(batch_size):
      cur_idx = idx_all[i]
      cur_val = val_all[i]
      cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
      cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
      ans = tf.concat([ans, cur_vec_tensor], 0)
      ans = tf.reshape(ans, [batch_size, dim_num])
      return ans

      @classmethod
      def idx_reform(cls, idx_all):
      batch_size = idx_all.shape[0].value
      return tf.reshape(idx_all, [batch_size, -1, 1])





      share|improve this answer























        0












        0








        0






        problem solved



        import tensorflow as tf
        from tensorflow import TensorShape, Dimension


        class GetVector:

        @classmethod
        def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
        batch_size = idx_all_0.shape[0].value
        '''
        cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
        cur_val = tf.placeholder(tf.float64, [batch_size, None])
        cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
        '''
        idx_all = cls.idx_reform(idx_all_0)

        ans =
        for i in range(batch_size):
        cur_idx = idx_all[i]
        cur_val = val_all[i]
        cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
        cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
        ans = tf.concat([ans, cur_vec_tensor], 0)
        ans = tf.reshape(ans, [batch_size, dim_num])
        return ans

        @classmethod
        def idx_reform(cls, idx_all):
        batch_size = idx_all.shape[0].value
        return tf.reshape(idx_all, [batch_size, -1, 1])





        share|improve this answer












        problem solved



        import tensorflow as tf
        from tensorflow import TensorShape, Dimension


        class GetVector:

        @classmethod
        def get_sparse_vector(cls, idx_all_0, val_all, dim_num):
        batch_size = idx_all_0.shape[0].value
        '''
        cur_idx = tf.placeholder(tf.int64, [batch_size, None, 1])
        cur_val = tf.placeholder(tf.float64, [batch_size, None])
        cur_vec = tf.placeholder(tf.float64, [batch_size, dim_num])
        '''
        idx_all = cls.idx_reform(idx_all_0)

        ans =
        for i in range(batch_size):
        cur_idx = idx_all[i]
        cur_val = val_all[i]
        cur_vec = tf.SparseTensor(cur_idx, cur_val, TensorShape(Dimension(dim_num)))
        cur_vec_tensor = tf.sparse_tensor_to_dense(cur_vec)
        ans = tf.concat([ans, cur_vec_tensor], 0)
        ans = tf.reshape(ans, [batch_size, dim_num])
        return ans

        @classmethod
        def idx_reform(cls, idx_all):
        batch_size = idx_all.shape[0].value
        return tf.reshape(idx_all, [batch_size, -1, 1])






        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Nov 13 at 7:40









        hvchys

        2127




        2127



























            draft saved

            draft discarded
















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53262744%2fhow-to-create-sparsevector-with-index-and-value-in-tensorflow-python%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            這個網誌中的熱門文章

            Barbados

            How to read a connectionString WITH PROVIDER in .NET Core?

            Node.js Script on GitHub Pages or Amazon S3