build numpy array from list of tuples









up vote
0
down vote

favorite
1












I want to convert a list of tuples into a numpy array. For example:



items = [(1, 2), (3, 4)]


using np.asarray(items) I get:



array([[1, 2],
[3, 4]])


but if I try to append the items individually:



new_array = np.empty(0)
for item in items:
new_array = np.append(new_array, item)


the new_array loses the original shape and becomes:



array([1., 2., 3., 4.])


I can get it to the shape I wanted using new_array.reshape(2, 2):



array([[1., 2.],
[3., 4.]])


but how would I get that shape without reshaping?










share|improve this question

















  • 4




    np.asarray() does what you want, so why are you looking for inefficient looping methods?
    – John Zwinck
    Nov 11 at 3:36










  • np.append has several booby traps. Did you read its docs? Better yet read its code. What's the original shape? new_array starts with a (0,) shape. items isn't an array so doesn't have a shape.
    – hpaulj
    Nov 11 at 4:08











  • Is the fact that it's a list of tuples instead of lists significant.
    – hpaulj
    Nov 11 at 4:16














up vote
0
down vote

favorite
1












I want to convert a list of tuples into a numpy array. For example:



items = [(1, 2), (3, 4)]


using np.asarray(items) I get:



array([[1, 2],
[3, 4]])


but if I try to append the items individually:



new_array = np.empty(0)
for item in items:
new_array = np.append(new_array, item)


the new_array loses the original shape and becomes:



array([1., 2., 3., 4.])


I can get it to the shape I wanted using new_array.reshape(2, 2):



array([[1., 2.],
[3., 4.]])


but how would I get that shape without reshaping?










share|improve this question

















  • 4




    np.asarray() does what you want, so why are you looking for inefficient looping methods?
    – John Zwinck
    Nov 11 at 3:36










  • np.append has several booby traps. Did you read its docs? Better yet read its code. What's the original shape? new_array starts with a (0,) shape. items isn't an array so doesn't have a shape.
    – hpaulj
    Nov 11 at 4:08











  • Is the fact that it's a list of tuples instead of lists significant.
    – hpaulj
    Nov 11 at 4:16












up vote
0
down vote

favorite
1









up vote
0
down vote

favorite
1






1





I want to convert a list of tuples into a numpy array. For example:



items = [(1, 2), (3, 4)]


using np.asarray(items) I get:



array([[1, 2],
[3, 4]])


but if I try to append the items individually:



new_array = np.empty(0)
for item in items:
new_array = np.append(new_array, item)


the new_array loses the original shape and becomes:



array([1., 2., 3., 4.])


I can get it to the shape I wanted using new_array.reshape(2, 2):



array([[1., 2.],
[3., 4.]])


but how would I get that shape without reshaping?










share|improve this question













I want to convert a list of tuples into a numpy array. For example:



items = [(1, 2), (3, 4)]


using np.asarray(items) I get:



array([[1, 2],
[3, 4]])


but if I try to append the items individually:



new_array = np.empty(0)
for item in items:
new_array = np.append(new_array, item)


the new_array loses the original shape and becomes:



array([1., 2., 3., 4.])


I can get it to the shape I wanted using new_array.reshape(2, 2):



array([[1., 2.],
[3., 4.]])


but how would I get that shape without reshaping?







python numpy






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 11 at 3:29









waspinator

1,47652745




1,47652745







  • 4




    np.asarray() does what you want, so why are you looking for inefficient looping methods?
    – John Zwinck
    Nov 11 at 3:36










  • np.append has several booby traps. Did you read its docs? Better yet read its code. What's the original shape? new_array starts with a (0,) shape. items isn't an array so doesn't have a shape.
    – hpaulj
    Nov 11 at 4:08











  • Is the fact that it's a list of tuples instead of lists significant.
    – hpaulj
    Nov 11 at 4:16












  • 4




    np.asarray() does what you want, so why are you looking for inefficient looping methods?
    – John Zwinck
    Nov 11 at 3:36










  • np.append has several booby traps. Did you read its docs? Better yet read its code. What's the original shape? new_array starts with a (0,) shape. items isn't an array so doesn't have a shape.
    – hpaulj
    Nov 11 at 4:08











  • Is the fact that it's a list of tuples instead of lists significant.
    – hpaulj
    Nov 11 at 4:16







4




4




np.asarray() does what you want, so why are you looking for inefficient looping methods?
– John Zwinck
Nov 11 at 3:36




np.asarray() does what you want, so why are you looking for inefficient looping methods?
– John Zwinck
Nov 11 at 3:36












np.append has several booby traps. Did you read its docs? Better yet read its code. What's the original shape? new_array starts with a (0,) shape. items isn't an array so doesn't have a shape.
– hpaulj
Nov 11 at 4:08





np.append has several booby traps. Did you read its docs? Better yet read its code. What's the original shape? new_array starts with a (0,) shape. items isn't an array so doesn't have a shape.
– hpaulj
Nov 11 at 4:08













Is the fact that it's a list of tuples instead of lists significant.
– hpaulj
Nov 11 at 4:16




Is the fact that it's a list of tuples instead of lists significant.
– hpaulj
Nov 11 at 4:16












2 Answers
2






active

oldest

votes

















up vote
1
down vote



accepted










Firstly you need to provide a correct shape to the array so that numpy could understand how to interpret the values provided to the append method.



Then, to prevent automatic flattening, specify the axis you wish to append on.



This code does what you intended to do:



import numpy as np

items = [(1,2),(3,4)]

new_array = np.ndarray((0,2))
for item in items:
new_array = np.append(new_array, [item], axis=0)

print(new_array) # [[1. 2.]
# [3. 4.]]





share|improve this answer




















  • Indeed this is the right way to use ` np.append`, if you must. Not that we encourage the iterative use of any concatenate family; it's too inefficient.
    – hpaulj
    Nov 11 at 4:52











  • @hpaulj Say that a user can't preallocate and has to dynamically grow an array over time (due to streaming data or the like). What's your own personal feeling as to the "best" pattern by which to carry out said dynamic array growth in the current version of numpy?
    – tel
    Nov 11 at 5:41






  • 1




    Collecting values in a list and performing one array build at the end is a time honored method. But the incremental concatenate might be better if you need an array, rather than a list, at intermediate stages (for stats or some other calculation). Other things being equal it comes down to timings - what's faster, or at least fast enough in a realistic case.
    – hpaulj
    Nov 11 at 7:11


















up vote
1
down vote













If you have a list of tuples, and you've decided you hate the standard array constructors (np.array, np.asarray, etc, which, as @JohnZwinck pointed out are probably the best answer) for some reason, the most efficient approach would be to preallocate the entire array and then assign to it:



items = [(1, 2), (3, 4)]
arr = np.empty((len(items), len(items[0])))

arr[...] = items


Even if what you want is to grow an array over time, row-by-row, it has been shown through detailed timings that you're usually better off just allocating a whole new array and then copying over the old values.



So given the above arr, by this approach the most efficient way to append a row would be:



newitem = (5, 6)
oldarr = arr
arr = np.empty((oldarr.shape[0] + 1, *oldarr.shape[1:]))

arr[:-1,:] = oldarr
arr[-1,:] = newitem





share|improve this answer






















    Your Answer






    StackExchange.ifUsing("editor", function ()
    StackExchange.using("externalEditor", function ()
    StackExchange.using("snippets", function ()
    StackExchange.snippets.init();
    );
    );
    , "code-snippets");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "1"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













     

    draft saved


    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53245597%2fbuild-numpy-array-from-list-of-tuples%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    1
    down vote



    accepted










    Firstly you need to provide a correct shape to the array so that numpy could understand how to interpret the values provided to the append method.



    Then, to prevent automatic flattening, specify the axis you wish to append on.



    This code does what you intended to do:



    import numpy as np

    items = [(1,2),(3,4)]

    new_array = np.ndarray((0,2))
    for item in items:
    new_array = np.append(new_array, [item], axis=0)

    print(new_array) # [[1. 2.]
    # [3. 4.]]





    share|improve this answer




















    • Indeed this is the right way to use ` np.append`, if you must. Not that we encourage the iterative use of any concatenate family; it's too inefficient.
      – hpaulj
      Nov 11 at 4:52











    • @hpaulj Say that a user can't preallocate and has to dynamically grow an array over time (due to streaming data or the like). What's your own personal feeling as to the "best" pattern by which to carry out said dynamic array growth in the current version of numpy?
      – tel
      Nov 11 at 5:41






    • 1




      Collecting values in a list and performing one array build at the end is a time honored method. But the incremental concatenate might be better if you need an array, rather than a list, at intermediate stages (for stats or some other calculation). Other things being equal it comes down to timings - what's faster, or at least fast enough in a realistic case.
      – hpaulj
      Nov 11 at 7:11















    up vote
    1
    down vote



    accepted










    Firstly you need to provide a correct shape to the array so that numpy could understand how to interpret the values provided to the append method.



    Then, to prevent automatic flattening, specify the axis you wish to append on.



    This code does what you intended to do:



    import numpy as np

    items = [(1,2),(3,4)]

    new_array = np.ndarray((0,2))
    for item in items:
    new_array = np.append(new_array, [item], axis=0)

    print(new_array) # [[1. 2.]
    # [3. 4.]]





    share|improve this answer




















    • Indeed this is the right way to use ` np.append`, if you must. Not that we encourage the iterative use of any concatenate family; it's too inefficient.
      – hpaulj
      Nov 11 at 4:52











    • @hpaulj Say that a user can't preallocate and has to dynamically grow an array over time (due to streaming data or the like). What's your own personal feeling as to the "best" pattern by which to carry out said dynamic array growth in the current version of numpy?
      – tel
      Nov 11 at 5:41






    • 1




      Collecting values in a list and performing one array build at the end is a time honored method. But the incremental concatenate might be better if you need an array, rather than a list, at intermediate stages (for stats or some other calculation). Other things being equal it comes down to timings - what's faster, or at least fast enough in a realistic case.
      – hpaulj
      Nov 11 at 7:11













    up vote
    1
    down vote



    accepted







    up vote
    1
    down vote



    accepted






    Firstly you need to provide a correct shape to the array so that numpy could understand how to interpret the values provided to the append method.



    Then, to prevent automatic flattening, specify the axis you wish to append on.



    This code does what you intended to do:



    import numpy as np

    items = [(1,2),(3,4)]

    new_array = np.ndarray((0,2))
    for item in items:
    new_array = np.append(new_array, [item], axis=0)

    print(new_array) # [[1. 2.]
    # [3. 4.]]





    share|improve this answer












    Firstly you need to provide a correct shape to the array so that numpy could understand how to interpret the values provided to the append method.



    Then, to prevent automatic flattening, specify the axis you wish to append on.



    This code does what you intended to do:



    import numpy as np

    items = [(1,2),(3,4)]

    new_array = np.ndarray((0,2))
    for item in items:
    new_array = np.append(new_array, [item], axis=0)

    print(new_array) # [[1. 2.]
    # [3. 4.]]






    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered Nov 11 at 3:52









    Eternal_flame-AD

    3126




    3126











    • Indeed this is the right way to use ` np.append`, if you must. Not that we encourage the iterative use of any concatenate family; it's too inefficient.
      – hpaulj
      Nov 11 at 4:52











    • @hpaulj Say that a user can't preallocate and has to dynamically grow an array over time (due to streaming data or the like). What's your own personal feeling as to the "best" pattern by which to carry out said dynamic array growth in the current version of numpy?
      – tel
      Nov 11 at 5:41






    • 1




      Collecting values in a list and performing one array build at the end is a time honored method. But the incremental concatenate might be better if you need an array, rather than a list, at intermediate stages (for stats or some other calculation). Other things being equal it comes down to timings - what's faster, or at least fast enough in a realistic case.
      – hpaulj
      Nov 11 at 7:11

















    • Indeed this is the right way to use ` np.append`, if you must. Not that we encourage the iterative use of any concatenate family; it's too inefficient.
      – hpaulj
      Nov 11 at 4:52











    • @hpaulj Say that a user can't preallocate and has to dynamically grow an array over time (due to streaming data or the like). What's your own personal feeling as to the "best" pattern by which to carry out said dynamic array growth in the current version of numpy?
      – tel
      Nov 11 at 5:41






    • 1




      Collecting values in a list and performing one array build at the end is a time honored method. But the incremental concatenate might be better if you need an array, rather than a list, at intermediate stages (for stats or some other calculation). Other things being equal it comes down to timings - what's faster, or at least fast enough in a realistic case.
      – hpaulj
      Nov 11 at 7:11
















    Indeed this is the right way to use ` np.append`, if you must. Not that we encourage the iterative use of any concatenate family; it's too inefficient.
    – hpaulj
    Nov 11 at 4:52





    Indeed this is the right way to use ` np.append`, if you must. Not that we encourage the iterative use of any concatenate family; it's too inefficient.
    – hpaulj
    Nov 11 at 4:52













    @hpaulj Say that a user can't preallocate and has to dynamically grow an array over time (due to streaming data or the like). What's your own personal feeling as to the "best" pattern by which to carry out said dynamic array growth in the current version of numpy?
    – tel
    Nov 11 at 5:41




    @hpaulj Say that a user can't preallocate and has to dynamically grow an array over time (due to streaming data or the like). What's your own personal feeling as to the "best" pattern by which to carry out said dynamic array growth in the current version of numpy?
    – tel
    Nov 11 at 5:41




    1




    1




    Collecting values in a list and performing one array build at the end is a time honored method. But the incremental concatenate might be better if you need an array, rather than a list, at intermediate stages (for stats or some other calculation). Other things being equal it comes down to timings - what's faster, or at least fast enough in a realistic case.
    – hpaulj
    Nov 11 at 7:11





    Collecting values in a list and performing one array build at the end is a time honored method. But the incremental concatenate might be better if you need an array, rather than a list, at intermediate stages (for stats or some other calculation). Other things being equal it comes down to timings - what's faster, or at least fast enough in a realistic case.
    – hpaulj
    Nov 11 at 7:11













    up vote
    1
    down vote













    If you have a list of tuples, and you've decided you hate the standard array constructors (np.array, np.asarray, etc, which, as @JohnZwinck pointed out are probably the best answer) for some reason, the most efficient approach would be to preallocate the entire array and then assign to it:



    items = [(1, 2), (3, 4)]
    arr = np.empty((len(items), len(items[0])))

    arr[...] = items


    Even if what you want is to grow an array over time, row-by-row, it has been shown through detailed timings that you're usually better off just allocating a whole new array and then copying over the old values.



    So given the above arr, by this approach the most efficient way to append a row would be:



    newitem = (5, 6)
    oldarr = arr
    arr = np.empty((oldarr.shape[0] + 1, *oldarr.shape[1:]))

    arr[:-1,:] = oldarr
    arr[-1,:] = newitem





    share|improve this answer


























      up vote
      1
      down vote













      If you have a list of tuples, and you've decided you hate the standard array constructors (np.array, np.asarray, etc, which, as @JohnZwinck pointed out are probably the best answer) for some reason, the most efficient approach would be to preallocate the entire array and then assign to it:



      items = [(1, 2), (3, 4)]
      arr = np.empty((len(items), len(items[0])))

      arr[...] = items


      Even if what you want is to grow an array over time, row-by-row, it has been shown through detailed timings that you're usually better off just allocating a whole new array and then copying over the old values.



      So given the above arr, by this approach the most efficient way to append a row would be:



      newitem = (5, 6)
      oldarr = arr
      arr = np.empty((oldarr.shape[0] + 1, *oldarr.shape[1:]))

      arr[:-1,:] = oldarr
      arr[-1,:] = newitem





      share|improve this answer
























        up vote
        1
        down vote










        up vote
        1
        down vote









        If you have a list of tuples, and you've decided you hate the standard array constructors (np.array, np.asarray, etc, which, as @JohnZwinck pointed out are probably the best answer) for some reason, the most efficient approach would be to preallocate the entire array and then assign to it:



        items = [(1, 2), (3, 4)]
        arr = np.empty((len(items), len(items[0])))

        arr[...] = items


        Even if what you want is to grow an array over time, row-by-row, it has been shown through detailed timings that you're usually better off just allocating a whole new array and then copying over the old values.



        So given the above arr, by this approach the most efficient way to append a row would be:



        newitem = (5, 6)
        oldarr = arr
        arr = np.empty((oldarr.shape[0] + 1, *oldarr.shape[1:]))

        arr[:-1,:] = oldarr
        arr[-1,:] = newitem





        share|improve this answer














        If you have a list of tuples, and you've decided you hate the standard array constructors (np.array, np.asarray, etc, which, as @JohnZwinck pointed out are probably the best answer) for some reason, the most efficient approach would be to preallocate the entire array and then assign to it:



        items = [(1, 2), (3, 4)]
        arr = np.empty((len(items), len(items[0])))

        arr[...] = items


        Even if what you want is to grow an array over time, row-by-row, it has been shown through detailed timings that you're usually better off just allocating a whole new array and then copying over the old values.



        So given the above arr, by this approach the most efficient way to append a row would be:



        newitem = (5, 6)
        oldarr = arr
        arr = np.empty((oldarr.shape[0] + 1, *oldarr.shape[1:]))

        arr[:-1,:] = oldarr
        arr[-1,:] = newitem






        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited Nov 11 at 4:12

























        answered Nov 11 at 4:06









        tel

        3,5161427




        3,5161427



























             

            draft saved


            draft discarded















































             


            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53245597%2fbuild-numpy-array-from-list-of-tuples%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            這個網誌中的熱門文章

            Barbados

            How to read a connectionString WITH PROVIDER in .NET Core?

            Node.js Script on GitHub Pages or Amazon S3