Mapping memory of one vector to another










2















I have a vector of binary data used by some middleware



std::vector<uint> data



The data inside this vector maps to a class Foo basically like that:



|uint|uint|uint|uint|uint|uint|uint|uint|uint|
| F o o | F o o | F o o |


Using a custom placement allocator (as described here Can I use an std::vector as a facade for a pre-allocated (raw) array?) I can now map the memory of the first vector to a vector of type std::vector<Foo>



But if I operate on the second vector the size of the first one is not updated.



Another approach would be to encapsulate the first vector std::vector<uint8> in a custom container which behaves like a std::vector<Foo> but this is a lot of effort (and I cannot use Boost).



Any ideas for an elegant solution?










share|improve this question






















  • 'But if I operate on the second vector the size of the first one is not updated.' - not only - if you need to re-allocate because of new elements, other vector won't get updated as well (instead, the link between both gets entirely lost). I assume the middleware is not under your control, is it?

    – Aconcagua
    Nov 15 '18 at 0:55












  • Yes, as @Aconcagua says, define 'operate on'. What changes to that vector do you need to make? Or, to take a more holistic view, what does a std::vector give you that makes you want to jump through these hoops in order to use it?

    – Paul Sanders
    Nov 15 '18 at 0:57












  • What operations do you need to do on vector<Foo>? I'd do something simple like auto start = reinterpret_cast<Foo*>(vec.data()); auto finish = reinterpret_cast<Foo*>(vec.data() + vec.size()); Then use start to finish as a range. Write utility functions for things like push_back.

    – Filipp
    Nov 15 '18 at 1:01











  • @Filipp That's more or less the custom container already mentioned... But actually, it appears to me the most 'elegant' solution anyway, even if there's quite some effort...

    – Aconcagua
    Nov 15 '18 at 1:05











  • @Aconcagua: What I would do to prevent the reallocation is to provide a custom allocator which keeps the memory mapping consistent (see the link). Still the sizes are not synchronized.

    – Andreas Pasternak
    Nov 15 '18 at 2:08















2















I have a vector of binary data used by some middleware



std::vector<uint> data



The data inside this vector maps to a class Foo basically like that:



|uint|uint|uint|uint|uint|uint|uint|uint|uint|
| F o o | F o o | F o o |


Using a custom placement allocator (as described here Can I use an std::vector as a facade for a pre-allocated (raw) array?) I can now map the memory of the first vector to a vector of type std::vector<Foo>



But if I operate on the second vector the size of the first one is not updated.



Another approach would be to encapsulate the first vector std::vector<uint8> in a custom container which behaves like a std::vector<Foo> but this is a lot of effort (and I cannot use Boost).



Any ideas for an elegant solution?










share|improve this question






















  • 'But if I operate on the second vector the size of the first one is not updated.' - not only - if you need to re-allocate because of new elements, other vector won't get updated as well (instead, the link between both gets entirely lost). I assume the middleware is not under your control, is it?

    – Aconcagua
    Nov 15 '18 at 0:55












  • Yes, as @Aconcagua says, define 'operate on'. What changes to that vector do you need to make? Or, to take a more holistic view, what does a std::vector give you that makes you want to jump through these hoops in order to use it?

    – Paul Sanders
    Nov 15 '18 at 0:57












  • What operations do you need to do on vector<Foo>? I'd do something simple like auto start = reinterpret_cast<Foo*>(vec.data()); auto finish = reinterpret_cast<Foo*>(vec.data() + vec.size()); Then use start to finish as a range. Write utility functions for things like push_back.

    – Filipp
    Nov 15 '18 at 1:01











  • @Filipp That's more or less the custom container already mentioned... But actually, it appears to me the most 'elegant' solution anyway, even if there's quite some effort...

    – Aconcagua
    Nov 15 '18 at 1:05











  • @Aconcagua: What I would do to prevent the reallocation is to provide a custom allocator which keeps the memory mapping consistent (see the link). Still the sizes are not synchronized.

    – Andreas Pasternak
    Nov 15 '18 at 2:08













2












2








2








I have a vector of binary data used by some middleware



std::vector<uint> data



The data inside this vector maps to a class Foo basically like that:



|uint|uint|uint|uint|uint|uint|uint|uint|uint|
| F o o | F o o | F o o |


Using a custom placement allocator (as described here Can I use an std::vector as a facade for a pre-allocated (raw) array?) I can now map the memory of the first vector to a vector of type std::vector<Foo>



But if I operate on the second vector the size of the first one is not updated.



Another approach would be to encapsulate the first vector std::vector<uint8> in a custom container which behaves like a std::vector<Foo> but this is a lot of effort (and I cannot use Boost).



Any ideas for an elegant solution?










share|improve this question














I have a vector of binary data used by some middleware



std::vector<uint> data



The data inside this vector maps to a class Foo basically like that:



|uint|uint|uint|uint|uint|uint|uint|uint|uint|
| F o o | F o o | F o o |


Using a custom placement allocator (as described here Can I use an std::vector as a facade for a pre-allocated (raw) array?) I can now map the memory of the first vector to a vector of type std::vector<Foo>



But if I operate on the second vector the size of the first one is not updated.



Another approach would be to encapsulate the first vector std::vector<uint8> in a custom container which behaves like a std::vector<Foo> but this is a lot of effort (and I cannot use Boost).



Any ideas for an elegant solution?







c++ vector mapping






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 15 '18 at 0:39









Andreas PasternakAndreas Pasternak

562314




562314












  • 'But if I operate on the second vector the size of the first one is not updated.' - not only - if you need to re-allocate because of new elements, other vector won't get updated as well (instead, the link between both gets entirely lost). I assume the middleware is not under your control, is it?

    – Aconcagua
    Nov 15 '18 at 0:55












  • Yes, as @Aconcagua says, define 'operate on'. What changes to that vector do you need to make? Or, to take a more holistic view, what does a std::vector give you that makes you want to jump through these hoops in order to use it?

    – Paul Sanders
    Nov 15 '18 at 0:57












  • What operations do you need to do on vector<Foo>? I'd do something simple like auto start = reinterpret_cast<Foo*>(vec.data()); auto finish = reinterpret_cast<Foo*>(vec.data() + vec.size()); Then use start to finish as a range. Write utility functions for things like push_back.

    – Filipp
    Nov 15 '18 at 1:01











  • @Filipp That's more or less the custom container already mentioned... But actually, it appears to me the most 'elegant' solution anyway, even if there's quite some effort...

    – Aconcagua
    Nov 15 '18 at 1:05











  • @Aconcagua: What I would do to prevent the reallocation is to provide a custom allocator which keeps the memory mapping consistent (see the link). Still the sizes are not synchronized.

    – Andreas Pasternak
    Nov 15 '18 at 2:08

















  • 'But if I operate on the second vector the size of the first one is not updated.' - not only - if you need to re-allocate because of new elements, other vector won't get updated as well (instead, the link between both gets entirely lost). I assume the middleware is not under your control, is it?

    – Aconcagua
    Nov 15 '18 at 0:55












  • Yes, as @Aconcagua says, define 'operate on'. What changes to that vector do you need to make? Or, to take a more holistic view, what does a std::vector give you that makes you want to jump through these hoops in order to use it?

    – Paul Sanders
    Nov 15 '18 at 0:57












  • What operations do you need to do on vector<Foo>? I'd do something simple like auto start = reinterpret_cast<Foo*>(vec.data()); auto finish = reinterpret_cast<Foo*>(vec.data() + vec.size()); Then use start to finish as a range. Write utility functions for things like push_back.

    – Filipp
    Nov 15 '18 at 1:01











  • @Filipp That's more or less the custom container already mentioned... But actually, it appears to me the most 'elegant' solution anyway, even if there's quite some effort...

    – Aconcagua
    Nov 15 '18 at 1:05











  • @Aconcagua: What I would do to prevent the reallocation is to provide a custom allocator which keeps the memory mapping consistent (see the link). Still the sizes are not synchronized.

    – Andreas Pasternak
    Nov 15 '18 at 2:08
















'But if I operate on the second vector the size of the first one is not updated.' - not only - if you need to re-allocate because of new elements, other vector won't get updated as well (instead, the link between both gets entirely lost). I assume the middleware is not under your control, is it?

– Aconcagua
Nov 15 '18 at 0:55






'But if I operate on the second vector the size of the first one is not updated.' - not only - if you need to re-allocate because of new elements, other vector won't get updated as well (instead, the link between both gets entirely lost). I assume the middleware is not under your control, is it?

– Aconcagua
Nov 15 '18 at 0:55














Yes, as @Aconcagua says, define 'operate on'. What changes to that vector do you need to make? Or, to take a more holistic view, what does a std::vector give you that makes you want to jump through these hoops in order to use it?

– Paul Sanders
Nov 15 '18 at 0:57






Yes, as @Aconcagua says, define 'operate on'. What changes to that vector do you need to make? Or, to take a more holistic view, what does a std::vector give you that makes you want to jump through these hoops in order to use it?

– Paul Sanders
Nov 15 '18 at 0:57














What operations do you need to do on vector<Foo>? I'd do something simple like auto start = reinterpret_cast<Foo*>(vec.data()); auto finish = reinterpret_cast<Foo*>(vec.data() + vec.size()); Then use start to finish as a range. Write utility functions for things like push_back.

– Filipp
Nov 15 '18 at 1:01





What operations do you need to do on vector<Foo>? I'd do something simple like auto start = reinterpret_cast<Foo*>(vec.data()); auto finish = reinterpret_cast<Foo*>(vec.data() + vec.size()); Then use start to finish as a range. Write utility functions for things like push_back.

– Filipp
Nov 15 '18 at 1:01













@Filipp That's more or less the custom container already mentioned... But actually, it appears to me the most 'elegant' solution anyway, even if there's quite some effort...

– Aconcagua
Nov 15 '18 at 1:05





@Filipp That's more or less the custom container already mentioned... But actually, it appears to me the most 'elegant' solution anyway, even if there's quite some effort...

– Aconcagua
Nov 15 '18 at 1:05













@Aconcagua: What I would do to prevent the reallocation is to provide a custom allocator which keeps the memory mapping consistent (see the link). Still the sizes are not synchronized.

– Andreas Pasternak
Nov 15 '18 at 2:08





@Aconcagua: What I would do to prevent the reallocation is to provide a custom allocator which keeps the memory mapping consistent (see the link). Still the sizes are not synchronized.

– Andreas Pasternak
Nov 15 '18 at 2:08












1 Answer
1






active

oldest

votes


















0














The situation is very different to the linked question. You aren't dealing with preallocated storage, you have data for objects already there. Furthermore, you wish to have both sides be able to modify the vector, or in other words they have shared ownership.



This isn't trivial. You can provide a "view class" that offers another set of operations on the same vector or have a family of free functions do the same. Either way they must have reference semantics.



A free function example



template<typename T>
void push_back_foo(std::vector<uint8_t>& vec, T&& t)

auto it = vec.insert(vec.end(), sizeof(Foo), );
new (&*it) Foostd::forward<T>(t);



And so on for all the other functions. They map operations on the Foo side to operations on the uint8_t side.



A side note: doing pointer arithmetic on the resulting vector is formally undefined behaviour because there is no array of Foo there. When dealing with overlaying binary data with a facade, it is generally understood that compilers won't burn down your house and such formalities are ignored.






share|improve this answer






















    Your Answer






    StackExchange.ifUsing("editor", function ()
    StackExchange.using("externalEditor", function ()
    StackExchange.using("snippets", function ()
    StackExchange.snippets.init();
    );
    );
    , "code-snippets");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "1"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53310858%2fmapping-memory-of-one-vector-to-another%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    The situation is very different to the linked question. You aren't dealing with preallocated storage, you have data for objects already there. Furthermore, you wish to have both sides be able to modify the vector, or in other words they have shared ownership.



    This isn't trivial. You can provide a "view class" that offers another set of operations on the same vector or have a family of free functions do the same. Either way they must have reference semantics.



    A free function example



    template<typename T>
    void push_back_foo(std::vector<uint8_t>& vec, T&& t)

    auto it = vec.insert(vec.end(), sizeof(Foo), );
    new (&*it) Foostd::forward<T>(t);



    And so on for all the other functions. They map operations on the Foo side to operations on the uint8_t side.



    A side note: doing pointer arithmetic on the resulting vector is formally undefined behaviour because there is no array of Foo there. When dealing with overlaying binary data with a facade, it is generally understood that compilers won't burn down your house and such formalities are ignored.






    share|improve this answer



























      0














      The situation is very different to the linked question. You aren't dealing with preallocated storage, you have data for objects already there. Furthermore, you wish to have both sides be able to modify the vector, or in other words they have shared ownership.



      This isn't trivial. You can provide a "view class" that offers another set of operations on the same vector or have a family of free functions do the same. Either way they must have reference semantics.



      A free function example



      template<typename T>
      void push_back_foo(std::vector<uint8_t>& vec, T&& t)

      auto it = vec.insert(vec.end(), sizeof(Foo), );
      new (&*it) Foostd::forward<T>(t);



      And so on for all the other functions. They map operations on the Foo side to operations on the uint8_t side.



      A side note: doing pointer arithmetic on the resulting vector is formally undefined behaviour because there is no array of Foo there. When dealing with overlaying binary data with a facade, it is generally understood that compilers won't burn down your house and such formalities are ignored.






      share|improve this answer

























        0












        0








        0







        The situation is very different to the linked question. You aren't dealing with preallocated storage, you have data for objects already there. Furthermore, you wish to have both sides be able to modify the vector, or in other words they have shared ownership.



        This isn't trivial. You can provide a "view class" that offers another set of operations on the same vector or have a family of free functions do the same. Either way they must have reference semantics.



        A free function example



        template<typename T>
        void push_back_foo(std::vector<uint8_t>& vec, T&& t)

        auto it = vec.insert(vec.end(), sizeof(Foo), );
        new (&*it) Foostd::forward<T>(t);



        And so on for all the other functions. They map operations on the Foo side to operations on the uint8_t side.



        A side note: doing pointer arithmetic on the resulting vector is formally undefined behaviour because there is no array of Foo there. When dealing with overlaying binary data with a facade, it is generally understood that compilers won't burn down your house and such formalities are ignored.






        share|improve this answer













        The situation is very different to the linked question. You aren't dealing with preallocated storage, you have data for objects already there. Furthermore, you wish to have both sides be able to modify the vector, or in other words they have shared ownership.



        This isn't trivial. You can provide a "view class" that offers another set of operations on the same vector or have a family of free functions do the same. Either way they must have reference semantics.



        A free function example



        template<typename T>
        void push_back_foo(std::vector<uint8_t>& vec, T&& t)

        auto it = vec.insert(vec.end(), sizeof(Foo), );
        new (&*it) Foostd::forward<T>(t);



        And so on for all the other functions. They map operations on the Foo side to operations on the uint8_t side.



        A side note: doing pointer arithmetic on the resulting vector is formally undefined behaviour because there is no array of Foo there. When dealing with overlaying binary data with a facade, it is generally understood that compilers won't burn down your house and such formalities are ignored.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Nov 15 '18 at 8:09









        Passer ByPasser By

        9,93432559




        9,93432559





























            draft saved

            draft discarded
















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53310858%2fmapping-memory-of-one-vector-to-another%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            這個網誌中的熱門文章

            Barbados

            How to read a connectionString WITH PROVIDER in .NET Core?

            Node.js Script on GitHub Pages or Amazon S3