Spacing calls to API in order to avoid flooding it









up vote
0
down vote

favorite












I'm building a trading bot which sometimes requires to make multiple calls to the Exchange's API (e.g. refresh N charts after reconnect). I'd like to have these calls time-spaced so I don't end up flooding the API (I still don't want to discard any call I need to make, so debounce/throttle is not what I'm looking for). Like this:



Source: [update chart 1, update chart 2, ..., update chart N]



Observable: [update chart 1 -> wait 1 second -> update chart 2 -> wait 1 second -> ... -> update chart N]



I'd expect this observable to complete in N - 1 seconds.



How can I build such Observable?
(I'm using rxjs ^6.3.3)



Note: I was exploring the idea of having a callAPI Subject wrapping all calls to the API and delay them somehow, but I can't figure this either.



Thanks.-



Update: Ok, I ended up using Bottleneck, like this:



const limiter = new Bottleneck(
minTime: 1000
)


Then: chartsToReset.map(async (chart) => await limiter.schedule(() => exchangeAPI.updateChart(chart)))










share|improve this question



























    up vote
    0
    down vote

    favorite












    I'm building a trading bot which sometimes requires to make multiple calls to the Exchange's API (e.g. refresh N charts after reconnect). I'd like to have these calls time-spaced so I don't end up flooding the API (I still don't want to discard any call I need to make, so debounce/throttle is not what I'm looking for). Like this:



    Source: [update chart 1, update chart 2, ..., update chart N]



    Observable: [update chart 1 -> wait 1 second -> update chart 2 -> wait 1 second -> ... -> update chart N]



    I'd expect this observable to complete in N - 1 seconds.



    How can I build such Observable?
    (I'm using rxjs ^6.3.3)



    Note: I was exploring the idea of having a callAPI Subject wrapping all calls to the API and delay them somehow, but I can't figure this either.



    Thanks.-



    Update: Ok, I ended up using Bottleneck, like this:



    const limiter = new Bottleneck(
    minTime: 1000
    )


    Then: chartsToReset.map(async (chart) => await limiter.schedule(() => exchangeAPI.updateChart(chart)))










    share|improve this question

























      up vote
      0
      down vote

      favorite









      up vote
      0
      down vote

      favorite











      I'm building a trading bot which sometimes requires to make multiple calls to the Exchange's API (e.g. refresh N charts after reconnect). I'd like to have these calls time-spaced so I don't end up flooding the API (I still don't want to discard any call I need to make, so debounce/throttle is not what I'm looking for). Like this:



      Source: [update chart 1, update chart 2, ..., update chart N]



      Observable: [update chart 1 -> wait 1 second -> update chart 2 -> wait 1 second -> ... -> update chart N]



      I'd expect this observable to complete in N - 1 seconds.



      How can I build such Observable?
      (I'm using rxjs ^6.3.3)



      Note: I was exploring the idea of having a callAPI Subject wrapping all calls to the API and delay them somehow, but I can't figure this either.



      Thanks.-



      Update: Ok, I ended up using Bottleneck, like this:



      const limiter = new Bottleneck(
      minTime: 1000
      )


      Then: chartsToReset.map(async (chart) => await limiter.schedule(() => exchangeAPI.updateChart(chart)))










      share|improve this question















      I'm building a trading bot which sometimes requires to make multiple calls to the Exchange's API (e.g. refresh N charts after reconnect). I'd like to have these calls time-spaced so I don't end up flooding the API (I still don't want to discard any call I need to make, so debounce/throttle is not what I'm looking for). Like this:



      Source: [update chart 1, update chart 2, ..., update chart N]



      Observable: [update chart 1 -> wait 1 second -> update chart 2 -> wait 1 second -> ... -> update chart N]



      I'd expect this observable to complete in N - 1 seconds.



      How can I build such Observable?
      (I'm using rxjs ^6.3.3)



      Note: I was exploring the idea of having a callAPI Subject wrapping all calls to the API and delay them somehow, but I can't figure this either.



      Thanks.-



      Update: Ok, I ended up using Bottleneck, like this:



      const limiter = new Bottleneck(
      minTime: 1000
      )


      Then: chartsToReset.map(async (chart) => await limiter.schedule(() => exchangeAPI.updateChart(chart)))







      javascript rxjs6






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 12 at 15:12

























      asked Nov 11 at 1:58









      Luciano Ropero

      84




      84






















          2 Answers
          2






          active

          oldest

          votes

















          up vote
          0
          down vote













          You may want to look at interval RxJS creation function in combination with mergeMap operator along these lines



          const updates = ['chart1', 'chart2', 'chart3'];

          // this function simulates fetching of data for the chart
          function getUpdatedChartData(chartName: string)
          return of(chart: chartName, data: [chartName + ' 1', chartName + ' 2', chartName + ' 3']).pipe(
          delay(Math.random() * 2000)
          )


          const theObservableIWantToBuild = interval(1000).pipe(
          take(updates.length),
          mergeMap(i => getUpdatedChartData(updates[i]).pipe(
          tap(chartData =>
          console.log(JSON.stringify(chartData, null, 3))
          // do something with chart data)
          ),
          map(() => updates[i])
          )),
          )


          You may see that in the simulated function getUpdatedChartData I have added a delay, just to make it slightly more realistic.



          As a general rule, Subjects are required in a relatively limited number of cases. Any time you think you need to use a Subject, ask yourself if there is not an equivalent solution which requires simply using operators.






          share|improve this answer





























            up vote
            0
            down vote













            You could use a function that has a Promise + setTimeout and a recursive function (something along the lines of):



            var runQueries = function(source) 
            if (source.length > 0)
            runQuery(source[0]).then(() => runQueries(source.slice(1));
            else
            console.log('done!')


            // this is just pseudo code...
            var runQuery = function(item)
            //var promise = new Promise
            //run code
            // setTimeout for 1000 to resolve promise
            // return promise



            Promise documentation on MDN



            edit



            If the iterating function is non-destructive with the array, and you and a properties queryRunning and queries (or something) you could do (I haven't used rxjs, so you'd have to modify code slightly for that format):



            var next = function(query) 
            // queries and queryRunning are properties on the thing doing these calls
            queries.push(query);
            if (!queryRunning)
            runQueries();


            var runQueries = function()
            if (queries.length > 0)
            queryRunning = true;
            runQuery(queries.shift()).then(() => runQueries();
            else
            queryRunning = false;
            console.log('done!')


            // this is just pseudo code...
            var runQuery = function(query)
            var promise1 = new Promise(function(resolve, reject)
            setTimeout(function()
            resolve('complete');
            , 1000);
            );
            //run code to handle query
            return promise1;






            share|improve this answer






















            • I was hoping for a rxjs-ish solution. Something I would be able to: next(apiRequest), next(apiRequest), next(apiRequest)... and the Observable would handle by itself.
              – Luciano Ropero
              Nov 11 at 11:57











            • I'm not familiar with rxjs, but I'll edit the above adding a method next() that would queue things up.
              – jacob.mccrumb
              Nov 11 at 14:23










            Your Answer






            StackExchange.ifUsing("editor", function ()
            StackExchange.using("externalEditor", function ()
            StackExchange.using("snippets", function ()
            StackExchange.snippets.init();
            );
            );
            , "code-snippets");

            StackExchange.ready(function()
            var channelOptions =
            tags: "".split(" "),
            id: "1"
            ;
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function()
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled)
            StackExchange.using("snippets", function()
            createEditor();
            );

            else
            createEditor();

            );

            function createEditor()
            StackExchange.prepareEditor(
            heartbeatType: 'answer',
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader:
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            ,
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            );



            );













             

            draft saved


            draft discarded


















            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53245197%2fspacing-calls-to-api-in-order-to-avoid-flooding-it%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            0
            down vote













            You may want to look at interval RxJS creation function in combination with mergeMap operator along these lines



            const updates = ['chart1', 'chart2', 'chart3'];

            // this function simulates fetching of data for the chart
            function getUpdatedChartData(chartName: string)
            return of(chart: chartName, data: [chartName + ' 1', chartName + ' 2', chartName + ' 3']).pipe(
            delay(Math.random() * 2000)
            )


            const theObservableIWantToBuild = interval(1000).pipe(
            take(updates.length),
            mergeMap(i => getUpdatedChartData(updates[i]).pipe(
            tap(chartData =>
            console.log(JSON.stringify(chartData, null, 3))
            // do something with chart data)
            ),
            map(() => updates[i])
            )),
            )


            You may see that in the simulated function getUpdatedChartData I have added a delay, just to make it slightly more realistic.



            As a general rule, Subjects are required in a relatively limited number of cases. Any time you think you need to use a Subject, ask yourself if there is not an equivalent solution which requires simply using operators.






            share|improve this answer


























              up vote
              0
              down vote













              You may want to look at interval RxJS creation function in combination with mergeMap operator along these lines



              const updates = ['chart1', 'chart2', 'chart3'];

              // this function simulates fetching of data for the chart
              function getUpdatedChartData(chartName: string)
              return of(chart: chartName, data: [chartName + ' 1', chartName + ' 2', chartName + ' 3']).pipe(
              delay(Math.random() * 2000)
              )


              const theObservableIWantToBuild = interval(1000).pipe(
              take(updates.length),
              mergeMap(i => getUpdatedChartData(updates[i]).pipe(
              tap(chartData =>
              console.log(JSON.stringify(chartData, null, 3))
              // do something with chart data)
              ),
              map(() => updates[i])
              )),
              )


              You may see that in the simulated function getUpdatedChartData I have added a delay, just to make it slightly more realistic.



              As a general rule, Subjects are required in a relatively limited number of cases. Any time you think you need to use a Subject, ask yourself if there is not an equivalent solution which requires simply using operators.






              share|improve this answer
























                up vote
                0
                down vote










                up vote
                0
                down vote









                You may want to look at interval RxJS creation function in combination with mergeMap operator along these lines



                const updates = ['chart1', 'chart2', 'chart3'];

                // this function simulates fetching of data for the chart
                function getUpdatedChartData(chartName: string)
                return of(chart: chartName, data: [chartName + ' 1', chartName + ' 2', chartName + ' 3']).pipe(
                delay(Math.random() * 2000)
                )


                const theObservableIWantToBuild = interval(1000).pipe(
                take(updates.length),
                mergeMap(i => getUpdatedChartData(updates[i]).pipe(
                tap(chartData =>
                console.log(JSON.stringify(chartData, null, 3))
                // do something with chart data)
                ),
                map(() => updates[i])
                )),
                )


                You may see that in the simulated function getUpdatedChartData I have added a delay, just to make it slightly more realistic.



                As a general rule, Subjects are required in a relatively limited number of cases. Any time you think you need to use a Subject, ask yourself if there is not an equivalent solution which requires simply using operators.






                share|improve this answer














                You may want to look at interval RxJS creation function in combination with mergeMap operator along these lines



                const updates = ['chart1', 'chart2', 'chart3'];

                // this function simulates fetching of data for the chart
                function getUpdatedChartData(chartName: string)
                return of(chart: chartName, data: [chartName + ' 1', chartName + ' 2', chartName + ' 3']).pipe(
                delay(Math.random() * 2000)
                )


                const theObservableIWantToBuild = interval(1000).pipe(
                take(updates.length),
                mergeMap(i => getUpdatedChartData(updates[i]).pipe(
                tap(chartData =>
                console.log(JSON.stringify(chartData, null, 3))
                // do something with chart data)
                ),
                map(() => updates[i])
                )),
                )


                You may see that in the simulated function getUpdatedChartData I have added a delay, just to make it slightly more realistic.



                As a general rule, Subjects are required in a relatively limited number of cases. Any time you think you need to use a Subject, ask yourself if there is not an equivalent solution which requires simply using operators.







                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited Nov 11 at 13:21

























                answered Nov 11 at 13:15









                Picci

                5,05252248




                5,05252248






















                    up vote
                    0
                    down vote













                    You could use a function that has a Promise + setTimeout and a recursive function (something along the lines of):



                    var runQueries = function(source) 
                    if (source.length > 0)
                    runQuery(source[0]).then(() => runQueries(source.slice(1));
                    else
                    console.log('done!')


                    // this is just pseudo code...
                    var runQuery = function(item)
                    //var promise = new Promise
                    //run code
                    // setTimeout for 1000 to resolve promise
                    // return promise



                    Promise documentation on MDN



                    edit



                    If the iterating function is non-destructive with the array, and you and a properties queryRunning and queries (or something) you could do (I haven't used rxjs, so you'd have to modify code slightly for that format):



                    var next = function(query) 
                    // queries and queryRunning are properties on the thing doing these calls
                    queries.push(query);
                    if (!queryRunning)
                    runQueries();


                    var runQueries = function()
                    if (queries.length > 0)
                    queryRunning = true;
                    runQuery(queries.shift()).then(() => runQueries();
                    else
                    queryRunning = false;
                    console.log('done!')


                    // this is just pseudo code...
                    var runQuery = function(query)
                    var promise1 = new Promise(function(resolve, reject)
                    setTimeout(function()
                    resolve('complete');
                    , 1000);
                    );
                    //run code to handle query
                    return promise1;






                    share|improve this answer






















                    • I was hoping for a rxjs-ish solution. Something I would be able to: next(apiRequest), next(apiRequest), next(apiRequest)... and the Observable would handle by itself.
                      – Luciano Ropero
                      Nov 11 at 11:57











                    • I'm not familiar with rxjs, but I'll edit the above adding a method next() that would queue things up.
                      – jacob.mccrumb
                      Nov 11 at 14:23














                    up vote
                    0
                    down vote













                    You could use a function that has a Promise + setTimeout and a recursive function (something along the lines of):



                    var runQueries = function(source) 
                    if (source.length > 0)
                    runQuery(source[0]).then(() => runQueries(source.slice(1));
                    else
                    console.log('done!')


                    // this is just pseudo code...
                    var runQuery = function(item)
                    //var promise = new Promise
                    //run code
                    // setTimeout for 1000 to resolve promise
                    // return promise



                    Promise documentation on MDN



                    edit



                    If the iterating function is non-destructive with the array, and you and a properties queryRunning and queries (or something) you could do (I haven't used rxjs, so you'd have to modify code slightly for that format):



                    var next = function(query) 
                    // queries and queryRunning are properties on the thing doing these calls
                    queries.push(query);
                    if (!queryRunning)
                    runQueries();


                    var runQueries = function()
                    if (queries.length > 0)
                    queryRunning = true;
                    runQuery(queries.shift()).then(() => runQueries();
                    else
                    queryRunning = false;
                    console.log('done!')


                    // this is just pseudo code...
                    var runQuery = function(query)
                    var promise1 = new Promise(function(resolve, reject)
                    setTimeout(function()
                    resolve('complete');
                    , 1000);
                    );
                    //run code to handle query
                    return promise1;






                    share|improve this answer






















                    • I was hoping for a rxjs-ish solution. Something I would be able to: next(apiRequest), next(apiRequest), next(apiRequest)... and the Observable would handle by itself.
                      – Luciano Ropero
                      Nov 11 at 11:57











                    • I'm not familiar with rxjs, but I'll edit the above adding a method next() that would queue things up.
                      – jacob.mccrumb
                      Nov 11 at 14:23












                    up vote
                    0
                    down vote










                    up vote
                    0
                    down vote









                    You could use a function that has a Promise + setTimeout and a recursive function (something along the lines of):



                    var runQueries = function(source) 
                    if (source.length > 0)
                    runQuery(source[0]).then(() => runQueries(source.slice(1));
                    else
                    console.log('done!')


                    // this is just pseudo code...
                    var runQuery = function(item)
                    //var promise = new Promise
                    //run code
                    // setTimeout for 1000 to resolve promise
                    // return promise



                    Promise documentation on MDN



                    edit



                    If the iterating function is non-destructive with the array, and you and a properties queryRunning and queries (or something) you could do (I haven't used rxjs, so you'd have to modify code slightly for that format):



                    var next = function(query) 
                    // queries and queryRunning are properties on the thing doing these calls
                    queries.push(query);
                    if (!queryRunning)
                    runQueries();


                    var runQueries = function()
                    if (queries.length > 0)
                    queryRunning = true;
                    runQuery(queries.shift()).then(() => runQueries();
                    else
                    queryRunning = false;
                    console.log('done!')


                    // this is just pseudo code...
                    var runQuery = function(query)
                    var promise1 = new Promise(function(resolve, reject)
                    setTimeout(function()
                    resolve('complete');
                    , 1000);
                    );
                    //run code to handle query
                    return promise1;






                    share|improve this answer














                    You could use a function that has a Promise + setTimeout and a recursive function (something along the lines of):



                    var runQueries = function(source) 
                    if (source.length > 0)
                    runQuery(source[0]).then(() => runQueries(source.slice(1));
                    else
                    console.log('done!')


                    // this is just pseudo code...
                    var runQuery = function(item)
                    //var promise = new Promise
                    //run code
                    // setTimeout for 1000 to resolve promise
                    // return promise



                    Promise documentation on MDN



                    edit



                    If the iterating function is non-destructive with the array, and you and a properties queryRunning and queries (or something) you could do (I haven't used rxjs, so you'd have to modify code slightly for that format):



                    var next = function(query) 
                    // queries and queryRunning are properties on the thing doing these calls
                    queries.push(query);
                    if (!queryRunning)
                    runQueries();


                    var runQueries = function()
                    if (queries.length > 0)
                    queryRunning = true;
                    runQuery(queries.shift()).then(() => runQueries();
                    else
                    queryRunning = false;
                    console.log('done!')


                    // this is just pseudo code...
                    var runQuery = function(query)
                    var promise1 = new Promise(function(resolve, reject)
                    setTimeout(function()
                    resolve('complete');
                    , 1000);
                    );
                    //run code to handle query
                    return promise1;







                    share|improve this answer














                    share|improve this answer



                    share|improve this answer








                    edited Nov 11 at 14:32

























                    answered Nov 11 at 4:32









                    jacob.mccrumb

                    454213




                    454213











                    • I was hoping for a rxjs-ish solution. Something I would be able to: next(apiRequest), next(apiRequest), next(apiRequest)... and the Observable would handle by itself.
                      – Luciano Ropero
                      Nov 11 at 11:57











                    • I'm not familiar with rxjs, but I'll edit the above adding a method next() that would queue things up.
                      – jacob.mccrumb
                      Nov 11 at 14:23
















                    • I was hoping for a rxjs-ish solution. Something I would be able to: next(apiRequest), next(apiRequest), next(apiRequest)... and the Observable would handle by itself.
                      – Luciano Ropero
                      Nov 11 at 11:57











                    • I'm not familiar with rxjs, but I'll edit the above adding a method next() that would queue things up.
                      – jacob.mccrumb
                      Nov 11 at 14:23















                    I was hoping for a rxjs-ish solution. Something I would be able to: next(apiRequest), next(apiRequest), next(apiRequest)... and the Observable would handle by itself.
                    – Luciano Ropero
                    Nov 11 at 11:57





                    I was hoping for a rxjs-ish solution. Something I would be able to: next(apiRequest), next(apiRequest), next(apiRequest)... and the Observable would handle by itself.
                    – Luciano Ropero
                    Nov 11 at 11:57













                    I'm not familiar with rxjs, but I'll edit the above adding a method next() that would queue things up.
                    – jacob.mccrumb
                    Nov 11 at 14:23




                    I'm not familiar with rxjs, but I'll edit the above adding a method next() that would queue things up.
                    – jacob.mccrumb
                    Nov 11 at 14:23

















                     

                    draft saved


                    draft discarded















































                     


                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function ()
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53245197%2fspacing-calls-to-api-in-order-to-avoid-flooding-it%23new-answer', 'question_page');

                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    這個網誌中的熱門文章

                    How to read a connectionString WITH PROVIDER in .NET Core?

                    Node.js Script on GitHub Pages or Amazon S3

                    Museum of Modern and Contemporary Art of Trento and Rovereto