Spacing calls to API in order to avoid flooding it
up vote
0
down vote
favorite
I'm building a trading bot which sometimes requires to make multiple calls to the Exchange's API (e.g. refresh N charts after reconnect). I'd like to have these calls time-spaced so I don't end up flooding the API (I still don't want to discard any call I need to make, so debounce/throttle is not what I'm looking for). Like this:
Source: [update chart 1, update chart 2, ..., update chart N]
Observable: [update chart 1 -> wait 1 second -> update chart 2 -> wait 1 second -> ... -> update chart N]
I'd expect this observable to complete in N - 1 seconds.
How can I build such Observable?
(I'm using rxjs ^6.3.3)
Note: I was exploring the idea of having a callAPI Subject wrapping all calls to the API and delay them somehow, but I can't figure this either.
Thanks.-
Update: Ok, I ended up using Bottleneck, like this:
const limiter = new Bottleneck(
minTime: 1000
)
Then: chartsToReset.map(async (chart) => await limiter.schedule(() => exchangeAPI.updateChart(chart)))
javascript rxjs6
add a comment |
up vote
0
down vote
favorite
I'm building a trading bot which sometimes requires to make multiple calls to the Exchange's API (e.g. refresh N charts after reconnect). I'd like to have these calls time-spaced so I don't end up flooding the API (I still don't want to discard any call I need to make, so debounce/throttle is not what I'm looking for). Like this:
Source: [update chart 1, update chart 2, ..., update chart N]
Observable: [update chart 1 -> wait 1 second -> update chart 2 -> wait 1 second -> ... -> update chart N]
I'd expect this observable to complete in N - 1 seconds.
How can I build such Observable?
(I'm using rxjs ^6.3.3)
Note: I was exploring the idea of having a callAPI Subject wrapping all calls to the API and delay them somehow, but I can't figure this either.
Thanks.-
Update: Ok, I ended up using Bottleneck, like this:
const limiter = new Bottleneck(
minTime: 1000
)
Then: chartsToReset.map(async (chart) => await limiter.schedule(() => exchangeAPI.updateChart(chart)))
javascript rxjs6
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I'm building a trading bot which sometimes requires to make multiple calls to the Exchange's API (e.g. refresh N charts after reconnect). I'd like to have these calls time-spaced so I don't end up flooding the API (I still don't want to discard any call I need to make, so debounce/throttle is not what I'm looking for). Like this:
Source: [update chart 1, update chart 2, ..., update chart N]
Observable: [update chart 1 -> wait 1 second -> update chart 2 -> wait 1 second -> ... -> update chart N]
I'd expect this observable to complete in N - 1 seconds.
How can I build such Observable?
(I'm using rxjs ^6.3.3)
Note: I was exploring the idea of having a callAPI Subject wrapping all calls to the API and delay them somehow, but I can't figure this either.
Thanks.-
Update: Ok, I ended up using Bottleneck, like this:
const limiter = new Bottleneck(
minTime: 1000
)
Then: chartsToReset.map(async (chart) => await limiter.schedule(() => exchangeAPI.updateChart(chart)))
javascript rxjs6
I'm building a trading bot which sometimes requires to make multiple calls to the Exchange's API (e.g. refresh N charts after reconnect). I'd like to have these calls time-spaced so I don't end up flooding the API (I still don't want to discard any call I need to make, so debounce/throttle is not what I'm looking for). Like this:
Source: [update chart 1, update chart 2, ..., update chart N]
Observable: [update chart 1 -> wait 1 second -> update chart 2 -> wait 1 second -> ... -> update chart N]
I'd expect this observable to complete in N - 1 seconds.
How can I build such Observable?
(I'm using rxjs ^6.3.3)
Note: I was exploring the idea of having a callAPI Subject wrapping all calls to the API and delay them somehow, but I can't figure this either.
Thanks.-
Update: Ok, I ended up using Bottleneck, like this:
const limiter = new Bottleneck(
minTime: 1000
)
Then: chartsToReset.map(async (chart) => await limiter.schedule(() => exchangeAPI.updateChart(chart)))
javascript rxjs6
javascript rxjs6
edited Nov 12 at 15:12
asked Nov 11 at 1:58
Luciano Ropero
84
84
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
up vote
0
down vote
You may want to look at interval
RxJS creation function in combination with mergeMap
operator along these lines
const updates = ['chart1', 'chart2', 'chart3'];
// this function simulates fetching of data for the chart
function getUpdatedChartData(chartName: string)
return of(chart: chartName, data: [chartName + ' 1', chartName + ' 2', chartName + ' 3']).pipe(
delay(Math.random() * 2000)
)
const theObservableIWantToBuild = interval(1000).pipe(
take(updates.length),
mergeMap(i => getUpdatedChartData(updates[i]).pipe(
tap(chartData =>
console.log(JSON.stringify(chartData, null, 3))
// do something with chart data)
),
map(() => updates[i])
)),
)
You may see that in the simulated function getUpdatedChartData
I have added a delay, just to make it slightly more realistic.
As a general rule, Subjects are required in a relatively limited number of cases. Any time you think you need to use a Subject, ask yourself if there is not an equivalent solution which requires simply using operators.
add a comment |
up vote
0
down vote
You could use a function that has a Promise + setTimeout and a recursive function (something along the lines of):
var runQueries = function(source)
if (source.length > 0)
runQuery(source[0]).then(() => runQueries(source.slice(1));
else
console.log('done!')
// this is just pseudo code...
var runQuery = function(item)
//var promise = new Promise
//run code
// setTimeout for 1000 to resolve promise
// return promise
Promise documentation on MDN
edit
If the iterating function is non-destructive with the array, and you and a properties queryRunning and queries (or something) you could do (I haven't used rxjs, so you'd have to modify code slightly for that format):
var next = function(query)
// queries and queryRunning are properties on the thing doing these calls
queries.push(query);
if (!queryRunning)
runQueries();
var runQueries = function()
if (queries.length > 0)
queryRunning = true;
runQuery(queries.shift()).then(() => runQueries();
else
queryRunning = false;
console.log('done!')
// this is just pseudo code...
var runQuery = function(query)
var promise1 = new Promise(function(resolve, reject)
setTimeout(function()
resolve('complete');
, 1000);
);
//run code to handle query
return promise1;
I was hoping for a rxjs-ish solution. Something I would be able to: next(apiRequest), next(apiRequest), next(apiRequest)... and the Observable would handle by itself.
– Luciano Ropero
Nov 11 at 11:57
I'm not familiar with rxjs, but I'll edit the above adding a method next() that would queue things up.
– jacob.mccrumb
Nov 11 at 14:23
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
You may want to look at interval
RxJS creation function in combination with mergeMap
operator along these lines
const updates = ['chart1', 'chart2', 'chart3'];
// this function simulates fetching of data for the chart
function getUpdatedChartData(chartName: string)
return of(chart: chartName, data: [chartName + ' 1', chartName + ' 2', chartName + ' 3']).pipe(
delay(Math.random() * 2000)
)
const theObservableIWantToBuild = interval(1000).pipe(
take(updates.length),
mergeMap(i => getUpdatedChartData(updates[i]).pipe(
tap(chartData =>
console.log(JSON.stringify(chartData, null, 3))
// do something with chart data)
),
map(() => updates[i])
)),
)
You may see that in the simulated function getUpdatedChartData
I have added a delay, just to make it slightly more realistic.
As a general rule, Subjects are required in a relatively limited number of cases. Any time you think you need to use a Subject, ask yourself if there is not an equivalent solution which requires simply using operators.
add a comment |
up vote
0
down vote
You may want to look at interval
RxJS creation function in combination with mergeMap
operator along these lines
const updates = ['chart1', 'chart2', 'chart3'];
// this function simulates fetching of data for the chart
function getUpdatedChartData(chartName: string)
return of(chart: chartName, data: [chartName + ' 1', chartName + ' 2', chartName + ' 3']).pipe(
delay(Math.random() * 2000)
)
const theObservableIWantToBuild = interval(1000).pipe(
take(updates.length),
mergeMap(i => getUpdatedChartData(updates[i]).pipe(
tap(chartData =>
console.log(JSON.stringify(chartData, null, 3))
// do something with chart data)
),
map(() => updates[i])
)),
)
You may see that in the simulated function getUpdatedChartData
I have added a delay, just to make it slightly more realistic.
As a general rule, Subjects are required in a relatively limited number of cases. Any time you think you need to use a Subject, ask yourself if there is not an equivalent solution which requires simply using operators.
add a comment |
up vote
0
down vote
up vote
0
down vote
You may want to look at interval
RxJS creation function in combination with mergeMap
operator along these lines
const updates = ['chart1', 'chart2', 'chart3'];
// this function simulates fetching of data for the chart
function getUpdatedChartData(chartName: string)
return of(chart: chartName, data: [chartName + ' 1', chartName + ' 2', chartName + ' 3']).pipe(
delay(Math.random() * 2000)
)
const theObservableIWantToBuild = interval(1000).pipe(
take(updates.length),
mergeMap(i => getUpdatedChartData(updates[i]).pipe(
tap(chartData =>
console.log(JSON.stringify(chartData, null, 3))
// do something with chart data)
),
map(() => updates[i])
)),
)
You may see that in the simulated function getUpdatedChartData
I have added a delay, just to make it slightly more realistic.
As a general rule, Subjects are required in a relatively limited number of cases. Any time you think you need to use a Subject, ask yourself if there is not an equivalent solution which requires simply using operators.
You may want to look at interval
RxJS creation function in combination with mergeMap
operator along these lines
const updates = ['chart1', 'chart2', 'chart3'];
// this function simulates fetching of data for the chart
function getUpdatedChartData(chartName: string)
return of(chart: chartName, data: [chartName + ' 1', chartName + ' 2', chartName + ' 3']).pipe(
delay(Math.random() * 2000)
)
const theObservableIWantToBuild = interval(1000).pipe(
take(updates.length),
mergeMap(i => getUpdatedChartData(updates[i]).pipe(
tap(chartData =>
console.log(JSON.stringify(chartData, null, 3))
// do something with chart data)
),
map(() => updates[i])
)),
)
You may see that in the simulated function getUpdatedChartData
I have added a delay, just to make it slightly more realistic.
As a general rule, Subjects are required in a relatively limited number of cases. Any time you think you need to use a Subject, ask yourself if there is not an equivalent solution which requires simply using operators.
edited Nov 11 at 13:21
answered Nov 11 at 13:15
Picci
5,05252248
5,05252248
add a comment |
add a comment |
up vote
0
down vote
You could use a function that has a Promise + setTimeout and a recursive function (something along the lines of):
var runQueries = function(source)
if (source.length > 0)
runQuery(source[0]).then(() => runQueries(source.slice(1));
else
console.log('done!')
// this is just pseudo code...
var runQuery = function(item)
//var promise = new Promise
//run code
// setTimeout for 1000 to resolve promise
// return promise
Promise documentation on MDN
edit
If the iterating function is non-destructive with the array, and you and a properties queryRunning and queries (or something) you could do (I haven't used rxjs, so you'd have to modify code slightly for that format):
var next = function(query)
// queries and queryRunning are properties on the thing doing these calls
queries.push(query);
if (!queryRunning)
runQueries();
var runQueries = function()
if (queries.length > 0)
queryRunning = true;
runQuery(queries.shift()).then(() => runQueries();
else
queryRunning = false;
console.log('done!')
// this is just pseudo code...
var runQuery = function(query)
var promise1 = new Promise(function(resolve, reject)
setTimeout(function()
resolve('complete');
, 1000);
);
//run code to handle query
return promise1;
I was hoping for a rxjs-ish solution. Something I would be able to: next(apiRequest), next(apiRequest), next(apiRequest)... and the Observable would handle by itself.
– Luciano Ropero
Nov 11 at 11:57
I'm not familiar with rxjs, but I'll edit the above adding a method next() that would queue things up.
– jacob.mccrumb
Nov 11 at 14:23
add a comment |
up vote
0
down vote
You could use a function that has a Promise + setTimeout and a recursive function (something along the lines of):
var runQueries = function(source)
if (source.length > 0)
runQuery(source[0]).then(() => runQueries(source.slice(1));
else
console.log('done!')
// this is just pseudo code...
var runQuery = function(item)
//var promise = new Promise
//run code
// setTimeout for 1000 to resolve promise
// return promise
Promise documentation on MDN
edit
If the iterating function is non-destructive with the array, and you and a properties queryRunning and queries (or something) you could do (I haven't used rxjs, so you'd have to modify code slightly for that format):
var next = function(query)
// queries and queryRunning are properties on the thing doing these calls
queries.push(query);
if (!queryRunning)
runQueries();
var runQueries = function()
if (queries.length > 0)
queryRunning = true;
runQuery(queries.shift()).then(() => runQueries();
else
queryRunning = false;
console.log('done!')
// this is just pseudo code...
var runQuery = function(query)
var promise1 = new Promise(function(resolve, reject)
setTimeout(function()
resolve('complete');
, 1000);
);
//run code to handle query
return promise1;
I was hoping for a rxjs-ish solution. Something I would be able to: next(apiRequest), next(apiRequest), next(apiRequest)... and the Observable would handle by itself.
– Luciano Ropero
Nov 11 at 11:57
I'm not familiar with rxjs, but I'll edit the above adding a method next() that would queue things up.
– jacob.mccrumb
Nov 11 at 14:23
add a comment |
up vote
0
down vote
up vote
0
down vote
You could use a function that has a Promise + setTimeout and a recursive function (something along the lines of):
var runQueries = function(source)
if (source.length > 0)
runQuery(source[0]).then(() => runQueries(source.slice(1));
else
console.log('done!')
// this is just pseudo code...
var runQuery = function(item)
//var promise = new Promise
//run code
// setTimeout for 1000 to resolve promise
// return promise
Promise documentation on MDN
edit
If the iterating function is non-destructive with the array, and you and a properties queryRunning and queries (or something) you could do (I haven't used rxjs, so you'd have to modify code slightly for that format):
var next = function(query)
// queries and queryRunning are properties on the thing doing these calls
queries.push(query);
if (!queryRunning)
runQueries();
var runQueries = function()
if (queries.length > 0)
queryRunning = true;
runQuery(queries.shift()).then(() => runQueries();
else
queryRunning = false;
console.log('done!')
// this is just pseudo code...
var runQuery = function(query)
var promise1 = new Promise(function(resolve, reject)
setTimeout(function()
resolve('complete');
, 1000);
);
//run code to handle query
return promise1;
You could use a function that has a Promise + setTimeout and a recursive function (something along the lines of):
var runQueries = function(source)
if (source.length > 0)
runQuery(source[0]).then(() => runQueries(source.slice(1));
else
console.log('done!')
// this is just pseudo code...
var runQuery = function(item)
//var promise = new Promise
//run code
// setTimeout for 1000 to resolve promise
// return promise
Promise documentation on MDN
edit
If the iterating function is non-destructive with the array, and you and a properties queryRunning and queries (or something) you could do (I haven't used rxjs, so you'd have to modify code slightly for that format):
var next = function(query)
// queries and queryRunning are properties on the thing doing these calls
queries.push(query);
if (!queryRunning)
runQueries();
var runQueries = function()
if (queries.length > 0)
queryRunning = true;
runQuery(queries.shift()).then(() => runQueries();
else
queryRunning = false;
console.log('done!')
// this is just pseudo code...
var runQuery = function(query)
var promise1 = new Promise(function(resolve, reject)
setTimeout(function()
resolve('complete');
, 1000);
);
//run code to handle query
return promise1;
edited Nov 11 at 14:32
answered Nov 11 at 4:32
jacob.mccrumb
454213
454213
I was hoping for a rxjs-ish solution. Something I would be able to: next(apiRequest), next(apiRequest), next(apiRequest)... and the Observable would handle by itself.
– Luciano Ropero
Nov 11 at 11:57
I'm not familiar with rxjs, but I'll edit the above adding a method next() that would queue things up.
– jacob.mccrumb
Nov 11 at 14:23
add a comment |
I was hoping for a rxjs-ish solution. Something I would be able to: next(apiRequest), next(apiRequest), next(apiRequest)... and the Observable would handle by itself.
– Luciano Ropero
Nov 11 at 11:57
I'm not familiar with rxjs, but I'll edit the above adding a method next() that would queue things up.
– jacob.mccrumb
Nov 11 at 14:23
I was hoping for a rxjs-ish solution. Something I would be able to: next(apiRequest), next(apiRequest), next(apiRequest)... and the Observable would handle by itself.
– Luciano Ropero
Nov 11 at 11:57
I was hoping for a rxjs-ish solution. Something I would be able to: next(apiRequest), next(apiRequest), next(apiRequest)... and the Observable would handle by itself.
– Luciano Ropero
Nov 11 at 11:57
I'm not familiar with rxjs, but I'll edit the above adding a method next() that would queue things up.
– jacob.mccrumb
Nov 11 at 14:23
I'm not familiar with rxjs, but I'll edit the above adding a method next() that would queue things up.
– jacob.mccrumb
Nov 11 at 14:23
add a comment |
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53245197%2fspacing-calls-to-api-in-order-to-avoid-flooding-it%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown