Hadoop Scheduling | 2 extremes | Availability and Scarcity of Resources









up vote
0
down vote

favorite












I have the following 6 Datanodes(dn):



  • dn1 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn2 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn3 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn4 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn5 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn6 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

Case 1 (Availability)



State of the system
===================
dn1 has 1 mapper running from another job Y; so 2 mapper slots are free
dn2 has 1 mapper running from another job Y; so 2 mapper slots are free
dn3 has 1 mapper running from another job Y; so 2 mapper slots are free
dn4 has 1 mapper running from another job Y; so 2 mapper slots are free
dn5 has 0 mappers running; so 3 mapper slots are free
dn6 has 0 mappers running; so 3 mapper slots are free
State of my input file
======================
I have a file that is distributed in 3 64MB blocks with RF 3 in the following way: 
R1(dn1,dn2,dn3) 
R2(dn2,dn3,dn4) 
R3(dn3,dn4,dn5)


When I run a job X on this file, 3 mappers need to be created corresponding to the 3 data blocks.



Questions



In FIFO: Is Job X still put on the queue waiting for Y to finish considering its a FIFO scheduler and other jobs are running even though "there are other mapper slots free in the same machine" or the FIFO logic kicks in only when no more resources are available in the system and the jobs consequently has to be put on the queue?
In Capacity Scheduler: What would the behavior be?
In Fair Share Scheduler: What would the behavior be?


Case 2 (Scarcity)



State of the system
===================
dn1 has 3 mappers running from another job Y; so 0 mapper slots are free
dn2 has 3 mappers running from another job Y; so 0 mapper slots are free
dn3 has 3 mappers running from another job Y; so 0 mapper slots are free
dn4 has 3 mappers running from another job Y; so 0 mapper slots are free
dn5 has 3 mappers running from another job Y; so 0 mapper slots are free
dn6 has 0 mappers running; so 3 mapper slots are free


I have a file that is distributed in 3 64MB blocks with RF 3 in the following way: 



R1(dn1,dn2,dn3) 
R2(dn2,dn3,dn4) 
R3(dn3,dn4,dn5)


When I run a job X on this file, 3 mappers need to be created corresponding to the 3 data blocks.



Questions



What happens now: 



 - Are the 3 mapper tasks created on dn6 (which does not have any of the data blocks of the input file yet) and corresponding data block transferred over the network from say dn1 to dn6?
 - If yes, does this same behaviour show in the case of all the three schedulers: FIFO/Capacity/Fair Share?
 - If no, then can you elaborate on the behaviour shown for this use case in case of:
 - FIFO Scheduler
 - Capacity Scheduler
 - Fair Share Scheduler





hadoop mapreduce hadoop2







share|improve this question























  • Datanodes don't control job placement. The Nodemanagers do. 6G on the machines also doesn't mean you can allocate 6G worth of MapReduce containers on them
    – cricket_007
    Nov 10 at 20:34










  • @cricket_007 I completely understand; I was using the term dn loosely for the slave machines. And yes, I agree 6G on the machines does not really translate to 6 containers...as you have other daemons running as well like DN, NM, AM etc....what I meant here was "6G 6 cores available for MR". If we abstract out the 6G and 6 core part, and just say that each machine has 3 Mapper and 3 Reducer Slots, then how do the schedulers behave in the 2 cases above?
    – Sheel Pancholi
    Nov 11 at 5:19











  • Well, FIFO is the easiest - it waits for jobs to complete before continuing. Capacity will perform bin-packing, and Fair will preempt/pause/kill other jobs, I believe
    – cricket_007
    Nov 11 at 7:49










  • Thanks @cricket_007. There are 2 cases. In the first case, there are enough mapper slots free in dn1 to dn4 (read slave 1 to slave 4) despite job Y running; are you saying FIFO still does not schedule the new job X? In the second case, irrespective of the scheduling type, there are no mapper slots available in dn1 to dn5. dn6 is free. But dn6 does not have the data blocks required for job X. Can't the job be scheduled to run on this free slave dn6 now and have data blocks/input splits from the relevant slave nodes be transferred over the network into the memory of the mapper tasks in dn6?
    – Sheel Pancholi
    Nov 11 at 9:29










  • Are we assuming that all "slots" require the same amount of memory to be put into them? And the input data is also consistently sized? I think FIFO packs in until no more resources left, then others are waiting. For the others, Capacity and Fair allow individual queues to used to set percentage of memory aside. Bursting workload queues are allowed to take resources away from the others, so it's not just based on single "slots" of any one node, rather all available slots of the whole cluster, for a given queue
    – cricket_007
    Nov 11 at 17:39














up vote
0
down vote

favorite












I have the following 6 Datanodes(dn):



  • dn1 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn2 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn3 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn4 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn5 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn6 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

Case 1 (Availability)



State of the system
===================
dn1 has 1 mapper running from another job Y; so 2 mapper slots are free
dn2 has 1 mapper running from another job Y; so 2 mapper slots are free
dn3 has 1 mapper running from another job Y; so 2 mapper slots are free
dn4 has 1 mapper running from another job Y; so 2 mapper slots are free
dn5 has 0 mappers running; so 3 mapper slots are free
dn6 has 0 mappers running; so 3 mapper slots are free
State of my input file
======================
I have a file that is distributed in 3 64MB blocks with RF 3 in the following way: 
R1(dn1,dn2,dn3) 
R2(dn2,dn3,dn4) 
R3(dn3,dn4,dn5)


When I run a job X on this file, 3 mappers need to be created corresponding to the 3 data blocks.



Questions



In FIFO: Is Job X still put on the queue waiting for Y to finish considering its a FIFO scheduler and other jobs are running even though "there are other mapper slots free in the same machine" or the FIFO logic kicks in only when no more resources are available in the system and the jobs consequently has to be put on the queue?
In Capacity Scheduler: What would the behavior be?
In Fair Share Scheduler: What would the behavior be?


Case 2 (Scarcity)



State of the system
===================
dn1 has 3 mappers running from another job Y; so 0 mapper slots are free
dn2 has 3 mappers running from another job Y; so 0 mapper slots are free
dn3 has 3 mappers running from another job Y; so 0 mapper slots are free
dn4 has 3 mappers running from another job Y; so 0 mapper slots are free
dn5 has 3 mappers running from another job Y; so 0 mapper slots are free
dn6 has 0 mappers running; so 3 mapper slots are free


I have a file that is distributed in 3 64MB blocks with RF 3 in the following way: 



R1(dn1,dn2,dn3) 
R2(dn2,dn3,dn4) 
R3(dn3,dn4,dn5)


When I run a job X on this file, 3 mappers need to be created corresponding to the 3 data blocks.



Questions



What happens now: 



 - Are the 3 mapper tasks created on dn6 (which does not have any of the data blocks of the input file yet) and corresponding data block transferred over the network from say dn1 to dn6?
 - If yes, does this same behaviour show in the case of all the three schedulers: FIFO/Capacity/Fair Share?
 - If no, then can you elaborate on the behaviour shown for this use case in case of:
 - FIFO Scheduler
 - Capacity Scheduler
 - Fair Share Scheduler





hadoop mapreduce hadoop2







share|improve this question























  • Datanodes don't control job placement. The Nodemanagers do. 6G on the machines also doesn't mean you can allocate 6G worth of MapReduce containers on them
    – cricket_007
    Nov 10 at 20:34










  • @cricket_007 I completely understand; I was using the term dn loosely for the slave machines. And yes, I agree 6G on the machines does not really translate to 6 containers...as you have other daemons running as well like DN, NM, AM etc....what I meant here was "6G 6 cores available for MR". If we abstract out the 6G and 6 core part, and just say that each machine has 3 Mapper and 3 Reducer Slots, then how do the schedulers behave in the 2 cases above?
    – Sheel Pancholi
    Nov 11 at 5:19











  • Well, FIFO is the easiest - it waits for jobs to complete before continuing. Capacity will perform bin-packing, and Fair will preempt/pause/kill other jobs, I believe
    – cricket_007
    Nov 11 at 7:49










  • Thanks @cricket_007. There are 2 cases. In the first case, there are enough mapper slots free in dn1 to dn4 (read slave 1 to slave 4) despite job Y running; are you saying FIFO still does not schedule the new job X? In the second case, irrespective of the scheduling type, there are no mapper slots available in dn1 to dn5. dn6 is free. But dn6 does not have the data blocks required for job X. Can't the job be scheduled to run on this free slave dn6 now and have data blocks/input splits from the relevant slave nodes be transferred over the network into the memory of the mapper tasks in dn6?
    – Sheel Pancholi
    Nov 11 at 9:29










  • Are we assuming that all "slots" require the same amount of memory to be put into them? And the input data is also consistently sized? I think FIFO packs in until no more resources left, then others are waiting. For the others, Capacity and Fair allow individual queues to used to set percentage of memory aside. Bursting workload queues are allowed to take resources away from the others, so it's not just based on single "slots" of any one node, rather all available slots of the whole cluster, for a given queue
    – cricket_007
    Nov 11 at 17:39












up vote
0
down vote

favorite









up vote
0
down vote

favorite











I have the following 6 Datanodes(dn):



  • dn1 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn2 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn3 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn4 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn5 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn6 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

Case 1 (Availability)



State of the system
===================
dn1 has 1 mapper running from another job Y; so 2 mapper slots are free
dn2 has 1 mapper running from another job Y; so 2 mapper slots are free
dn3 has 1 mapper running from another job Y; so 2 mapper slots are free
dn4 has 1 mapper running from another job Y; so 2 mapper slots are free
dn5 has 0 mappers running; so 3 mapper slots are free
dn6 has 0 mappers running; so 3 mapper slots are free
State of my input file
======================
I have a file that is distributed in 3 64MB blocks with RF 3 in the following way: 
R1(dn1,dn2,dn3) 
R2(dn2,dn3,dn4) 
R3(dn3,dn4,dn5)


When I run a job X on this file, 3 mappers need to be created corresponding to the 3 data blocks.



Questions



In FIFO: Is Job X still put on the queue waiting for Y to finish considering its a FIFO scheduler and other jobs are running even though "there are other mapper slots free in the same machine" or the FIFO logic kicks in only when no more resources are available in the system and the jobs consequently has to be put on the queue?
In Capacity Scheduler: What would the behavior be?
In Fair Share Scheduler: What would the behavior be?


Case 2 (Scarcity)



State of the system
===================
dn1 has 3 mappers running from another job Y; so 0 mapper slots are free
dn2 has 3 mappers running from another job Y; so 0 mapper slots are free
dn3 has 3 mappers running from another job Y; so 0 mapper slots are free
dn4 has 3 mappers running from another job Y; so 0 mapper slots are free
dn5 has 3 mappers running from another job Y; so 0 mapper slots are free
dn6 has 0 mappers running; so 3 mapper slots are free


I have a file that is distributed in 3 64MB blocks with RF 3 in the following way: 



R1(dn1,dn2,dn3) 
R2(dn2,dn3,dn4) 
R3(dn3,dn4,dn5)


When I run a job X on this file, 3 mappers need to be created corresponding to the 3 data blocks.



Questions



What happens now: 



 - Are the 3 mapper tasks created on dn6 (which does not have any of the data blocks of the input file yet) and corresponding data block transferred over the network from say dn1 to dn6?
 - If yes, does this same behaviour show in the case of all the three schedulers: FIFO/Capacity/Fair Share?
 - If no, then can you elaborate on the behaviour shown for this use case in case of:
 - FIFO Scheduler
 - Capacity Scheduler
 - Fair Share Scheduler





hadoop mapreduce hadoop2







share|improve this question















I have the following 6 Datanodes(dn):



  • dn1 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn2 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn3 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn4 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn5 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

  • dn6 - 6 cores 6GB - 3 Map Slots and 3 Reduce Slots

Case 1 (Availability)



State of the system
===================
dn1 has 1 mapper running from another job Y; so 2 mapper slots are free
dn2 has 1 mapper running from another job Y; so 2 mapper slots are free
dn3 has 1 mapper running from another job Y; so 2 mapper slots are free
dn4 has 1 mapper running from another job Y; so 2 mapper slots are free
dn5 has 0 mappers running; so 3 mapper slots are free
dn6 has 0 mappers running; so 3 mapper slots are free
State of my input file
======================
I have a file that is distributed in 3 64MB blocks with RF 3 in the following way: 
R1(dn1,dn2,dn3) 
R2(dn2,dn3,dn4) 
R3(dn3,dn4,dn5)


When I run a job X on this file, 3 mappers need to be created corresponding to the 3 data blocks.



Questions



In FIFO: Is Job X still put on the queue waiting for Y to finish considering its a FIFO scheduler and other jobs are running even though "there are other mapper slots free in the same machine" or the FIFO logic kicks in only when no more resources are available in the system and the jobs consequently has to be put on the queue?
In Capacity Scheduler: What would the behavior be?
In Fair Share Scheduler: What would the behavior be?


Case 2 (Scarcity)



State of the system
===================
dn1 has 3 mappers running from another job Y; so 0 mapper slots are free
dn2 has 3 mappers running from another job Y; so 0 mapper slots are free
dn3 has 3 mappers running from another job Y; so 0 mapper slots are free
dn4 has 3 mappers running from another job Y; so 0 mapper slots are free
dn5 has 3 mappers running from another job Y; so 0 mapper slots are free
dn6 has 0 mappers running; so 3 mapper slots are free


I have a file that is distributed in 3 64MB blocks with RF 3 in the following way: 



R1(dn1,dn2,dn3) 
R2(dn2,dn3,dn4) 
R3(dn3,dn4,dn5)


When I run a job X on this file, 3 mappers need to be created corresponding to the 3 data blocks.



Questions



What happens now: 



 - Are the 3 mapper tasks created on dn6 (which does not have any of the data blocks of the input file yet) and corresponding data block transferred over the network from say dn1 to dn6?
 - If yes, does this same behaviour show in the case of all the three schedulers: FIFO/Capacity/Fair Share?
 - If no, then can you elaborate on the behaviour shown for this use case in case of:
 - FIFO Scheduler
 - Capacity Scheduler
 - Fair Share Scheduler





hadoop mapreduce hadoop2




hadoop mapreduce hadoop2






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 11 at 9:35

























asked Nov 10 at 15:28









Sheel Pancholi

429




429











  • Datanodes don't control job placement. The Nodemanagers do. 6G on the machines also doesn't mean you can allocate 6G worth of MapReduce containers on them
    – cricket_007
    Nov 10 at 20:34










  • @cricket_007 I completely understand; I was using the term dn loosely for the slave machines. And yes, I agree 6G on the machines does not really translate to 6 containers...as you have other daemons running as well like DN, NM, AM etc....what I meant here was "6G 6 cores available for MR". If we abstract out the 6G and 6 core part, and just say that each machine has 3 Mapper and 3 Reducer Slots, then how do the schedulers behave in the 2 cases above?
    – Sheel Pancholi
    Nov 11 at 5:19











  • Well, FIFO is the easiest - it waits for jobs to complete before continuing. Capacity will perform bin-packing, and Fair will preempt/pause/kill other jobs, I believe
    – cricket_007
    Nov 11 at 7:49










  • Thanks @cricket_007. There are 2 cases. In the first case, there are enough mapper slots free in dn1 to dn4 (read slave 1 to slave 4) despite job Y running; are you saying FIFO still does not schedule the new job X? In the second case, irrespective of the scheduling type, there are no mapper slots available in dn1 to dn5. dn6 is free. But dn6 does not have the data blocks required for job X. Can't the job be scheduled to run on this free slave dn6 now and have data blocks/input splits from the relevant slave nodes be transferred over the network into the memory of the mapper tasks in dn6?
    – Sheel Pancholi
    Nov 11 at 9:29










  • Are we assuming that all "slots" require the same amount of memory to be put into them? And the input data is also consistently sized? I think FIFO packs in until no more resources left, then others are waiting. For the others, Capacity and Fair allow individual queues to used to set percentage of memory aside. Bursting workload queues are allowed to take resources away from the others, so it's not just based on single "slots" of any one node, rather all available slots of the whole cluster, for a given queue
    – cricket_007
    Nov 11 at 17:39
















  • Datanodes don't control job placement. The Nodemanagers do. 6G on the machines also doesn't mean you can allocate 6G worth of MapReduce containers on them
    – cricket_007
    Nov 10 at 20:34










  • @cricket_007 I completely understand; I was using the term dn loosely for the slave machines. And yes, I agree 6G on the machines does not really translate to 6 containers...as you have other daemons running as well like DN, NM, AM etc....what I meant here was "6G 6 cores available for MR". If we abstract out the 6G and 6 core part, and just say that each machine has 3 Mapper and 3 Reducer Slots, then how do the schedulers behave in the 2 cases above?
    – Sheel Pancholi
    Nov 11 at 5:19











  • Well, FIFO is the easiest - it waits for jobs to complete before continuing. Capacity will perform bin-packing, and Fair will preempt/pause/kill other jobs, I believe
    – cricket_007
    Nov 11 at 7:49










  • Thanks @cricket_007. There are 2 cases. In the first case, there are enough mapper slots free in dn1 to dn4 (read slave 1 to slave 4) despite job Y running; are you saying FIFO still does not schedule the new job X? In the second case, irrespective of the scheduling type, there are no mapper slots available in dn1 to dn5. dn6 is free. But dn6 does not have the data blocks required for job X. Can't the job be scheduled to run on this free slave dn6 now and have data blocks/input splits from the relevant slave nodes be transferred over the network into the memory of the mapper tasks in dn6?
    – Sheel Pancholi
    Nov 11 at 9:29










  • Are we assuming that all "slots" require the same amount of memory to be put into them? And the input data is also consistently sized? I think FIFO packs in until no more resources left, then others are waiting. For the others, Capacity and Fair allow individual queues to used to set percentage of memory aside. Bursting workload queues are allowed to take resources away from the others, so it's not just based on single "slots" of any one node, rather all available slots of the whole cluster, for a given queue
    – cricket_007
    Nov 11 at 17:39















Datanodes don't control job placement. The Nodemanagers do. 6G on the machines also doesn't mean you can allocate 6G worth of MapReduce containers on them
– cricket_007
Nov 10 at 20:34




Datanodes don't control job placement. The Nodemanagers do. 6G on the machines also doesn't mean you can allocate 6G worth of MapReduce containers on them
– cricket_007
Nov 10 at 20:34












@cricket_007 I completely understand; I was using the term dn loosely for the slave machines. And yes, I agree 6G on the machines does not really translate to 6 containers...as you have other daemons running as well like DN, NM, AM etc....what I meant here was "6G 6 cores available for MR". If we abstract out the 6G and 6 core part, and just say that each machine has 3 Mapper and 3 Reducer Slots, then how do the schedulers behave in the 2 cases above?
– Sheel Pancholi
Nov 11 at 5:19





@cricket_007 I completely understand; I was using the term dn loosely for the slave machines. And yes, I agree 6G on the machines does not really translate to 6 containers...as you have other daemons running as well like DN, NM, AM etc....what I meant here was "6G 6 cores available for MR". If we abstract out the 6G and 6 core part, and just say that each machine has 3 Mapper and 3 Reducer Slots, then how do the schedulers behave in the 2 cases above?
– Sheel Pancholi
Nov 11 at 5:19













Well, FIFO is the easiest - it waits for jobs to complete before continuing. Capacity will perform bin-packing, and Fair will preempt/pause/kill other jobs, I believe
– cricket_007
Nov 11 at 7:49




Well, FIFO is the easiest - it waits for jobs to complete before continuing. Capacity will perform bin-packing, and Fair will preempt/pause/kill other jobs, I believe
– cricket_007
Nov 11 at 7:49












Thanks @cricket_007. There are 2 cases. In the first case, there are enough mapper slots free in dn1 to dn4 (read slave 1 to slave 4) despite job Y running; are you saying FIFO still does not schedule the new job X? In the second case, irrespective of the scheduling type, there are no mapper slots available in dn1 to dn5. dn6 is free. But dn6 does not have the data blocks required for job X. Can't the job be scheduled to run on this free slave dn6 now and have data blocks/input splits from the relevant slave nodes be transferred over the network into the memory of the mapper tasks in dn6?
– Sheel Pancholi
Nov 11 at 9:29




Thanks @cricket_007. There are 2 cases. In the first case, there are enough mapper slots free in dn1 to dn4 (read slave 1 to slave 4) despite job Y running; are you saying FIFO still does not schedule the new job X? In the second case, irrespective of the scheduling type, there are no mapper slots available in dn1 to dn5. dn6 is free. But dn6 does not have the data blocks required for job X. Can't the job be scheduled to run on this free slave dn6 now and have data blocks/input splits from the relevant slave nodes be transferred over the network into the memory of the mapper tasks in dn6?
– Sheel Pancholi
Nov 11 at 9:29












Are we assuming that all "slots" require the same amount of memory to be put into them? And the input data is also consistently sized? I think FIFO packs in until no more resources left, then others are waiting. For the others, Capacity and Fair allow individual queues to used to set percentage of memory aside. Bursting workload queues are allowed to take resources away from the others, so it's not just based on single "slots" of any one node, rather all available slots of the whole cluster, for a given queue
– cricket_007
Nov 11 at 17:39




Are we assuming that all "slots" require the same amount of memory to be put into them? And the input data is also consistently sized? I think FIFO packs in until no more resources left, then others are waiting. For the others, Capacity and Fair allow individual queues to used to set percentage of memory aside. Bursting workload queues are allowed to take resources away from the others, so it's not just based on single "slots" of any one node, rather all available slots of the whole cluster, for a given queue
– cricket_007
Nov 11 at 17:39

















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













 

draft saved


draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53240428%2fhadoop-scheduling-2-extremes-availability-and-scarcity-of-resources%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes















 

draft saved


draft discarded















































 


draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53240428%2fhadoop-scheduling-2-extremes-availability-and-scarcity-of-resources%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







-

這個網誌中的熱門文章

Barbados

圖片
For other uses, see Barbados (disambiguation). Not to be confused with Barbuda. Coordinates: 13°10′N 59°33′W  /  13.167°N 59.550°W  / 13.167; -59.550 Barbados Flag Coat of arms Motto:  "Pride and Industry" Anthem:  In Plenty and In Time of Need Capital and largest city Bridgetown 13°06′N 59°37′W  /  13.100°N 59.617°W  / 13.100; -59.617 Official languages English Recognised regional languages Bajan Creole Ethnic groups (est. 2010 [1] ) 92.4% black 3.1% multiracial 2.7% white 1.3% East Indian 0.5% other/unspecified Religion 75.6% Christian 2.5% other 20.6% none 1.2% unspecified [1] Demonym(s) Barbadian Bajan (colloquial) Government Unitary parliamentary constitutional monarchy • Monarch Elizabeth II •  Governor-General Dame Sandra Mason • Prime Minister Mia Mottley Legislature Parliament • Upper house Senate • Lower house House of Assembly Independence • From the United Kingdom 30 November 1966 Area • Total 439 km 2 (169 sq mi) (183rd) • Water (%) Negligib
閱讀完整內容

How to read a connectionString WITH PROVIDER in .NET Core?

圖片
6 I added .AddJsonFile("Connections.json", optional: true, reloadOnChange: true) in public Startup(IHostingEnvironment env) Connections.json contains: "ConnectionStrings": "DefaultConnection": "Server=(localdb)\mssqllocaldb;Database=DATABASE;Trusted_Connection=True;MultipleActiveResultSets=true", "COR-W81-101": "Data Source=DATASOURCE;Initial Catalog=P61_CAFM_Basic;User Id=USERID;Password=PASSWORD;Persist Security Info=False;MultipleActiveResultSets=False;Packet Size=4096;", "COR-W81-100": "Data Source=DATASOURCE;Initial Catalog=Post_PS;User Id=USERID;Password=PASSWORD;Persist Security Info=False;MultipleActiveResultSets=False;Packet Size=4096;", "MSEDGEWIN10": "Data Source=DATASOURCE; Initial Catalog=COR_Basic; Persist Security Info=False;Integrated Security=true;MultipleActiveResultSets=False;Packet Size=4096;Application Name="COR_Basic"", "server&qu
閱讀完整內容

Node.js Script on GitHub Pages or Amazon S3

圖片
0 I'm wondering if I can run a Node.js script in my Jekyll page from GitHub Pages or Amazon S3? I think it can't run on GitHub Pages since it doesn't support server side code. Not too sure. The code is below: var Airtable = require('airtable'); var base = new Airtable(apiKey: 'YOUR_API_KEY').base('appAnZVyYqusNPV5Q'); base('Invitee list').select( // Selecting the first 3 records in Complete list: maxRecords: 3, view: "Complete list" ).eachPage(function page(records, fetchNextPage) // This function (`page`) will get called for each page of records. records.forEach(function(record) console.log('Retrieved', record.get('Email')); ); // To fetch the next page of records, call `fetchNextPage`. // If there are more records, `page` will get called again. // If there are no more records, `done` will get called. fetchNextPage(); , function done(err) if (err) console.error(err); return; ); node.js
閱讀完整內容