Mixing
How does number of workers in pytorch actually work?
1- If num_workers is 2, Does that mean that it will put 2 batches in the RAM and send 1 of them to the GPU or Does it put 3 batches in the RAM then sends 1 of them to the GPU?
2- What does actually happen when the number of workers is higher than the number of CPU cores? I tried it and it worked fine but How does it work?(I thought that the maximum number of workers I can choose is the number of cores)
3- If I set num_workers to 3 and during the training there were no batches in the memory for the GPU, Does the main process waits for its workers to read the batches or Does it read a single batch(without waiting for the workers)?
pytorch
asked Jan 1 at 19:23
floydfloyd
221111
add a comment |
1- If num_workers is 2, Does that mean that it will put 2 batches in the RAM and send 1 of them to the GPU or Does it put 3 batches in the RAM then sends 1 of them to the GPU?
2- What does actually happen when the number of workers is higher than the number of CPU cores? I tried it and it worked fine but How does it work?(I thought that the maximum number of workers I can choose is the number of cores)
3- If I set num_workers to 3 and during the training there were no batches in the memory for the GPU, Does the main process waits for its workers to read the batches or Does it read a single batch(without waiting for the workers)?
pytorch
asked Jan 1 at 19:23
floydfloyd
221111
add a comment |
1- If num_workers is 2, Does that mean that it will put 2 batches in the RAM and send 1 of them to the GPU or Does it put 3 batches in the RAM then sends 1 of them to the GPU?
2- What does actually happen when the number of workers is higher than the number of CPU cores? I tried it and it worked fine but How does it work?(I thought that the maximum number of workers I can choose is the number of cores)
3- If I set num_workers to 3 and during the training there were no batches in the memory for the GPU, Does the main process waits for its workers to read the batches or Does it read a single batch(without waiting for the workers)?
pytorch
asked Jan 1 at 19:23
floydfloyd
221111
1- If num_workers is 2, Does that mean that it will put 2 batches in the RAM and send 1 of them to the GPU or Does it put 3 batches in the RAM then sends 1 of them to the GPU?
2- What does actually happen when the number of workers is higher than the number of CPU cores? I tried it and it worked fine but How does it work?(I thought that the maximum number of workers I can choose is the number of cores)
3- If I set num_workers to 3 and during the training there were no batches in the memory for the GPU, Does the main process waits for its workers to read the batches or Does it read a single batch(without waiting for the workers)?
pytorch
pytorch
asked Jan 1 at 19:23
floydfloyd
221111
asked Jan 1 at 19:23
floydfloyd
221111
edited Jan 1 at 19:37
floyd
asked Jan 1 at 19:23
floydfloyd
221111
asked Jan 1 at 19:23
floydfloyd
221111
asked Jan 1 at 19:23
floydfloyd
221111
221111
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
- When
num_workers>0, only these workers will retrieve data, main process won't. So whennum_workers=2you have at most 2 workers simultaneously putting data into RAM, not 3. - Well our CPU can usually run like 100 processes without trouble and these worker processes aren't special in anyway, so having more workers than cpu cores is ok. But is it efficient? it depends on how busy your cpu cores are for other tasks, speed of cpu, speed of your hard disk etc. In short, its complicated, so setting workers to number of cores is like good rule of thumb, nothing more.
- Nope. Remember
DataLoaderdoesn't just randomly return from what's available in RAM right now, it usesbatch_samplerto decide which batch to return next. Each batch is assigned to a worker, and main process will wait until the desired batch is retrieved by assigned worker.
Lastly to clarify, it isn't DataLoader's job to send anything directly to GPU, you explicitly call cuda() for that, or modify Dataset's __getitem__() method.
answered Jan 2 at 6:33
Shihab ShahriarShihab Shahriar
1,392714
Thank you so much. that was really really helpful
– floyd
Jan 2 at 12:36
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53998282%2fhow-does-number-of-workers-in-pytorch-actually-work%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
- When
num_workers>0, only these workers will retrieve data, main process won't. So whennum_workers=2you have at most 2 workers simultaneously putting data into RAM, not 3. - Well our CPU can usually run like 100 processes without trouble and these worker processes aren't special in anyway, so having more workers than cpu cores is ok. But is it efficient? it depends on how busy your cpu cores are for other tasks, speed of cpu, speed of your hard disk etc. In short, its complicated, so setting workers to number of cores is like good rule of thumb, nothing more.
- Nope. Remember
DataLoaderdoesn't just randomly return from what's available in RAM right now, it usesbatch_samplerto decide which batch to return next. Each batch is assigned to a worker, and main process will wait until the desired batch is retrieved by assigned worker.
Lastly to clarify, it isn't DataLoader's job to send anything directly to GPU, you explicitly call cuda() for that, or modify Dataset's __getitem__() method.
answered Jan 2 at 6:33
Shihab ShahriarShihab Shahriar
1,392714
Thank you so much. that was really really helpful
– floyd
Jan 2 at 12:36
add a comment |
- When
num_workers>0, only these workers will retrieve data, main process won't. So whennum_workers=2you have at most 2 workers simultaneously putting data into RAM, not 3. - Well our CPU can usually run like 100 processes without trouble and these worker processes aren't special in anyway, so having more workers than cpu cores is ok. But is it efficient? it depends on how busy your cpu cores are for other tasks, speed of cpu, speed of your hard disk etc. In short, its complicated, so setting workers to number of cores is like good rule of thumb, nothing more.
- Nope. Remember
DataLoaderdoesn't just randomly return from what's available in RAM right now, it usesbatch_samplerto decide which batch to return next. Each batch is assigned to a worker, and main process will wait until the desired batch is retrieved by assigned worker.
Lastly to clarify, it isn't DataLoader's job to send anything directly to GPU, you explicitly call cuda() for that, or modify Dataset's __getitem__() method.
answered Jan 2 at 6:33
Shihab ShahriarShihab Shahriar
1,392714
Thank you so much. that was really really helpful
– floyd
Jan 2 at 12:36
add a comment |
- When
num_workers>0, only these workers will retrieve data, main process won't. So whennum_workers=2you have at most 2 workers simultaneously putting data into RAM, not 3. - Well our CPU can usually run like 100 processes without trouble and these worker processes aren't special in anyway, so having more workers than cpu cores is ok. But is it efficient? it depends on how busy your cpu cores are for other tasks, speed of cpu, speed of your hard disk etc. In short, its complicated, so setting workers to number of cores is like good rule of thumb, nothing more.
- Nope. Remember
DataLoaderdoesn't just randomly return from what's available in RAM right now, it usesbatch_samplerto decide which batch to return next. Each batch is assigned to a worker, and main process will wait until the desired batch is retrieved by assigned worker.
Lastly to clarify, it isn't DataLoader's job to send anything directly to GPU, you explicitly call cuda() for that, or modify Dataset's __getitem__() method.
answered Jan 2 at 6:33
Shihab ShahriarShihab Shahriar
1,392714
- When
num_workers>0, only these workers will retrieve data, main process won't. So whennum_workers=2you have at most 2 workers simultaneously putting data into RAM, not 3. - Well our CPU can usually run like 100 processes without trouble and these worker processes aren't special in anyway, so having more workers than cpu cores is ok. But is it efficient? it depends on how busy your cpu cores are for other tasks, speed of cpu, speed of your hard disk etc. In short, its complicated, so setting workers to number of cores is like good rule of thumb, nothing more.
- Nope. Remember
DataLoaderdoesn't just randomly return from what's available in RAM right now, it usesbatch_samplerto decide which batch to return next. Each batch is assigned to a worker, and main process will wait until the desired batch is retrieved by assigned worker.
Lastly to clarify, it isn't DataLoader's job to send anything directly to GPU, you explicitly call cuda() for that, or modify Dataset's __getitem__() method.
answered Jan 2 at 6:33
Shihab ShahriarShihab Shahriar
1,392714
edited yesterday
answered Jan 2 at 6:33
Shihab ShahriarShihab Shahriar
1,392714
answered Jan 2 at 6:33
Shihab ShahriarShihab Shahriar
1,392714
answered Jan 2 at 6:33
Shihab ShahriarShihab Shahriar
1,392714
1,392714
Thank you so much. that was really really helpful
– floyd
Jan 2 at 12:36
add a comment |
Thank you so much. that was really really helpful
– floyd
Jan 2 at 12:36
Thank you so much. that was really really helpful
– floyd
Jan 2 at 12:36
Thank you so much. that was really really helpful
– floyd
Jan 2 at 12:36
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53998282%2fhow-does-number-of-workers-in-pytorch-actually-work%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
