Mixing
Force NPM to Install All Dependencies from Custom, Cloned Repositories
This is more of an architectural approach question than a specific programming issue.
The situation:
Due to issues like the left pad incident and dependencies that are releasing breaking changes without changing their version numbers, some build failures can occur without changing any project code or npm configuration information such as package.json and package-lock.json. My build system will run npm install on the repository and pull down the dependencies before each build. Even with a package-lock.json file committed and in play, the build can randomly fail (and recently has failed) due to changes in dependencies that I do not control.
It has been decided, corporately, that we shall no longer allow this to happen.
Considering the options, the easiest solution seems to be simply committing the node_modules folder with a known good configuration. However, I fear this may cause issues due to the different OS usage between developer clients and build systems, Mac and Linux, respectively. Apparently some dependencies will make environment-specific compilations upon install, and committing them will result in the wrong binaries being present on the build system.
Alternatively, I've been looking for a way to mass clone not just our dependencies, but our dependencies' dependencies in some automated fashion to our local repositories. In theory, I could get my dependencies from my clones of the repository on both the dev clients and build servers, assured that the dependencies cannot change in any way, no matter how deep, unless I decide to change them manually. However, though I can certainly specify dependencies relative to a git repo URL in package.json, the package-lock.json file is the only file that actually contains all of the dependencies. And it does not reference git repo URLs, but some "resolved" URL that references a .tgz file provided by npmjs.org.
Is there a way to force my NPM commands or the package-lock.json file to use custom repos for ALL dependencies, and point them to our clones?
Am I missing some other method of achieving this goal? How have you solved this problem?
Most articles and StackOverflow questions on the subject seem to either endorse the idea that NPM, combined with a checked in package-lock.json will always solve this problem, which is incorrect in our case. Or they recommend committing the node_modules folder (to which there is fanatical opposition from others).
npm clone dependency-management
asked Nov 22 '18 at 5:56
DwightDwight
6071917
add a comment |
This is more of an architectural approach question than a specific programming issue.
The situation:
Due to issues like the left pad incident and dependencies that are releasing breaking changes without changing their version numbers, some build failures can occur without changing any project code or npm configuration information such as package.json and package-lock.json. My build system will run npm install on the repository and pull down the dependencies before each build. Even with a package-lock.json file committed and in play, the build can randomly fail (and recently has failed) due to changes in dependencies that I do not control.
It has been decided, corporately, that we shall no longer allow this to happen.
Considering the options, the easiest solution seems to be simply committing the node_modules folder with a known good configuration. However, I fear this may cause issues due to the different OS usage between developer clients and build systems, Mac and Linux, respectively. Apparently some dependencies will make environment-specific compilations upon install, and committing them will result in the wrong binaries being present on the build system.
Alternatively, I've been looking for a way to mass clone not just our dependencies, but our dependencies' dependencies in some automated fashion to our local repositories. In theory, I could get my dependencies from my clones of the repository on both the dev clients and build servers, assured that the dependencies cannot change in any way, no matter how deep, unless I decide to change them manually. However, though I can certainly specify dependencies relative to a git repo URL in package.json, the package-lock.json file is the only file that actually contains all of the dependencies. And it does not reference git repo URLs, but some "resolved" URL that references a .tgz file provided by npmjs.org.
Is there a way to force my NPM commands or the package-lock.json file to use custom repos for ALL dependencies, and point them to our clones?
Am I missing some other method of achieving this goal? How have you solved this problem?
Most articles and StackOverflow questions on the subject seem to either endorse the idea that NPM, combined with a checked in package-lock.json will always solve this problem, which is incorrect in our case. Or they recommend committing the node_modules folder (to which there is fanatical opposition from others).
npm clone dependency-management
asked Nov 22 '18 at 5:56
DwightDwight
6071917
add a comment |
This is more of an architectural approach question than a specific programming issue.
The situation:
Due to issues like the left pad incident and dependencies that are releasing breaking changes without changing their version numbers, some build failures can occur without changing any project code or npm configuration information such as package.json and package-lock.json. My build system will run npm install on the repository and pull down the dependencies before each build. Even with a package-lock.json file committed and in play, the build can randomly fail (and recently has failed) due to changes in dependencies that I do not control.
It has been decided, corporately, that we shall no longer allow this to happen.
Considering the options, the easiest solution seems to be simply committing the node_modules folder with a known good configuration. However, I fear this may cause issues due to the different OS usage between developer clients and build systems, Mac and Linux, respectively. Apparently some dependencies will make environment-specific compilations upon install, and committing them will result in the wrong binaries being present on the build system.
Alternatively, I've been looking for a way to mass clone not just our dependencies, but our dependencies' dependencies in some automated fashion to our local repositories. In theory, I could get my dependencies from my clones of the repository on both the dev clients and build servers, assured that the dependencies cannot change in any way, no matter how deep, unless I decide to change them manually. However, though I can certainly specify dependencies relative to a git repo URL in package.json, the package-lock.json file is the only file that actually contains all of the dependencies. And it does not reference git repo URLs, but some "resolved" URL that references a .tgz file provided by npmjs.org.
Is there a way to force my NPM commands or the package-lock.json file to use custom repos for ALL dependencies, and point them to our clones?
Am I missing some other method of achieving this goal? How have you solved this problem?
Most articles and StackOverflow questions on the subject seem to either endorse the idea that NPM, combined with a checked in package-lock.json will always solve this problem, which is incorrect in our case. Or they recommend committing the node_modules folder (to which there is fanatical opposition from others).
npm clone dependency-management
asked Nov 22 '18 at 5:56
DwightDwight
6071917
This is more of an architectural approach question than a specific programming issue.
The situation:
Due to issues like the left pad incident and dependencies that are releasing breaking changes without changing their version numbers, some build failures can occur without changing any project code or npm configuration information such as package.json and package-lock.json. My build system will run npm install on the repository and pull down the dependencies before each build. Even with a package-lock.json file committed and in play, the build can randomly fail (and recently has failed) due to changes in dependencies that I do not control.
It has been decided, corporately, that we shall no longer allow this to happen.
Considering the options, the easiest solution seems to be simply committing the node_modules folder with a known good configuration. However, I fear this may cause issues due to the different OS usage between developer clients and build systems, Mac and Linux, respectively. Apparently some dependencies will make environment-specific compilations upon install, and committing them will result in the wrong binaries being present on the build system.
Alternatively, I've been looking for a way to mass clone not just our dependencies, but our dependencies' dependencies in some automated fashion to our local repositories. In theory, I could get my dependencies from my clones of the repository on both the dev clients and build servers, assured that the dependencies cannot change in any way, no matter how deep, unless I decide to change them manually. However, though I can certainly specify dependencies relative to a git repo URL in package.json, the package-lock.json file is the only file that actually contains all of the dependencies. And it does not reference git repo URLs, but some "resolved" URL that references a .tgz file provided by npmjs.org.
Is there a way to force my NPM commands or the package-lock.json file to use custom repos for ALL dependencies, and point them to our clones?
Am I missing some other method of achieving this goal? How have you solved this problem?
Most articles and StackOverflow questions on the subject seem to either endorse the idea that NPM, combined with a checked in package-lock.json will always solve this problem, which is incorrect in our case. Or they recommend committing the node_modules folder (to which there is fanatical opposition from others).
npm clone dependency-management
npm clone dependency-management
asked Nov 22 '18 at 5:56
DwightDwight
6071917
asked Nov 22 '18 at 5:56
DwightDwight
6071917
edited Nov 22 '18 at 6:02
Dwight
asked Nov 22 '18 at 5:56
DwightDwight
6071917
asked Nov 22 '18 at 5:56
DwightDwight
6071917
asked Nov 22 '18 at 5:56
DwightDwight
6071917
6071917
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53424687%2fforce-npm-to-install-all-dependencies-from-custom-cloned-repositories%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53424687%2fforce-npm-to-install-all-dependencies-from-custom-cloned-repositories%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
