Install by default, “optional” dependencies in Python (setuptools)
Is there a way to specify optional dependencies for a Python package that should be installed by default from pip
but for which an install should not be considered a failure if they cannot be installed?
I know that I can specify install_requires
so that the packages will be installed for the 90% of users using OSes that can easily install certain optional dependencies, and I also know I can specify extra_require
to specify that users can declare they want a full install to get these features, but I haven't found a way to make a default pip
install try to install the packages but not complain if they cannot be installed.
(The particular package I'd like to update the setuptools
and setup.py
for is called music21
for which 95% of the tools can be run without matplotlib, IPython, scipy, pygame, some obscure audio tools etc. but the package gains extra abilities and speed if these packages are installed, and I'd prefer to let people have these abilities by default but not report errors if they cannot be installed)
python pip setuptools music21
add a comment |
Is there a way to specify optional dependencies for a Python package that should be installed by default from pip
but for which an install should not be considered a failure if they cannot be installed?
I know that I can specify install_requires
so that the packages will be installed for the 90% of users using OSes that can easily install certain optional dependencies, and I also know I can specify extra_require
to specify that users can declare they want a full install to get these features, but I haven't found a way to make a default pip
install try to install the packages but not complain if they cannot be installed.
(The particular package I'd like to update the setuptools
and setup.py
for is called music21
for which 95% of the tools can be run without matplotlib, IPython, scipy, pygame, some obscure audio tools etc. but the package gains extra abilities and speed if these packages are installed, and I'd prefer to let people have these abilities by default but not report errors if they cannot be installed)
python pip setuptools music21
add a comment |
Is there a way to specify optional dependencies for a Python package that should be installed by default from pip
but for which an install should not be considered a failure if they cannot be installed?
I know that I can specify install_requires
so that the packages will be installed for the 90% of users using OSes that can easily install certain optional dependencies, and I also know I can specify extra_require
to specify that users can declare they want a full install to get these features, but I haven't found a way to make a default pip
install try to install the packages but not complain if they cannot be installed.
(The particular package I'd like to update the setuptools
and setup.py
for is called music21
for which 95% of the tools can be run without matplotlib, IPython, scipy, pygame, some obscure audio tools etc. but the package gains extra abilities and speed if these packages are installed, and I'd prefer to let people have these abilities by default but not report errors if they cannot be installed)
python pip setuptools music21
Is there a way to specify optional dependencies for a Python package that should be installed by default from pip
but for which an install should not be considered a failure if they cannot be installed?
I know that I can specify install_requires
so that the packages will be installed for the 90% of users using OSes that can easily install certain optional dependencies, and I also know I can specify extra_require
to specify that users can declare they want a full install to get these features, but I haven't found a way to make a default pip
install try to install the packages but not complain if they cannot be installed.
(The particular package I'd like to update the setuptools
and setup.py
for is called music21
for which 95% of the tools can be run without matplotlib, IPython, scipy, pygame, some obscure audio tools etc. but the package gains extra abilities and speed if these packages are installed, and I'd prefer to let people have these abilities by default but not report errors if they cannot be installed)
python pip setuptools music21
python pip setuptools music21
asked Nov 19 '18 at 13:53
Michael Scott Cuthbert
1,74421233
1,74421233
add a comment |
add a comment |
3 Answers
3
active
oldest
votes
Not a perfect solution by any means, but you could setup a post-install script to try to install the packages, something like this:
from distutils.core import setup
from distutils import debug
from setuptools.command.install import install
class PostInstallExtrasInstaller(install):
extras_install_by_default = ['matplotlib', 'nothing']
@classmethod
def pip_main(cls, *args, **kwargs):
def pip_main(*args, **kwargs):
raise Exception('No pip module found')
try:
from pip import main as pip_main
except ImportError:
from pip._internal import main as pip_main
ret = pip_main(*args, **kwargs)
if ret:
raise Exception(f'Exitcode {ret}')
return ret
def run(self):
for extra in self.extras_install_by_default:
try:
self.pip_main(['install', extra])
except Exception as E:
print(f'Optional package {extra} not installed: {E}')
else:
print(f"Optional package {extra} installed")
return install.run(self)
setup(
name='python-package-ignore-extra-dep-failures',
version='0.1dev',
packages=['somewhat',],
license='Creative Commons Attribution-Noncommercial-Share Alike license',
install_requires=['requests',],
extras_require={
'extras': PostInstallExtrasInstaller.extras_install_by_default,
},
cmdclass={
'install': PostInstallExtrasInstaller,
},
)
This seems like a great idea I hadn’t thought of. Worthy of the bounty if nothing simpler emerges.
– Michael Scott Cuthbert
Nov 30 '18 at 3:28
1
Beware that post-install commands are useless withpip
, except that you explicilty instruct your users to install viapip install pkgname --no-binary=pkgname
. This will only work when invokingpython setup.py install
directly.
– hoefling
Nov 30 '18 at 16:15
1
Please don't recommend that users importpip
: it is a command line tool, and doesn't support any importable API. Doing so causes breakage and confusion for users, and headache forpip
developers when they need to move things around internally (hence yourtry
/except
-- thepip._internal
module indicates that this is internal topip
and should not be imported.)
– Dustin Ingram
Dec 1 '18 at 5:36
add a comment |
The simplest way to do this is by adding a custom install command that simply shells out to pip to install the "optional" packages. In your setup.py
:
import sys
import subprocess
from setuptools import setup
from setuptools.command.install import install
class MyInstall(install):
def run(self):
subprocess.call([sys.executable, "-m", "pip", "install", "whatever"])
install.run(self)
setup(
...
cmdclass={
'install': MyInstall,
},
)
Like hoefling
said above, this will only work if you publish a source distribution (.tar.gz
or .zip
). It will not work if you publish your package as a built distribution (.whl
).
add a comment |
This might be what you are looking for. It's appears to be a built in feature of setup tools that allows you to declare "optional dependencies".
https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies
Ex
setup(
name="Project-A",
...
extras_require={
'PDF': ["ReportLab>=1.2", "RXP"],
'reST': ["docutils>=0.3"],
}
)
these unfortunately do not install by default. They're a great way for power users to decide what to install, but not helpful for most users.
– Michael Scott Cuthbert
Dec 6 '18 at 3:07
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53376135%2finstall-by-default-optional-dependencies-in-python-setuptools%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
Not a perfect solution by any means, but you could setup a post-install script to try to install the packages, something like this:
from distutils.core import setup
from distutils import debug
from setuptools.command.install import install
class PostInstallExtrasInstaller(install):
extras_install_by_default = ['matplotlib', 'nothing']
@classmethod
def pip_main(cls, *args, **kwargs):
def pip_main(*args, **kwargs):
raise Exception('No pip module found')
try:
from pip import main as pip_main
except ImportError:
from pip._internal import main as pip_main
ret = pip_main(*args, **kwargs)
if ret:
raise Exception(f'Exitcode {ret}')
return ret
def run(self):
for extra in self.extras_install_by_default:
try:
self.pip_main(['install', extra])
except Exception as E:
print(f'Optional package {extra} not installed: {E}')
else:
print(f"Optional package {extra} installed")
return install.run(self)
setup(
name='python-package-ignore-extra-dep-failures',
version='0.1dev',
packages=['somewhat',],
license='Creative Commons Attribution-Noncommercial-Share Alike license',
install_requires=['requests',],
extras_require={
'extras': PostInstallExtrasInstaller.extras_install_by_default,
},
cmdclass={
'install': PostInstallExtrasInstaller,
},
)
This seems like a great idea I hadn’t thought of. Worthy of the bounty if nothing simpler emerges.
– Michael Scott Cuthbert
Nov 30 '18 at 3:28
1
Beware that post-install commands are useless withpip
, except that you explicilty instruct your users to install viapip install pkgname --no-binary=pkgname
. This will only work when invokingpython setup.py install
directly.
– hoefling
Nov 30 '18 at 16:15
1
Please don't recommend that users importpip
: it is a command line tool, and doesn't support any importable API. Doing so causes breakage and confusion for users, and headache forpip
developers when they need to move things around internally (hence yourtry
/except
-- thepip._internal
module indicates that this is internal topip
and should not be imported.)
– Dustin Ingram
Dec 1 '18 at 5:36
add a comment |
Not a perfect solution by any means, but you could setup a post-install script to try to install the packages, something like this:
from distutils.core import setup
from distutils import debug
from setuptools.command.install import install
class PostInstallExtrasInstaller(install):
extras_install_by_default = ['matplotlib', 'nothing']
@classmethod
def pip_main(cls, *args, **kwargs):
def pip_main(*args, **kwargs):
raise Exception('No pip module found')
try:
from pip import main as pip_main
except ImportError:
from pip._internal import main as pip_main
ret = pip_main(*args, **kwargs)
if ret:
raise Exception(f'Exitcode {ret}')
return ret
def run(self):
for extra in self.extras_install_by_default:
try:
self.pip_main(['install', extra])
except Exception as E:
print(f'Optional package {extra} not installed: {E}')
else:
print(f"Optional package {extra} installed")
return install.run(self)
setup(
name='python-package-ignore-extra-dep-failures',
version='0.1dev',
packages=['somewhat',],
license='Creative Commons Attribution-Noncommercial-Share Alike license',
install_requires=['requests',],
extras_require={
'extras': PostInstallExtrasInstaller.extras_install_by_default,
},
cmdclass={
'install': PostInstallExtrasInstaller,
},
)
This seems like a great idea I hadn’t thought of. Worthy of the bounty if nothing simpler emerges.
– Michael Scott Cuthbert
Nov 30 '18 at 3:28
1
Beware that post-install commands are useless withpip
, except that you explicilty instruct your users to install viapip install pkgname --no-binary=pkgname
. This will only work when invokingpython setup.py install
directly.
– hoefling
Nov 30 '18 at 16:15
1
Please don't recommend that users importpip
: it is a command line tool, and doesn't support any importable API. Doing so causes breakage and confusion for users, and headache forpip
developers when they need to move things around internally (hence yourtry
/except
-- thepip._internal
module indicates that this is internal topip
and should not be imported.)
– Dustin Ingram
Dec 1 '18 at 5:36
add a comment |
Not a perfect solution by any means, but you could setup a post-install script to try to install the packages, something like this:
from distutils.core import setup
from distutils import debug
from setuptools.command.install import install
class PostInstallExtrasInstaller(install):
extras_install_by_default = ['matplotlib', 'nothing']
@classmethod
def pip_main(cls, *args, **kwargs):
def pip_main(*args, **kwargs):
raise Exception('No pip module found')
try:
from pip import main as pip_main
except ImportError:
from pip._internal import main as pip_main
ret = pip_main(*args, **kwargs)
if ret:
raise Exception(f'Exitcode {ret}')
return ret
def run(self):
for extra in self.extras_install_by_default:
try:
self.pip_main(['install', extra])
except Exception as E:
print(f'Optional package {extra} not installed: {E}')
else:
print(f"Optional package {extra} installed")
return install.run(self)
setup(
name='python-package-ignore-extra-dep-failures',
version='0.1dev',
packages=['somewhat',],
license='Creative Commons Attribution-Noncommercial-Share Alike license',
install_requires=['requests',],
extras_require={
'extras': PostInstallExtrasInstaller.extras_install_by_default,
},
cmdclass={
'install': PostInstallExtrasInstaller,
},
)
Not a perfect solution by any means, but you could setup a post-install script to try to install the packages, something like this:
from distutils.core import setup
from distutils import debug
from setuptools.command.install import install
class PostInstallExtrasInstaller(install):
extras_install_by_default = ['matplotlib', 'nothing']
@classmethod
def pip_main(cls, *args, **kwargs):
def pip_main(*args, **kwargs):
raise Exception('No pip module found')
try:
from pip import main as pip_main
except ImportError:
from pip._internal import main as pip_main
ret = pip_main(*args, **kwargs)
if ret:
raise Exception(f'Exitcode {ret}')
return ret
def run(self):
for extra in self.extras_install_by_default:
try:
self.pip_main(['install', extra])
except Exception as E:
print(f'Optional package {extra} not installed: {E}')
else:
print(f"Optional package {extra} installed")
return install.run(self)
setup(
name='python-package-ignore-extra-dep-failures',
version='0.1dev',
packages=['somewhat',],
license='Creative Commons Attribution-Noncommercial-Share Alike license',
install_requires=['requests',],
extras_require={
'extras': PostInstallExtrasInstaller.extras_install_by_default,
},
cmdclass={
'install': PostInstallExtrasInstaller,
},
)
answered Nov 30 '18 at 2:34
mingaleg
1,201920
1,201920
This seems like a great idea I hadn’t thought of. Worthy of the bounty if nothing simpler emerges.
– Michael Scott Cuthbert
Nov 30 '18 at 3:28
1
Beware that post-install commands are useless withpip
, except that you explicilty instruct your users to install viapip install pkgname --no-binary=pkgname
. This will only work when invokingpython setup.py install
directly.
– hoefling
Nov 30 '18 at 16:15
1
Please don't recommend that users importpip
: it is a command line tool, and doesn't support any importable API. Doing so causes breakage and confusion for users, and headache forpip
developers when they need to move things around internally (hence yourtry
/except
-- thepip._internal
module indicates that this is internal topip
and should not be imported.)
– Dustin Ingram
Dec 1 '18 at 5:36
add a comment |
This seems like a great idea I hadn’t thought of. Worthy of the bounty if nothing simpler emerges.
– Michael Scott Cuthbert
Nov 30 '18 at 3:28
1
Beware that post-install commands are useless withpip
, except that you explicilty instruct your users to install viapip install pkgname --no-binary=pkgname
. This will only work when invokingpython setup.py install
directly.
– hoefling
Nov 30 '18 at 16:15
1
Please don't recommend that users importpip
: it is a command line tool, and doesn't support any importable API. Doing so causes breakage and confusion for users, and headache forpip
developers when they need to move things around internally (hence yourtry
/except
-- thepip._internal
module indicates that this is internal topip
and should not be imported.)
– Dustin Ingram
Dec 1 '18 at 5:36
This seems like a great idea I hadn’t thought of. Worthy of the bounty if nothing simpler emerges.
– Michael Scott Cuthbert
Nov 30 '18 at 3:28
This seems like a great idea I hadn’t thought of. Worthy of the bounty if nothing simpler emerges.
– Michael Scott Cuthbert
Nov 30 '18 at 3:28
1
1
Beware that post-install commands are useless with
pip
, except that you explicilty instruct your users to install via pip install pkgname --no-binary=pkgname
. This will only work when invoking python setup.py install
directly.– hoefling
Nov 30 '18 at 16:15
Beware that post-install commands are useless with
pip
, except that you explicilty instruct your users to install via pip install pkgname --no-binary=pkgname
. This will only work when invoking python setup.py install
directly.– hoefling
Nov 30 '18 at 16:15
1
1
Please don't recommend that users import
pip
: it is a command line tool, and doesn't support any importable API. Doing so causes breakage and confusion for users, and headache for pip
developers when they need to move things around internally (hence your try
/except
-- the pip._internal
module indicates that this is internal to pip
and should not be imported.)– Dustin Ingram
Dec 1 '18 at 5:36
Please don't recommend that users import
pip
: it is a command line tool, and doesn't support any importable API. Doing so causes breakage and confusion for users, and headache for pip
developers when they need to move things around internally (hence your try
/except
-- the pip._internal
module indicates that this is internal to pip
and should not be imported.)– Dustin Ingram
Dec 1 '18 at 5:36
add a comment |
The simplest way to do this is by adding a custom install command that simply shells out to pip to install the "optional" packages. In your setup.py
:
import sys
import subprocess
from setuptools import setup
from setuptools.command.install import install
class MyInstall(install):
def run(self):
subprocess.call([sys.executable, "-m", "pip", "install", "whatever"])
install.run(self)
setup(
...
cmdclass={
'install': MyInstall,
},
)
Like hoefling
said above, this will only work if you publish a source distribution (.tar.gz
or .zip
). It will not work if you publish your package as a built distribution (.whl
).
add a comment |
The simplest way to do this is by adding a custom install command that simply shells out to pip to install the "optional" packages. In your setup.py
:
import sys
import subprocess
from setuptools import setup
from setuptools.command.install import install
class MyInstall(install):
def run(self):
subprocess.call([sys.executable, "-m", "pip", "install", "whatever"])
install.run(self)
setup(
...
cmdclass={
'install': MyInstall,
},
)
Like hoefling
said above, this will only work if you publish a source distribution (.tar.gz
or .zip
). It will not work if you publish your package as a built distribution (.whl
).
add a comment |
The simplest way to do this is by adding a custom install command that simply shells out to pip to install the "optional" packages. In your setup.py
:
import sys
import subprocess
from setuptools import setup
from setuptools.command.install import install
class MyInstall(install):
def run(self):
subprocess.call([sys.executable, "-m", "pip", "install", "whatever"])
install.run(self)
setup(
...
cmdclass={
'install': MyInstall,
},
)
Like hoefling
said above, this will only work if you publish a source distribution (.tar.gz
or .zip
). It will not work if you publish your package as a built distribution (.whl
).
The simplest way to do this is by adding a custom install command that simply shells out to pip to install the "optional" packages. In your setup.py
:
import sys
import subprocess
from setuptools import setup
from setuptools.command.install import install
class MyInstall(install):
def run(self):
subprocess.call([sys.executable, "-m", "pip", "install", "whatever"])
install.run(self)
setup(
...
cmdclass={
'install': MyInstall,
},
)
Like hoefling
said above, this will only work if you publish a source distribution (.tar.gz
or .zip
). It will not work if you publish your package as a built distribution (.whl
).
edited Dec 1 '18 at 5:43
answered Dec 1 '18 at 5:32
Dustin Ingram
2,8441125
2,8441125
add a comment |
add a comment |
This might be what you are looking for. It's appears to be a built in feature of setup tools that allows you to declare "optional dependencies".
https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies
Ex
setup(
name="Project-A",
...
extras_require={
'PDF': ["ReportLab>=1.2", "RXP"],
'reST': ["docutils>=0.3"],
}
)
these unfortunately do not install by default. They're a great way for power users to decide what to install, but not helpful for most users.
– Michael Scott Cuthbert
Dec 6 '18 at 3:07
add a comment |
This might be what you are looking for. It's appears to be a built in feature of setup tools that allows you to declare "optional dependencies".
https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies
Ex
setup(
name="Project-A",
...
extras_require={
'PDF': ["ReportLab>=1.2", "RXP"],
'reST': ["docutils>=0.3"],
}
)
these unfortunately do not install by default. They're a great way for power users to decide what to install, but not helpful for most users.
– Michael Scott Cuthbert
Dec 6 '18 at 3:07
add a comment |
This might be what you are looking for. It's appears to be a built in feature of setup tools that allows you to declare "optional dependencies".
https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies
Ex
setup(
name="Project-A",
...
extras_require={
'PDF': ["ReportLab>=1.2", "RXP"],
'reST': ["docutils>=0.3"],
}
)
This might be what you are looking for. It's appears to be a built in feature of setup tools that allows you to declare "optional dependencies".
https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies
Ex
setup(
name="Project-A",
...
extras_require={
'PDF': ["ReportLab>=1.2", "RXP"],
'reST': ["docutils>=0.3"],
}
)
answered Dec 6 '18 at 1:30
Samy Bencherif
741619
741619
these unfortunately do not install by default. They're a great way for power users to decide what to install, but not helpful for most users.
– Michael Scott Cuthbert
Dec 6 '18 at 3:07
add a comment |
these unfortunately do not install by default. They're a great way for power users to decide what to install, but not helpful for most users.
– Michael Scott Cuthbert
Dec 6 '18 at 3:07
these unfortunately do not install by default. They're a great way for power users to decide what to install, but not helpful for most users.
– Michael Scott Cuthbert
Dec 6 '18 at 3:07
these unfortunately do not install by default. They're a great way for power users to decide what to install, but not helpful for most users.
– Michael Scott Cuthbert
Dec 6 '18 at 3:07
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53376135%2finstall-by-default-optional-dependencies-in-python-setuptools%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown