Styjun

I came across an error when trying to install and test my package.

The error was caused by conflicting versions of transitive dependencies of two packages - one in setup_requires (twine) and another in install_requires (tensorflow).

Both transitively depend on bleach, each with a different version.

Installing the package with pip install -e . or python setup.py develop works as expected.

When running python setup.py test, an error occurs (note: this happens only when running tests after installing. If the package was not installed first, there is no error).

Traceback (most recent call last):
 File "setup.py", line 3, in <module>
 setup()
 File "/usr/local/lib/python3.6/site-packages/setuptools/__init__.py", line 145, in setup
 return distutils.core.setup(**attrs)
 File "/usr/local/lib/python3.6/distutils/core.py", line 148, in setup
 dist.run_commands()
 File "/usr/local/lib/python3.6/distutils/dist.py", line 955, in run_commands
 self.run_command(cmd)
 File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
 cmd_obj.run()
 File "/root/test/.eggs/pytest_runner-4.4-py3.6.egg/ptr.py", line 189, in run
 with self.project_on_sys_path():
 File "/usr/local/lib/python3.6/contextlib.py", line 81, in __enter__
 return next(self.gen)
 File "/usr/local/lib/python3.6/site-packages/setuptools/command/test.py", line 166, in project_on_sys_path
 require('%s==%s' % (ei_cmd.egg_name, ei_cmd.egg_version))
 File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 900, in require
 needed = self.resolve(parse_requirements(requirements))
 File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 791, in resolve
 raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (bleach 3.1.0 (/root/test/.eggs/bleach-3.1.0-py3.6.egg), Requirement.parse('bleach==1.5.0'), 'tensorboard')

Steps to reproduce

I get the error with this minimal package structure:

my-package
|-- src/
| |-- module.py (empty)
|-- setup.py
|-- setup.cfg

setup.cfg looks like this:

[metadata]
name = test
version = 0.0.1

[options]
package_dir=
 = src

packages=find:

setup_requires =
 setuptools >= 40.0.0
 pip >= 10
 pytest-runner >= 2.0
 twine >= 1.0.0

install_requires =
 tensorflow == 1.6.0


tests_require =
 pytest >=3.0, <4.0

[options.packages.find]
where=src

[aliases]
test=pytest

setup.py look like this:

from setuptools import setup

setup()

Steps:

1. Run pip install -e .
   


2. Run python setup.py test
   

Environment:

• 
  python:3.6 Docker image

What I think is happening

As far as I can tell, when running the commands (both installing and testing), packages from setup_requires are installed and egg files are placed in .eggs directory.

When running python setup.py test, setuptools installs all packages from install_requires but sees that there is already an egg for one of the dependencies and uses that.

I'm not sure what the exact mechanism is.

Possible solutions

• Use newer version of tensorflow. This solved the conflict


• Or: remove twine from setup_requires - I realised it is not required for setup


• Or: Do not install the package locally. Only run python setup.py test
  

Questions

Although I was able to solve the problem in this specific case, I would still like to know more about what caused it in the first place.

• At what point is the .eggs directory created?


• Why the install works fine but tests fail?


• Why tests fail only after install?


• Is there a way to prevent the conflict from happening?