ClobberWarnings when installing pylint for conda-build tests #27

pmspire · 2019-02-13T19:06:22Z

Issue:

Apologies in advance if this ends up having nothing to do with this feedstock... Maybe it's a conda-build problem, but I didn't find anything appropriate in their GitHub issues and thought I might get some insight here. I'd be grateful for any input.

When building a package that uses pylint for tests, I'm getting warnings/errors like

ClobberWarning: This transaction has incompatible packages due to a shared path.
  packages: conda-forge::backports-1.0-py_2, conda-forge::configparser-3.7.1-py27_0
  path: 'lib/python2.7/site-packages/backports/__init__.py'

ClobberWarning: This transaction has incompatible packages due to a shared path.
  packages: conda-forge::backports-1.0-py_2, conda-forge::configparser-3.7.1-py27_0
  path: 'lib/python2.7/site-packages/backports/__init__.pyc'

ERROR:conda.stderr:
ClobberWarning: Conda was asked to clobber an existing path.
  source path: /home/pmadden/miniconda3/pkgs/configparser-3.7.1-py27_0/lib/python2.7/site-packages/backports/__init__.py
  target path: /home/pmadden/miniconda3/conda-bld/foo_1550082840264/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python2.7/site-packages/backports/__init__.py

ERROR:conda.stderr:
ClobberWarning: Conda was asked to clobber an existing path.
  source path: /home/pmadden/miniconda3/pkgs/configparser-3.7.1-py27_0/lib/python2.7/site-packages/backports/__init__.pyc
  target path: /home/pmadden/miniconda3/conda-bld/foo_1550082840264/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python2.7/site-packages/backports/__init__.pyc

Here's the recipe:

% cat a.py
print

% cat run_test.sh 
/bin/true

% cat meta.yaml 
package:
  name: foo
  version: 0.0.1

requirements:
  build:
    - python 2.7
  run:
    - python 2.7

source:
  path: .

test:
  requires:
    - pylint
  source_files:
    - a.py

I'm building with

conda build -c conda-forge --override-channels .

I get the same warnings (some claim to be errors, but the build seems to proceed in any case) with a Miniconda3 as with Miniconda2. However, if I change the python version from 2.7 to 3.7 in meta.yaml, the warnings/errors disappear.

I actually get the same behavior if I do conda build . and so use defaults packages instead of conda-forge, but I thought I might make better progress asking here about this.

Environment (conda list):

$ conda list

# packages in environment at /home/pmadden/miniconda2:
#
# Name                    Version                   Build  Channel
asn1crypto                0.24.0                   py27_0  
backports                 1.0                      py27_1  
backports.functools_lru_cache 1.5                      py27_1  
beautifulsoup4            4.7.1                    py27_1  
bzip2                     1.0.6                h14c3975_5  
ca-certificates           2019.1.23                     0  
certifi                   2018.11.29               py27_0  
cffi                      1.11.5           py27he75722e_1  
chardet                   3.0.4                    py27_1  
conda                     4.6.3                    py27_0  
conda-build               3.17.8                   py27_0  
conda-env                 2.6.0                         1  
contextlib2               0.5.5                    py27_0  
cryptography              2.4.2            py27h1ba5d50_0  
enum34                    1.1.6                    py27_1  
filelock                  3.0.10                   py27_0  
futures                   3.2.0                    py27_0  
glob2                     0.6                      py27_1  
icu                       58.2                 h9c2bf20_1  
idna                      2.8                      py27_0  
ipaddress                 1.0.22                   py27_0  
jinja2                    2.10                     py27_0  
libarchive                3.3.3                h5d8350f_5  
libedit                   3.1.20170329         h6b74fdf_2  
libffi                    3.2.1                hd88cf55_4  
libgcc-ng                 8.2.0                hdf63c60_1  
liblief                   0.9.0                h7725739_2  
libstdcxx-ng              8.2.0                hdf63c60_1  
libxml2                   2.9.9                he19cac6_0  
lz4-c                     1.8.1.2              h14c3975_0  
lzo                       2.10                 h49e0be7_2  
markupsafe                1.1.0            py27h7b6447c_0  
ncurses                   6.1                  he6710b0_1  
openssl                   1.1.1a               h7b6447c_0  
patchelf                  0.9                  he6710b0_3  
pip                       18.1                     py27_0  
pkginfo                   1.5.0.1                  py27_0  
psutil                    5.5.0            py27h7b6447c_0  
py-lief                   0.9.0            py27h7725739_2  
pycosat                   0.6.3            py27h14c3975_0  
pycparser                 2.19                     py27_0  
pyopenssl                 18.0.0                   py27_0  
pysocks                   1.6.8                    py27_0  
python                    2.7.15               h9bab390_6  
python-libarchive-c       2.8                      py27_6  
pytz                      2018.9                   py27_0  
pyyaml                    3.13             py27h14c3975_0  
readline                  7.0                  h7b6447c_5  
requests                  2.21.0                   py27_0  
ruamel_yaml               0.15.46          py27h14c3975_0  
scandir                   1.9.0            py27h14c3975_0  
setuptools                40.6.3                   py27_0  
six                       1.12.0                   py27_0  
soupsieve                 1.7.1                    py27_0  
sqlite                    3.26.0               h7b6447c_0  
tk                        8.6.8                hbc83047_0  
tqdm                      4.29.1                     py_0  
urllib3                   1.24.1                   py27_0  
wheel                     0.32.3                   py27_0  
xz                        5.2.4                h14c3975_4  
yaml                      0.1.7                had09818_2  
zlib                      1.2.11               h7b6447c_3  
zstd                      1.3.7                h0b5b093_0

Details about conda and system ( conda info ):

$ conda info

     active environment : base
    active env location : /home/pmadden/miniconda2
            shell level : 1
       user config file : /home/pmadden/.condarc
 populated config files : 
          conda version : 4.6.3
    conda-build version : 3.17.8
         python version : 2.7.15.final.0
       base environment : /home/pmadden/miniconda2  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/pmadden/miniconda2/pkgs
                          /home/pmadden/.conda/pkgs
       envs directories : /home/pmadden/miniconda2/envs
                          /home/pmadden/.conda/envs
               platform : linux-64
             user-agent : conda/4.6.3 requests/2.21.0 CPython/2.7.15 Linux/4.15.0-45-generic ubuntu/18.04.2 glibc/2.27
                UID:GID : 1000:1000
             netrc file : /home/pmadden/.netrc
           offline mode : False

The text was updated successfully, but these errors were encountered:

jakirkham · 2019-02-13T21:24:08Z

Thanks @pmspire. Sounds like the issue is in configparser.

Can you please try installing configparser and backports into a fresh environment and see if the issue persists? If so, could you please raise that as an issue on the configparser feedstock?

pmspire · 2019-02-14T23:17:31Z

I tried, but wasn't able, to reproduce this issue only via conda create and/or conda install commands. It seems to happen only via conda build, and in the test phase. But I did manage to reproduce it without pylint, and so opened issue conda-forge/configparser-feedstock#19.

jakirkham · 2019-02-19T03:19:43Z

Thanks @pmspire. There's probably a flag that we would need to add to the conda command to get it to complain about packages clobbering each other. Though your reproducer seems fine to me. Added a comment upstream based on what you found there.

pmspire · 2019-02-19T17:06:16Z

conda-forge/configparser-feedstock#20 is open to potentially address this upstream.

jakirkham · 2019-02-19T22:44:24Z

Thanks @pmspire. Have merged that now. Can you please retest after the new configparser packages are built?

pmspire · 2019-02-19T22:48:56Z

Will do. Is there a better way for me to know when new packages are available than to check with conda search?

pmspire · 2019-02-19T23:18:26Z

The 3.7.1 build 1 package became available, and I can confirm that the ClobberWarnings are gone when I build my test recipe, which indeed pulls in the 3.7.1-py37_1 package. I think this is ok to close if you're happy, too.

pmspire · 2019-02-19T23:19:11Z

Hang on, I'm in the wrong issue! Let me double-check this reproducer, with pylint.

pmspire · 2019-02-19T23:20:55Z

Yes, confirmed: The recipe mentioned at the top of this issue also now builds without the ClobberWarning messages.

jakirkham · 2019-02-24T01:43:46Z

Thanks @pmspire :)

sebastian-luna-valero · 2019-11-12T20:49:26Z

Hi,

I am experiencing a similar issue when updating this feedstock:
https://github.com/conda-forge/libvips-feedstock/pull/4/checks?check_run_id=298253696

The error is:

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... 
ClobberWarning: Conda was asked to clobber an existing path.
  source path: /opt/conda/pkgs/libvips-8.8.1-h9154f7a_0/lib//
  target path: /home/conda/feedstock_root/build_artifacts/libvips_1573585782684/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib//



done
ERROR:conda.core.link:An error occurred while installing package 'local::libvips-8.8.1-h9154f7a_0'.
ERROR conda.core.link:_execute(700): An error occurred while installing package 'local::libvips-8.8.1-h9154f7a_0'.
Rolling back transaction: ...working... done
Traceback (most recent call last):
  File "/opt/conda/bin/conda-build", line 11, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 469, in main
    execute(sys.argv[1:])
  File "/opt/conda/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 460, in execute
    verify=args.verify, variants=args.variants)
  File "/opt/conda/lib/python3.7/site-packages/conda_build/api.py", line 209, in build
    notest=notest, need_source_download=need_source_download, variants=variants)
  File "/opt/conda/lib/python3.7/site-packages/conda_build/build.py", line 2350, in build_tree
    test(pkg, config=metadata.config.copy(), stats=stats)
  File "/opt/conda/lib/python3.7/site-packages/conda_build/build.py", line 2155, in test
    is_conda=metadata.name() == 'conda')
  File "/opt/conda/lib/python3.7/site-packages/conda_build/environ.py", line 868, in create_env
    execute_actions(actions, index)
  File "/opt/conda/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
    return f(*args, **kwds)
  File "/opt/conda/lib/python3.7/site-packages/conda/plan.py", line 321, in execute_actions
    execute_instructions(plan, index, verbose)
  File "/opt/conda/lib/python3.7/site-packages/conda/plan.py", line 533, in execute_instructions
    cmd(state, arg)
  File "/opt/conda/lib/python3.7/site-packages/conda/instructions.py", line 73, in UNLINKLINKTRANSACTION_CMD
    unlink_link_transaction.execute()
  File "/opt/conda/lib/python3.7/site-packages/conda/core/link.py", line 249, in execute
    self._execute(tuple(concat(interleave(itervalues(self.prefix_action_groups)))))
  File "/opt/conda/lib/python3.7/site-packages/conda/core/link.py", line 716, in _execute
    rollback_excs,
conda.CondaMultiError: [Errno 21] Is a directory: '/opt/conda/pkgs/libvips-8.8.1-h9154f7a_0/lib//'
()

I haven't been able to find any help out there, could you please let us know your thoughts?

Best regards,
Sebastian

jakirkham · 2019-11-12T21:04:55Z

@sebastian-luna-valero, please open an issue on that feedstock. Thanks!

sebastian-luna-valero · 2019-11-13T07:03:43Z

Thanks @jakirkham

I am one of the maintainers of the other feedstock, and I don't know how to proceed. Please let me know how to ask for help.

jakirkham · 2019-11-13T08:22:51Z

Would open the issue on the other feedstock and cc core for help 🙂

pmspire mentioned this issue Feb 14, 2019

ClobberWarning via simple conda-build recipe conda-forge/configparser-feedstock#19

Closed

jakirkham closed this as completed Feb 24, 2019

sebastian-luna-valero mentioned this issue Nov 12, 2019

update version to 8.8.3 conda-forge/libvips-feedstock#4

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ClobberWarnings when installing pylint for conda-build tests #27

ClobberWarnings when installing pylint for conda-build tests #27

pmspire commented Feb 13, 2019

jakirkham commented Feb 13, 2019

pmspire commented Feb 14, 2019

jakirkham commented Feb 19, 2019

pmspire commented Feb 19, 2019

jakirkham commented Feb 19, 2019

pmspire commented Feb 19, 2019

pmspire commented Feb 19, 2019

pmspire commented Feb 19, 2019

pmspire commented Feb 19, 2019

jakirkham commented Feb 24, 2019

sebastian-luna-valero commented Nov 12, 2019

jakirkham commented Nov 12, 2019

sebastian-luna-valero commented Nov 13, 2019

jakirkham commented Nov 13, 2019

ClobberWarnings when installing pylint for conda-build tests #27

ClobberWarnings when installing pylint for conda-build tests #27

Comments

pmspire commented Feb 13, 2019

jakirkham commented Feb 13, 2019

pmspire commented Feb 14, 2019

jakirkham commented Feb 19, 2019

pmspire commented Feb 19, 2019

jakirkham commented Feb 19, 2019

pmspire commented Feb 19, 2019

pmspire commented Feb 19, 2019

pmspire commented Feb 19, 2019

pmspire commented Feb 19, 2019

jakirkham commented Feb 24, 2019

sebastian-luna-valero commented Nov 12, 2019

jakirkham commented Nov 12, 2019

sebastian-luna-valero commented Nov 13, 2019

jakirkham commented Nov 13, 2019