tar: Error opening archive: Unrecognized archive format when copying wheels to host

[mcleantom]

Description

Hey, I am building my python library using cibuildwheel and the wheels look like they are being built okay but it is failing at the last stage:

Successfully built arl_pyinterp        
+ /opt/python/cp38-cp38/bin/python 
-c 'import sys, json, glob; json.dump(glob.glob('"'"'/tmp/cibuildwheel/built_wheel/*.whl'"'"'), sys.stdout)'
+ rm -rf /tmp/cibuildwheel/repaired_wheel
+ mkdir -p /tmp/cibuildwheel/repaired_wheel


✓ 103.18sRepairing wheel...

+ sh -c 'auditwheel repair -w /tmp/cibuildwheel/repaired_wheel /tmp/cibuildwheel/built_wheel/arl_pyinterp-0.0.1-cp312-cp312-manylinux_2_17_x86_64.whl'  
INFO:auditwheel.main_repair:Repairing arl_pyinterp-0.0.1-cp312-cp312-manylinux_2_17_x86_64.whl
INFO:auditwheel.wheeltools:Previous filename tags: manylinux_2_17_x86_64      
INFO:auditwheel.wheeltools:New filename tags: manylinux_2_17_x86_64, manylinux2014_x86_64
INFO:auditwheel.wheeltools:Previous WHEEL info tags: cp312-cp312-manylinux_2_17_x86_64
INFO:auditwheel.wheeltools:New WHEEL info tags: cp312-cp312-manylinux_2_17_x86_64, cp312-cp312-manylinux2014_x86_64  
INFO:auditwheel.main_repair:
Fixed-up wheel written to /tmp/cibuildwheel/repaired_wheel/arl_pyinterp-0.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
+ /opt/python/cp38-cp38/bin/python -c 'import sys, json, glob; json.dump(glob.glob('"'"'/tmp/cibuildwheel/repaired_wheel/*.whl'"'"'), sys.stdout)'
+ mkdir -p /output
+ mv /tmp/cibuildwheel/repaired_wheel/arl_pyinterp-0.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl /output

✓ 1.17s
✓ cp312-manylinux_x86_64 finished in 104.51s
Copying wheels back to host...


tar: Error opening archive: Unrecognized archive format
417-4fca-a86a-59c4c8a41001 tar -cC /output -
 ✕ 2.06sError: Command docker exec -i cibuildwheel-8e0c5eb7-f417-4fca-a86a-59c4c8a41001 tar -cC /output -f - . | tar -xf - failed with code 1. None

I am not sure why this is happening, any pointers on how I could debug this?

I am installing my wheels on a windows platform running the command:

cibuildwheel --platform linux .

and my pyproject.toml file looks like:

[build-system]
requires = ["scikit-build-core>=0.3.3", "pybind11"]
build-backend = "scikit_build_core.build"

[project]
name = "arl_pyinterp"
version = "0.0.1"
description = "A minimal example package with scikit build core and vcpkg"
readme = "README.md"
authors = [
    { name = "Tom McLean",  email = "mcleantom97@gmail.com" }
]

requires-python = ">=3.11"
classifiers = [
  "Development Status :: 4 - Beta",
  "License :: OSI Approved :: MIT License",
  "Programming Language :: Python :: 3 :: Only",
  "Programming Language :: Python :: 3.7",
  "Programming Language :: Python :: 3.8",
  "Programming Language :: Python :: 3.9",
  "Programming Language :: Python :: 3.10",
  "Programming Language :: Python :: 3.11",
  "Programming Language :: Python :: 3.12",
]

[tool.scikit-build]
cmake.verbose = true
logging.level = "DEBUG"
cmake.args = [
    "-DCMAKE_TOOLCHAIN_FILE=./vcpkg/scripts/buildsystems/vcpkg.cmake",
    "-DVCPKG_INSTALLED_DIR=./vcpkg_installed",
    "-DBUILD_PYTHON_EXTENSION=ON"
]

[tool.pytest.ini_options]
minversion = "6.0"
addopts = ["-ra", "--showlocals", "--strict-markers", "--strict-config"]
xfail_strict = true
log_cli_level = "INFO"
filterwarnings = [
  "error",
]
testpaths = ["tests"]

[tool.cibuildwheel]
build = "cp*"
skip = ["cp*-musllinux*", "pp*"]
archs = ["x86_64", "AMD64", "arm64"]
manylinux-x86_64-image = "tmp"

[tool.cibuildwheel.linux]
archs = ["x86_64"]

[tool.cibuildwheel.windows]
archs = ['AMD64']

and my custom docker image looks like:

FROM quay.io/pypa/manylinux2014_x86_64:latest

ENV VCPKG_INSTALLED_DIR=/project/vcpkg

RUN yum -y update && \
    yum -y install curl zip unzip && \
    yum install -y https://packages.endpointdev.com/rhel/7/os/x86_64/endpoint-repo.x86_64.rpm && \
    yum groupinstall -y "Development tools" && \
    yum install -y centos-release-scl && \
    yum-config-manager --enable rhel-server-rhscl-7-rpms && \
    yum install -y devtoolset-11 python3 perl-IPC-Cmd m4 && \
    yum makecache && yum -y install lcov && \
    yum -y install autoconf automake autoconf-archive

RUN git clone https://github.com/microsoft/vcpkg.git /src/vcpkg && \
    /src/vcpkg/bootstrap-vcpkg.sh && \
    mkdir -p /project/vcpkg && \
    mv /src/vcpkg/* /project/vcpkg/ && \
    ln -s /project/vcpkg/vcpkg /usr/local/bin/vcpkg && \
    /project/vcpkg/vcpkg version

If you need any more information please let me know!

Thanks

Build log

No response

CI config

No response

[mcleantom]

Actually it was just because I was running it from within windows, running it within linux runs fine.

[joerick]

Yeah, we're relying on system tar here, so that might not work on windows. Actually we could check the docker daemon version and only use the tar copying mode on Docker Server versions before 24.0.0, apparently docker cp should work now. moby/moby#44210 .