[bug] protoc is searched within /usr/bin path

[AnatoliiShablov]

Describe the bug

I have system installed protoc. And while protobuf::libprotobuf is looking for conan provided library protobuf::protoc LOCATION is /usr/bin/protoc

How to reproduce it

Minimal example that produces such problem:

CMakeLists.txt

cmake_minimum_required(VERSION 3.24)

project(prototest)

set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

find_package(protobuf REQUIRED CONFIG)
# I've tried different options like protobuf/Protobuf CONFIG/MODULE
get_target_property(protoc1 protobuf::protoc LOCATION)

message(FATAL_ERROR "${protoc1}") # Here i see /usr/bin/protoc

conanfile.py

from conan import ConanFile

class my_package(ConanFile):
    name = "my_package"
    version = "1.0.0"
    settings = "os", "arch", "compiler", "build_type"
    generators = "CMakeDeps", "CMakeToolchain"
    # Tried with and without toolchain
    
    def requirements(self):
        self.requires("protobuf/5.27.0")

    def build_requirements(self):
        self.tool_requires("protobuf/<host_version>")

I tried to trace the error and in file protobuf-conan-protoc-target.cmake the order to obtain excutable is

find_program(PROTOC_PROGRAM NAMES protoc PATHS ENV PATH NO_DEFAULT_PATH) # Here we path PATH variable were protoc is placed
find_program(PROTOC_PROGRAM NAMES protoc) # Here we just looking for protoc like on normal system
# And only after this we are trying to use hardcoded path to conan package
set(PROTOC_PROGRAM "${CMAKE_CURRENT_LIST_DIR}/../../../bin/protoc${CMAKE_EXECUTABLE_SUFFIX}")
# When I don't have protoc in system i'm coming to the last variant set(PROTOC_PROGRAM... and it's fits

I'm not sure but maybe it's enough to set(PROTOC_PROGRAM .... But I don't know how it will affect crosscompilation

Here's the log

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=14
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/cc', 'cpp': '/usr/bin/c++'}
tools.cmake.cmaketoolchain:generator=Unix Makefiles

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=23
compiler.libcxx=libstdc++11
compiler.version=13
os=Linux


======== Computing dependency graph ========
Graph root
    conanfile.py (my_package/1.0.0): /home/shablov/Projects/prototest/conanfile.py
Requirements
    abseil/20240116.2#33c6b1fe8f2c5d96074682eae31bed66 - Cache
    protobuf/5.27.0#ccce9aa25886556c6d66c77b2be4d806 - Cache
    zlib/1.3.1#f52e03ae3d251dec704634230cd806a2 - Cache
Build requirements
    abseil/20240116.2#33c6b1fe8f2c5d96074682eae31bed66 - Cache
    protobuf/5.27.0#ccce9aa25886556c6d66c77b2be4d806 - Cache
    zlib/1.3.1#f52e03ae3d251dec704634230cd806a2 - Cache
Resolved version ranges
    zlib/[>=1.2.11 <2]: zlib/1.3.1

======== Computing necessary packages ========
abseil/20240116.2: Main binary package 'd1737f1ec9a769c4e540e98a71b868c81aca8fc0' missing
abseil/20240116.2: Checking 9 compatible configurations
abseil/20240116.2: Found compatible package '60d979b0a7d7272ce4719856e16fef14b991f4c9': compiler.cppstd=23
protobuf/5.27.0: Main binary package '796e2afb20b3da0893f215b66bc3683e00121029' missing
protobuf/5.27.0: Checking 9 compatible configurations
protobuf/5.27.0: Found compatible package '0c016fb820a3bdb494a62a8bfb2633b63120a30d': compiler.cppstd=23
Requirements
    abseil/20240116.2#33c6b1fe8f2c5d96074682eae31bed66:60d979b0a7d7272ce4719856e16fef14b991f4c9#04cc9dc933e28f393b60261df4470375 - Cache
    protobuf/5.27.0#ccce9aa25886556c6d66c77b2be4d806:0c016fb820a3bdb494a62a8bfb2633b63120a30d#1bc758961ffbef54347598301efbf3fc - Cache
    zlib/1.3.1#f52e03ae3d251dec704634230cd806a2:6980ffdedf2f062ab47388192e39ae3988cb5678#13a225a52e1fc48dfd9b2c3565b1afe7 - Cache
Build requirements
    protobuf/5.27.0#ccce9aa25886556c6d66c77b2be4d806:65e01f2a48be3b1bfa0606ee9a6fee71f70bf0f0#b495b9ac0f1e41a7838e95e033068bde - Cache
Skipped binaries
    abseil/20240116.2, zlib/1.3.1

======== Installing packages ========
abseil/20240116.2: Already installed! (1 of 4)
zlib/1.3.1: Already installed! (2 of 4)
protobuf/5.27.0: Already installed! (3 of 4)
protobuf/5.27.0: Already installed! (4 of 4)
WARN: deprecated: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X:
WARN: deprecated:     'cpp_info.names' used in: zlib/1.3.1, abseil/20240116.2, protobuf/5.27.0
WARN: deprecated:     'cpp_info.build_modules' used in: abseil/20240116.2, protobuf/5.27.0
WARN: deprecated:     'cpp_info.filenames' used in: protobuf/5.27.0
WARN: deprecated:     'env_info' used in: protobuf/5.27.0

======== Finalizing install (deploy, generators) ========
conanfile.py (my_package/1.0.0): Writing generators to /home/shablov/Projects/prototest/build/conan
conanfile.py (my_package/1.0.0): Generator 'CMakeDeps' calling 'generate()'
conanfile.py (my_package/1.0.0): CMakeDeps necessary find_package() and targets for your CMakeLists.txt
    find_package(protobuf)
    target_link_libraries(... protobuf::protobuf)
conanfile.py (my_package/1.0.0): Generator 'CMakeToolchain' calling 'generate()'
conanfile.py (my_package/1.0.0): CMakeToolchain generated: conan_toolchain.cmake
conanfile.py (my_package/1.0.0): CMakeToolchain: Preset 'conan-release' added to CMakePresets.json.
    (cmake>=3.23) cmake --preset conan-release
    (cmake<3.23) cmake <path> -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake  -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_BUILD_TYPE=Release
conanfile.py (my_package/1.0.0): CMakeToolchain generated: /home/shablov/Projects/prototest/build/conan/CMakePresets.json
conanfile.py (my_package/1.0.0): CMakeToolchain generated: /home/shablov/Projects/prototest/CMakeUserPresets.json
conanfile.py (my_package/1.0.0): Generating aggregated env files
conanfile.py (my_package/1.0.0): Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh']
Install finished successfully
CMake Error at CMakeLists.txt:12 (message):
  /usr/bin/protoc

[memsharded]

Hi @AnatoliiShablov

Thanks for your feedback

I have just tried your above files, in my Windows machine with:

$ conan install .
$ cmake --preset conan-default
CMake Error at CMakeLists.txt:12 (message):
  C:/Users/memsharded/.conan2/p/proto1bc625ad405e0/p/bin/protoc.exe

So in this case, it seems to be working fine.
Are you using these commands?

[memsharded]

Seems to work well in Linux Ubuntu too:

cmake --preset conan-release
Preset CMake variables:

  CMAKE_BUILD_TYPE="Release"
  CMAKE_POLICY_DEFAULT_CMP0091="NEW"
  CMAKE_TOOLCHAIN_FILE:FILEPATH="conan_toolchain.cmake"

Preset environment variables:

  DYLD_LIBRARY_PATH="/home/memsharded/.conan2/p/proto04bc8484cb9eb/p/lib:"
  LD_LIBRARY_PATH="/home/memsharded/.conan2/p/proto04bc8484cb9eb/p/lib:"
  PATH="/home....

-- Using Conan toolchain: /mnt/c/Users/Diego/conanws/kk/protobuf/conan_toolchain.cmake
-- Conan toolchain: Defining architecture flag: -m64
-- Conan toolchain: C++ Standard 17 with extensions ON
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
...
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Warning: Standard CMAKE_CXX_STANDARD value defined in conan_toolchain.cmake to 17 has been modified to 20 by /mnt/c/Users/Diego/conanws/kk/protobuf/CMakeLists.txt
-- Conan: Component target declared 'utf8_range::utf8_range'
...
-- Conan: Component target declared 'absl::flags_parse'
-- Conan: Target declared 'abseil::abseil'
-- Conan: Including build module from '/home/memsharded/.conan2/p/abseifc59d59abd8d7/p/lib/cmake/conan_trick/cxx_std.cmake'
-- Conan: Including build module from '/home/memsharded/.conan2/p/proto04bc8484cb9eb/p/lib/cmake/protobuf/protobuf-generate.cmake'
-- Conan: Including build module from '/home/memsharded/.conan2/p/proto04bc8484cb9eb/p/lib/cmake/protobuf/protobuf-module.cmake'
-- Conan: Including build module from '/home/memsharded/.conan2/p/proto04bc8484cb9eb/p/lib/cmake/protobuf/protobuf-options.cmake'
-- Conan: Including build module from '/home/memsharded/.conan2/p/proto04bc8484cb9eb/p/lib/cmake/protobuf/protobuf-conan-protoc-target.cmake'

CMake Error at CMakeLists.txt:12 (message):
  /home/memsharded/.conan2/p/proto04bc8484cb9eb/p/bin/protoc

[memsharded]

I have also created a /usr/bin/protoc executable, and still the same one inside the Conan package is fund

[AnatoliiShablov]

Thanks for you reply @memsharded

(Just for info my OS is Linux openSUSE Tumbleweed)

I've done this

1. Removed conan
2. Removed .conan2 cache
3. Installed it back
4. run conan profile detect
   Now I have clean installation
5. Run same commands as you (except build missing)

conan install --build=missing .
cmake --preset conan-release

And everything went fine I've got /home/shablov/.conan2/p/b/proto3d1ce850776ec/p/bin/protoc as path for protoc

But then I've tried to use conan_provider.cmake (downloaded right now) as I usualy do (sorry, forgot to say that I'm using conan_provider.cmake) and bumped to the same error

...
-- Conan: Component target declared 'absl::log_flags'
-- Conan: Component target declared 'absl::flags_parse'
-- Conan: Target declared 'abseil::abseil'
-- Conan: Including build module from '/home/shablov/.conan2/p/b/abseibacc6622e0d7d/p/lib/cmake/conan_trick/cxx_std.cmake'
-- Conan: Including build module from '/home/shablov/.conan2/p/b/proto3d1ce850776ec/p/lib/cmake/protobuf/protobuf-generate.cmake'
-- Conan: Including build module from '/home/shablov/.conan2/p/b/proto3d1ce850776ec/p/lib/cmake/protobuf/protobuf-module.cmake'
-- Conan: Including build module from '/home/shablov/.conan2/p/b/proto3d1ce850776ec/p/lib/cmake/protobuf/protobuf-options.cmake'
-- Conan: Including build module from '/home/shablov/.conan2/p/b/proto3d1ce850776ec/p/lib/cmake/protobuf/protobuf-conan-protoc-target.cmake'
CMake Error at CMakeLists.txt:12 (message):
  /usr/bin/protoc

[AnatoliiShablov]

BTW configure command:

mkdir build
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_PROJECT_TOP_LEVEL_INCLUDES=conan_provider.cmake

[memsharded]

But then I've tried to use conan_provider.cmake

That is the issue. The conan_provider.cmake is not able to inject the necessary toolchain to provide some extra paths definition that makes things like find_program() to work. It is one of the known current limitations of the cmake-conan integration with the provider (there are others, like the impossibility to inject environment from dependencies and tool-requires). We are working to try to improve it with the new "incubating" CMakeDeps generator (see https://docs.conan.io/2/incubating.html), but it is a kind of chicken and egg problem, Conan cannot define certain things until it is called, but if Conan is not called first, and CMake is running first, then CMake doesn't have the necessary information to locate some aspects from dependencies like having find_program() to find executables, as the CMake "dependency providers" only intercept find_package() calls.

So in summary, this is not a bug in Conan, it works as expected. It is a know limitation of cmake-conan integration via CMake providers (due mostly to the nature of CMake providers, though we are trying to workaround it)

[AnatoliiShablov]

Oh, understood. But as I can see that only will be problem for find_program usage. And if find_package like protobuf/grpc (don't know about other) will provide target for executable like I mentioned earlier hard-pathes into conan package. It's will be enough to get location with get_target_property. And included functions like generate_protobuf are using protobuf::protoc. So it's fit here too. But again, maybe there will be problems with cross-build, where we need to provide host tool, probably it's have to be builded for host and target have to locate to it.
Maybe it's will be better to close this issue. Probably the right one is conan-io/cmake-conan#669

[memsharded]

Oh, understood. But as I can see that only will be problem for find_program usage. And if find_package like protobuf/grpc (don't know about other) will provide target for executable like I mentioned earlier hard-pathes into conan package. It's will be enough to get location with get_target_property. And included functions like generate_protobuf are using protobuf::protoc. So it's fit here too. But again, maybe there will be problems with cross-build, where we need to provide host tool, probably it's have to be builded for host and target have to locate to it.

Yes, that is correct, there is also ongoing work in the incubating CMakeDeps to generate the executable targets directly from definitions from cpp_info.exe in dependencies, which will reduce the need for find_program() (still there might be existing CMakeLists.txt that use find_program(), so that will still work in some cases with the incubating CMakeDeps, but not all, because it requires a find_package() first)

Yes, lets close the issue, keep tuned to that ticket and to new releases of the incubating CMakeDeps, this should be solved when it is matured and get out of incubating state. Thanks for the feedback!