issues sym-linking catkin's new local_setup.sh

[rhaschke]

I get the following error since local_setup.sh was recently introduced in ros/catkin#993

Cannot symlink from /homes/rhaschke/src/ros/devel/.private/rviz/local_setup.sh to existing file /homes/rhaschke/src/ros/devel/./local_setup.sh
Warning: Source hash: 1b5f4fec545a06a4cc4723db5116e3f2
Warning: Dest hash: 51a457f5bad6a3827efef461047f77e8

As pointed out in ros/catkin#994 (comment), this looks like catkin-tools needs some adaptation too.

I see the same issues in Travis, building MoveIt, but interestingly only in this build job using clang as a compiler but not in [this one]https://travis-ci.org/ros-planning/moveit/jobs/502614339#L543) using gcc.

[NikolausDemmel]

Btw, this seems a relevant code section:

catkin_tools/catkin_tools/jobs/catkin.py

Lines 634 to 643 in 118bf7c

	DEVEL_LINK_BLACKLIST = DEVEL_LINK_PREBUILD_BLACKLIST + [
	os.path.join('etc', 'catkin', 'profile.d', '05.catkin_make.bash'),
	os.path.join('etc', 'catkin', 'profile.d', '05.catkin_make_isolated.bash'),
	os.path.join('etc', 'catkin', 'profile.d', '05.catkin-test-results.sh'),
	'env.sh',
	'setup.bash',
	'setup.zsh',
	'setup.sh',
	'_setup_util.py',
	]

But I'm not sure if simply adding it to the blacklist is enough. It might be that the prebuild package needs to be updated as well, and I'm not sure if the local_setup.py from the individual packages need to somehow be aggregated or if one is enough.

[dirk-thomas]

Will either of you (@rhaschke, @NikolausDemmel, @wjwwood - or anybody else) be able to look into this and work on releasing a new version?

[NikolausDemmel]

I won't, sorry.

[rhaschke]

Me neither, sorry.

[wjwwood]

No, sorry I will not either. Ever since the decision was made to use colcon for everything moving forward, I've avoided spending time on build tools, including catkin_tools.

Hopefully, someone who uses it everyday will spend some time fixing it up. I'm happy to help by giving access/permissions to more people (@mikepurvis has access already).

[rhaschke]

@dirk-thomas, please have a look at #539.

[dirk-thomas]

I have pretty much no knowledge of the internals of catkin_tools and don't know how this blacklist is used / what it is needed for. If it make the build pass where it was failing without the patch that sounds good to me 👍

[rhaschke]

@dirk-thomas, my question is primarily: what is the role of these new local_setup.sh files?
When / for what they are needed?

[dirk-thomas]

The local_setup.* files allow to only updated the environment with this specific workspace. The setup.* files on the other hand implicitly also source the chained workspaces. So using the local setup files the user can explicitly choose which workspaces to chain.

One use case is to build ROS 1 workspaces with colcon - see colcon/colcon-ros#54 .

[rhaschke]

What happens in a catkin_make_isolated scenario: I guess all packages attempt to create a local_setup file. How they are merged for the local workspace?

[dirk-thomas]

The local setup files are not treated differently from the previously existing setup files.

[NikolausDemmel]

It seems they differ indeed just as much as the setup.* files

$ diff devel/.private/rosmake/setup.sh devel/.private/catkin/setup.sh
14c14
< : ${_CATKIN_SETUP_DIR:=/Users/demmeln/work/ros/test_local_setup_ws/devel/.private/rosmake}
---
> : ${_CATKIN_SETUP_DIR:=/Users/demmeln/work/ros/test_local_setup_ws/devel/.private/catkin}
$ diff devel/.private/rosmake/local_setup.sh devel/.private/catkin/local_setup.sh
6c6
< : ${_CATKIN_SETUP_DIR:=/Users/demmeln/work/ros/test_local_setup_ws/devel/.private/rosmake}
---
> : ${_CATKIN_SETUP_DIR:=/Users/demmeln/work/ros/test_local_setup_ws/devel/.private/catkin}

and catkin_tools already just links the catkin version of setup.* in merged devel mode, which means in particular that that file also already has the "wrong" _CATKIN_SETUP_DIR, namely the one from catkin's isolated devel space:

$ grep "private/catkin" devel/setup.sh
: ${_CATKIN_SETUP_DIR:=/Users/demmeln/work/ros/test_local_setup_ws/devel/.private/catkin}

So maybe doing the same for local_setup.* is reasonable as a fix for now (I have no idea what implications a "wrong" _CATKIN_SETUP_DIR has...).

The only thing to check is that the local_setup.* are generated the same when catkin is not part of the build (overlay workspace). I.e. we need to check that the prebuild package does the right thing.

[varunagrawal]

Just encountered this issue for Kinetic (yeah I need to upgrade), and the fix is to simply run pip install -U catkin_tools to v0.4.5 and stuff works.