Move crt & headers to common target prefix #10634

elmarco · 2022-01-31T08:32:18Z

For historical reasons, headers were installed under
${MINGW_PREFIX}/${MINGW_CHOST}, but there is no reason to use a
sub-directory of the target prefix.

This align with other mingw target directory structures in Fedora,
Debian ...

Fixes #10630

For historical reasons, crt was installed under ${MINGW_PREFIX}/${MINGW_CHOST}, but there is no reason to use a sub-directory of the target prefix. This aligned with other mingw target directory structures in Fedora, Debian ... Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>

For historical reasons, headers were installed under ${MINGW_PREFIX}/${MINGW_CHOST}, but there is no reason to use a sub-directory of the target prefix. This aligned with other mingw target directory structures in Fedora, Debian ... Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>

lazka · 2022-01-31T08:34:52Z

hm, don't gcc and clang also reference this?

elmarco · 2022-01-31T08:45:14Z

hm, don't gcc and clang also reference this?

${MINGW_PREFIX}/${MINGW_CHOST}/ is part of default include & library search path. But the file names used in specs (or elsewhere?) are not absolute, so we can move them to the common target prefix.

elmarco · 2022-01-31T08:47:16Z

According to my simple search, there are about 20 packages that should be rebuilt.

Thanks! They don't have to be rebuilt right away, but they may need to be updated. I see some cases where the path shouldn't be hardcoded, but rely on compiler default search path instead.

lazka · 2022-01-31T08:51:28Z

I've asked @Alexpux if he remembers why things are the way they are right now.

elmarco · 2022-01-31T08:52:45Z

Thanks! They don't have to be rebuilt right away, but they may need to be updated. I see some cases where the path shouldn't be hardcoded, but rely on compiler default search path instead.

No, They should be rebuilt to make sure that this change doesn't break anything.

That's a distribution policy, it's not strictly required for end users. Many distributions accept a temporary FTBFS state (fail to build from source). This happend every now and then because a new compiler or library dependency regression.

Is there a way to trigger a mass-rebuild of all the packages for testing?

mstorsjo · 2022-01-31T08:54:34Z

This align with other mingw target directory structures in Fedora, Debian ...

This argument I don't quite understand. When not cross compiling, it's ok to assume that files in <toolchain>/{include,lib} are for the current default target, but when cross compiling, the default would be to have those files in <toolchain>/<cross-triple>/{include,lib} - just like on Debian, /usr/bin/x86_64-w64-mingw32-gcc uses files in /usr/x86_64-w64-mingw32/{include,lib} (which aren't mixed in under /usr/include etc).

For msys2, I guess this change would be ok, as all toolchains there are mainly non-cross, but I don't see the argument about likeness with how cross compilers are set up in at least Debian (I'm not familiar with Fedora).

elmarco · 2022-01-31T09:04:08Z

This align with other mingw target directory structures in Fedora, Debian ...

This argument I don't quite understand. When not cross compiling, it's ok to assume that files in <toolchain>/{include,lib} are for the current default target, but when cross compiling, the default would be to have those files in <toolchain>/<cross-triple>/{include,lib} - just like on Debian, /usr/bin/x86_64-w64-mingw32-gcc uses files in /usr/x86_64-w64-mingw32/{include,lib} (which aren't mixed in under /usr/include etc).

On msys2, ${MINGW_PREFIX} is the equivalent of (for ex) Debian ~~/usr/bin/x86_64-w64-mingw32-gcc~~ /usr/x86_64-w64-mingw32-gcc, right?

On Debian, Fedora etc, mingw crt & headers, and various libraries are all installed under the same prefix.

There is no need for an extra sub-directory ${MINGW_PREFIX}/${MINGW_CHOST}.

(having the same directory structure allows to install msys2/mingw packages on Linux and link /usr/bin/x86_64-w64-mingw32-gcc -> /mingw64 (under a user namespace), and you don't need to package all the mingw-* libraries to cross-compile on your Linux distro anymore. Well, that's in theory, I am doing some tests to check that)

mstorsjo · 2022-01-31T09:09:52Z

On msys2, ${MINGW_PREFIX} is the equivalent of (for ex) Debian /usr/bin/x86_64-w64-mingw32-gcc, right?

I presume you mean the equivalent of /usr/x86_64-w64-mingw32?

There is no need for an extra sub-directory ${MINGW_PREFIX}/${MINGW_CHOST}.

(having the same directory structure allows to install msys2/mingw packages on Linux and link /usr/bin/x86_64-w64-mingw32-gcc -> /mingw64 (under a user namespace), and you don't need to package all the mingw-* libraries to cross-compile on your Linux distro anymore. Well, that's in theory, I am doing some tests to check that)

Ok, sure, that usecase makes sense. And yes, there's a maybe unusal split in msys2 today with the toolchain's files being in /mingw64/x86_64-w64-mingw32 while third party packages are installed directly in /mingw64. Avoiding that can probably useful.

lazka · 2022-01-31T11:32:30Z

Some feedback on mingw-w64 IRC:

 09:47 <Biswa96> Is this right thing to do? https://github.com/msys2/MINGW-packages/pull/10634
09:53 <@jon_y> Biswa96: yeah --prefix == sysroot is fine
09:57 <lazka> I wonder if there could be any filename conflicts with random packages
09:57 <lazka> I guess I could write a script to check
09:57 <@jon_y> or packages installing different headers due to bitness difference
09:58 <wbs> yeah in principle it should be fine, but with 2000 packages in msys2, there's certainly something that can break due to excessive assumptions somewhere

ok, let's give this a try then.

There are more packages using that prefix, binutils and winpthreads, etc
I'd like to check if there are any potential file conflicts with other packages in /lib and /include, just to be sure

elmarco · 2022-01-31T11:34:57Z

ok, let's give this a try then.

* There are more packages using that prefix, binutils and winpthreads, etc

* I'd like to check if there are any potential file conflicts with other packages in /lib and /include, just to be sure

thanks, I am not experienced with pacman, but let me know if I can help.

lazka · 2022-02-05T11:53:12Z

Here is a list of file conflicts created by the move:

mingw-w64-x86_64-freetds /mingw64/include/odbcss.h
mingw-w64-x86_64-glbinding /mingw64/include/KHR/khrplatform.h
mingw-w64-x86_64-hdf4 /mingw64/include/error.h
mingw-w64-x86_64-libgdiplus /mingw64/lib/libgdiplus.a
mingw-w64-x86_64-mesa /mingw64/include/KHR/khrplatform.h
mingw-w64-x86_64-pupnp /mingw64/lib/libupnp.a
mingw-w64-x86_64-unixodbc /mingw64/include/odbcinst.h
mingw-w64-x86_64-unixodbc /mingw64/include/sql.h
mingw-w64-x86_64-unixodbc /mingw64/include/sqlext.h
mingw-w64-x86_64-unixodbc /mingw64/include/sqltypes.h
mingw-w64-x86_64-unixodbc /mingw64/include/sqlucode.h

If anyone has any idea how to handle these, go ahead.

elmarco · 2022-02-07T08:41:21Z

Here is a list of file conflicts created by the move:
If anyone has any idea how to handle these, go ahead.

Ok, some ideas below. I suppose ideally you would want to fix all before moving the crt & headers.

mingw-w64-x86_64-freetds /mingw64/include/odbcss.h

Hi @freddy77 ! the file name conflicts with a windows system header. can it be renamed or moved upstream?

mingw-w64-x86_64-glbinding /mingw64/include/KHR/khrplatform.h
mingw-w64-x86_64-mesa /mingw64/include/KHR/khrplatform.h

Existing conflict.

They look like the same file, we should ask upstream to use the system one.

mingw-w64-x86_64-hdf4 /mingw64/include/error.h

The file doesn't seem to be included by the rest of the headers. Furthermore the symbols are not exported by the dll.

mingw-w64-x86_64-libgdiplus /mingw64/lib/libgdiplus.a
mingw-w64-x86_64-pupnp /mingw64/lib/libupnp.a

Those 2, perhaps msys2 doesn't have to ship the static library?

For pupnp, the library name doesn't follow package name. Maybe they can consider a rename for future releases..

mingw-w64-x86_64-unixodbc /mingw64/include/odbcinst.h
mingw-w64-x86_64-unixodbc /mingw64/include/sql.h
mingw-w64-x86_64-unixodbc /mingw64/include/sqlext.h
mingw-w64-x86_64-unixodbc /mingw64/include/sqltypes.h
mingw-w64-x86_64-unixodbc /mingw64/include/sqlucode.h

Given that they are explicitly consistent with the MS headers, they should use the system headers when available.

fziglio · 2022-02-07T09:58:59Z

Here is a list of file conflicts created by the move:
If anyone has any idea how to handle these, go ahead.

Ok, some ideas below. I suppose ideally you would want to fix all before moving the crt & headers.

mingw-w64-x86_64-freetds /mingw64/include/odbcss.h

Hi @freddy77 ! the file name conflicts with a windows system header. can it be renamed or moved upstream?

They both provides the same header so I think renaming it would make no sense. The idea of this header was providing it for systems not having it. But if the system provides it (that is Windows target) using the system one should be the same (they should be compatibles). So I think the perfect solution to fix the issue would be for FreeTDS not installing the header if already provided by the system. Not sure 100% if they are really compatibles perfectly. In particular some functions exported by Windows ODBC driver manager are not available by Unix ODBC driver managers so I use some static inline definition in this file to override them, but this should not be necessary on Windows targets (the FreeTDS ODBC driver exports the needed functions for the DM).

elmarco · 2022-02-07T11:28:59Z

For mesa, see https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14019

elmarco · 2022-02-07T12:21:26Z

glbinding header conflict was fixed in v3.3 cginternals/glbinding#299

elmarco · 2022-02-07T13:32:27Z

glbinding header conflict was fixed in v3.3 cginternals/glbinding#299

The latest release is 3.1.0, there is no mention of a new release anywhere! When was it released?

Oh, it's a branch actually, apparently forked from master a few weeks ago: https://github.com/cginternals/glbinding/commits/v3.3

@scheibel did you release 3.2.0 and 3.3.0 ?

elmarco · 2022-02-09T18:54:23Z

For hdf4:
HDFGroup/hdf4#17

I can make a msys2 backport if that helps. In general, I think it should use --includedir=$include/hdf4 and rely on .pc to give the adjusted path.

elmarco · 2022-02-09T19:21:45Z

For unixodbc: I tried to compile with the system headers, but it fails. I suggest we use --includedir=$include/unixodbc, there is already a .pc file.

scheibel · 2022-02-10T16:34:00Z

@scheibel did you release 3.2.0 and 3.3.0 ?

We haven't officially released a version 3.2.x or 3.3.x, although we absolutely should do so. You can think of our two branches v3.2 and v3.3 as release candidates for those versions.
Are there any time constraints I should be aware of?
Unfortunately, I have a lot going on right now that is getting in the way of a clean release.
Would a tag, pre-release, or an unofficial release be of any help?

elmarco · 2022-02-15T20:22:01Z

Many thanks @lazka ! What do you do of the remaining conflicts?

lazka · 2022-02-16T03:38:14Z

According to my scripts there aren't any left now.

See #10634

lazka · 2022-02-17T07:47:11Z

There is some potential fallout from this (limited, since I can't reproduce):

Though no clue right now.

revelator · 2022-03-29T22:46:28Z

python2 build also broken, looks for system headers in old path and refuses to pick them up from current.
guess we need a patch ?.

elmarco added 2 commits January 31, 2022 12:14

lazka added 4 commits February 5, 2022 12:04

winstorecompat: install under ${MINGW_PREFIX}

45b7baf

windows-default-manifest: install under ${MINGW_PREFIX}

c556a6c

winpthreads: install under ${MINGW_PREFIX}

c25b7de

binutils: install under ${MINGW_PREFIX}

07b287d

bump pkgrel for crt/headers

07694f5

lazka mentioned this pull request Feb 7, 2022

freetds: update to 1.3.7 #10693

Merged

elmarco mentioned this pull request Feb 7, 2022

odbcss.h conflicts with Windows SDK header FreeTDS/freetds#454

Closed

elmarco mentioned this pull request Feb 9, 2022

Repository cleanups lurcher/unixODBC#98

Merged

lazka merged commit d5cf1c1 into msys2:master Feb 15, 2022

lazka added a commit that referenced this pull request Feb 16, 2022

gcc: rebuild for system header prefix change

876a448

See #10634

lazka mentioned this pull request Feb 16, 2022

Fix conflicts with mingw-w64 #10741

Merged

lazka mentioned this pull request Feb 17, 2022

gcc seems broken #10761

Open

naveen521kk mentioned this pull request Feb 21, 2022

Cabal is broken after updating mingw-w64-x86_64-crt-git package #10837

Open

Kha mentioned this pull request Mar 22, 2022

chore: CI: fix MinGW library root leanprover/lean4#1060

Merged

MehdiChinoune mentioned this pull request Mar 25, 2022

unable to bootstrap gcc after path change #11095

Closed

jeremyd2019 mentioned this pull request Apr 14, 2022

clang: messes up its search path if a prefixed gcc is on the PATH #11495

Open

mstorsjo mentioned this pull request Dec 16, 2023

CLANG64/x86_64-w64-mingw32-clang link error if C:\msys64\mingw64\bin is in PATH #19279

Closed

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move crt & headers to common target prefix #10634

Move crt & headers to common target prefix #10634

elmarco commented Jan 31, 2022

lazka commented Jan 31, 2022 •

edited

Loading

elmarco commented Jan 31, 2022

elmarco commented Jan 31, 2022

lazka commented Jan 31, 2022

elmarco commented Jan 31, 2022 •

edited

Loading

mstorsjo commented Jan 31, 2022

elmarco commented Jan 31, 2022 •

edited

Loading

mstorsjo commented Jan 31, 2022

lazka commented Jan 31, 2022

elmarco commented Jan 31, 2022

lazka commented Feb 5, 2022 •

edited

Loading

elmarco commented Feb 7, 2022

fziglio commented Feb 7, 2022

elmarco commented Feb 7, 2022

elmarco commented Feb 7, 2022

elmarco commented Feb 7, 2022

elmarco commented Feb 9, 2022 •

edited

Loading

elmarco commented Feb 9, 2022

scheibel commented Feb 10, 2022 •

edited

Loading

elmarco commented Feb 15, 2022

lazka commented Feb 16, 2022

lazka commented Feb 17, 2022 •

edited

Loading

revelator commented Mar 29, 2022

Move crt & headers to common target prefix #10634

Move crt & headers to common target prefix #10634

Conversation

elmarco commented Jan 31, 2022

lazka commented Jan 31, 2022 • edited Loading

elmarco commented Jan 31, 2022

elmarco commented Jan 31, 2022

lazka commented Jan 31, 2022

elmarco commented Jan 31, 2022 • edited Loading

mstorsjo commented Jan 31, 2022

elmarco commented Jan 31, 2022 • edited Loading

mstorsjo commented Jan 31, 2022

lazka commented Jan 31, 2022

elmarco commented Jan 31, 2022

lazka commented Feb 5, 2022 • edited Loading

elmarco commented Feb 7, 2022

fziglio commented Feb 7, 2022

elmarco commented Feb 7, 2022

elmarco commented Feb 7, 2022

elmarco commented Feb 7, 2022

elmarco commented Feb 9, 2022 • edited Loading

elmarco commented Feb 9, 2022

scheibel commented Feb 10, 2022 • edited Loading

elmarco commented Feb 15, 2022

lazka commented Feb 16, 2022

lazka commented Feb 17, 2022 • edited Loading

revelator commented Mar 29, 2022

lazka commented Jan 31, 2022 •

edited

Loading

elmarco commented Jan 31, 2022 •

edited

Loading

elmarco commented Jan 31, 2022 •

edited

Loading

lazka commented Feb 5, 2022 •

edited

Loading

elmarco commented Feb 9, 2022 •

edited

Loading

scheibel commented Feb 10, 2022 •

edited

Loading

lazka commented Feb 17, 2022 •

edited

Loading