The changes of `symbol.tsv` is not applied until cleaning (Windows) #932

tats-u · 2024-05-19T09:24:43Z

Description

A clear and concise description of what the bug is.

I'm trying to modify src\data\symbol\symbol.tsv, but its changes are difficult to be reflected because they require us to clean the build.

Steps to reproduce

Steps to reproduce the behavior:

python build_mozc.py gyp
python build_mozc.py build -c Release package
Edit src\data\symbol\symbol.tsv
python build_mozc.py build -c Release package
Install the generated Mozc64.msi & Enable Mozc
Try the change

Expected behavior

A clear and concise description of what you expected to happen.

The change is applied

Actual behavior

A clear and concise description of what you actually got.

The change is not applied

Screenshots

If applicable, add screenshots to help explain your problem.

Version or commit-id

[e.g. Mozc-2.28.4960.100+24.11.oss or d50a8b9]
You can get the version string by converting "Version" or "ばーじょん".

6ba364e (master)

Environment

OS: [e.g. Ubuntu 20.04, macOS 13.1, etc]
IMF (for Linux) [e.g. Ibus, Fcitx, etc]
Related Applications [e.g. Chromium 109, gedit, etc]

Windows 11 (on Windows Sandbox)
(Notepad)

Investigations

Whether this issue happens on both Ibus and other IMF (e.g. Fcitx, uim).
- yes, no, N/A
Whether this issue happens on other IMEs (e.g. Anthy, SKK).
- yes, no, N/A
What applications this issue happens on (e.g. Chromium, gedit).
- list of applications
What applications this issue does not happen on (e.g. Chromium, gedit).
- list of applications, or not-found
(optional) What versions or commit-ids this issue did not happen on
(e.g. Mozc-2.28.4960.100+24.11.oss).
- list of versions or commit-ids, or no-investigation.

Additional context

Add any other context about the problem here.

We have to rebuild after cleaning once (python build_mozc.py clean) to apply the changes in symbol.tsv.

Solutions:

Migrate to Bazel completely (Use bazel to build Mozc for Windows #948)
Fix GYP (will not done by maintainers)

The text was updated successfully, but these errors were encountered:

hiroyuki-komatsu · 2024-07-21T06:51:24Z

Hi tats-u,
Thank you for your report.

I guess we might need to modify data_manager/data_manager.gypi or data_manager/oss/oss_data_manager_base.gyp (or other .gyp / .gypi files).

https://github.com/google/mozc/blob/master/src/data_manager/oss/oss_data_manager_base.gyp
https://github.com/google/mozc/blob/master/src/data_manager/data_manager_base.gypi

Since we are migrating the build system from GYP to Bazel (#948), we would not address this limitation.
If you are willing to fix it, we would appreciate it and review your PR.

Best,

tats-u · 2024-07-21T07:47:24Z

@hiroyuki-komatsu Will Mozc64.msi be built by Bazel in the future, won't it?
If so I think I can wait for it and will link this issue to that one.

hiroyuki-komatsu · 2024-07-21T08:35:21Z

Yes, we want to build .msi files with Bazel in the future.

tats-u · 2024-07-21T13:49:08Z

I see.

hiroyuki-komatsu · 2024-10-05T14:23:36Z

We have supported Bazel build for Windows.

Please check the instruction.
https://github.com/google/mozc/blob/master/docs/build_mozc_in_windows.md#build-with-bazel-experimental

I will close this issue in a week if no issues are found.
Thank you,

tats-u · 2024-10-05T15:07:34Z

Without [Console]::OutputEncoding = [Console]::InputEncoding = [Text.Encoding]::UTF8, I got:

INFO: From Compiling absl/random/internal/randen_round_keys.cc [for tool]:
randen_round_keys.cc
INFO: From Compiling absl/flags/internal/commandlineflag.cc [for tool]:
commandlineflag.cc
ERROR: I:/mozc/src/gui/base/BUILD.bazel:74:12: Executing genrule //gui/base:base_files failed: (Exit -1): bash.exe failed: error executing Genrule command (from target //gui/base:base_files) C:\Users\tatsu\scoop\apps\msys2\current\usr\bin\bash.exe -c ... (remaining 1 argument skipped)
Action failed to execute: java.io.IOException: ERROR: src/main/native/windows/process.cc(202): CreateProcessW("C:\Users\tatsu\scoop\apps\msys2\current\usr\bin\bash.exe" -c "source external/bazel_tools/tools/genrule/genrule-setup.sh; external/_main~_repo_rules~qt_win/bin/moc.exe -p $(dirname gui/base/window_title_modifier.h) -o bazel-out/x64_windows-opt/bin/gui/base/moc_window_title_modifier.cc gui/base/window_title_modifier.h"): 謖・ｮ壹＆繧後◆繝輔ぃ繧､繝ｫ縺瑚ｦ九▽縺九ｊ縺ｾ縺帙ｓ縲・
 (error: 2)
Target //:package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 35.236s, Critical Path: 1.11s
INFO: 1346 processes: 1315 internal, 31 local.

PS> java --version
openjdk 11.0.23 2024-04-16 LTS
OpenJDK Runtime Environment Microsoft-9394293 (build 11.0.23+9-LTS)
OpenJDK 64-Bit Server VM Microsoft-9394293 (build 11.0.23+9-LTS, mixed mode, sharing)

PS> gcm java -all | select Version,Source

Version   Source
-------   ------
11.0.23.0 C:\Program Files\Microsoft\jdk-11.0.23.9-hotspot\bin\java.exe
21.0.3.0  C:\Program Files\Eclipse Adoptium\jdk-21.0.3.9-hotspot\bin\java.exe

tats-u · 2024-10-05T15:11:07Z

Even with [Console]::OutputEncoding = [Console]::InputEncoding = [Text.Encoding]::UTF8 I got:

INFO: From Compiling base/win32/hresult.cc:
hresult.cc
ERROR: I:/mozc/src/gui/base/BUILD.bazel:85:12: Executing genrule //gui/base:qrc_tr failed: (Exit -1): bash.exe failed: error executing Genrule command (from target //gui/base:qrc_tr) C:\Users\tatsu\scoop\apps\msys2\current\usr\bin\bash.exe -c ... (remaining 1 argument skipped)
Action failed to execute: java.io.IOException: ERROR: src/main/native/windows/process.cc(202): CreateProcessW("C:\Users\tatsu\scoop\apps\msys2\current\usr\bin\bash.exe" -c "source external/bazel_tools/tools/genrule/genrule-setup.sh; external/_main~_repo_rules~qt_win/bin/rcc.exe -o bazel-out/x64_windows-opt/bin/gui/base/qrc_tr.cc -name qrc_tr gui/base/tr.qrc"): 指定されたファイルが見つかりません。
 (error: 2)

Note: not yet terminated by this error.

I wonder I should remove the stale Java 11.

PS. stucked at:

[2,264 / 4,377] Compiling transliteration/transliteration.cc; 468s local

Bazel Ctrl+C handler; shutting down.
[2,264 / 4,377] Compiling transliteration/transliteration.cc; 517s local

tats-u · 2024-10-05T15:19:45Z

After removing Java 11 and git clean -f, stucked with Extracting Bazel installation....

yukawa · 2024-10-05T15:37:15Z

ERROR: I:/mozc/src/gui/base/BUILD.bazel:85:12: Executing genrule //gui/base:qrc_tr failed: (Exit -1): bash.exe failed: error executing Genrule command (from target //gui/base:qrc_tr) C:\Users\tatsu\scoop\apps\msys2\current\usr\bin\bash.exe -c ... (remaining 1 argument skipped)
Action failed to execute: java.io.IOException: ERROR: src/main/native/windows/process.cc(202): CreateProcessW("C:\Users\tatsu\scoop\apps\msys2\current\usr\bin\bash.exe" -c "source external/bazel_tools/tools/genrule/genrule-setup.sh; external/_main~_repo_rules~qt_win/bin/rcc.exe -o bazel-out/x64_windows-opt/bin/gui/base/qrc_tr.cc -name qrc_tr gui/base/tr.qrc"): 指定されたファイルが見つかりません。
 (error: 2)

Most likely this error means bazel failed to find bash.exe in your system. Check out the following section for details.

https://bazel.build/install/windows#bazel_does_not_find_bash_or_bashexe

tats-u · 2024-10-05T15:49:39Z

I see. We should add the link to https://bazel.build/install/windows?hl=ja#install-compilers in the Wiki page.
Should Java be 11, not 21? Java 11 is more than 5 years old.

yukawa · 2024-10-05T15:57:45Z

Should Java be 11, not 21? Java 11 is more than 5 years old.

My understanding is that recent versions of Bazel (or bazelisk) download JRE automatically, so you in general don't need to install Java by yourself.

Note also that we will definitely make sure that our Bazel build continues to succeed on the GitHub Actions. Setting up an environment similar enough to it may help.

https://github.com/actions/runner-images/blob/main/images/windows/Windows2022-Readme.md

mozc/.github/workflows/windows.yaml

Lines 77 to 133 in ace3145

    
             build_bazel: 
        
               # https://github.com/actions/runner-images/blob/main/images/windows/Windows2022-Readme.md 
        
               runs-on: windows-2022 
        
               timeout-minutes: 120 
        
               steps: 
        
                 - name: Checkout 
        
                   uses: actions/checkout@v4 
        
                   with: 
        
                     submodules: 'recursive' 
        
                 - name: Set up pip 
        
                   shell: cmd 
        
                   working-directory: .\src 
        
                   run: | 
        
                     python -m pip install six requests 
        
                 - name: Try to restore update_deps cache 
        
                   uses: actions/cache@v4 
        
                   with: 
        
                     path: src/third_party_cache 
        
                     key: update_deps-${{ runner.os }}-${{ hashFiles('src/build_tools/update_deps.py') }} 
        
                 - name: Install Dependencies 
        
                   shell: cmd 
        
                   working-directory: .\src 
        
                   # This command uses src/third_party_cache as the download cache. 
        
                   run: | 
        
                     python build_tools/update_deps.py 
        
                 - name: Biuld Qt 
        
                   shell: cmd 
        
                   working-directory: .\src 
        
                   run: | 
        
                     python build_tools/build_qt.py --release --confirm_license 
        
                 # This is needed only for GitHub Actions where free disk space is quite limited. 
        
                 - name: Delete Qt src to save disk space 
        
                   shell: cmd 
        
                   working-directory: .\src 
        
                   run: | 
        
                     rmdir third_party\qt_src /s /q 
        
                 - name: build artifacts 
        
                   shell: cmd 
        
                   working-directory: .\src 
        
                   env: 
        
                     ANDROID_NDK_HOME: "" 
        
                   run: | 
        
                     bazel --bazelrc=windows.bazelrc build --config oss_windows --config release_build package 
        
                 - name: upload Mozc64_bazel.msi 
        
                   uses: actions/upload-artifact@v4 
        
                   with: 
        
                     name: Mozc64_bazel.msi 
        
                     path: src/bazel-bin/win32/installer/Mozc64.msi 
        
                     if-no-files-found: warn

tats-u · 2024-10-06T02:53:39Z

I tried MSYS2 via WinGet but it wasn't be recognized.
I had to reinstall it using Scoop.

However, I confirmed the fix of this issue.

tats-u closed this as completed Oct 6, 2024

tats-u mentioned this issue Oct 6, 2024

Java "11" is not required (especially for Windows) bazelbuild/bazel#23882

Open

tats-u mentioned this issue Oct 14, 2024

Improve Windows build instruction #1079

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The changes of `symbol.tsv` is not applied until cleaning (Windows) #932

The changes of `symbol.tsv` is not applied until cleaning (Windows) #932

tats-u commented May 19, 2024 •

edited

Loading

hiroyuki-komatsu commented Jul 21, 2024

tats-u commented Jul 21, 2024 •

edited

Loading

hiroyuki-komatsu commented Jul 21, 2024

tats-u commented Jul 21, 2024

hiroyuki-komatsu commented Oct 5, 2024

tats-u commented Oct 5, 2024

tats-u commented Oct 5, 2024 •

edited

Loading

tats-u commented Oct 5, 2024

yukawa commented Oct 5, 2024 •

edited

Loading

tats-u commented Oct 5, 2024 •

edited

Loading

yukawa commented Oct 5, 2024 •

edited

Loading

tats-u commented Oct 6, 2024

The changes of symbol.tsv is not applied until cleaning (Windows) #932

The changes of symbol.tsv is not applied until cleaning (Windows) #932

Comments

tats-u commented May 19, 2024 • edited Loading

Description

Steps to reproduce

Expected behavior

Actual behavior

Screenshots

Version or commit-id

Environment

Investigations

Additional context

hiroyuki-komatsu commented Jul 21, 2024

tats-u commented Jul 21, 2024 • edited Loading

hiroyuki-komatsu commented Jul 21, 2024

tats-u commented Jul 21, 2024

hiroyuki-komatsu commented Oct 5, 2024

tats-u commented Oct 5, 2024

tats-u commented Oct 5, 2024 • edited Loading

tats-u commented Oct 5, 2024

yukawa commented Oct 5, 2024 • edited Loading

tats-u commented Oct 5, 2024 • edited Loading

yukawa commented Oct 5, 2024 • edited Loading

tats-u commented Oct 6, 2024

The changes of `symbol.tsv` is not applied until cleaning (Windows) #932

The changes of `symbol.tsv` is not applied until cleaning (Windows) #932

tats-u commented May 19, 2024 •

edited

Loading

tats-u commented Jul 21, 2024 •

edited

Loading

tats-u commented Oct 5, 2024 •

edited

Loading

yukawa commented Oct 5, 2024 •

edited

Loading

tats-u commented Oct 5, 2024 •

edited

Loading

yukawa commented Oct 5, 2024 •

edited

Loading