Binary distribution of executables or LightGBM exectuable of latest stable binaries release

[tobigithub]

Executive summary: a repository of monthly stable binary LightGBM executables would be useful.

I am able to compile LightGBM under WIN and LINUX, but the increasing complexity
with CUDA and OpenCL dependencies and drivers (GPU graphic card accelerators)
really irks me. https://github.com/Microsoft/LightGBM/wiki/Installation-Guide
I am wondering if there is way to share stable working binaries with SHA-256 hashes for CPU, GPU and MPI.

Most of the Microsoft code is shared as binaries, why push people back into bananaware times
(https://en.wikipedia.org/wiki/Perpetual_beta) and forcing them to compile their own code.
I understand that there will be code changes and improvements with every release, I just don't see
the point of forcing people to compile endless dependencies. Plus providing executables may improve the user base and number of contributors.

There is an Azure VM for deep learning on Ubuntu and CentOS and Windows
https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-data-science-dsvm-ubuntu-intro but no download for standalone and direct use on a local machine for lightGBM.

[wxchan]

I was thinking the same issue last weekend when I saw the release binary version of xgboost on http://www.picnet.com.au/blogs/guido/post/2016/09/22/xgboost-windows-x64-binaries-for-download/.

[Laurae2]

We can use Travis and AppVeyor to output file artifacts so every Pull Request has a binary for Windows & Linux, OpenCL and no OpenCL (but those would be locked to Intel/AMD).

Travis: https://stackoverflow.com/questions/12343452/how-to-publish-artifacts-in-travis-ci/33109519#33109519

AppVeyor: https://www.appveyor.com/docs/packaging-artifacts/

But we need people to contribute to make this possible. We also need to get AppVeyor setup in the case of Windows. Otherwise, if someone might volunteer to have servers (with all configurations) compiling binaries and host the files, at the risk of users (malware, etc.) that would be great.

welcome to contribute @tobigithub if you are able to.

As for the steps required to compile xgboost / LightGBM from scratch, they are not that large and are actually easy enough to perform both on Linux and Windows. There are existing repositories with over 1,000 dependencies.

I just don't see the point of forcing people to compile endless dependencies.

LightGBM does not have many/endless dependencies.

Also, for maximum performance and tuning to your CPU/GPU, it is recommended to compile by yourself the tools. If squeezing 3% training time on one month is about 1 day worth of savings, then it is better to compile than using premade binaries.

Note that for GPU:

• OpenCL implementation varies by companies (Intel, AMD, NVIDIA's OpenCLs are not interoperable for most scenarios).
• OpenCL versions may have deprecated functions, or may not follow OpenCL standards (especially Intel / NVIDIA).

Check the GPU targeting here: https://github.com/Microsoft/LightGBM/blob/master/docs/GPU-Targets.md

[tobigithub]

@Laurae2,
happy to help, never used those services, but looking into that how to integrate CUDA for WIN in AppVeyor and also in Travis seems even worse, also not sure if the free cloud services there offer native CUDA support, they usually requires paid subscriptions.

Also consider, that there is no containerized CUDA version for Windows, they will not run in VMs, plus local MPI versions and MinGW, boost and other dependencies create a lot of trouble to maintain with 10 different dependencies and environmental settings that are unrelated to lightGBM. So, sure I would be able to compile lightGBM, but then everything else brakes.

If you look at the installation instructions here https://github.com/Microsoft/LightGBM/blob/master/docs/GPU-Windows.md

I consider the lightGBM CUDA and OpenCL setup very complicated, not only getting all the CUDA and OPENCL dependencies, but VS2013, the DDLs, boost, mingw, endless environmental variables etc. Why not share the bred crumbs of the compiled EXE? That would be really nice, maybe not like in a production environment, but still helpful.

[guolinke]

you can get the binaries from the github release now: https://github.com/Microsoft/LightGBM/releases

The gpu version is still unavailable, due to these CI cannot support CUDA.

BTW, here is a simple version for installing GPU version in windows: https://github.com/Microsoft/LightGBM/wiki/Installation-Guide#windows-2 .

[rspadim]

please gpu support

[guolinke]

Ping @StrikerRUS and @huanzhang12

[StrikerRUS]

As @huanzhang12 mentioned here, we can build CUDA version at Travis side but cannot test it.

@Laurae2's answer is still actual, except this phrase about publishing CI artifacts:

But we need people to contribute to make this possible.

Done.

[guolinke]

@StrikerRUS BTW, current python package is uploaded by me manually, can we change it to auto publish when push a new tag ?

[StrikerRUS]

@guolinke Are you talking about PyPI?

It's possible in theory, but I'm not sure about the possibility of "iterative" uploading. I mean, we should upload from both sources: Travis and Appveyor corresponding platform versions.

[guolinke]

@StrikerRUS I think pypi supports that iterative update.
BTW, can you give me your pypi username ? I can add you to the maintainer.

[guolinke]

@StrikerRUS BTW, I want to move the CI to vsts-ci, which support Linux, OSX and Windows, and more parallel jobs.

[StrikerRUS]

@guolinke

BTW, can you give me your pypi username ? I can add you to the maintainer.

I haven't registered there yet. Should I?

BTW, I want to move the CI to vsts-ci, which support Linux, OSX and Windows, and more parallel jobs.

Sounds promising. Unfortunately, I've never heard about this service. Will read about it and try to help you.

[guolinke]

@StrikerRUS
I tried the vsts-ci, but it seems unstable now.
I think we can continue to use Travis and Appveyor for now.

[StrikerRUS]

@guolinke Ah, OK.
Also, I've noticed that vsts-ci is private, so contributors will be not able to see what's wrong with their PRs in case of tests fail.

[guolinke]

@StrikerRUS
refer to https://blogs.msdn.microsoft.com/devops/2018/04/27/vsts-public-projects-limited-preview/, we can set the build to public.

[guolinke]

@StrikerRUS
I met a error when running test on vsts:

2018-06-30T00:58:34.1076149Z Traceback (most recent call last):
2018-06-30T00:58:34.1101523Z   File "plot_example.py", line 62, in <module>
2018-06-30T00:58:34.1133791Z     graph.render(view=True)
2018-06-30T00:58:34.1149551Z   File "/usr/share/miniconda/envs/test-env/lib/python3.6/site-packages/graphviz/files.py", line 182, in render
2018-06-30T00:58:34.1165557Z     self._view(rendered, self._format)
2018-06-30T00:58:34.1180815Z   File "/usr/share/miniconda/envs/test-env/lib/python3.6/site-packages/graphviz/files.py", line 218, in _view
2018-06-30T00:58:34.1259903Z     view_method(filepath)
2018-06-30T00:58:34.1275792Z   File "/usr/share/miniconda/envs/test-env/lib/python3.6/site-packages/graphviz/backend.py", line 223, in view_unixoid
2018-06-30T00:58:34.1292151Z     subprocess.Popen(['xdg-open', filepath])
2018-06-30T00:58:34.1308176Z   File "/usr/share/miniconda/envs/test-env/lib/python3.6/subprocess.py", line 709, in __init__
2018-06-30T00:58:34.1381537Z     restore_signals, start_new_session)
2018-06-30T00:58:34.1397485Z   File "/usr/share/miniconda/envs/test-env/lib/python3.6/subprocess.py", line 1344, in _execute_child
2018-06-30T00:58:34.1447136Z     raise child_exception_type(errno_num, err_msg, err_filename)
2018-06-30T00:58:34.1462570Z FileNotFoundError: [Errno 2] No such file or directory: 'xdg-open': 'xdg-open'

any idea about how to fix it ?

[StrikerRUS]

@guolinke

set the build to public.

Wow, just one radio button, great!

any idea about how to fix it ?

Actually, this error is presented at Travis too, see, for example,
https://travis-ci.org/Microsoft/LightGBM/jobs/398216604#L1108
Fortunately, Travis doesn't treat it as error.

This error means that CI instance cannot open file with rendered Graphviz graph (pdf by default). So, you can either just make vsts ignore it, or install any pdf reader (at least, I believe in it; maybe it's more complicated than it seems).

[guolinke]

@huanzhang12
When I build GPU version at vsts ci, it met a error. The log is
log_128_65.zip

The docker file of running agent is https://github.com/Microsoft/vsts-agent-docker/blob/master/ubuntu/16.04/standard/Dockerfile

[guolinke]

@StrikerRUS can we ignore that exception at python code ?

[StrikerRUS]

@guolinke We can not open rendered graph, but it's an example, so it should be demonstrative...

We can change the file format. Can it help? png, for instance. Does vsts have any photo viewer installed?

---

GPU error log says that these two paths should be set properly:
https://github.com/Microsoft/LightGBM/blob/a6e878e2fc6e7f545921cbe337cc511fbd1f500d/.travis.yml#L48-L49

2018-06-30T01:15:50.6784413Z -- Looking for CL_VERSION_2_2
2018-06-30T01:15:50.7046622Z -- Looking for CL_VERSION_2_2 - not found
2018-06-30T01:15:50.7103886Z -- Looking for CL_VERSION_2_1
2018-06-30T01:15:50.7258379Z -- Looking for CL_VERSION_2_1 - not found
2018-06-30T01:15:50.7272231Z -- Looking for CL_VERSION_2_0
2018-06-30T01:15:50.7405597Z -- Looking for CL_VERSION_2_0 - not found
2018-06-30T01:15:50.7468101Z -- Looking for CL_VERSION_1_2
2018-06-30T01:15:50.7605502Z -- Looking for CL_VERSION_1_2 - not found
2018-06-30T01:15:50.7619974Z -- Looking for CL_VERSION_1_1
2018-06-30T01:15:50.7793103Z -- Looking for CL_VERSION_1_1 - not found
2018-06-30T01:15:50.7829945Z -- Looking for CL_VERSION_1_0
2018-06-30T01:15:50.7983612Z -- Looking for CL_VERSION_1_0 - not found

UPD:
I've note that .vsts-ci.yml doesn't contain the following parts of GPU test preparation phase:
https://github.com/Microsoft/LightGBM/blob/a6e878e2fc6e7f545921cbe337cc511fbd1f500d/.travis/setup.sh#L17-L19
https://github.com/Microsoft/LightGBM/blob/a6e878e2fc6e7f545921cbe337cc511fbd1f500d/.travis/setup.sh#L27-L34

[StrikerRUS]

@guolinke Oh, wait! What about sed?

sed -i 's/graph.render(view=True)/graph.render(view=False)/' examples/python-guide/plot_example.py

[guolinke]

refer to https://github.com/Microsoft/LightGBM/blob/vsts/.vsts-ci/setup.sh#L15

[guolinke]

@StrikerRUS @huanzhang12
The previous version has a bug, thus it cannot install amd sdk...
However, it still get the same error after fix it..

2018-06-30T16:11:00.1440588Z ../tests/c_api_test/test_.py ssssss..vsts-ci/test.sh: line 95:  2008 Segmentation fault      (core dumped) pytest ${BUILD_REPOSITORY_LOCALPATH}

log_125_68.zip

[guolinke]

@StrikerRUS
another error when install gpu from pip:

2018-06-30T16:09:28.0257643Z Installing collected packages: lightgbm
2018-06-30T16:09:28.0271815Z   Created temporary directory: /tmp/pip-record-gozayl_j
2018-06-30T16:09:28.0285588Z   Running setup.py install for lightgbm: started
2018-06-30T16:09:28.0304490Z     Running command /usr/share/miniconda/envs/test-env/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-req-build-9w5fxnud/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-gozayl_j/install-record.txt --single-version-externally-managed --compile --gpu --boost_root=/usr/share/miniconda/envs/test-env/ --opencl-include-dir=/root/AMDAPPSDK/include/
2018-06-30T16:09:28.0325154Z     usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
2018-06-30T16:09:28.0340250Z        or: -c --help [cmd1 cmd2 ...]
2018-06-30T16:09:28.0356749Z        or: -c --help-commands
2018-06-30T16:09:28.0373834Z        or: -c cmd --help
2018-06-30T16:09:28.0380750Z 
2018-06-30T16:09:28.0395180Z     error: option --boost_root not recognized
2018-06-30T16:09:28.0408773Z     Running setup.py install for lightgbm: finished with status 'error'

related command: https://github.com/Microsoft/LightGBM/blob/vsts/.vsts-ci/test.sh#L73

[StrikerRUS]

@guolinke
The option should be --boost-root, not --boost_root.

[guolinke]

@StrikerRUS Thank you so much !
BTW, there is a strange error for the check-the-docs.
50.txt

[StrikerRUS]

@guolinke

Seems there are some problems with URLs with anchors.

I think the reason is in different version of LinkChecker:

VSTS:

2018-07-06T07:37:34.6264751Z LinkChecker 9.3 Copyright (C) 2000-2014 Bastian Kleineidam

Travis:

LinkChecker 8.6 Copyright (C) 2000-2013 Bastian Kleineidam

[StrikerRUS]

@guolinke Can you please comment this line and see whether problem persists?
https://github.com/Microsoft/LightGBM/blob/2e93cdab9eee02d4d7f5cb3b6b31128dec94e25e/docs/conf.py#L58

[huanzhang12]

@guolinke My fix for GPU build was pushed into the vsts-fix branch. It was a Boost compatibility issue.
I saw you are working on building binaries with vsts. How does it work?