Outdated docs for hacking on sandstorm shell #3047

dmitry-sher · 2018-02-27T12:27:55Z

UPDATE

Easiest way to hack on sandstorm shell is as follows:

install latest VirtualBox (5.2.6 is good)
import VM, made by @xet7 . read How to compile Sandstorm? #3036 for details. VM link
set up VM's networking as appropriate (i prefer NAT with some port like host's 4222 as guest 22 and host's 127.0.0.1:6080 -> guest's 0.0.0.0:6080). ssh to VM with wekan:wekan ssh -p 4222 wekan@localhost. You can add your keys ssh-copy-id -p 4222 wekan@localhost
update /opt/sandstorm/sandstorm.conf in VM. set BIND_IP=0.0.0.0. comment out #UPDATE_CHANNEL=dev. restart sandstorm sudo service sandstorm restart. Now you can access sandstorm from host OS on port 6080
add wekan user to sandstorm group. sudo usermod -a -G sandstorm wekan. logout and login back for changes to take place.
go to ~/repos/sandstorm and make update
run sandstorm dev-shell from the repo root.

Below are my previous attempts to do it on Ubuntu 16.04.3. You can safely skip it.

The commands

cd shell
sudo service sandstorm stop-fe
./run-dev.sh

Are not working anymore. Current command is sandstorm dev-shell in project root.

If you run it, you would see a multitude of problems.

1a. Preparations to run hacking.

cd shell
meteor npm install

When trying to run sandstorm dev-shell. From the latest master commit 760faa2

=> Errors prevented startup:                  
   
   While processing files with fourseven:scss (for target web.browser):
   /client/styles/shell.scss: Scss compiler error: File to import:
   {}/client/styles/_icons.scss not found in file:
   /home/sherd/sandstorm/shell/{}/client/styles/shell.scss

As you can see, the file is missing in the sources:
https://github.com/sandstorm-io/sandstorm/tree/master/shell/client/styles.
It's generated. See how to generate it below.

You would miss capnp.node and capnp.js. You can find them in /opt/sandstorm/latest/node_modules. Copy everything from that folder to shell/node_modules.
There are some missing capnp files, like payments.capnp. You can find all current capnp files in project in src/sandstorm/. Copy them to shell/node_modules/sandstorm.

The text was updated successfully, but these errors were encountered:

dmitry-sher · 2018-02-27T12:31:42Z

ok nice... the steps to get that _icons.scss:

go to icons in project root
run npm install
ensure you have grunt: npm install -g grunt
run grunt

dmitry-sher · 2018-02-27T13:54:34Z

Problem is much bigger than that.
That whole part of the docs about installing from source is outdated.
Requirements changed a lot.
I will write down what i had to do in order to get it running, if i will succeed.

Before running make or any other steps from Option 4: Installing from Source, do these (all commands are for Ubuntu 16.04):

Install cmake: sudo apt-get install cmake -y. Install clang-6.0:

wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
sudo apt-add-repository "deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-6.0 main"
sudo apt-get update
sudo apt-get install -y clang-6.0

Install Go.

sudo add-apt-repository ppa:gophers/archive
sudo apt-get update
sudo apt-get install golang-1.9-go

Add /usr/lib/go-1.9/bin to PATH.

dmitry-sher · 2018-02-27T14:08:08Z

Right now i cannot make boringssl because it wants clang-6.0 and the clang-6.0 doesn't want to install on my ubuntu 16.04 because...

E: Failed to fetch http://apt.llvm.org/xenial/dists/llvm-toolchain-xenial-6.0/main/binary-amd64/Packages.gz  Hash Sum mismatch

update: here is relevant bug from llvm bugzilla. it seems closed.
https://bugs.llvm.org/show_bug.cgi?id=36396

update 2: looks it can be a CDN problem. another run and apt-get update worked.

xet7 · 2018-02-27T14:23:35Z

@dmitry-sher

I made Debian 9 VirtualBox Appliance where Wekan and Sandstorm compile successfully:
https://github.com/wekan/wekan/releases

There is Wekan-Sandstorm.ova.

dmitry-sher · 2018-02-27T14:55:04Z

Ok, now the sandstorm is not compiling. Not that I think it's really necessary. I just thought that maybe, if you want to hack on shell, you need to get it make-able first. I will try to continue.

✘ compile: sandstorm/gateway.c++
    /ekam-provider/canonical/sandstorm/gateway.c++:364:1: warning: control may 
      reach end of non-void function [-Wreturn-type]
    }
    ^
    /ekam-provider/canonical/sandstorm/gateway.c++:831:42: error: no member 
      named 'try_emplace' in 'std::map<kj::StringPtr, sandstorm::GatewayService:
      :ForeignHostnameEntry, std::less<kj::StringPtr>, std::allocator<std::pair<
      const kj::StringPtr, sandstorm::GatewayService::ForeignHostnameEntry> > >'
        auto insertResult = foreignHostnames.try_emplace(key, kj::mv(entry));

update try_emplace is c++17 thing. Maybe something is wrong with my compiler..

update 2 i've updated gcc to 7.2.0:

sudo apt-get install build-essential software-properties-common -y && \
sudo add-apt-repository ppa:ubuntu-toolchain-r/test -y && \
sudo apt-get update -y && \
sudo apt-get install gcc-7 g++-7 -y && \
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 60 --slave /usr/bin/g++ g++ /usr/bin/g++-7 && \
sudo update-alternatives --config gcc

now we have different error.

dmitry-sher · 2018-02-27T15:41:48Z

When I try to run make in sandstorm project root with gcc-7.2.0 I get this error:

✘ compile: sandstorm/sandstorm-http-bridge.c++
    In file included from /ekam-provider/canonical/sandstorm/sandstorm-http-
      bridge.c++:36:
    In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/7.2.0/../../../..
      /include/c++/7.2.0/map:60:
    In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/7.2.0/../../../..
      /include/c++/7.2.0/bits/stl_tree.h:63:
    In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/7.2.0/../../../..
      /include/c++/7.2.0/bits/stl_algobase.h:67:
    In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/7.2.0/../../../..
      /include/c++/7.2.0/bits/stl_iterator.h:66:
    /usr/bin/../lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0/
      bits/ptr_traits.h:133:9: error: 'rebind' declared as a pointer to a 
      reference of type 'sandstorm::SessionContext::Client &'
            using rebind = _Up*;

update the error is meaningless. It should be smth with this part of sandstorm-http-bridge.c++:

std::map<kj::StringPtr, SessionContext::Client&> sessions;

see that & there? is that wrong? i'm not a c++ ninja for almost 15 years now...

dmitry-sher · 2018-02-27T15:46:13Z

@xet7 i downloaded Wekan-Sandstorm-0.77.ova - 4.1 GB . For Developers compiling Wekan, Sandstorm, Wekan for Sandstorm, etc. two times and each time when i try to import it into my VirtualBox 5.1.20 r114629 (Qt5.6.2) on MacOS Sierra 10.12.6 (16G29) it fails with following error:

Error reading OVA '/Users/dmitrysher/Downloads/chrome/Wekan-Sandstorm-0.77.ova' (VERR_TAR_UNEXPECTED_EOS).

File size is 4398810112 bytes, sha1 hash ba1f92b2577a32bce7ee4a42b696a0a7f7191599 made with command

$ shasum -v
5.84
$ shasum /Users/dmitrysher/Downloads/chrome/Wekan-Sandstorm-0.77.ova
ba1f92b2577a32bce7ee4a42b696a0a7f7191599

dmitry-sher · 2018-02-27T16:59:45Z

What a nice catch-22.

~/sandstorm/shell$ sandstorm dev-shell
sandstorm dev-shell: please run this from the root of your Sandstorm source tree
Try 'sandstorm dev-shell --help' for more information.
sherd@osboxes:~/sandstorm/shell$ cd ..
sherd@osboxes:~/sandstorm$ sandstorm dev-shell --help
Usage: sandstorm dev-shell [<option>...] [<meteor-arg>...]

Runs the Sandstorm shell in development mode. For use in developing Sandstorm
itself. Must be run from the `shell` subdirectory of the Sandstorm source code.

Do you see that?
If you run the command from project root, it starts to run but eventually fails. And help message asks to run the command from shell subdir.
But when you run it from shell subdir, it asks to run command from project root.

dmitry-sher · 2018-02-27T17:08:41Z

I think i will give up for today. It's already evening in moscow. Short summary of the day:

docs on site are obsolete
source docs are incomplete
source docs and help messages are contradictory
VM from @xet7 is not working
new dev-shell command when i run it from project root is not working at my VM due to multitude of problems, the last problem is message kj/filesystem-disk-unix.c++:1668: warning: PWD environment variable doesn't match current directory; pwd = /home/sherd/sandstorm. It should be somehow connected to what dir it is running from.. somehow..
i was ready to dive into c++ code and fix or debug those problems, but i couldn't build the sandstorm c++ code because of some c++17 magic that i don't understand.

xet7 · 2018-02-27T19:46:37Z

@dmitry-sher

Your VirtualBox version 5.1.20 r114629 (Qt5.6.2) for Mac OS X is really old.

That VM has been done yesterday with newest VirtualBox 5.2.6, you can
download it from https://www.virtualbox.org , it is compatible with that VM.

You can also check SHA256SUMS.txt that your download is not corrupted:
https://releases.wekan.team/

For me downloading that VM, importing to VirtualBox 5.2.6 and starting it works fine.

Current install Sandstorm from source instructions are at:
https://docs.sandstorm.io/en/latest/install/#option-4-installing-from-source

My previous issue about compiling sandstorm is #3036 , dependencies are currently these. On Debian 9 clang-6.0 is at Debian package repos.

Currently I have these capnproto packages installed, you may need some of those maybe, if those are not installed with with that sudo apt line.

ii  capnproto                             0.6.1-1                             amd64        tool for working with the Cap'n Proto data interchange format
ii  libcap-dev:amd64                      1:2.25-1.2                          amd64        POSIX 1003.1e capabilities (development)
ii  libcap-ng0:amd64                      0.7.7-3.1+b1                        amd64        An alternate POSIX capabilities library
ii  libcap2:amd64                         1:2.25-1.2                          amd64        POSIX 1003.1e capabilities (library)
ii  libcap2-bin                           1:2.25-1.2                          amd64        POSIX 1003.1e capabilities (utilities)
ii  libcapnp-0.6.1                        0.6.1-1                             amd64        Cap'n Proto C++ library
ii  libcapnp-dev                          0.6.1-1                             amd64        Cap'n Proto C++ library (development files)

Also because Debian did not have sudo installed, I installed it:

su
(password here)
apt install sudo
exit

Then continue installing:

sudo apt install build-essential libcap-dev xz-utils zip \
unzip strace curl clang clang-6.0 discount git python zlib1g-dev golang \
cmake autoconf

mkdir ~/go

mkdir ~/repos
cd ~/repos
git clone https://github.com/sandstorm-io/sandstorm.git
cd sandstorm
make

For that VirtualBox VM, I did install Sandstorm normally, without compiling it from source, from https://sandstorm.io/install . I did install it in dev mode. I did also compile sandstorm from source on that VM. To prevent Sandstorm from auto-updating, dev channel needs to be commented out at /opt/sandstorm/sandstorm.conf . Then install compiled sandstorm version, more info at:
https://docs.sandstorm.io/en/latest/install/#option-4-installing-from-source

npm for Debian is possible to install from here:
https://nodejs.org/en/download/package-manager/

@kentonv uses Debian for developing Sandstorm.

dmitry-sher · 2018-02-28T07:15:19Z

Thnx @xet7 , upgrading VirtualBox helped. My bad.

dmitry-sher · 2018-02-28T07:26:04Z

Anyway, the problem with hacking the shell is the same:

running sandstorm dev-shell from repo root brings the warning kj/filesystem-disk-unix.c++:1668: warning: PWD environment variable doesn't match current directory; pwd = /home/wekan/repos/sandstorm and some broken pipe error
running sandstorm dev-shell from the shell subfolder brings error sandstorm dev-shell: please run this from the root of your Sandstorm source tree

Full log

wekan@wekan-sandstorm:~/repos/sandstorm$ sandstorm dev-shell
[[[[[ ~/repos/sandstorm/shell ]]]]]           

=> Started proxy.                             
=> Meteor 1.6.1 is available. Update this project with 'meteor update'.
W20180228-09:20:49.375(2)? (STDERR) kj/filesystem-disk-unix.c++:1668: warning: PWD environment variable doesn't match current directory; pwd = /home/wekan/repos/sandstorm
W20180228-09:20:49.938(2)? (STDERR) Unhandled exception:  { Error: sendmsg(fd, &msg, 0): Broken pipe
W20180228-09:20:49.939(2)? (STDERR)     at Object.exports.connect (/home/wekan/repos/sandstorm/node_modules/capnp.js:265:35)
W20180228-09:20:49.940(2)? (STDERR)     at core.js (server/core.js:662:40)
W20180228-09:20:49.940(2)? (STDERR)     at fileEvaluate (packages/modules-runtime.js:343:9)
W20180228-09:20:49.941(2)? (STDERR)     at require (packages/modules-runtime.js:238:16)
W20180228-09:20:49.942(2)? (STDERR)     at /home/wekan/repos/sandstorm/shell/.meteor/local/build/programs/server/app/app.js:17299:1
W20180228-09:20:49.943(2)? (STDERR)     at infos.forEach.info (/home/wekan/repos/sandstorm/shell/.meteor/local/build/programs/server/boot.js:414:13)
W20180228-09:20:49.944(2)? (STDERR)     at Array.forEach (<anonymous>)
W20180228-09:20:49.944(2)? (STDERR)     at /home/wekan/repos/sandstorm/shell/.meteor/local/build/programs/server/boot.js:413:9
W20180228-09:20:49.945(2)? (STDERR)     at /home/wekan/repos/sandstorm/shell/.meteor/local/build/programs/server/boot.js:463:5
W20180228-09:20:49.945(2)? (STDERR)     at Function.run (/home/wekan/repos/sandstorm/shell/.meteor/local/build/programs/server/profile.js:510:12)
W20180228-09:20:49.946(2)? (STDERR)     at /home/wekan/repos/sandstorm/shell/.meteor/local/build/programs/server/boot.js:462:11
W20180228-09:20:49.946(2)? (STDERR)   cppFile: 'node-capnp/capnp.cc',
W20180228-09:20:49.947(2)? (STDERR)   line: 2062,
W20180228-09:20:49.948(2)? (STDERR)   kjType: 'disconnected' }

dmitry-sher · 2018-02-28T08:25:40Z

After i made make update, sandstorm dev-shell worked from the project root.

urakagi · 2018-03-06T03:16:12Z

Hi @dmitry-sher, I'm also facing sandstorm-http-bridge.c++: std::map<kj::StringPtr, SessionContext::Client&> sessions; problem. How did you solve this? I have trouble about localization but I'm not able to build sandstorm now.

dmitry-sher · 2018-03-06T06:32:54Z

@urakagi I gave up and used pre-built VM. Read first comment: #3047 (comment) . BTW, what OS are you using?

I have seen a difference between my initial VM and xet7's one: I had Ubuntu 16.03 with gcc 7.2.0. xet7's: Debian 9 with gcc 7.3.0

urakagi · 2018-03-07T06:05:40Z

I'm using Ubuntu 16.04. I was able to build up hmm.

kentonv · 2018-03-11T21:55:28Z

Hi all,

I eliminated the Clang dependency because it trips people up. The build now downloads Clang binaries directly from the Chromium project, so you don't have to install it from apt.llvm.org or whatever. See commit 4463c3f.

I've verified that with this change, the build works correctly in a fresh Debian Stretch VM.

Michael-S · 2018-03-11T23:30:35Z

Very nice, thank you. Separately, I have a list of Sandstorm dependencies other than Clang for Fedora 27. Do you think it's worth adding that to docs/install.md underneath the Debian/Ubuntu section?

ocdtrekkie · 2018-03-16T23:49:49Z

@Michael-S It's probably going to be useful to someone, probably wouldn't hurt to have it there, and it may end up helping people with other distros too.

kentonv · 2018-03-17T16:41:42Z

@Michael-S Yes, that would be great.

Michael-S · 2018-03-17T21:17:44Z

Okay, done with #3060

ocdtrekkie · 2020-01-15T16:58:42Z

I think I can close this, as fixes were made, and the PR that Michael made after this was also merged.

dmitry-sher changed the title ~~missing _icons.scss~~ Outdated docs for hacking on sandstorm shell Feb 27, 2018

This was referenced Mar 2, 2018

Clang 6 is now required, minor docs update. #3048

Closed

Clang v6 doc tweak #3049

Closed

ocdtrekkie closed this as completed Jan 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Outdated docs for hacking on sandstorm shell #3047

Outdated docs for hacking on sandstorm shell #3047

dmitry-sher commented Feb 27, 2018 •

edited

Loading

dmitry-sher commented Feb 27, 2018

dmitry-sher commented Feb 27, 2018 •

edited

Loading

dmitry-sher commented Feb 27, 2018 •

edited

Loading

xet7 commented Feb 27, 2018

dmitry-sher commented Feb 27, 2018 •

edited

Loading

dmitry-sher commented Feb 27, 2018 •

edited

Loading

dmitry-sher commented Feb 27, 2018 •

edited

Loading

dmitry-sher commented Feb 27, 2018

dmitry-sher commented Feb 27, 2018 •

edited

Loading

xet7 commented Feb 27, 2018

dmitry-sher commented Feb 28, 2018

dmitry-sher commented Feb 28, 2018 •

edited

Loading

dmitry-sher commented Feb 28, 2018

urakagi commented Mar 6, 2018

dmitry-sher commented Mar 6, 2018

urakagi commented Mar 7, 2018

kentonv commented Mar 11, 2018

Michael-S commented Mar 11, 2018

ocdtrekkie commented Mar 16, 2018

kentonv commented Mar 17, 2018

Michael-S commented Mar 17, 2018

ocdtrekkie commented Jan 15, 2020

Outdated docs for hacking on sandstorm shell #3047

Outdated docs for hacking on sandstorm shell #3047

Comments

dmitry-sher commented Feb 27, 2018 • edited Loading

UPDATE

Below are my previous attempts to do it on Ubuntu 16.04.3. You can safely skip it.

dmitry-sher commented Feb 27, 2018

dmitry-sher commented Feb 27, 2018 • edited Loading

dmitry-sher commented Feb 27, 2018 • edited Loading

xet7 commented Feb 27, 2018

dmitry-sher commented Feb 27, 2018 • edited Loading

dmitry-sher commented Feb 27, 2018 • edited Loading

dmitry-sher commented Feb 27, 2018 • edited Loading

dmitry-sher commented Feb 27, 2018

dmitry-sher commented Feb 27, 2018 • edited Loading

xet7 commented Feb 27, 2018

dmitry-sher commented Feb 28, 2018

dmitry-sher commented Feb 28, 2018 • edited Loading

dmitry-sher commented Feb 28, 2018

urakagi commented Mar 6, 2018

dmitry-sher commented Mar 6, 2018

urakagi commented Mar 7, 2018

kentonv commented Mar 11, 2018

Michael-S commented Mar 11, 2018

ocdtrekkie commented Mar 16, 2018

kentonv commented Mar 17, 2018

Michael-S commented Mar 17, 2018

ocdtrekkie commented Jan 15, 2020

dmitry-sher commented Feb 27, 2018 •

edited

Loading

dmitry-sher commented Feb 27, 2018 •

edited

Loading

dmitry-sher commented Feb 27, 2018 •

edited

Loading

dmitry-sher commented Feb 27, 2018 •

edited

Loading

dmitry-sher commented Feb 27, 2018 •

edited

Loading

dmitry-sher commented Feb 27, 2018 •

edited

Loading

dmitry-sher commented Feb 27, 2018 •

edited

Loading

dmitry-sher commented Feb 28, 2018 •

edited

Loading