Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crystal Compiler can't target asm.js #535

Closed
keplersj opened this issue Apr 13, 2015 · 26 comments
Closed

Crystal Compiler can't target asm.js #535

keplersj opened this issue Apr 13, 2015 · 26 comments

Comments

@keplersj
Copy link
Contributor

_crystal build hello.cr --single-module --target "asmjs-unknown-emscripten"_

No available targets are compatible with this triple, see -version for the available targets.
*raise<String>:NoReturn +70 [26303048]
*Crystal::TargetMachine::create<String, String, Bool>:LLVM::TargetMachine +219 [26303048]
*Crystal::Compiler#compile<Crystal::Compiler, Array(Crystal::Compiler::Source), String>:Crystal::Compiler::Result +214 [26303048]
*Crystal::Command::run<Array(String)>:(Crystal::Compiler::Result | File | Bool | CFileIO | Nil) +243 [26303048]
__crystal_main +10024 [26303048]
main +48 [26303048]
__libc_start_main +245 [26303048]
_start +41 [26303048]

_crystal --version_

Crystal 0.6.1 [48461ba] (Wed Mar  4 22:54:56 UTC 2015)

_cat hello.cr_

subject = "world"
puts "Hello #{subject}!"

I was attempting an experiment to build a simple Crystal script, compile it, and transpile the Object file into JavaScript using Emscripten. However when I was using emcc to transpile the .o created in the .crystal folder I had gotten this error::

WARNING  root: .crystal/home/kepler/Developer/CrystalJS/hello.cr/main.o is not valid LLVM bitcode
ERROR    root: no input files
note that input files without a known suffix are ignored, make sure your input files end with one of: ('.c', '.C', '.cpp', '.cxx', '.cc', '.c++', '.CPP', '.CXX', '.CC', '.C++', '.m', '.mm', '.bc', '.o', '.obj', '.dylib', '.so', '.a', '.ll', '.h', '.hxx', '.hpp', '.hh', '.H', '.HXX', '.HPP', '.HH')

After seeing this error I attempted to use the .bc file created in the same directory as well, and got this error:

WARNING: Linking two modules of different data layouts: '/home/kepler/.emscripten_cache/libc.bc' is 'e-p:32:32-i64:64-v128:32:128-n32-S128' whereas '/home/kepler/Developer/CrystalJS/.crystal/home/kepler/Developer/CrystalJS/hello.cr/main.bc' is 'e-i64:64-f80:128-n8:16:32:64'
WARNING: Linking two modules of different target triples: /home/kepler/.emscripten_cache/libc.bc' is 'asmjs-unknown-emscripten' whereas '/home/kepler/Developer/CrystalJS/.crystal/home/kepler/Developer/CrystalJS/hello.cr/main.bc' is 'x86_64-unknown-linux-gnu'
Value:   %4 = call %CFileIO @"*CFileIO::new<Pointer(Void)>:CFileIO"(i8* %3)
LLVM ERROR: Unrecognized struct value
Traceback (most recent call last):
  File "/usr/local/emsdk_portable/emscripten/master/emcc", line 1301, in <module>
    shared.Building.llvm_opt(final, link_opts)
  File "/usr/local/emsdk_portable/emscripten/master/tools/shared.py", line 1461, in llvm_opt
    assert os.path.exists(target), 'Failed to run llvm optimizations: ' + output
AssertionError: Failed to run llvm optimizations: 

I'll keep poking around to see if there is a fix I can do on my side, outside of the compiler, but I thought I'd pass this on to you guys to see what you might be able to do about it. There is of course the issue that asm.js is not an upstreamed backend in the LLVM yet, so I would wholly understand if you guys cannot do anything until then.

@jhass
Copy link
Member

jhass commented Apr 13, 2015

I tried fooling around a bit by compiling crystal against emscripten-fastcomp, but without much luck (it compiled, but it doesn't seem to have the target available). But even if we get past this point, in Crystal head pretty much every program depends on libevent, libpcl and libpcre, so these would need to be ported first anyway.

@ysbaddaden
Copy link
Contributor

You may use the --prelude=empty to skip loading src/prelude.rb which will prevent loading the core library. You're down to the raw, with only the Crystal syntax available (not even puts is available), but it can help to start compiling some basic code, and to only enable some parts of the core library.

@keplersj
Copy link
Contributor Author

keplersj commented May 18, 2016

I've made some progress fairly recently.

_cat number.cr_

1

_./bin/crystal build --mcpu asmjs --emit llvm-ir --verbose --single-module --prelude=empty number.cr_

Using compiled compiler at .build/crystal
'asmjs' is not a recognized processor for this target (ignoring processor)
'asmjs' is not a recognized processor for this target (ignoring processor)
'asmjs' is not a recognized processor for this target (ignoring processor)
'asmjs' is not a recognized processor for this target (ignoring processor)
'asmjs' is not a recognized processor for this target (ignoring processor)
'asmjs' is not a recognized processor for this target (ignoring processor)
cc -o "/Users/kepler/Developer/crystal-lang/crystal/number" "${@}"  -rdynamic  _main.o

_cat number.ll_

; ModuleID = 'main_module'
target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-apple-macosx"

@ARGC_UNSAFE = global i32 0
@ARGV_UNSAFE = global i8** null

define internal i32 @__crystal_main(i32 %argc, i8** %argv) {
entry:
  store i32 %argc, i32* @ARGC_UNSAFE
  store i8** %argv, i8*** @ARGV_UNSAFE
  ret i32 1
}

declare i32 @printf(i8*, ...)

; Function Attrs: uwtable
define i32 @main(i32 %argc, i8** %argv) #0 {
entry:
  %0 = call i32 @__crystal_main(i32 %argc, i8** %argv)
  ret i32 0
}

attributes #0 = { uwtable "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf"="true" }

_emcc -o number.js -rdynamic number.ll_

warning: Linking two modules of different data layouts: '/Users/kepler/.emscripten_cache/asmjs/libc.bc' is 'e-p:32:32-i64:64-v128:32:128-n32-S128' whereas '/var/folders/kt/npxm5bgn7_97lg4r5k91p1bc0000gn/T/tmphmCu4C/number_0.o' is 'e-m:o-i64:64-f80:128-n8:16:32:64-S128'

warning: Linking two modules of different target triples: /Users/kepler/.emscripten_cache/asmjs/libc.bc' is 'asmjs-unknown-emscripten' whereas '/var/folders/kt/npxm5bgn7_97lg4r5k91p1bc0000gn/T/tmphmCu4C/number_0.o' is 'x86_64-apple-macosx'

warning: Linking two modules of different data layouts: '/Users/kepler/.emscripten_cache/asmjs/dlmalloc.bc' is 'e-p:32:32-i64:64-v128:32:128-n32-S128' whereas '/var/folders/kt/npxm5bgn7_97lg4r5k91p1bc0000gn/T/tmphmCu4C/number_0.o' is 'e-m:o-i64:64-f80:128-n8:16:32:64-S128'

warning: Linking two modules of different target triples: /Users/kepler/.emscripten_cache/asmjs/dlmalloc.bc' is 'asmjs-unknown-emscripten' whereas '/var/folders/kt/npxm5bgn7_97lg4r5k91p1bc0000gn/T/tmphmCu4C/number_0.o' is 'x86_64-apple-macosx'

warning: incorrect target triple 'x86_64-apple-macosx' (did you use emcc/em++ on all source files and not clang directly?)

_cat number.js_

Huge file, showing only transpiled relevant Crystal code:

function ___crystal_main($argc,$argv) {
 $argc = $argc|0;
 $argv = $argv|0;
 var label = 0, sp = 0;
 sp = STACKTOP;
 HEAP32[31] = $argc;
 HEAP32[32] = $argv;
 return 1;
}
function _main($argc,$argv) {
 $argc = $argc|0;
 $argv = $argv|0;
 var label = 0, sp = 0;
 sp = STACKTOP;
 (___crystal_main($argc,$argv)|0);
 return 0;
}

@keplersj
Copy link
Contributor Author

From what I can tell Emscripten comes with a version of musl for its libc, so Crystal should now be able to support it thanks to @ysbaddaden's posix project.

With an ABI it seems to me that the compiler might now have to be coded to switch between Emscripten LLVM and the regular System LLVM. Both have their own linker.

@keplersj
Copy link
Contributor Author

I just realized that Emscripten comes with a tool to aid building projects with Makefiles. I got an interesting result after running Crystal through it. I'll look more into how the library dependencies should be handled soon.

_emmake make_

/usr/local/Cellar/emscripten/1.36.3/libexec/em++ -c -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc `/usr/local/bin/llvm-config-3.6 --cxxflags`
/usr/local/Cellar/emscripten/1.36.3/libexec/emcc  -fPIC   -c -o src/ext/sigfault.o src/ext/sigfault.c
ar -rcs src/ext/libcrystal.a src/ext/sigfault.o
warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: warning for library: src/ext/libcrystal.a the table of contents is empty (no object file members in the library define global symbols)
CRYSTAL_CONFIG_PATH=`pwd`/src ./bin/crystal build  -o .build/crystal src/compiler/crystal.cr -D without_openssl -D without_zlib
WARNING:root:_main.o is not valid LLVM bitcode
WARNING:root:emcc: cannot find library "event"
WARNING:root:emcc: cannot find library "pcre"
WARNING:root:emcc: cannot find library "gc"
WARNING:root:emcc: cannot find library "pthread"
WARNING:root:emcc: cannot find library "iconv"
WARNING:root:emcc: cannot find library "dl"
--: /Users/kepler/.cache/crystal/crystal-run-macro-run-_Users_kepler_Developer_crystal-lang_crystal_src_ecr_process.cr.tmp: Permission denied
Error in ./src/compiler/crystal.cr:3: instantiating 'Crystal::Command:Class#run()'

Crystal::Command.run
                 ^~~

instantiating 'run(Array(String))'

in ./src/compiler/crystal/command.cr:38: instantiating 'Crystal::Command#run()'

    new(options).run
                 ^~~

in ./src/compiler/crystal/command.cr:54: instantiating 'init()'

        init
        ^~~~

in ./src/compiler/crystal/command.cr:150: instantiating 'Crystal::Init:Module#run(Array(String))'

    Init.run(options)
         ^~~

in ./src/compiler/crystal/tools/init.cr:41: instantiating 'Crystal::Init::InitProject#run()'

      InitProject.new(config).run
                              ^~~

in ./src/compiler/crystal/tools/init.cr:153: instantiating 'Array(Crystal::Init::View+:Class)#each()'

        views.each do |view|
              ^~~~

in ./src/array.cr:813: instantiating 'each_index()'

    each_index do |i|
    ^~~~~~~~~~

in ./src/array.cr:813: instantiating 'each_index()'

    each_index do |i|
    ^~~~~~~~~~

in ./src/compiler/crystal/tools/init.cr:153: instantiating 'Array(Crystal::Init::View+:Class)#each()'

        views.each do |view|
              ^~~~

in ./src/compiler/crystal/tools/init.cr:154: instantiating 'Crystal::Init::View+#render()'

          view.new(config).render
                           ^~~~~~

in ./src/compiler/crystal/tools/init.cr:131: instantiating 'to_s()'

        File.write(full_path, to_s)
                              ^~~~

in ./src/object.cr:75: instantiating 'String:Class#build()'

    String.build do |io|
           ^~~~~

in ./src/string.cr:234: instantiating 'String::Builder:Class#build(Int32)'

    String::Builder.build(capacity) do |builder|
                    ^~~~~

in ./src/string.cr:234: instantiating 'String::Builder:Class#build(Int32)'

    String::Builder.build(capacity) do |builder|
                    ^~~~~

in ./src/object.cr:75: instantiating 'String:Class#build()'

    String.build do |io|
           ^~~~~

in ./src/object.cr:76: instantiating 'to_s(String::Builder)'

      to_s io
      ^~~~

in macro 'embed' /Users/kepler/Developer/crystal-lang/crystal/src/ecr/macros.cr:81, line 1:

  1.     {{ run("ecr/process", "/Users/kepler/Developer/crystal-lang/crystal/src/compiler/crystal/tools/init/template/example_spec.cr.ecr", "__io__") }}
  2.   

    {{ run("ecr/process", "/Users/kepler/Developer/crystal-lang/crystal/src/compiler/crystal/tools/init/template/example_spec.cr.ecr", "__io__") }}
    ^

expanding macro
in macro 'embed' /Users/kepler/Developer/crystal-lang/crystal/src/ecr/macros.cr:81, line 1:

  1.     {{ run("ecr/process", "/Users/kepler/Developer/crystal-lang/crystal/src/compiler/crystal/tools/init/template/example_spec.cr.ecr", "__io__") }}
  2.   

    {{ run("ecr/process", "/Users/kepler/Developer/crystal-lang/crystal/src/compiler/crystal/tools/init/template/example_spec.cr.ecr", "__io__") }}
       ^~~

Error executing run: ecr/process "/Users/kepler/Developer/crystal-lang/crystal/src/compiler/crystal/tools/init/template/example_spec.cr.ecr" "__io__"

Got:



make: *** [.build/crystal] Error 1

@rdp
Copy link
Contributor

rdp commented Oct 21, 2016

I had the thought recently that it might be nice to target emscripten/asm.js with crystal :) (or the JVM? but that's a bit tricker). Interesting ideas...

@mverzilli
Copy link

Now with Web Assembly in the picture, does it still make sense to target asm.js? Unless someone still strongly believes otherwise, I'd vote to close this issue.

@keplersj
Copy link
Contributor Author

@mverzilli It's the same backend

@mverzilli
Copy link

That was fast :). Thanks for the clarification!

@keplersj
Copy link
Contributor Author

Of course! #3634 should still work with WebASM due to LLVM awesomeness, the only standing issue is the dependencies like Boehm

@catmando
Copy link

catmando commented Jul 3, 2018

The ruby world now has a stable near perfect ruby -> js transpiler. Of course stdlib stuff like threads, filemgt, etc etc and a very fee language features are simply not possible. However the subset makes it possible to code complete websites in a single language.

A emscriptem targeted crystal subset would be a logical and highly desirable next step for a lot of people using opal and ruby.

@keplersj any further news on your efforts?

@keplersj
Copy link
Contributor Author

keplersj commented Jul 4, 2018

Compiling the language with an asm.js or wasm backend isn't entirely too difficult. I imagine that most of the code from my asm.js branch should still be applicable. The difficult part comes in with the Crystal standard library and garbage collection. If you create the subset you're talking about @catmando it should be more than possible to compile and execute it.

@catmando
Copy link

catmando commented Jul 5, 2018

@keplersj tx for quick reply. GC is written in C correct? So can it just also be run through emscriptem?

@refi64
Copy link
Contributor

refi64 commented Jul 5, 2018

I don't believe so. The Boehm GC relies on a lot of platform-specific, and IIRC some of it is also assembler. You'd basically have to redo the whole thing.

@keplersj
Copy link
Contributor Author

keplersj commented Jul 5, 2018

@catmando If you create your own prelude with the subset you imagine, this technique I use above with emscripten should work to output runnable asm.js or wasm. That technique above certainly isn't foolproof, but it doesn't require a custom build of the compiler. Could be worth a try, might be helpful for building that web-focus subset you imagine.

@rdp
Copy link
Contributor

rdp commented Jul 6, 2018

Maybe don't need GC because javascript land takes care of it for you? Not too familiar with asm.js... :)

@asterite
Copy link
Member

asterite commented Jul 6, 2018

The GC has very little to do here. The problem is fibers and context switching via inline assembler.

@RX14
Copy link
Member

RX14 commented Jul 7, 2018

Neither wasm or asm.js well-support coroutines or multiple stacks, which makes porting fibers impractical. wasm has support for this timetabled, I think we'll be waiting for that and GC. And probably not asm.js - at least not with fibers.

@rdp
Copy link
Contributor

rdp commented Jul 9, 2018

I guess rust can target wasm, what does it do with threads, just run single threaded, anybody know? Also wonder how golang does GC in wasm if it targets it...

@mmKALLL
Copy link

mmKALLL commented Jul 23, 2018

I started speccing a VM for WASM written in Crystal, but compiling Crystal into WASM seems like a much more practical project. I'll be keeping an eye out on this topic, and would like to contribute later on. Are there any actionable plans brewing at the moment?

@RX14
Copy link
Member

RX14 commented Jul 23, 2018

@mmKALLL no, we're waiting for wasm to mature.

@rdp
Copy link
Contributor

rdp commented Sep 5, 2018

Can we close this in favor of #829 since "asm.js is old and wasm is its enw replacement" maybe? :)

@keplersj
Copy link
Contributor Author

keplersj commented Sep 5, 2018

I'm gonna say no. In an ideal world the Crystal compiler would be able to output both asm.js and wasm, letting the user decide which output they need. If a user still needs to support browsers pre-wasm, or even just older JavaScript runtimes that support ES5 (with or without asm.js optimizations), they should still be able to output their codebases.

@catmando
Copy link

catmando commented Jun 28, 2019

FYI: I just put a discussion topic around this general topic: https://forum.crystal-lang.org/t/crystal-js-transpiler/903

@straight-shoota
Copy link
Member

Closing. asm.js is nowhere on the roadmap and probably obsolete with WASM support (which has already partly landed).

@straight-shoota straight-shoota closed this as not planned Won't fix, can't repro, duplicate, stale Oct 27, 2022
@workingjubilee
Copy link

Correct, asm.js support has been removed from the EMCC toolchain1, which many people were using for their support for this, and it is considered deprecated2.

Footnotes

  1. https://github.com/emscripten-core/emscripten/issues/18013

  2. https://developer.mozilla.org/en-US/docs/Games/Tools/asm.js

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests