Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Emit mixin forwarders by default? #2563

Closed
smarter opened this issue May 29, 2017 · 8 comments
Closed

Emit mixin forwarders by default? #2563

smarter opened this issue May 29, 2017 · 8 comments

Comments

@smarter
Copy link
Member

smarter commented May 29, 2017

This is what Scala 2.12 does, for startup performance reasons (http://www.scala-lang.org/blog/2016/07/08/trait-method-performance.html). Currently we're only doing it for JUnit-annotated methods (#2557).
/cc @DarkDimius

@DarkDimius
Copy link
Contributor

@smarter, I still believe that the results of this blog-post are not true for jit-ed code and we should perform a more close study.

@DarkDimius
Copy link
Contributor

While this may help startup time, but I feel this would only worsen peak performance

@smarter
Copy link
Member Author

smarter commented May 29, 2017

scala/scala#5429 says:

  • Startup is slower without forwarders
  • For a large program with a short running time (compiling a small file) the difference can be significant (> 40%)
  • Hot performance is not affected
  • There are probably multiple underlying causes (CHA, populating vtables), we don't have a clear picture

And also:

We could not identify any performance issues due to having two forwarders. Note that the JVM will unconditionally inline small methods (< 35 bytes) without querying any heuristics, the forwarders fall into this category.

@DarkDimius
Copy link
Contributor

Citing your citation:

we don't have a clear picture

I don't think we should make a decision here before having a clear picture. The picture that is desciben in the blogpost i believe is not what happens in JVM.

And, to cite the update to this blogpost:

The micro-benchmark analyzed in this blog post is not the root cause of the performance regression observed when running the Scala compiler. It just shows one example where the use of default methods can have unexpected consequences on performance.

the 20% slowdown was due to different reason.

@lrytz
Copy link
Member

lrytz commented May 29, 2017

I totally agree that the discussion in the blog post cannot be related to what happens to the compiler performance. However, the decision to merge scala/scala#5429 was not based on the blog post, but on the benchmarks shown in the PR discussion: scala/scala#5429 (comment) and later comments

@DarkDimius
Copy link
Contributor

@lrytz, thanks for the link.

I think that we should NOT do this, as this can hide correctness bugs. We should make sure that our predicates on when to emmit forwarders for correctess is exercised.
When we'll have a stable release, we should reconsider. By that time jdk9 is likely to ship and a lot of fixes for performance of default methods will be in.

@smarter
Copy link
Member Author

smarter commented May 29, 2017

When we'll have a stable release, we should reconsider

Well, hopefully before we have a stable release because switching -Xmixin-force-forwarders on/off is not serialization-compatible (cf discussion in scala/scala#5429).

I still believe that the results of this blog-post are not true for jit-ed code and we should perform a more close study.

What kind of "close study" do you have in mind? If we run http://github.com/dotty-staging/compiler-benchmark and see that mixin forwarders improve cold performance without impacting hot performance, would that be enough?

I think that we should NOT do this, as this can hide correctness bugs. We should make sure that our predicates on when to emmit forwarders for correctess is exercised.

One easy way to work around this is to run the tests twice, with mixing forwarders on or off.

@DarkDimius
Copy link
Contributor

Also, our startup time won't be affected much.
Afaik the milestone release is going to use scala.collections.* compiled by scalac.
When we'll be preparing a bootstrapped stable release we should reconsider.

What kind of "close study" do you have in mind?

I think we should understand how it works, not try to guess it. @lrytz and scalac-team tried to do this in the linked PR, but studies of that kind are always hard to perform, due to debug versions of JVM behaving very differently performance-wise. The right discussion seems to be here: scala/scala-dev#243, we should instrument the optimized build of hotspot to see if our assumptions are correct.

I think that we should NOT do this, as this can hide correctness bugs. We should make sure that our predicates on when to emmit forwarders for correctess is exercised.

One easy way to work around this is to run the tests twice, with mixing forwarders on or off.

While I agree, I don't recall triggering bugs by emitting more forwarders rather then less.

smarter added a commit to smarter/scala that referenced this issue Feb 13, 2019
Just like what we did for concat in
scala#7696, we add a dummy parameter list
to avoid signature clashes between mixin forwarders, but this change
should be less controversial since it only affects a deprecated method.

Note that this isn't actually needed to compile with Dotty master yet
because we only emit mixin forwarders when required to, but we will soon
switch to emitting them all the time like scalac
does (scala/scala3#2563), therefore we need
to get the fix in first (and most importantly in time for 2.13).
smarter added a commit to smarter/scala that referenced this issue Feb 14, 2019
Just like what we did for concat in
scala#7696, we add a dummy parameter list
to avoid signature clashes between mixin forwarders, but this change
should be less controversial since it only affects a deprecated method.

Note that this isn't actually needed to compile with Dotty master yet
because we only emit mixin forwarders when required to, but we will soon
switch to emitting them all the time like scalac
does (scala/scala3#2563), therefore we need
to get the fix in first (and most importantly in time for 2.13).
smarter added a commit to smarter/scala that referenced this issue Feb 17, 2019
Just like what we did for concat in
scala#7696, we add a dummy parameter list
to avoid signature clashes between mixin forwarders, but this change
should be less controversial since it only affects a deprecated method.

Note that this isn't actually needed to compile with Dotty master yet
because we only emit mixin forwarders when required to, but we will soon
switch to emitting them all the time like scalac
does (scala/scala3#2563), therefore we need
to get the fix in first (and most importantly in time for 2.13).
smarter added a commit to dotty-staging/dotty that referenced this issue Mar 12, 2019
This is what Scala 2.12+ does for cold performance reasons (see scala#5928
for more details) and we should align ourselves with them when possible.

About two years ago in scala#2563, Dmitry objected that we may not need to do
that if we found another way to get performance back, or if newer JDKs
improved the performance of default method resolution. It doesn't look
like these things have happened so far (but there's some recent glimmer
of hope here: https://bugs.openjdk.java.net/browse/JDK-8036580).

Dmitry also said "I don't recall triggering bugs by emitting more
forwarders rather then less.", but in fact since then I've found one
case where the standard library failed to compile with extra forwarders
causing name clashes, requiring a non-binary-compatible change:
scala/scala@e3ef657.
As of scala#6079 this is no longer a problem since we now emit mixin
forwarders after erasure like scalac, but it still seems prudent to emit
as many forwarders as scalac to catch potential name clash issues.
smarter added a commit to dotty-staging/dotty that referenced this issue Mar 12, 2019
This is what Scala 2.12+ does for cold performance reasons (see scala#5928
for more details) and we should align ourselves with them when possible.

About two years ago in scala#2563, Dmitry objected that we may not need to do
that if we found another way to get performance back, or if newer JDKs
improved the performance of default method resolution. It doesn't look
like these things have happened so far (but there's some recent glimmer
of hope here: https://bugs.openjdk.java.net/browse/JDK-8036580).

Dmitry also said "I don't recall triggering bugs by emitting more
forwarders rather then less.", but in fact since then I've found one
case where the standard library failed to compile with extra forwarders
causing name clashes, requiring a non-binary-compatible change:
scala/scala@e3ef657.
As of scala#6079 this is no longer a problem since we now emit mixin
forwarders after erasure like scalac, but it still seems prudent to emit
as many forwarders as scalac to catch potential name clash issues.
smarter added a commit to dotty-staging/dotty that referenced this issue Mar 18, 2019
This is what Scala 2.12+ does for cold performance reasons (see scala#5928
for more details) and we should align ourselves with them when possible.

About two years ago in scala#2563, Dmitry objected that we may not need to do
that if we found another way to get performance back, or if newer JDKs
improved the performance of default method resolution. It doesn't look
like these things have happened so far (but there's some recent glimmer
of hope here: https://bugs.openjdk.java.net/browse/JDK-8036580).

Dmitry also said "I don't recall triggering bugs by emitting more
forwarders rather then less.", but in fact since then I've found one
case where the standard library failed to compile with extra forwarders
causing name clashes, requiring a non-binary-compatible change:
scala/scala@e3ef657.
As of scala#6079 this is no longer a problem since we now emit mixin
forwarders after erasure like scalac, but it still seems prudent to emit
as many forwarders as scalac to catch potential name clash issues.
smarter added a commit that referenced this issue Mar 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants