Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stable getters #1518

Open
eernstg opened this issue Mar 13, 2021 · 130 comments
Open

Stable getters #1518

eernstg opened this issue Mar 13, 2021 · 130 comments
Labels
feature Proposed language feature that solves one or more problems field-promotion Issues related to addressing the lack of field promotion

Comments

@eernstg
Copy link
Member

eernstg commented Mar 13, 2021

Stable Getters

Promotion of fields is probably the most prominent request we have in response to the release of null safety. One approach to improve support for promotion could be to ensure that a particular getter will return the same value if invoked multiple times, that is, the getter is stable.

Note that the underlying concept is immutability: A stable getter will implement an immutable property of the given object (if it's an instance getter, otherwise it's just an immutable property). This is a concept which is useful both because it enables promotion, but also for a number of other reasons related to software correctness and readability.

abstract class A {
  stable num get n;
}

class B1 implements A {
  final num n; // OK.
  B(this.n);
}

class B2 implements A {
  late final int n = aComplexExpression; // OK.
}

class B3 implements A {
  num get n => someExpression; // Error, unless `someExpression` is stable.
}

The modifier stable can be specified on any final instance variable or instance getter declaration.

When stable is present in a declaration where the syntax also admits final, final is implied and may be omitted.

A compile-time error occurs if an instance getter declaration directly or indirectly overrides another getter declaration, where each of them can be explicitly declared or implicitly induced by a variable declaration, when the overridden declaration is marked stable and the overriding declaration is not a stable declaration.

In other words, when a getter has been marked stable then it and every override must be stable. Note that the overridden declaration can be in a superclass, including mixins, or it can be in an implemented class.

  • A final non-local variable (late or non-late) is a stable declaration.
  • A getter with a body of the form => e where e is stable is a stable declaration.
  • A getter with a block body is stable if it contains exactly one return statement return e; if e is a stable expression that does not contain any reference to a local variable, and if the body cannot complete normally (that is, it can't also return null implicitly).

The modifier stable can be specified on a library getter, and on a getter marked static and declared in a class, mixin, or extension. A compile-time error occurs if said getter does not have a stable body. The modifier stable can also be specified on a variable that implicitly induces one of these kinds of getters.

A stable expression is defined as follows:

  • At a location with access to this, this is a stable expression.
  • If e is a stable expression with static type T and m is a stable getter or a method in the interface of T then e.m is stable.
  • A constant expression is stable.
  • A library variable (aka top-level variable) with the modifier stable is stable (with or without late).
  • A variable with the modifiers static and stable declared in a class, mixin, or extension is stable (with or without late).
  • A final local variable is stable.
  • A local variable which is never mutated is stable (we could improve on this).
  • An expression of type Never is stable.
  • There is a form of stable expression corresponding to each form of constant expression that contains an identifier that stands for a subexpression. For instance, if e1 and e2 are stable and e1 has static type num, int, double, or String, then e1 + e2 is stable; and if b, e1, and e2 are stable, then b ? e1 : e2 is stable.

Note that if a final local variable declared in a function f occurs in the body of a function literal or local function declared in f then it may have been created during any execution of the enclosing function body. So we can have two instances of () => x where x is a final local variable declared in the body of f, and they can return different values because the function objects were obtained by evaluating the same function literal expression during different runs of f. However, it is still true that multiple evaluations of x in the same function literal body or local function body will yield the same value every time it is evaluated during the execution of a specific function object. So even here it's OK to say that the local variable is stable, and, e.g., allow promotions like () => x is int ? x.isEven : false.

We can now safely promote any stable expression. Here is a basic example:

class D {
  A a; // `A` was declared in the initial example.
  D(this.a);
}

void main() {
  final d = complexExpressionOfTypeD;
  if (d.a.n is int) { // `d.a.n` promoted, can be auto-shadowed.
    d.a.n.isEven; // OK.
  }
}

As a design trade-off, we can choose to allow compilers to cache the value of any stable expression. The argument in favor of this approach is that it will allow code to be expressed using the straightforward expressions (like x.y) and have good performance (because the value is cached into an implicitly created local variable). The argument against doing it is that it may be confusing for developers that an expression like x.y isn't evaluated again and again when its value is used multiple times in the code, because it is implicitly turned into an occurrence of that implicitly created local variable.


Final Getters

This is a variant of the proposal above: It does not introduce a new keyword stable. It is focused on the same concept, but strives to make that concept the semantics of existing syntax. As a result, it is a breaking change, because it makes certain existing programs an error (when a final variable declaration is overridden by a getter that isn't stable). But it recognizes the claim which has been made several times about stable getters: "That's the way it should have worked from the beginning".

In this variant, a getter can be explicitly declared to be final (inspired by this comment):

abstract class A {
  final num get n;
}

class B1 implements A {
  final num n; // OK.
  B(this.n);
}

class B2 implements A {
  late final int n = aComplexExpression; // OK.
}

class B3 implements A {
  num get n => someExpression; // Error, unless `someExpression` is final.
}

The modifier final on a getter implies that it is enforced that multiple invocations of that getter on the same receiver will evaluate to the same object (it may also throw, but it won't return two different objects).

The getter of a variable declaration that includes final is a final getter. This is the breaking change, but the assumption is that it will not break much code, because such getters are rarely overridden by a getter that may return different values for different invocations with the same receiver.

In order to allow the old semantics to be expressed, we introduce the following new syntax: The getter of an instance variable declaration that includes final followed by var is not a final getter.

class B {
  final var int g; // With `var`, `get g` below is not an error.
  A(this.g);
}

int offset = 10;

class C extends B {
  get g => ++offset + super.g;
}

A compile-time error occurs if an instance getter declaration directly or indirectly overrides another getter declaration, where each of them can be explicitly declared or implicitly induced by a variable declaration, when the overridden declaration is a final getter, and the overriding declaration can not be a final getter.

In other words, when a getter is final then it and every override of it must be final. Note that the overridden declaration can be in a superclass, including mixins, or it can be in an implemented class.

  • A non-local final variable (late or non-late) without var implicitly induces a final getter.
  • (A non-local final variable with var does not induce a final getter.)
  • A getter with a body of the form => e where e is a final expression can be a final getter.
  • A getter with a block body can be final if it contains exactly one return statement return e; if e is a final expression that does not contain any reference to a local variable, and if the body cannot complete normally (that is, it can't also return null implicitly).

A final expression is defined as follows:

  • At a location with access to this, this is a final expression.
  • If e is a final expression with static type T and m is a final getter or a method in the interface of T then e.m is final.
  • A constant expression is final.
  • A final library variable (aka top-level variable) is final (with or without late).
  • A variable with the modifiers static and final, declared in a class, mixin, or extension is final (with or without late).
  • A final local variable is final.
  • A local variable which is never mutated is final (we could improve on this).
  • An expression of type Never is final.
  • There is a form of final expression corresponding to each form of constant expression that contains an identifier that stands for a subexpression. For instance, if e1 and e2 are final and e1 has static type num, int, double, or String, then e1 + e2 is final; and if b, e1, and e2 are final, then b ? e1 : e2 is final.

Note that if a final local variable declared in a function f occurs in the body of a function literal or local function declared in f then it may have been created during any execution of the enclosing function body. So we can have two instances of () => x where x is a final local variable declared in the body of f, and they can return different values because the function objects were obtained by evaluating the same function literal expression during different runs of f. However, it is still true that multiple evaluations of x in the same function literal body or local function body will yield the same value every time it is evaluated during the execution of a specific function object. So even here it's OK to say that the local variable is final, and, e.g., allow promotions like () => x is int ? x.isEven : false.

We can now promote any final expression. Here is a basic example:

class D {
  A a; // `A` was declared in the initial example.
  D(this.a);
}

void main() {
  final d = complexExpressionOfTypeD;
  if (d.a.n is int) { // `d.a.n` promoted, can be auto-shadowed.
    d.a.n.isEven; // OK.
  }
}

As a design trade-off, we can choose to allow compilers to cache the value of any final expression. The argument in favor of this approach is that it will allow code to be expressed using the straightforward expressions (like x.y) and have good performance (because the value is cached into an implicitly created local variable). The argument against doing it is that it may be confusing for developers that an expression like x.y isn't evaluated again and again when its value is used multiple times in the code, because it is implicitly turned into an occurrence of that implicitly created local variable.

Note that this proposal makes it a breaking change to change a top-level or static variable from a final variable to a non-final getter. However, this is justified because it actually changes the semantics of the declarations. For instance, a final top-level variable can be cached and/or promoted, but the non-final getter cannot be cached nor promoted, and the reasoning that developers need to apply when they are writing code that uses it will also need to be reconsidered if we change the variable to a non-final getter. On the other hand, it is not a breaking change to change a final top-level variable to a final getter, i.e., we do not get boxed into the implementation choice of making it a variable.


[Edit Mar 14 2021: Fleshed out the definition of stable expressions. Mar 21: Added more stable expression forms. Apr 7: Require stable modifier on library/class variables for them to be stable. Edit Jun 28 2021: Wrote the 'Final Getters' variant of this proposal. Edit Sep 10 2023: Fix a mistake reported by @leafpetersen: The body of a stable/final getter cannot return an expression that contains a local variable.]

@eernstg eernstg added feature Proposed language feature that solves one or more problems field-promotion Issues related to addressing the lack of field promotion labels Mar 13, 2021
@lrhn
Copy link
Member

lrhn commented Mar 13, 2021

I don't think this actually solves most of the problems people will have with promoting instance variables.
You need promotion for mutable variables too, and if you need to whether a value is null, then perhaps even likely that the variable is also mutable

Also, if you know the variable is stable, by declaration or just by promise, then it's always safe to cache it in a local variable, without worrying about concurrent modifications. In that case, any feature which introduces a local variable copy of the value is just as safe, and it can work with mutable variables too.

(Also, can I haz stable getters? 😁 )

@eernstg
Copy link
Member Author

eernstg commented Mar 13, 2021

😁

It's a trade-off: Shadowing variables will allow the developer to cache a non-local getter result relatively easily and safely. That's possible even in the case where the non-local getter is not stable (it could still be correct according to the assumptions of the software to cache its value and keep using it for a while), and it is a documented choice, in the sense that the shadow keyword (or something like that) would serve to indicate what's going on.

But declaring that a getter is stable establishes that fact as a design choice at the declaration. This gives us a semantic guarantee which can be conceptualized (it's like "this property of the object is immutable"), and that influences all the code which is written to use that getter.

On top of this conceptual benefit, we get the ability to (auto)shadow the getter and promote it.

This makes stable getters a rather principled device that also yields practical benefits, whereas shadowing is motivated by practical benefits, with foundations that are more shaky (we are reusing a cached value for a while, and that may or may not be correct in the given context).

@eernstg
Copy link
Member Author

eernstg commented Mar 14, 2021

For a getter to be declared stable, it must return the same result every time, and in particular identical(x, x) must evaluate to true. But it's stronger than that, because that would hold for a mutable local variable x as well.

I think it's likely to be a massively breaking change to use stable as the default.

However, when an immutable variable can be declared with stable rather than final then the syntactic cost is going to be quite low (stable final int i; is still allowed, but stable int i; will probably be much more common).

We'd basically migrate by performing a global search-and-replace from final to stable on variables, and see what breaks. 😁

@eernstg
Copy link
Member Author

eernstg commented Mar 14, 2021

(1) Implement stable getters. (2) Hack the result to artificially add stable to every final instance variable. (3) Count errors on some body of software, e.g., github. ;-)

I think no language has the notion of stable getters. The crucial property is that it returns the same result every time, but this is usually achieved by making it the foundation of the language (pure functional programming), or by preventing overrides (e.g., Java and C++ variables do not have a notion of overriding, getters and setters must be written manually). The interesting bit here is that a stable getter can be a lot more expressive than just a fixed memory cell read.

@eernstg
Copy link
Member Author

eernstg commented Mar 15, 2021

insert random bugs

We will never run out of PhD projects that someone should do. 😸

@esDotDev
Copy link

esDotDev commented Mar 17, 2021

The downside of this is you need to write stable everywhere, just to get back to what is basically default (and desired) behavior in most other typed languages. It also closes off the field for a future developer to sub-class which is always a bad thing since you can never fully predict future use-cases.

From a high level, it still seems most correct to me, to make the edge case the edge case.

  1. A developer who wants to do this rare thing of 'intercepting' a field, must declare his intent there or compiler complains:
    void get int i => sometimesReturnNull() // Error, if you want to replace a field with a method, use intercept
    void intercept get int i => sometimesReturnNull() // Compiles ok
  2. With that keyword in place, compiler can then walk the inheritence chain of any field, and see if it has been intercepted. (still don't really understand why compiler can't do this now...)
  3. Compiler can then give a reasonable message in the very rare case a field faills promotion "We can not promote this field, as it has been intercepted in Class Foo".

@Levi-Lesches
Copy link

Levi-Lesches commented Mar 17, 2021

Error, if you want to replace a field with a method, use intercept

That's kinda tricky. Consider the following:

int? get maybeNull => math.Random().nextBool() ? 1 : null;

There's no explicit method call, but this is exactly the situation Dart is preventing by refusing to promote fields.

On a broader note, I keep getting confused between this issue and #1514. Are we trying to provide a solution for promoting fields, or defining a new (maybe default) behavior for getters? If the latter is just a means to an end, I'd say that IMO, it's not worth the trouble of just adding a local variable to cache the value for promotion, and being a little careful when using this.

@esDotDev
Copy link

esDotDev commented Mar 17, 2021

here's no explicit method call, but this is exactly the situation Dart is preventing by refusing to promote fields.

I don't see the issue if we just said that getters and methods can not be promoted. The glaring issue here is a developer declaring a field, and dart telling them it can't check their field for null. It just makes no sense from an authoring perspective.

Conversely, it makes sense, on its face, why you could not trust a getter to provide the same result each time.

I know under the hood, dart doesn't make much of a distinction here, but developers use these semantics with clear intent.

Unless maybeNull is overriding a base class? Then it should be forced to be this, so everything is declared in the open:

@override @intercept
int? get maybeNull => math.Random().nextBool() ? 1 : null;

@eernstg
Copy link
Member Author

eernstg commented Mar 17, 2021

@Levi-Lesches wrote:

this issue and #1514. Are we trying to provide a solution for promoting fields,
or defining a new (maybe default) behavior for getters?

The motivation for stable getters (this issue) is to allow developers to express a relevant software property in a way that is handled soundly by the static analysis: "with a given receiver, this getter will return the same value each time". As a consequence, it allows for promotion of any stable expression (including promotion of fields), and it allows for caching. This kind of caching can be handled by the compiler, it is guaranteed to be correct.

#1514 and shadow variables is about developer-controlled caching: In that case there is no guarantee that the caching is correct, it might as well be a bug to ignore modifications of the underlying variable. So that kind of caching must be requested by the developer. But if we decide that it is appropriate to use a shadow variable then the mechanism automatically makes the caching safe, because every write to the shadowing variable (if allowed) is automatically followed by a write to the shadowed variable.

@eernstg
Copy link
Member Author

eernstg commented Mar 17, 2021

@esDotDev wrote:

I don't see the issue if we just said that getters and methods can not be promoted

The core issue here is the semantics of instance variables. You can find many, many examples where software people express the rule that instance variables should be private. For instance, https://stackoverflow.com/a/14399978/5603708 says (in a question which is tagged OOP and C++, but referring to Java as well):

Public variables are usually discouraged, and the better form is to make
all variables private and access them with getters and setters

The rationale is typically that it creates too many constraints on the implementation of a given interface if it has been cast in concrete that a specific member is a storage location in the object. So we should write methods (getters and setters) and let clients call them, they shouldn't access the instance variable directly.

Dart was designed to eliminate the need to write all those getters and setters by making them mandatory. It's often possible for a compiler to eliminate the getter/setter invocations, but you still have the encapsulation because you won't have to edit your declarations and all the call sites (that may occur in code that you don't control) in order to introduce getters and setters, you just have them by default.

So if we can't promote getters then we can't promote instance variables!

In contrast, this proposal is all about enabling promotion of getters (hence instance variables), without reducing their ability to support overriding and arbitrary computations.

I believe the concept of a stable getter is useful from a software engineering perspective: you're declaring immutability, not a specific implementation/representation.

@Levi-Lesches
Copy link

@eernstg So basically we're looking for a shortcut for the following:

class Temp {
  int? get maybeNull => 3;
  
  int roundNumber() {
    final int? value = maybeNull;  // <-- stable getters would make this unnecessary
    if (value == null) return 0;
    else return value.round();  // <-- can be promoted now
  }
}

That's why I made the comparison to shadowing -- it seems that just adding this one line fixes both problems.

@eernstg
Copy link
Member Author

eernstg commented Mar 19, 2021

@Levi-Lesches wrote:

we're looking for a shortcut for the following:

class Temp {
  int? get maybeNull => 3;
  
  int roundNumber() {
    final int? value = maybeNull;  // <-- stable getters would make this unnecessary
    if (value == null) return 0;
    else return value.round();  // <-- can be promoted now
  }
}

No, stable getters are actually quite different.

stable int get x; is an abstract declaration for which it is guaranteed that, with a given receiver, it will return the same value each time it is invoked. This property is relevant conceptually (it is an immutability property), it is relevant to software correctness (you can use this information when reasoning about the software), it is pragmatically convenient (because it enables promotion), and it is relevant to performance (because it will allow compilers to cache the value implicitly at any call site).

Other forms like stable int x; have the same properties as the abstract declaration stable int get x;, and it also specifies an implementation (in this case: a portion of storage in the object layout). Other implementations are possible, e.g., it could return a constant, or it could return any stable expression.

When a getter is marked stable, however, it must satisfy the constraint: If we call it twice on the same receiver it must return the same value (and the compiler must be able to verify that fact statically).

I don't see any indication in the example above that maybeNull should have any of the immutability properties. If maybeNull doesn't satisfy the constraint that a stable getter specifies then, obviously, we can't declare it as a stable getter.

On the other hand, the example maps directly to a shadowing variable:

class Temp {
  int? get maybeNull => 3;
  int roundNumber() {
    shadow maybeNull;
    return maybeNull == null ? 0 : maybeNull.round();
  }
}

@eernstg
Copy link
Member Author

eernstg commented Mar 19, 2021

@tatumizer wrote:

I think your definition of stable getters is trying to achieve too much. Much
weaker condition identical(x, x) might do.

That is true, but the motivation for stable getters is the conceptual content: I would like to be able to declare explicitly that a particular getter has the stated immutability property, and then I'd like to allow for all the expressive power we can have while preserving that property.

So I'm not going for the best possible fit for instance variable promotion, I'm going for a useful language concept. With that, I'm noting that stable getters are highly relevant to instance variable promotion because (1) any stable expression can be promoted (and that's a lot more than just a single getter invocation), and (2) any stable expression can be cached (implicitly, by the compiler). We just need to write stable once, and then there might be thousands of call sites where caching turns out to be a useful optimization which will then occur.

Another thing is that identical(x, x) has very few implications (it is a very weak property). @lrhn already mentioned function invocations involving getters:

class C {
  int? x;
  C(this.x);

  void Function(int) get use {
    x = null;
    return (int i) {};
  }

  void foo() {
    if (x != null) {
      use(x!); // `x` _is_ actually null, so we can't omit `!`.
    }
  }
}

void main() => C(2).foo();

@esDotDev
Copy link

esDotDev commented Mar 19, 2021

After reading the proposal more closely it is quite nice, and does knock off the majority of problem cases as they relate to flutter. All widgets params are final, for example, so right off the hop you are probably addressing 80% of the headaches with this.

I see how this is adding to the language with a useful feature, whereas shadow really feels like a workaround with no great intrinsic value.

But, if all stable fields must be final, then something like this wouldn't work?

int? counter;

void increment(){
  if(counter == null) return;
  counter++;
}

Also, if shadow allows the implementation of auto-shadow-by-default, with the explicit shadow meant for edge cases where local caching is needd, then shadow does, in some roundabout way, make the language much easier to use, both in the main use case, but also the unstable getter edge case, and you'd virtually never use it or see it. I'm probably mis-understanding this application of it tho...

@esDotDev
Copy link

esDotDev commented Mar 19, 2021

@eernstg: I think your definition of stable getters is trying to achieve too much. Much weaker condition identical(x, x) might do.
Consider 2 examples:

if (x != null) {
   use(x);
}

The compiler complains here. The user is stunned! The explanation involves the notion of getters: you see, there can be a getter for x, and on each invocation, it can return a different value. Totally unintuitive explanation.
Compare it with a Situation 2:

if (x != null) {
   someOtherMethod();
   use(x);
}

Here, the compiler also complains. But the explanation is much simpler: someOtherMethod may change the value of x. The refusal of the compiler to promote x is not that surprising - probably, the same would happen in any language having no concept of getters.

This is the perfect description of the confusion devs are encountering. No one will call the compiler "stupid" in the latter, I'm seeing constant complaining about the former because it's just non-obvious and no one gets it. I receive a whole lot of these when I try and explain how it works: 😕

@eernstg
Copy link
Member Author

eernstg commented Mar 19, 2021

@esDotDev wrote:

something like this wouldn't work?

int? counter;

void increment(){
  if (counter == null) return;
  counter++;
}

True, that wouldn't work: We can't trust counter to remain non-null if we evaluate it twice. If we are willing to take the responsibility (that is, we promise that it isn't a bug to cache counter) then we can use a shadow (with the #1210 abbreviation):

int? counter;

void increment(){
  if (shadow: counter != null) counter++;
}

no one gets it

In some cases a compiler/analyzer is just forced to stop tracking down partial solutions to problems that are undecidable in general, so in some cases the compiler/analyzer will deny a truth that developers may find obvious. However, in cases where we actually have unsoundness it's just so much more important that the compiler/analyzer can detect that a given assumption can be false. This will be true in some of those situations where the unsoundness isn't obvious...

@esDotDev
Copy link

esDotDev commented Mar 19, 2021

Cool, just wanted to make sure I was understanding properly. I don't think it would be a major issue, at least it's a lot better than what we have now :)

In some cases a compiler/analyzer is just forced to stop tracking down partial solutions to problems

Totally understand, its just this specific case is so non-obvious will continue to jump out at developers until it is addressed.

@eernstg
Copy link
Member Author

eernstg commented Mar 19, 2021

@tatumizer wrote:

the word stable sounds weaker than final to me.

Agreed, 'final' is rather definitive. However, 'final' has had a specific meaning in Dart for many years, and we still need that semantics (say, in a declaration of a top-level variable which doesn't admit assignments). So we do need a new syntax, and stable seems reasonable to me.

Question: is this getter (x) stable?

final late int _x1;
final late int _x2;
final late bool cond;
stable get x => cond? _x1 : _x2;

The final variables must be library or instance variables, because otherwise we couldn't have a getter at the same level.

First, if they are instance members then we can't make x stable, because _x1 and _x2 could be overridden by any getter.

If they are top-level declarations then we could in principle allow x to be marked stable. I haven't included rules about a conditional expression being stable if all three subexpressions are stable, but it would be sound to do so. As usual, we just need to decide on how many decidable subproblems of an undecidable problem we can and will support.

@Levi-Lesches
Copy link

Levi-Lesches commented Mar 19, 2021

I was also about to bring up late -- would a late final variable carry the same semantics? I tried it in Dartpad, and you can't assign a custom getter to a late variable, and if you combine it with final, then you can only set the value once.

class Temp {
  late final int? tbd;
  
  /// The compiler knows by this point whether the value has been set or is still null. 
  /// Since it's final, the value won't change -- whatever it is now it will be through the rest of the function. 
  int roundValue() {
    if (tbd == null) return 0;
    else return tbd.round();
  }
}

It seems like we're looking for a final field for which you can't provide your own (non-stable) getter. The problem is that final fields do provide a getter. late fixes this by not allowing you to do have a getter, which ensures that the value cannot change by itself, and making it late final ensures that the value is never changed externally.

It doesn't quite make sense to me to allow users to provide their own stable getters -- it's hard to statically check that the value won't change (which I believe is the same curse that plagues const). What we're really looking for is a way to let users define a value once, and never let it change, which is why late final should fit perfectly.

@eernstg
Copy link
Member Author

eernstg commented Mar 19, 2021

@Levi-Lesches wrote:

would a late final variable carry the same semantics?

Yes. A final instance variable has an implicitly induced getter which can be stable (so it's OK to override a stable getter with a final instance variable and it is OK to mark a final instance variable as stable, in which case final is implied an may be omitted). This is true for a late variable as well as a non-late one, both when the variable has an initializer and when it doesn't have an initializer.

(Various other rules differ in those cases, but all those kinds of variables do satisfy the requirement that their getter will return the same value each time, for the same receiver).

It doesn't quite make sense to me to allow users to provide their own stable getters

It's not so hard to specify a range of different implementations which are allowed: The notion of a stable expression already allows a lot of different implementations, and we could easily add a few extra cases (for instance, if b, e1 and e2 are stable expressions then b ? e1 : e2 is a stable expression).

Also, it would surely be inconvenient if we couldn't use a constant, e.g., stable get s => 'Hello, world!';.

The compiler knows by this point whether the value has been set or is still null.

This comment is misleading: The value of a late variable with no initializer and a nullable type isn't initially null, the variable actually doesn't have a value. So the following throws, rather than printing null:

var b = false;

void main() {
  late final int? i;
  if (b) i = 10;
  print(i); // Throws!
}

@Levi-Lesches
Copy link

Right, I mentioned late final instead of just final since late would preclude you from adding a getter, but if we're willing to look into stable expressions, then we don't need late. You've convinced me that stable expressions are useful, but we'd want to be careful that it doesn't fall into the same trap as const, where some expressions should be const, but the compiler can't prove it (see dart-lang/sdk#3059).

Out of curiousity, is there a natural way to use the final keyword instead of stable? final is a concept that Dart users already understand, whereas it took me and some others a few reads of this issue to understand it. Like you noted, a final field is already stable, so for getters, how about final get? Currently, it throws an error, so it wouldn't be a breaking change. On the other hand, as I write this I realize that subclasses an override final into a non-stable implementation, so maybe we should have stable.

class Temp {
  final int var1 = 1;  // stable (except for subclasses)
  int get var2 => 2;  // not stable, but should be
  final int get var3 => 3;  // this feels natural IMO
  stable int var4 => 4;  // will always be stable, even if overriden
}

@eernstg
Copy link
Member Author

eernstg commented Mar 20, 2021

late would preclude you from adding a getter,

As an instance variable, and in the same class or mixin? That actually depends on the form: A non-local late variable with an initializing expression induces a getter (and no setter), but when there is no initializing expression it induces both a getter and a setter. So it's only the latter that will preclude the declaration of a setter in the same class. However, they both preclude the declaration of a getter, so I'm not quite sure which getter you're referring to.

class C {
  late final int i = 14;
  set i(_) {} // OK.
  late final int j;
  set j(_) {} // Error.
}

(By the way, that error wasn't implemented consistently, cf. dart-lang/sdk#45398, but it is an error).

be careful that it doesn't fall into the same trap as const

stable is a lot easier than const: If you want to use an expression e in a location where a constant expression is required (as a subexpression of another constant, or as a default value of a parameter, etc) then you can't work around it when e isn't constant, you've simply lost.

If there is an expression e that you wish to use as the body of a stable getter, but the compiler claims that e isn't stable, then you can simply evaluate it once and for all and use the cached value. The same trick will work for local variables:

class C {
  // Assume that this won't work:
  stable get g => e; // Error "`e` is not stable".

  // Then you can do this:
  stable late final g = e; // OK, will evaluate `e` once, when first used.
}

You don't have to include stable here, but the point is that you can include stable as shown, and that will force every overriding declaration of g to be stable as well.

is there a natural way to use the final keyword instead of stable?

We'd use final when we do not wish to enforce that the getter is stable, that is, when we by design expect values to differ when we invoke the getter several times:

abstract class Subject {
  double get age; // Not stable!
}

class Person implements Subject {
  double get age => DateTime.now().difference(birthTime).inMilliseconds / 1000;
  // ...
}

class Deity implements Subject {
  final age = double.infinity; // But we can use a final variable here.
}

a final field is already stable

It's the other way around: A stable instance variable must be final, because we certainly can't expect that the value is the same in multiple evaluations if the variable is mutable. But stable is stronger than final, because it constrains all overrides to be stable as well.

I suspect that a large proportion of all public final instance variables could be stable, and program correctness would benefit from doing that. We could basically do a search-and-replace on instance variables and change final to stable, and then check what breaks. ;-)

@Levi-Lesches
Copy link

If there is an expression e that you wish to use as the body of a stable getter, but the compiler claims that e isn't stable, then you can simply evaluate it once and for all and use the cached value.

That's a really good point. So the problems of const should never apply here, which is nice.

Back to my point on late final from earlier, I played around on Dartpad and got this:

class Temp {
  late int value = Random().nextInt(5);
}

class Temp2 extends Temp {
  @override
  int get value => Random().nextInt(5);
}

void main() {
  Temp temp = Temp();
  print(temp.value);  // 4 (guaranteed to be statistically random -- https://xkcd.com/221/)
  print(temp.value);  // 4
  print(temp.value);  // 4
  Temp2 temp2 = Temp2();
  print(temp2.value);  // 1
  print(temp2.value);  // 2
  print(temp2.value);  // 3
}

So late does give us equivalent behavior to stable -- the late ensures that the random variable is computed once (not accounting for mutabitility, see below). The only problem is that subclasses can extend it and make it non-late. Would stable simply enforce this behavior for subclasses? In that case, maybe we can modify the semantics of late to do this by default? IMO, the above behavior is very unintuitive and shouldn't be allowed.

A stable instance variable must be final, because we certainly can't expect that the value is the same in multiple evaluations if the variable is mutable.

But the problem with null-safe promotion of fields is often with mutable fields, it's just that the value is guaranteed not to change in the getter itself. If stable fields won't define a setter, how can we work with such a case?

class Person {
  /// Can't fix this with `late`, since it can be set to null later.
  /// But on the other hand, this getter never changes the value.
  String? favoriteColor;
  
  String introduction() {
    if (favoriteColor == null) {
      return "I don't have a favorite color yet/anymore";
    } else {
      return favoriteColor.toUpperCase();  // Can't promote!
    }
  }
}

void main() {
  final Person person = Person();
  print(person.introduction());  // I don't have a favorite color yet
  person.favoriteColor = "blue";
  print(person.introduction());  // BLUE
  person.favoriteColor = null;
  print(person.introduction());  // I don't have a favorite color anymore
}

It sounds like the definition of stable as "never changing once it's set" is more equivalent to late final, whereas a definition of "simply calling the getter twice in a row does not change the value" would help the compiler promote where necessary. On the other hand, one can argue this is simply a case where if-vars (#1201) should be used and has nothing to do with stable getters. Thoughts?

@eernstg
Copy link
Member Author

eernstg commented Mar 21, 2021

So late does give us equivalent behavior to stable -- the late
ensures that the random variable is computed once

It is true that a late variable with an initializing expression will get the value computed by evaluating that expression when it is first read, and it will keep that value during its whole life span.

However, that's not the same thing as a stable declaration (variable or getter): If a getter or variable is declared stable then it is guaranteed that this declaration as well as every overriding declaration will have the stability property (that is, if it is evaluated multiple times then we get the same value each time).

the problem with null-safe promotion of fields is often with mutable fields

That's true. Stable getters are not relevant in that case, but shadow variables could be used (#1514).

But anyone who wishes to use elements of a functional programming style would probably have many instance variables that are final, and they should probably be stable, because it's part of the intended design that the value doesn't change. So even though you could override such a final variable by a getter that returns different values from time to time (if it is just final, not stable, that is), it would probably be a bug to do so.

this getter never changes the value

It is of course possible for a getter to change the value that it returns (var _counter = 0; and int get counter => _counter++;), but I don't think that's a very common case. The really important issue is that any method which is executed between two evaluations of a given getter could run an unlimited amount of code that is not known at compile-time, and that code could modify the value returned by said getter.

@Levi-Lesches
Copy link

So would you say, for my favoriteColor example, that shadowing/if-vars is the proper solution? If so, I agree.

To that note, can you provide a real-world example for stable getters? My intuition is that it's a replacement for const values that need to be evaluated at runtime -- is that correct? As you noted, it's stricter than final, so I'm having trouble thinking of it in terms of something I would do today.

@eernstg
Copy link
Member Author

eernstg commented Mar 22, 2021

for my favoriteColor example ... shadowing/if-vars is the proper solution?

Yes, for that example I'd like to have shadow and #1210, such that we could do this:

class Person {
  String? favoriteColor; // Can be null, can be mutated.
  
  String introduction() {
    if (shadow: favoriteColor == null) {
      return "I don't have a favorite color yet/anymore";
    } else {
      return favoriteColor.toUpperCase();  // Promoted.
    }
  }
}

For real-world usage, I just checked that flutter/flutter/lib contains 5912 declarations matching ^\ \ final\b, and a manual inspection of about one hundred of them seems to confirm that a really large proportion of those are instance variables.

I expect that almost all of those can be changed from final to stable (or perhaps that stable modifier can be added to an abstract getter in some supertype, which would be even better).

For every variable/getter which is changed to stable we'll have the benefit that it can be promoted in client code. So if x is a stable instance variable declared by C then x can be promoted for code in C and its subtypes, and e.x can be promoted anywhere, as long as e is a stable expression of type C or a subtype thereof.

That seems pretty real-worldish to me.

@Levi-Lesches
Copy link

Now that you mention Flutter, would this be a good example of when to use stable?

class MyWidget extends StatelessWidget {
  final Widget child;  // --> stable Widget child
  final String? label;  // --> stable String? label
  const MyWidget(this.child, this.label);
  // build() here
}

As an addition, is it fair to add that any class marked with @immutable (such as StatelessWidget) should have all its fields as stable?

@lrhn
Copy link
Member

lrhn commented Oct 12, 2022

For noSuchMethod, we could say that an implicit nSM-forwarder for a stable getter will be implemented as caching:

final TheType theName = noSuchMethod(Invocation.getter(#theName));

which will cache the first successful result returned (after successfully downcasting to TheType). This enforces stability.

I guess mockito codegen can do something similar.

@lrhn
Copy link
Member

lrhn commented Oct 12, 2022

About stable expressions being an extension of potentially constant expressions ... maybe we could extend potentially constant expressions to also allow accessing stable getters of potentially constant values.

@eernstg
Copy link
Member Author

eernstg commented Oct 12, 2022

an implicit nSM-forwarder for a stable getter will be implemented as caching

Yep, that's what I just said earlier today during a discussion about final getters. It would need to be late, but with that it should just work.

@leafpetersen
Copy link
Member

A note in passing. I believe the following is a program which defines a getter x which passes the criteria for stability above, but which returns different values on subsequent invocations:

class A {
  late final int _x;

  // A getter with a block body containing exactly one return statement return
  // e; which occurs at the top level of the function body and where e is stable
  // is a stable declaration.
  int get x {
    late int cache;
    try {
      cache = _x;
    } catch (e) {
      _x = 99;
      cache = 0;
    }
    final int r = cache;
    // A final local variable is stable
    return r;
  }
  }
void main() {
  A a = A();
  print(a.x);
  print(a.x);
}

I think that at a minimum the definition of stable expression needs to be adjusted such that it is closed under substitution of values for local variables (both final and mutable).

@Levi-Lesches
Copy link

Well in that case, wouldn't this technically count?

class A {
  int get b {
    // Final local variable is stable
    final int x = Random().nextBool() ? 1 : 2;
    // Returning a stable expression is stable
    return x;
  }
}

I thought the point of marking something as stable is that it gains late final semantics -- the value is cached by Dart, making the value unchanging regardless of how its getter is implemented.

@leafpetersen
Copy link
Member

leafpetersen commented Sep 7, 2023

I thought the point of marking something as stable is that it gains late final semantics -- the value is cached by Dart, making the value unchanging regardless of how its getter is implemented.

This is not what is proposed above. The proposal above is attempting to define a sub-language which is statically guaranteed to always produce the same result, so no caching is required.

@lrhn
Copy link
Member

lrhn commented Sep 7, 2023

I think that at a minimum the definition of stable expression needs to be adjusted ...

Agree. For a getter to be stable, its returned value needs to be stable globally.

A final local variable is stable within its scope. It's not the same variable the next time the same code gets executed.

Which means we probably need a hierarchy of stability:

  • globally/statically stable. Every execution of this code gives the same value.
  • instance stable. Every execution of this code with the same this object gives the same value.
  • locally stable. Every execution of this code within the same scope gives the same value.

Or more generally: contextually stable, aka. Something with a context dependency gives the same result when executed within the "same context".

A getter is not instance stable just because the expression it returns is locally stable. The expression had to be instance stable too.

@eernstg
Copy link
Member Author

eernstg commented Sep 10, 2023

@leafpetersen wrote:

I believe the following is a program which defines a getter x which passes the criteria for stability above, but which returns different values on subsequent invocations:

Good catch, that's indeed true!

I fixed that loophole in the implementation of the lint which was used to investigate the breakage associated with the 'final getters' variant of this mechanism, but apparently I forgot to fix it here. Here is a shorter way to use the same loophole:

class A {
  final int get x {
    final now = DateTime.now().millisecondsSinceEpoch;
    return now;
  }
}

void main() {
  A a = A();
  print(a.x);
  /* ... do something to waste at least one millisecond */
  print(a.x);
}

We need to distinguish between expressions that are stable based on having the same object as this (or, for a top-level or static getter, just being stable) and expressions that are only stable during the execution of a given function body. We could say that the former expressions are just 'stable' and the latter are 'body-stable', because the stability is only guaranteed for one specific execution of one specific function body.

We have to require that return e; in a stable/final getter must have a stable e, but in order to promote (or possibly cache) an expression it is enough that it is body-stable. I adjusted the proposal to have this extra constraint.

@Levi-Lesches wrote:

wouldn't this technically count?

True, that's enough, too.

@lrhn wrote:

we probably need a hierarchy of stability: ... globally/instance/locally

Indeed. I think it's enough to separate out 'locally stable' which is what I called 'body stable' in the update of the proposal where I fixed this loophole.

Consider a member m which isn't necessarily stable/final itself, it just uses stability of some other declarations to get some promotions. For instance, m could be a method.

In order to be able to promote a stable/final expression in the body of m, it just needs to be body stable.

The other two concepts don't seem to be needed separately: A getter needs to return a stable expression in order to be stable (doesn't matter whether it is instance stable or globally stable). If e is a body-stable expression then e.g is body-stable as well if g is a stable getter. So I agree that the concepts of globally vs. instance stable make sense, but I don't think we have to talk about it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Proposed language feature that solves one or more problems field-promotion Issues related to addressing the lack of field promotion
Projects
None yet
Development

No branches or pull requests

10 participants