Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support itblock for Prism::Translation::Parser #3481

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

koic
Copy link
Contributor

@koic koic commented Feb 26, 2025

Summary

itblock node is added to support the it block parameter syntax introduced in Ruby 3.4.

$ ruby -Ilib -rprism -rprism/translation/parser34 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { it }"; \
                                                      p Prism::Translation::Parser34.new.tokenize(buffer)[0]'
s(:itblock,
  s(:send, nil, :proc), :it,
  s(:lvar, :it))

This node design is similar to the numblock node, which was introduced for the numbered parameter syntax in Ruby 2.7.

$ ruby -Ilib -rprism -rprism/translation/parser34 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { _1 }"; \
                                                      p Prism::Translation::Parser34.new.tokenize(buffer)[0]'
s(:numblock,
  s(:send, nil, :proc), 1,
  s(:lvar, :_1))

The difference is that while numbered parameters can have multiple parameters, the it block parameter syntax allows only a single parameter.

In Ruby 3.3, the conventional node prior to the it block parameter syntax is returned.

$ ruby -Ilib -rprism -rprism/translation/parser33 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { it }"; \
                                                      p Prism::Translation::Parser33.new.tokenize(buffer)[0]'
s(:block,
  s(:send, nil, :proc),
  s(:args),
  s(:send, nil, :it))

Development Note

The Parser gem does not yet support the it block parameter syntax. This is the first case where Prism's node design precedes that of the Parser gem. When implementing whitequark/parser#962, this node design will need to be taken into consideration.

cc @whitequark @iliabylich

@koic koic force-pushed the support_itblock_for_translation_parser branch 2 times, most recently from ff8045e to 7e0e098 Compare February 26, 2025 05:01
@@ -0,0 +1,7 @@
-> do it + 42 end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is actually not really tested, only Parser33 is run in the test suite. I thought about it a bit before, and probably a separate directory for new syntax would make sense (or something along those lines).

You probably need to assert specifically against the ast anyways, as it only compares to the parser gem, and there is nothing to compare against in this case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now, I've added a test to check the :itblock node type, which is currently provided only by Prism::Translation::Parser. Since it is essentially an extension of the Parser gem, I've added it to parser_test.rb.

Additionally, I've implemented a mechanism to compare node compatibility with the Parser::Ruby34, but since the Parser gem doesn't yet support the it block parameter, it is currently skipped.

@koic koic force-pushed the support_itblock_for_translation_parser branch 4 times, most recently from 4eff4d4 to 15475ac Compare February 26, 2025 09:56
## Summary

`itblock` node is added to support the `it` block parameter syntax introduced in Ruby 3.4.

```console
$ ruby -Ilib -rprism -rprism/translation/parser34 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { it }"; \
                                                      p Prism::Translation::Parser34.new.tokenize(buffer)[0]'
s(:itblock,
  s(:send, nil, :proc), :it,
  s(:lvar, :it))
```

This node design is similar to the `numblock` node, which was introduced for the numbered parameter syntax in Ruby 2.7.

```
$ ruby -Ilib -rprism -rprism/translation/parser34 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { _1 }"; \
                                                      p Prism::Translation::Parser34.new.tokenize(buffer)[0]'
s(:numblock,
  s(:send, nil, :proc), 1,
  s(:lvar, :_1))
```

The difference is that while numbered parameters can have multiple parameters, the `it` block parameter syntax allows only a single parameter.

In Ruby 3.3, the conventional node prior to the `it` block parameter syntax is returned.

```console
$ ruby -Ilib -rprism -rprism/translation/parser33 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { it }"; \
                                                      p Prism::Translation::Parser33.new.tokenize(buffer)[0]'
s(:block,
  s(:send, nil, :proc),
  s(:args),
  s(:send, nil, :it))
```

## Development Note

The Parser gem does not yet support the `it` block parameter syntax. This is the first case where Prism's node design precedes that of the Parser gem.
When implementing whitequark/parser#962, this node design will need to be taken into consideration.
@koic koic force-pushed the support_itblock_for_translation_parser branch from 15475ac to 722a9c7 Compare February 26, 2025 09:59
@@ -242,5 +261,11 @@ def assert_equal_comments(expected_comments, actual_comments)
"actual: #{actual_comments.inspect}"
}
end

def assert_prism_only_node(actual_ast, fixture_path)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's doubtful if parser will ever support 3.4 and this seems highly specific. It also only works because this is valid syntax to the parser gem, if some new syntax gets added you can't test this via this mechanism.

I think I would just add new specific tests for this syntax. If the parser gem does end up supporting it, then these test cases can just be imported. Something like this:

# Pseudo-code like
def test_it_syntax
  parser = Prism:Parser::Translator::Ruby34.new
  expected = s(:itblock, s(:begin, s(:itarg)))
  assert_equal(expected, parser.tokenize(it_fixture)[0].to_sexp)
end

Then, you can extend the existing it.txt (stuff in fixtures/whitequark is pulled from upstream) and explicitly assert against it. What do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants