Skip to content

Commit

Permalink
Support itblock for Prism::Translation::Parser
Browse files Browse the repository at this point in the history
## Summary

`itblock` node is added to support the `it` block parameter syntax introduced in Ruby 3.4.

```console
$ ruby -Ilib -rprism -rprism/translation/parser34 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { it }"; \
                                                      p Prism::Translation::Parser34.new.tokenize(buffer)[0]'
s(:itblock,
  s(:send, nil, :proc), :it,
  s(:lvar, :it))
```

This node design is similar to the `numblock` node, which was introduced for the numbered parameter syntax in Ruby 2.7.

```
$ ruby -Ilib -rprism -rprism/translation/parser34 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { _1 }"; \
                                                      p Prism::Translation::Parser34.new.tokenize(buffer)[0]'
s(:numblock,
  s(:send, nil, :proc), 1,
  s(:lvar, :_1))
```

The difference is that while numbered parameters can have multiple parameters, the `it` block parameter syntax allows only a single parameter.

In Ruby 3.3, the conventional node prior to the `it` block parameter syntax is returned.

```console
$ ruby -Ilib -rprism -rprism/translation/parser33 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { it }"; \
                                                      p Prism::Translation::Parser33.new.tokenize(buffer)[0]'
s(:block,
  s(:send, nil, :proc),
  s(:args),
  s(:send, nil, :it))
```

## Development Note

The Parser gem does not yet support the `it` block parameter syntax. This is the first case where Prism's node design precedes that of the Parser gem.
When implementing whitequark/parser#962, this node design will need to be taken into consideration.
  • Loading branch information
koic committed Feb 26, 2025
1 parent ad7d82c commit 722a9c7
Show file tree
Hide file tree
Showing 6 changed files with 238 additions and 5 deletions.
48 changes: 48 additions & 0 deletions lib/prism/translation/parser/builder.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,55 @@ class Parser
# A builder that knows how to convert more modern Ruby syntax
# into whitequark/parser gem's syntax tree.
class Builder < ::Parser::Builders::Default
# It represents the `it` block argument, which is not yet implemented in the Parser gem.
def itarg
n(:itarg, [:it], nil)
end

# The following three lines have been added to support the `it` block parameter syntax in the source code below.
#
# if args.type == :itarg
# block_type = :itblock
# args = :it
#
# https://github.com/whitequark/parser/blob/v3.3.7.1/lib/parser/builders/default.rb#L1122-L1155
def block(method_call, begin_t, args, body, end_t)
_receiver, _selector, *call_args = *method_call

if method_call.type == :yield
diagnostic :error, :block_given_to_yield, nil, method_call.loc.keyword, [loc(begin_t)]
end

last_arg = call_args.last
if last_arg && (last_arg.type == :block_pass || last_arg.type == :forwarded_args)
diagnostic :error, :block_and_blockarg, nil, last_arg.loc.expression, [loc(begin_t)]
end

if args.type == :itarg
block_type = :itblock
args = :it
elsif args.type == :numargs
block_type = :numblock
args = args.children[0]
else
block_type = :block
end

if [:send, :csend, :index, :super, :zsuper, :lambda].include?(method_call.type)
n(block_type, [ method_call, args, body ],
block_map(method_call.loc.expression, begin_t, end_t))
else
# Code like "return foo 1 do end" is reduced in a weird sequence.
# Here, method_call is actually (return).
actual_send, = *method_call
block =
n(block_type, [ actual_send, args, body ],
block_map(actual_send.loc.expression, begin_t, end_t))

n(method_call.type, [ block ],
method_call.loc.with_expression(join_exprs(method_call, block)))
end
end
end
end
end
Expand Down
2 changes: 1 addition & 1 deletion lib/prism/translation/parser/compiler.rb
Original file line number Diff line number Diff line change
Expand Up @@ -1157,7 +1157,7 @@ def visit_it_local_variable_read_node(node)
# -> { it }
# ^^^^^^^^^
def visit_it_parameters_node(node)
builder.args(nil, [], nil, false)
builder.itarg
end

# foo(bar: baz)
Expand Down
7 changes: 7 additions & 0 deletions test/prism/fixtures/whitequark/it_arg_after_34.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
-> do it + 42 end

-> { it + 42 }

m do it + 42 end

m { it + 42 }
31 changes: 28 additions & 3 deletions test/prism/ruby/parser_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
verbose, $VERBOSE = $VERBOSE, nil
require "parser/ruby33"
require "prism/translation/parser33"
require "parser/ruby34"
require "prism/translation/parser34"
rescue LoadError
# In CRuby's CI, we're not going to test against the parser gem because we
# don't want to have to install it. So in this case we'll just skip this test.
Expand Down Expand Up @@ -55,6 +57,12 @@ def ==(other)

module Prism
class ParserTest < TestCase
IT_BLOCK_PARAMETER_FIXTURES = [
'whitequark/it_arg_after_34.txt'
]

RUBY34_FIXTURES = IT_BLOCK_PARAMETER_FIXTURES

# These files contain code that is being parsed incorrectly by the parser
# gem, and therefore we don't want to compare against our translation.
skip_incorrect = [
Expand Down Expand Up @@ -86,6 +94,10 @@ class ParserTest < TestCase
# Regex with \c escape
"unescaping.txt",
"seattlerb/regexp_esc_C_slash.txt",

# Please remove this once the Parser gem supports the `it` block parameter syntax:
# https://github.com/whitequark/parser/issues/962
"whitequark/it_arg_after_34.txt"
]

# These files are either failing to parse or failing to translate, so we'll
Expand Down Expand Up @@ -150,7 +162,13 @@ def assert_equal_parses(fixture, compare_asts: true, compare_tokens: true, compa
buffer = Parser::Source::Buffer.new(fixture.path, 1)
buffer.source = fixture.read

parser = Parser::Ruby33.new
whitequark_parser, translation_parser = if RUBY34_FIXTURES.include?(fixture.path)
[Parser::Ruby34, Prism::Translation::Parser34]
else
[Parser::Ruby33, Prism::Translation::Parser33]
end

parser = whitequark_parser.new
parser.diagnostics.consumer = ->(*) {}
parser.diagnostics.all_errors_are_fatal = true

Expand All @@ -161,8 +179,7 @@ def assert_equal_parses(fixture, compare_asts: true, compare_tokens: true, compa
return
end

actual_ast, actual_comments, actual_tokens =
ignore_warnings { Prism::Translation::Parser33.new.tokenize(buffer) }
actual_ast, actual_comments, actual_tokens = ignore_warnings { translation_parser.new.tokenize(buffer) }

if expected_ast == actual_ast
if !compare_asts && !Fixture.custom_base_path?
Expand All @@ -183,6 +200,8 @@ def assert_equal_parses(fixture, compare_asts: true, compare_tokens: true, compa
elsif compare_asts
assert_equal expected_ast, actual_ast, -> { assert_equal_asts_message(expected_ast, actual_ast) }
end

assert_prism_only_node(fixture.path, actual_ast)
end

def assert_equal_asts_message(expected_ast, actual_ast)
Expand Down Expand Up @@ -242,5 +261,11 @@ def assert_equal_comments(expected_comments, actual_comments)
"actual: #{actual_comments.inspect}"
}
end

def assert_prism_only_node(actual_ast, fixture_path)
if IT_BLOCK_PARAMETER_FIXTURES.include?(fixture_path)
assert_equal :itblock, actual_ast.children.first.type
end
end
end
end
6 changes: 5 additions & 1 deletion test/prism/ruby/ruby_parser_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,11 @@ class RubyParserTest < TestCase
"whitequark/pattern_matching_single_match.txt",
"whitequark/ruby_bug_12402.txt",
"whitequark/ruby_bug_14690.txt",
"whitequark/space_args_block.txt"
"whitequark/space_args_block.txt",

# Please remove this once the Parser gem supports the `it` block parameter syntax:
# https://github.com/whitequark/parser/issues/962
"whitequark/it_arg_after_34.txt"
]

# https://github.com/seattlerb/ruby_parser/issues/344
Expand Down
149 changes: 149 additions & 0 deletions test/prism/snapshots/whitequark/it_arg_after_34.txt

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 722a9c7

Please sign in to comment.