Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lark allows both '?' and '_' on the same rule #1173

Closed
chanicpanic opened this issue Aug 1, 2022 · 2 comments
Closed

Lark allows both '?' and '_' on the same rule #1173

chanicpanic opened this issue Aug 1, 2022 · 2 comments

Comments

@chanicpanic
Copy link
Contributor

Describe the bug

Lark allows ? and _ to be used on the same rule. As they both control the inlining of children, I would expect them to be mutually exclusive. In fact, including both of them on the same rule can cause an error during tree building.

To Reproduce

grammar = r"""
start: _a
!?_a: "A"
"""

p = Lark(grammar)
p.parse("A")
  File "/___/lark/lark.py", line 628, in parse
    return self.parser.parse(text, start=start, on_error=on_error)
  File "/___/lark/parser_frontends.py", line 96, in parse
    return self.parser.parse(stream, chosen_start, **kw)
  File "/___/lark/parsers/earley.py", line 292, in parse
    return transformer.transform(solutions[0])
  File "/___/lark/parsers/earley_forest.py", line 395, in transform
    self.visit(root)
  File "/___/lark/parsers/earley_forest.py", line 532, in visit
    super(ForestToParseTree, self).visit(root)
  File "/___/lark/parsers/earley_forest.py", line 336, in visit
    vpno(current)
  File "/___/lark/parsers/earley_forest.py", line 633, in visit_packed_node_out
    super(ForestToParseTree, self).visit_packed_node_out(node)
  File "/___/lark/parsers/earley_forest.py", line 445, in visit_packed_node_out
    self._visit_node_out_helper(node, self.transform_packed_node)
  File "/___/lark/parsers/earley_forest.py", line 433, in _visit_node_out_helper
    transformed = method(node, self.data[id(node)])
  File "/___/lark/parsers/earley_forest.py", line 617, in transform_packed_node
    return self._cache.setdefault(id(node), self._call_rule_func(node, children))
  File "/___/lark/parsers/earley_forest.py", line 563, in _call_rule_func
    return self.callbacks[node.rule](data)
  File "/___/lark/parse_tree_builder.py", line 146, in __call__
    filtered = children[i].children
AttributeError: 'Token' object has no attribute 'children'
@erezsh
Copy link
Member

erezsh commented Aug 1, 2022

Good catch!

I guess we should just throw an error when they are used together.

erezsh added a commit that referenced this issue Oct 7, 2022
erezsh added a commit that referenced this issue Oct 7, 2022
erezsh added a commit that referenced this issue Oct 7, 2022
@erezsh
Copy link
Member

erezsh commented Oct 7, 2022

Thanks for reporting it!

@erezsh erezsh closed this as completed Oct 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants