Exceptions being swallowed if reported in a Hook

[dkowis]

     [java] Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
     [java]     at java.util.ArrayList.RangeCheck(ArrayList.java:547)
     [java]     at java.util.ArrayList.remove(ArrayList.java:387)
     [java]     at gherkin.formatter.PrettyFormatter.printStep(PrettyFormatter.java:166)
     [java]     at gherkin.formatter.PrettyFormatter.result(PrettyFormatter.java:159)
     [java]     at cucumber.runtime.RuntimeWorld.runHookIfTagsMatch(RuntimeWorld.java:77)
     [java]     at cucumber.runtime.RuntimeWorld.runHooks(RuntimeWorld.java:59)
     [java]     at cucumber.runtime.RuntimeWorld.runAfterHooksAndDisposeBackendWorlds(RuntimeWorld.java:43)
     [java]     at cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:39)
     [java]     at cucumber.runtime.model.CucumberScenarioOutline.run(CucumberScenarioOutline.java:37)
     [java]     at cucumber.runtime.Runtime.run(Runtime.java:75)
     [java]     at cucumber.runtime.Runtime.run(Runtime.java:67)
     [java]     at cucumber.cli.Main.run(Main.java:78)
     [java]     at cucumber.cli.Main.main(MERROR HAPPENED, and then it nuked the exception stack!
     [java] EXCEPTION: 
     [java] org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `failed?' for nil:NilClass
     [java]     at #<Class:0x2d4c8822>.(root)(com/rackspace/incident/features/ruby/step_definitions/env.rb:33)
     [java]     at org.jruby.RubyKernel.instance_exec(org/jruby/RubyKernel.java:2079)
     [java] ain.java:64)ber::Runtime::JRuby::HookDefinition.execute(/cucumber/runtime/jruby/dsl.rb:60)

So in that stack trace there's a bit of code that I added to the RuntimeWorld at line 69 to print the stack trace of the Throwable before it's handed off to the result, because that's the last place in the main stack trace about the Index Out of Bounds Exception.

The cause is a undefined method "failed?" in the After Hook being executed (in Jruby, because somehow the ScenarioResult object isn't being set, but that's a different issue.) So when the reporter gets ahold of this, and tries to output the error, it cannot, because there are no more steps for it to report on, and so it runs over the Array.

This was while running the Jruby CLI feature executor.

[aslakhellesoy]

Yeah there are still a few bugs in the pretty formatter. It's not well tested, and could do with some more edge-casy unit tests. -And there is some experimental logic in there for printing the same line several times (before and after execution) that relies on special TTY ANSI characters (for up). This isn't going anywhere and can be removed. Removing this first would make it easier to fix other bugs like this one.

[aslakhellesoy]

I'm not sure how to reproduce this bug so it can be fixed. Please submit more detail.

[dkowis]

Okay, I'll try to build a sample project that demonstrates this after I get done building a sample project for work that has problems with hazelcast :(

[dkowis]

Okay, I've got a repo that reproduces the issue: https://github.com/dkowis/Cucumber-JVM-Ruby-Example/tree/error_swallow

Hopefully that will be enough to describe it.

I think if we were to redo the pretty formatter along the lines of previous discussion [1] the issue would be solved, as the issue is that it tries to pop something off the stack that was never added (there's no line for after hooks).

1: I think we were talking about rewriting it so it'd make two passes through the feature file so it knew where to align things, as well as not doing the fancy shell codes to jump up a line. I don't remember all the details though.

[klausbayrhammer]

We had a similar problem when using custom SpringObjectFactory.

When we had exceptions during the startup of our spring-context the exception had been swallowed which resulted in a NPE in the PrettyFormatter.

[dkowis]

@klausbayrhammer yeah I just had the same problem, no custom SpringObjectFactory, but somehow we have duplicate spring beans, probably due to something that changed in how cucumber-jvm loads stuff :) But the problem is still that the exception is swallowed.

@aslakhellesoy do you have enough data using my sample project as well as the other examples listed here?

I could probably work on this as well, but I haven't been in the code in a while, and real life is really really busy right now :(

[aslakhellesoy]

I'm very busy myself at the moment, so some help on this would be great. The codebase hasn't changed that much since you were in it I think.

In any case - this looks like a bug in gherkin's PrettyFromatter.

[dkowis]

Yeah, I'm looking at seeing what I can do to fix it without rewriting the entire pretty formatter.

[dkowis]

cucumber/gherkin#172

Filed this issue, because some changes are also necessary to the JSON formatter as well, sadly.

[dkowis]

I see that this is fixed in my sample branch. Looks good to me. Thanks for all the hard work!

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.