If you’ve been keeping up with the news in the Java world lately, you’ve probably heard that the latest Java 8 build released by Oracle, Java 8u11 (and Java 7u65), introduced errors and broke some popular 3rd party tools such as ZeroTurnaround’s JRebel, Javassist, Google’s Guice, and even Groovy itself.
Fracture

The errors spewed by the JVM are long and verbose, but in essence they look something like this:

Exception in thread "main" java.lang.VerifyError: Bad method call from inside of a branch
Exception Details:
Location:
com/takipi/tests/dc/DepthCounter.()V @10: invokespecial
…

The reason these errors suddenly started appearing stems from the fact that the bytecode verifier in the latest updates is a bit more strict than that of previous versions. Unlike previous versions, it does not allow calls to super-constructors from within branched code.

Let’s break it down.

Java bytecode and the bytecode verifier

Bytecode is the intermediate language the JVM actually executes, and in which compiled .class files are written. The JVM’s machine code, if you will.

All JVM-based languages are compiled into bytecode, from Java, through Scala, Groovy, Clojure, and so on. The JVM doesn’t know and doesn’t care what the source language was — it only knows bytecode.

I’m not going to go into how bytecode works, as it is a subject worthy of a post (or several posts) of its own, but just to get a feel for what bytecode looks like — take this simple Java method for example:

int add(int x, int y) {
    int z = x + y;
    return z;
}

When compiled, its bytecode looks like this:

ILOAD x
ILOAD y
IADD
ISTORE z
ILOAD z
IRETURN

When the JVM loads a class file from the classpath into memory, it must first make sure the bytecode is valid and that the code is structured correctly. It basically checks if the code that’s being loaded can actually be executed. If the bytecode is good, the class is successfully loaded into memory; otherwise, a VerifyError is thrown, just like the one at the beginning of the post.

This process is called bytecode verification, and the part of the JVM that’s responsible for it is the bytecode verifier.

Why did it break?

In order for bytecode to pass verification, it must adhere to a set of rules defined in the class file format specification. As the JVM was originally designed with the Java programming language in mind, many of these rules are directly derived from Java rules and constraints.

One such well-known constraint in the Java language, is that the very first thing you must do in a constructor, before doing anything else, is call either super(…) or this(…). Any piece of code before that — and your code won’t compile. Even when you don’t explicitly write super(), the compiler implicitly inserts it for you at the very beginning of the constructor.

A special case of this constraint which can be seen in the bytecode verification rules, states that a constructor must not end its execution without calling its superclass’s constructor (or one of its own overloads), otherwise leaving this not fully initialized. This imposes certain limitations on how branching can be performed around the call to super(…) or this(…).

However, it turns out that up until these recent JDK updates, this constraint had not been fully enforced. This means, that although no Java compiler would ever let you compile this code:

public static class ClassyClass {
    public ClassyClass() {
        if (checkSomething()) {
            super();
        } else {
            ...
        }
    }
}

… The equivalent bytecode could pass verification.

ALOAD this
INVOKESTATIC checkSomething() : boolean
IFEQ L1
INVOKESPECIAL super() : void
GOTO L2
L1: ...
...
L2: RETURN

You can see in the simplified bytecode above that there are both an invocation (INVOKESTATIC) and even a branch (IFEQ — “if equal”) taking place before the first call to the super-constructor (INVOKESPECIAL).

Bear in mind that although the above code is not legal Java, and thus no Java compiler would ever produce the equivalent bytecode — there are plenty of other tools which potentially could, such as compilers of other JVM languages which do not follow Java’s constraints, and many other tools such as bytecode instrumentation libraries. The ability to execute code before the call to super can be pretty useful!

However, Java 8 update 11 brought with it a stricter bytecode verifier, one which rejects classes that use such constructs in their bytecode, and causes verification errors to be thrown and JVMs to crash.

On one hand, the new verifier is loyal to the spec, making sure our JVMs are safe from bad code. On the other hand, many tools which utilize bytecode instrumentation, such as debuggers and aspect weavers (AOP) often make use of constructs such as the above.

How to solve it?

A fix to the bytecode verifier has already been committed, but it hasn’t been released yet. However, many of the affected tools and projects are already releasing fixed versions and workarounds.

In the meantime, if you happen to encounter one of these errors, you can try starting your JVM with the -noverify command line argument. This option instructs the JVM to skip bytecode verification when loading classes.

Niv is co-founder and VP R&D at OverOps. In his free time Niv is an avid basketball fan and a student of languages.
  • Andreas Fester

    Great article – just a minor comment: I assume “IFEQ L2” needs to be “IFEQ L1” in “… The equivalent bytecode”, right?

    • http://www.takipi.com/ Niv Steingarten

      Hey Andreas, thanks for the feedback!
      You are correct of course — fixed.

  • Michael Rasmussen

    You say the same constraint exists in the jvms, linking to the structural constraints part, but where exactly do you see this constraint specified?

    All I see is that you cannot:
    * access members on uninitialized-this before the call to super (except for setting fields defined on the same class)
    * have uninitialized-this on the stack/locals if you’re using JSR/RET (which is not the same as branching)
    * have any uninitialized objects when returning
    * initialize the same object twice

    • http://www.takipi.com/ Niv Steingarten

      Hi Michael,

      You are correct that the general Java constraint mentioned in the paragraph before is not reflected in the bytecode verification rules, and rightfully so — it is not, in itself, a constraint. I wanted to keep the post focused and high-level, and thus avoided going into details of the specific rule I was referring to. I changed the post to reflect this more clearly, in any case.

      The specific constraint at hand is the one which does not allow inserting branching code before the call to ‘super’, if these might allow any execution path to skip the call to the superclass’s initializer and thus leave the object not fully initialized.

      The update to the bytecode verifier tried to enforce this by tracing bytecode, keeping constant track of the furthest point in the bytecode into which execution can branch. It rejected classes which branched over these ‘super’ calls, where it couldn’t be determined that all execution paths would indeed leave ‘this’ fully initialized. Apparently this new process turned out to be a bit of an overshot, and has since been reverted.

  • troyv

    You could almost consider this a break and backward compatibility. I understand Oracle’s goal of not breaking backward compatibility when coming up with new features in new releases, but this could be considered an example of the fact they already do this. I imagine developers of the tools you listed are scurrying to update the code to match the new bytecode verifier, just like anyone other developers would be if Java had some breaks in compatibility with new features.

    • http://www.takipi.com/ Niv Steingarten

      Yes, indeed most of the teams behind the tools that broke reacted rather quickly and released fixes to deal with the new verifier. It appears though, as if the fix has already been reverted in the OpenJDK codebase.

  • Manikandan Nedunchezhiyan

    Great post.. I’m getting the same exception while running my test cases in powermock and mockito. I don’t want to use -noVerify or -XX:-UseSplitVerifier. Is there any other way to fix the issue without that?? Also is powermock working on this to get it fixed in their next release??