Lies, damned lies and methods implementing specialized generic interfaces

You know that I’ve been spoiled by spending years in a land with reified generics. I’ve heard Daniel Spiewak, in an episode of “The Scalawags“, say that he prefers erased generics because they permit more freedom in the typesystem, and Martin Odersky also swears by them, so I’m not ready to condemn them yet. But they can sure tie one in knots.

Here’s a recount of a recent experience of mine, suitably distilled to its essentials. Follows javap disassembly for a completely boring pair of interface and implementation.

Compiled from ""
public interface Interface {
  public abstract boolean toBoolean(java.lang.String);
Compiled from ""
public class Implementation implements Interface {
  public Implementation();
       0: aload_0
       1: invokespecial #1                  // Method java/lang/Object."<init>":()V
       4: return

  public boolean toBoolean(java.lang.String);
       0: iconst_1
       1: ireturn

Surely you must be yawning by now. Let’s spice it up by introducing some Java 8-like base interface. toBoolean looks like Predicate#test, doesn’t it? But this time I’ll also show you the original Java, because you’re in for a surprise.

public interface Predicate<T> {
  public abstract boolean test(T);

public interface Interface2 extends Predicate<java.lang.String> {

public class Implementation2 implements Interface2 {
 public boolean test(String s) {
  return true;
Compiled from ""
public class Implementation2 implements Interface2 {
  public Implementation2();
       0: aload_0
       1: invokespecial #1                  // Method java/lang/Object."<init>":()V
       4: return

  public boolean test(java.lang.String);
       0: iconst_1
       1: ireturn

  public boolean test(java.lang.Object);
       0: aload_0
       1: aload_1
       2: checkcast     #2                  // class java/lang/String
       5: invokevirtual #3                  // Method test:(Ljava/lang/String;)Z
       8: ireturn

What the…? Apparently, the mere act of defining Interface2 as a specialization of Predicate, actually specialized the signature of test, so as to allow uses of “real” test(String). But, at the same time, the compiler injected an erased test that calls the specialized one. That’s how I found it. I was creating the implementation with Javassist, and the Javassist compiler did not perform this sleigh-of-hand. Which was OK until I tried to call the method and got an abstract method error. Not a very common sight, is it?

Oh, and you cannot do this explicitly with Java code, because trying to do so gives you an error. error: name clash: test(Object) in OtherImplementation2b and test(T) in Predicate have the same erasure, yet neither overrides the other
 public boolean test(Object o) {
  where T is a type-variable:
    T extends Object declared in interface Predicate
1 error

boolean test(java.lang.Object) should have been final, but isn’t. Even so, the aforementioned error does not let you override it, anyway. You can only override the specialized method.

Ultimately, it is not just compilers for “other” languages who need to jump into hoops to get around the limitations of Generics. This is a case of the Java compiler itself jumping through hoops. Remember that the Javassist-created class compiled fine and was loaded fine, but the error stroke at runtime. Beware!

UPDATE: These phantom methods are called “bridge” methods, and they are utilized in all places where the typesystem needs to express things the JVM cannot. They sport the ACC_BRIDGE flag.


Generics Face-Off, Nugget #1

My last post was a bit overly enthusiastic about Scala. To counterbalance it, I present another “nugget” in this post, inspired by my slim experience on using Scala together with Java. You see, Scala aims to remedy some shortcomings of the Java typesystem, in particular its generics, which makes it somewhat uncomfortable, at times, to bridge the two. And the face-off you read in the title, will be between Scala and another JVM language, Kotlin, which aims more towards being a better Java than providing a wholly different solution. Let’s see how they do, using the scale used by Catch-22‘s Colonel Cathcart.

Java’s wacky use of Generics

Suppose there’s a generic Java class, named Something, presented here as a bare-bones example.

public class Something<T> {
    public String toString() {
        return "Something";

And here’s the rub. Java allows you to “cop out” of Generics and use either the erased type, or an existential type, in the following meaningless way (and I’m not making this up).

public class JTest {
    public static Something<?> getSomething() {
        return new Something<String>();

Here’s some Scala code to process an instance of Something<String>.

class STest {
    def workWithSomethingString(s : Something[String]) {

Suppose you know that the particular instance is a Something<String> (the way you know it lies certainly outside of the type system). Going from Java to Scala produces an unchecked warning, as it should.

    public static void scalaDriver() {
        STest sTest = new STest();
        //flagged as unckecked

Enter Kotlin.

class KTest {
    fun workWithSomethingString(s:Something<String>) {

Same thing here.

    public static void kotlinDriver() {
        KTest ktest = new KTest();
        ktest.workWithSomethingString((Something<String>) getSomething());
        //flagged as unckecked

Both fail to cope with Java’s incongruent use of Generics. Hence, a black eye for both, in that regard. Hint: at the call site, it’s the Java compiler that runs the show. There’s really no way to win. Let’s see what we get if we give full control of the calling site to the other compilers.

Reified generics

If we disregard direct interconnection with Java, for a moment, both Scala and Kotlin want to provide some solution to being able to recover type information that is lost by erasure.

Scala does it using TypeTags and such, in various ways which employ the compiler to supply information about generic types from the calling site to the called method.

Below is one way to do it. The first call to workWithSomething delegates correctly to workWithSomethingString.

    def passTheBuck() {
        workWithSomething(new Something[String])
        workWithSomething(new Something[Int])

    def workWithSomething[T](s : Something[T])(implicit t : TypeTag[T]) {
        if( t.tpe.typeConstructor =:= typeOf[String] ) {
        } else {
            print("\nCannot handle\n")

How about Kotlin? Kotlin has promised reified generics, but it’s still in the roadmap. Black eye for Kotlin, feather in Scala’s cap.

Things don’t look so good for Kotlin, so far. In the next nugget, let’s see if it manages to recoup, using its distinguishing feature of mixed-site variance.

It may quack like a duck

This post comes after a long hiatus. I have family matters to attend to and a new job, to boot. And I made a promise to myself to not let this blog devolve to rants without substance. I was hoping to have something interesting to say on the subject of working with column constraints that Higher Order Sql will produce, at least for the usual case of constraints defined using literal values, but I don’t have anything ready yet. So I devised a new category, “Nugget”, for small posts in which I will try to convey interesting information, nevertheless.

For this one, I will draw upon my recent studies to focus on a Scala feature which, I have seen, is not given much attention. You see, my new job is in a Java shop. And I have found that adding a little Scala makes using Java almost bearable, even before moving to Scala wholesale. You might find my words too scornful but the truth of the matter is that Java was, well, the Java of the 90s, but is now the Cobol of the 00s. It is progressing at a glacial pace, to the point that I found it practically unchanged after I spent seven years working with .Net. Its signal-to-noise ratio is very low (really? after all these years I still have to Alt-Ins to produce thousands of getters and setters that clobber the rest of my code?) and it’s fallen behind after all other modern languages. It does have the most complete ecosystem, but this is a property of running in the JVM, it makes no difference whether the many useful libraries have been written in Java. Except that, since most of them have been written in Java, they make too much use of mutation, practically blocking Java from leaping to the multicore era.

Enough with the ranting. The subject of this post is Scala structural types (BTW, you can read a nice exposé on the Scala typesystem in this article). Structural types are not used much, but they enable type-safe Duck Typing, as in the following example.

import scala.language.reflectiveCalls

class Foo(val x:Int) {
    def addToMe(i:Int) = i+x

class Bar(val x:Int) {
    def addToMe(i:Int) = i+x

object test {
    def test(o : { def addToMe(i:Int) }) = {

The { def addToMe(i:Int) } part is where one describes a type that has an addToMe method that takes an Int. Why would anyone want to do that? Why, to show it to their friends who use Python or Ruby! I’m kidding… A case I found recently is the following: in QueryDsl, NumberExpression and ComparableExpression both implement the same API methods for generating comparisons (e.g, goe(), gt() ) and they don’t inherit them from some common ancestor, so, in Java, one needs to duplicate code to deal with both of them, code which is absolutely the same except for the type of the expression.

It is very instructive to look at how Scala does it (spoiler: the same way you’d do it youself, only without the grief). Class test$ (Scala makes it for the singleton object) has the following contents.

public final class test$ {
  public static final test$ MODULE$;
  public static {};
  public static java.lang.reflect.Method reflMethod$Method1(java.lang.Class);
  public void test(java.lang.Object);

Method reflMethod$Method1 is used to find the addToMe method of the target, and method test calls it.

  public static java.lang.reflect.Method reflMethod$Method1(java.lang.Class);
       0: getstatic     #35                 // Field reflPoly$Cache1:Ljava/lang/ref/SoftReference;
       3: invokevirtual #42                 // Method java/lang/ref/SoftReference.get:()Ljava/lang/Object;
       6: checkcast     #44                 // class scala/runtime/MethodCache
       9: astore_1      
      10: aload_1       
      11: ifnonnull     33
      14: new           #25                 // class scala/runtime/EmptyMethodCache
      17: dup           
      18: invokespecial #28                 // Method scala/runtime/EmptyMethodCache."<init>":()V
      21: astore_1      
      22: new           #23                 // class java/lang/ref/SoftReference
      25: dup           
      26: aload_1       
      27: invokespecial #31                 // Method java/lang/ref/SoftReference."<init>":(Ljava/lang/Object;)V
      30: putstatic     #35                 // Field reflPoly$Cache1:Ljava/lang/ref/SoftReference;
      33: aload_1       
      34: aload_0       
      35: invokevirtual #47                 // Method scala/runtime/MethodCache.find:(Ljava/lang/Class;)Ljava/lang/reflect/Method;
      38: astore_2      
      39: aload_2       
      40: ifnull        45
      43: aload_2       
      44: areturn       
      45: getstatic     #52                 // Field scala/runtime/ScalaRunTime$.MODULE$:Lscala/runtime/ScalaRunTime$;
      48: aload_0       
      49: ldc           #54                 // String addToMe
      51: getstatic     #21                 // Field reflParams$Cache1:[Ljava/lang/Class;
      54: invokevirtual #58                 // Method java/lang/Class.getMethod:(Ljava/lang/String;[Ljava/lang/Class;)Ljava/lang/reflect/Method;
      57: invokevirtual #62                 // Method scala/runtime/ScalaRunTime$.ensureAccessible:(Ljava/lang/reflect/Method;)Ljava/lang/reflect/Method;
      60: astore_2      
      61: new           #23                 // class java/lang/ref/SoftReference
      64: dup           
      65: aload_1       
      66: aload_0       
      67: aload_2       
      68: invokevirtual #66                 // Method scala/runtime/MethodCache.add:(Ljava/lang/Class;Ljava/lang/reflect/Method;)Lscala/runtime/MethodCache;
      71: invokespecial #31                 // Method java/lang/ref/SoftReference."<init>":(Ljava/lang/Object;)V
      74: putstatic     #35                 // Field reflPoly$Cache1:Ljava/lang/ref/SoftReference;
      77: aload_2       
      78: areturn       

  public void test(java.lang.Object);
       0: aload_1       
       1: astore_2      
       2: aload_2       
       3: invokevirtual #80                 // Method java/lang/Object.getClass:()Ljava/lang/Class;
       6: invokestatic  #82                 // Method reflMethod$Method1:(Ljava/lang/Class;)Ljava/lang/reflect/Method;
       9: aload_2       
      10: iconst_1      
      11: anewarray     #4                  // class java/lang/Object
      14: dup           
      15: iconst_0      
      16: bipush        42
      18: invokestatic  #88                 // Method scala/runtime/BoxesRunTime.boxToInteger:(I)Ljava/lang/Integer;
      21: aastore       
      22: invokevirtual #92                 // Method java/lang/reflect/Method.invoke:(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
      25: pop           
      26: getstatic     #98                 // Field scala/runtime/BoxedUnit.UNIT:Lscala/runtime/BoxedUnit;
      29: pop           
      30: return        
      31: astore_3      
      32: aload_3       
      33: invokevirtual #102                // Method java/lang/reflect/InvocationTargetException.getCause:()Ljava/lang/Throwable;
      36: athrow        
    Exception table:
       from    to  target type
           2    25    31   Class java/lang/reflect/InvocationTargetException

There you go, my first “Nugget”, until I can get back on track and produce more substantial content. Scala has a lot more where this came from, and I can’t urge you strongly enough to have a look at it, particularly if you work with Java. Make no mistake. Scala is not F#. It is a modern, object-oriented language with a strong functional inclination, but it’s not a functional language (although Scalaz tries to make it more of one). So I won’t be abandoning F#, but Scala is definitely the best language on the JVM at this time (it’s the only strongly-typed of the non-Javas), having very advanced interoperability with Java (you can actually have mixed-language projects). And, guess what! I already know two people who have enrolled for Martin Odersky’s course on learning FP using Scala. What are you waiting for?