Improving Java support for 64-bit.

April 15, 2011

Overview

There are a number of features in Java which are limited to 32-bit. Some of these are limitations of the language, some are limitations of the byte-code.

As servers are getting larger, some have 1 TB of main memory, I believe these limitations will have to be addressed soon.

32-bit features in Java and its libraries

These feature/libraries use int type for sizes.

array.length

This limitation is in the language and the defintion of the byte code. The biggest problem to changing this is to support backward compatibility.

One way around this it to provide two arraylength instructions. The existing one would throw a RuntimeException if the length is greater than Integer.MAX_VALUE, as a second one which would return the length as a long. New code can use the long length while existing code would work, provided it is only given arrays which it can handle.

In Oracle Java 6 the only way to allocate a large chunk of memory is to use Unsafe.allocateMemory()

Unsafe unsafe = getUnsafe();
final long size = 24L * 1024 * 1024 * 1024;
long ds = unsafe.allocateMemory(size); // Create a 24 GB array.
try {
    long start1 = System.nanoTime();
    for (long i = 0; i < size; i += 8) {
        final long ptr = ds + i;
        unsafe.putDouble(ptr, unsafe.getDouble(ptr) + 1);
    }
    long time1 = System.nanoTime() - start1;
    System.out.println("Average Unsafe ++ time " + 1000 * time1 / size + " ps.");
    System.out.printf("Total time was %.2f%n" , time1 / 1e9);
} finally {
    unsafe.freeMemory(ds);
}

Collection.size() and Map.size()

Collection.size() and Map.size() returns an int value. Fortunately, at the JVM level, the return type is part of the method signature so older code which expects an int explicilty asks for it. Newer code could expect a long return value in the manner below.

The defined behaviour for collections with more than Integer.MAX_VALUE elements is to return Integer.MAX_VALUE. Java could have two methods, one for backward compatibility which is called by compiled code and another which can be called by new code.

// this would called by default.

public <T extends Long> long size() {
    return 1L << 40;
}

// this would called by 32-bit based code. 
public <T extends Integer> int size() {
    long size = this.<Long>size();
    return size > Integer.MAX_VALUE ? Integer.MAX_VALUE : (int) size;
}

public static void main(String... args) {
    Main main = new Main();
    System.out.println("main.<Integer>size(): "+main.<Integer>size());
    System.out.println("main.<Long>size(): "+main.<Long>size());
}

BTW: This compiles in Java 6 update 24 due to a bug/feature in the compiler and prints.

main.<Integer>size(): 2147483647
main.<Long>size(): 1099511627776

Memory mapping of files

Memory mapped files use MappedByteBuffer which extends Buffer. Buffer's capacity, length and position all use int values.

This could be changed to support long return types (in the manner suggested for Collection) and take long arguments for position() etc using overloading.

Interestingly, all the underlying native methods support long values and you can access these using reflections, however it would be far better if this were supported via public interfaces.

Vanilla Java