Signing Java Objects for Secure Transfer

In distributed J2EE applications or in any application where you need to transfer Java objects to another system then there is always a security risk where the object can be intercepted which can result in data theft/loss. Especially in Serialization, (where the object is a physical file in the native file system) when the serialized Java objects are sent through the network, whoever knows the type of the object can always read it.

In this article, we will build two simple applications, one which generates the object, the keys (public & private) and signs the object with the private key. Other application which verifies the signed object in other end over the network or another application in the same machine. Both these apps can run independently in different machines. For signing the object we will be using Public-Key cryptography. This is one of the most widely used standards to sign data along with DSA & SHA1PRNG (cryptographically strong pseudo-random number generator (PRNG)). Public-Key cryptography is a asymmetric key algorithm, where the key used to encrypt a message is not the same as the key used to decrypt it.

This is the class diagram of the applications which we will be building. This article will be divided into two parts, the first part we will sign the object (serialized) and in the second part, we will verify it.

Sign the Java Object

First of all we need a class which will generate a public and private key. We will create a class named SecurityUtil which will generate those based on DSA (we can use RSA or any other algorithm as long as its available) and we will generate a cryptographically strong pseudo-random number generator (PRNG) which can be clubbed along with DSA (SHA1PRNG). The strength of the key will be 1024.

protected KeyPair generateKey () throws Exception {
    KeyPairGenerator keyPairGen = KeyPairGenerator.getInstance("DSA");
    SecureRandom secureRandom = SecureRandom.getInstance("SHA1PRNG");
    keyPairGen.initialize(1024,secureRandom);
    KeyPair keyPair = keyPairGen.generateKeyPair();
    return keyPair;
}

Next we will create a class named EmployeeValueObject which is nothing but a POJO with a HashMap getter/setter. This will be the object which we will be transferring over the network/application. Since we serialize the object before transferring, this class should implement Serializable.

public class EmployeeValueObject implements Serializable {
    HashMap employeeSalary = new HashMap();
    public void setSalary (HashMap employeeSalary){
        this.employeeSalary = employeeSalary;
    }

    public HashMap getSalary () {
        return employeeSalary;
    }
}

Now we have all the supporting classes which we need and let’s start building the main application. Let’s call this class EmployeeDetails and this will create an object for the POJO which we created in our previous step and populate with some data. In addition to that, we will sign the POJO object and then serialize to a file. In this example we will be also serializing the public key to transfer to the other end. Note: In production implementations, both these objects shouldn’t be sent at the same time. The application at the other end should already have the public key)

Let’s create the POJO and populate with some data in the HashMap.

EmployeeValueObject employeeVO = new EmployeeValueObject();
employeeVO.setSalary(populateData());

private static HashMap populateData (){
    HashMap employeeSalary = new HashMap ();
    employeeSalary.put("3", "Johns, Galvin D. --> $18,000");
    employeeSalary.put("4", "Weber, Murphy I. --> $5,000");

    return employeeSalary;
}

Now let’s generate the public and private keys from SecutityUtil and sign the POJO which we created in the above step.

KeyPair keyPair = new SecurityUtil().generateKey();
PrivateKey privateKey = keyPair.getPrivate();
PublicKey publicKey = keyPair.getPublic();

Signature digitalSignature = Signature.getInstance(privateKey.getAlgorithm());
SignedObject digitalSignedObj =
    new SignedObject(employeeVO, privateKey, digitalSignature);

Now digitalSignedObj is a digitally signed data with the private key which we generated. Now let’s serialize this object for the secure transfer.

ileOutputStream serializedFileOutput = new FileOutputStream("employee.ser");
ObjectOutputStream serializedObjOutput = new ObjectOutputStream(serializedFileOutput);
serializedObjOutput.writeObject(digitalSignedObj);
serializedObjOutput.close();
serializedFileOutput.close();

We will also serialize the public key so that for this example we can send both of them to another machine to verify. Note: In production implementations, both these objects shouldn’t be sent at the same time. The application at the other end should already have the public key)

serializedFileOutput = new FileOutputStream("publickey.ser");
serializedObjOutput = new ObjectOutputStream(serializedFileOutput);
serializedObjOutput.writeObject(publicKey);
serializedObjOutput.close();
serializedFileOutput.close();

This will complete the creation of application one. When you run this application, it will create two new files in the same directory. employee.ser – which is the signed and serialized POJO (Salary details) & publickey.ser – public key to verify the POJO. Now using the appropriate protocol send these files to the other application (remote or local) and let’s start building the verification part.

Verification & De-Serializing the Java Object

As a start we have the files employee.ser & publickey.ser. Let’s start building up the class to verify and de-serialize these files. Let’s name this class DecryptEmployee. The following code should de-serialize the objects.

FileInputStream serializedPublicKeyIn = new FileInputStream("publicKey.ser");
ObjectInputStream serializedPublicKey = new ObjectInputStream(serializedPublicKeyIn);
PublicKey publicKey = (PublicKey) serializedPublicKey.readObject();

FileInputStream serializedEmployeeIn = new FileInputStream("employee.ser");
ObjectInputStream serializedEmployee = new ObjectInputStream(serializedEmployeeIn);
SignedObject digitalSignedObj = (SignedObject) serializedEmployee.readObject();

Since the public key was not signed, publicKey variable will be readable. But the employee POJO was signed, so we are reading the object as a SignedObject. Let’s move forward and verify this.

Signature digitalSignature = Signature.getInstance(publicKey.getAlgorithm());
boolean decryptFlag = digitalSignedObj.verify(publicKey, digitalSignature);

The decryptFlag contains the status of the verification. If the public key is incorrect or if the object was tampered, then this will return false and we won’t be able to verify the object. If its true then everything looks good and we can successfully verify the POJO and print the values from HashMap.

if(decryptFlag) {
    EmployeeValueObject employeeVO = (EmployeeValueObject) digitalSignedObj.getObject();
    HashMap employeeSalary = (HashMap) employeeVO.getSalary();
    Collection collHashMap = employeeSalary.values();
    Iterator collectionIterator = collHashMap.iterator();
    while (collectionIterator.hasNext()) {
        System.out.println(collectionIterator.next());
    }
} else {
    System.out.println ("Decryption Failed. Please check the Keys.");
}

If you run this application, we will get an output similar to below.

This can be used in any sensitive application to make sure that the objects which are transferred over the network are safe.

UPDATE: SignedObject signs the object, but it doesn’t encrypt it. So if you need encryption, you can use the Cipher class in Java.

Recover from Out of Memory Errors in Java

In Java, as any other programming language there is a restriction of amount of memory any program can use. In languages like C the memory is limited to amount of RAM the operating system allocates to the applications or the user space. Since Java applications are technically running in the Java Virtual Machine (JVM), the applications have memory allocated by the JVM. Due to that we can start java applications with the amount of memory we need by using the –Xms and –Xmx command line parameters (Given the JVM has the memory to spare).

In Java, the memory allocation is handled by JVM. We create the objects and JVM decides where to keep those objects in heap. In languages like C we can use methods like malloc to dynamically allocate memory for objects (variables) and when we are done, the appropriate clean up methods should be called to release the memory space. There are advantages in each method, and discussing them will take another separate post.

In this post, I am going to explain how to recover from the Out of Memory errors. When a Java class faces out of memory errors, we should try to recover from those errors by reducing or blocking the service rather than crashing the application. In many cases when the application crashes due to out of memory, JVM is also impacted (which may be running other applications).

You can ask why can’t we catch the exception. First of all, catching the exception means that the error has already occurred and in complex systems letting the error happen will be costly. This method will prevent you from the error itself. When the memory is lower than the defined threshold, the code is not even executed and we are taking recovery measures to free up some memory. Also if your application is using all the memory allocated to JVM, when out of memory error occurs it will end up crashing JVM so there is no point catching the exception since your application would have been crashed.

To recover from the out of memory errors, we need to simulate the error first and then find a solution to recover. So this post consists of two major parts.

Simulate Out of Memory Error

Simulating the error is pretty simple, we will create a lot of array objects in a loop for which JVM will allocate memory spaces. Once the objects reaches a threshold, the JVM throws out of memory errors. But with the current desktops having at least 1GB of RAM (with at least 100M as the max limit to JVM) we need to create thousands of objects to simulate the error. So as I mentioned in the start of this post we will use the –Xms and –Xmx options to start the program with a maximum of 2M so that we can easily simulate the error.

Create a Java Class with the following code and start with the parameter java -Xms2m –Xmx3m

public void fillMemory() throws Exception {
    int voidSpace=20;
    for (int outerIterator=1;outerIterator<50;outerIterator++) {
        System.out.println ("Iteration " + outerIterator + " Free Mem: "
            + Runtime.getRuntime().freeMemory());
        int innerIterator=10;
        int[] memoryFillIntVar=new int[voidSpace];
        do {
            memoryFillIntVar[innerIterator]=0;
            innerIterator--;
        } while(innerIterator>0);
        voidSpace = voidSpace * 10;
    }
}

When you run this class (MemoryTest.java), you will get an output which will be similar to below.

D:\temp\>java -Xms2m -Xmx3m MemoryTest
Iteration 1 Free Mem: 1826368
Iteration 2 Free Mem: 1826368
Iteration 3 Free Mem: 1826368
Iteration 4 Free Mem: 1818352
Iteration 5 Free Mem: 1738336
Iteration 6 Free Mem: 1100952
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at MemoryTest.fillMemory(MemoryTest.java:22)
        at MemoryTest.main(MemoryTest.java:12)

Now we have successfully simulated the out of memory error. Let’s continue to the next part where I will show how to recover from these errors.

Recover from the error

To recover from the error we will be using one of the utility classes from Apache Derby V10.6 Internals called LowMemory. The class has the below 2 methods drawing our intrest.

void setLowMemory() - Sets a low memory watermark where the owner of this object just hit an OutOfMemoryError.

boolean isLowMemory() - Returns true if a low memory water mark has been set and the current free memory is lower than it.

So we should use setLowMemory to set a threshold and once set, whenever we call the isLowMemory it will return true/false based on the available memory. The isLowMemory will allow an low memory watermark to be valid for five seconds after it was set. This stops an incorrect limit being set for ever. This could occur if other threads were freeing memory when we called Runtime.getRuntime().freeMemory().

Now let’s rewrite our earlier MemoryTest program to use these methods to recover from the error. The modification is simple, we just need to encapsulate the object creation in a if condition which checks for the memory before creating the objects. The modified code is given below.

public class MemoryTest {
    public static void main(String[] args) throws Exception {
        MemoryTest memoryTest = new MemoryTest ();
        LowMemory lowMemory = new LowMemory();
        long[] memoryFillLongVar = new long[70000];
        lowMemory.setLowMemory();
        memoryTest.fillMemory(lowMemory);
    }
    public void fillMemory(LowMemory lowMemory) throws Exception {
        int voidSpace=20;
        for (int outerIterator=1;outerIterator<50;outerIterator++) {
            System.out.println ("Iteration " + outerIterator + " Free Mem: "
                + Runtime.getRuntime().freeMemory());
            int innerIterator=10;
            if (!lowMemory.isLowMemory()) {
                int[] memoryFillIntVar=new int[voidSpace];
                do {
                    memoryFillIntVar[innerIterator]=0;
                    innerIterator--;
                } while(innerIterator>0);
                    voidSpace = voidSpace * 10;
            } else {
                System.out.println ("Memory lower than threshold to
                    continue. Exiting the loop.");
                break;
            }
        }
    }
}

Let’s run the modified code with the same command line arguments. The output is given below. Note that the class file for LowMemory.java is present in the same directory as of MemoryTest.

D:\temp\>java -Xms2m -Xmx3m MemoryTest
Iteration 1 Free Mem: 1349576
Iteration 2 Free Mem: 1349576
Iteration 3 Free Mem: 1349576
Iteration 4 Free Mem: 1341560
Iteration 5 Free Mem: 1261544
Iteration 6 Free Mem: 461528
Memory lower than threshold to continue. Exiting the loop.

This will be very useful while writing distributed J2EE applications which involves huge databases and processing of a large amount of data within a class. This will ensure there will be no data corruption and ensure the JVM (which may host multiple applications) is able to recover from the memory errors. You can always call the GC and runFinalization once this error occurs to free some memory.

Just as an additional note, in C these kind of errors can be handled as the example code given below.

int *pointer = malloc(3 * sizeof(int));
if(pointer == NULL) {
    fprintf(stderr, "Out of memory");
    exit(1);
}