using GraalVM & AWS Lambda in Java for cold start problems

EPAM Anywhere isn't just a platform with remote IT jobs. It's a vibrant and supportive community of software engineers, quality assurance specialists, UX designers, and other professionals. We’ve never found an issue they didn't know a solution to — our tutorials library speaks for itself! 

For this tutorial, we asked Aleksandr Filichkin, Lead Software Engineer and Technical Lead at EPAM Anywhere, to share his practical experience with Java and AWS. Aleksandr is a big AWS fan with five years of production experience and AWS certification. 

He'll explain how to use GraalVM and AWS Lambda in Java to solve the cold start problem.

We'll test a REST service that saves queries to a DynamoDB database, using AWS Lambda. It's a viable solution since AWS Lambda offers terrific scalability and Java support. Besides, we only have to pay for usage, which is budget-friendly. 

Now let's see how we can use AWS Lambda with GraalVM to resolve Java’s cold start problem and significantly improve performance.

AWS Lambda is a compute service that allows you to run code for almost any type of application or backend service and handles all of the administration processes, which includes the maintenance of server and operating system. It's a budget-friendly solution because AWS Lambda only charges you for the time when the service is running. In addition, AWS Lambda offers great speed and performance capabilities, and seamlessly integrates with other AWS services.

Here's a scheme of the AWS Lambda operating cycle:

Version 1: plain Java w/o improvements

• Java 11
• AWS SDK-V2 for DynamoDB (extended DynamoDb client)
• No DI (Spring, Dagger, etc)
• No special frameworks

Check the code implementation on GitHub.

The handler is LambdaV1.java

Result:

• Duration: 10845.21 ms
• Billed Duration: 10846 ms
• Memory: 256 MB
• Max Memory Used: 168 MB
• Init Duration: 2650.86 ms

Version 2: plain Java with improvements

• Java 11
• AWS SDK-V2 for Dynamodb
• No DI (Spring, etc)
• No special frameworks
• Utilize CPU burst on startup (move everything to static, warm-up DynamoDB client)
• Reduce dependencies (exclude Netty)
• Specify AWS Regions
• Specify Credential Provider

Code implementation

Also, check out the handler LabmdaV2.java

Result:

• Duration: 4037.08 ms
• Billed Duration: 4038 ms
• Memory: 256 MB
• Max Memory Used: 170 MB
• Init Duration: 3604.04 ms

As you can see, the billable time has been reduced by 2.5 times. Let's review our final solution with AWS Lambda custom runtime and GraalVM.

What is AWS Lambda Custom runtime?

AWS Lambda Custom runtime was introduced in 2018. The runtime comes in a function.zip file with a bootstrap shell script or binary executable file (compiled for Amazon Linux). You can implement runtime in any programming language and include it in your function's deployment package in the form of an executable file.

GraalVM is a Java and JDK virtual machine based on HotSpot/OpenJDK. Out of the box, the tool offers fast Java execution, execution of programs written in platform-dependent languages, support for multiple programming languages, JVM application augmentation, and more.

GraalVM comes in two editions:

Community edition. This version is available for free, and you can use it even in commercial projects. It's built from the GraalVM sources available on GitHub. The community edition provides distributions based on OpenJDK 11 for Linux, macOS, and Windows platforms on x86 64-bit systems, and for Linux on ARM 64-bit systems.

Enterprise edition. This edition provides additional performance, security, and scalability. It's an optimal choice for applications in production.

AOT vs JIT: Startup Time

JIT:

• Load JVM executables
• Load classes from the file system
• Verify bytecodes
• Start interpreting
• Run static initializers
• First-tier compilation
• Gather profiling feedback
• Second tier compilation (C2 or GraalVM)
• Finally, run with the best machine code

AOT:

• Load executable with a prepared heap
• Immediately start with the best machine code

Version 3: AWS Lambda Custom Runtime + GraalVM

Check the code implementation here.

To build a native binary, let's use Docker:  GraalVM-Dockerfile

Result:

• Duration: 372.73 ms
• Billed Duration: 704 ms
• Memory: 256 MB
• Max Memory Used: 90 MB
• Init Duration: 330.61 ms

GraalVM Native drawbacks:

Even though GraalVM can be a lifesaver when it comes to Java's cold start problem, it comes with some limitations, including:

• Manual/explicit mapping for reflections
• Not all libraries can be compiled (closed-world assumption)
• Slow (CPU intensive) build time
• Large size of binary file
• Only Serial GC is supported for the GraalVM CE version

Useful GraalVM tips:

• Use JVM agentlib to track all usages of dynamic features of an execution on a regular Java VM:$JAVA_HOME/bin/java -agentlib:native-image-agent=config-output-dir=path
• Use Dashboard to analyze the binary file 
• For logger, use slf4j-simple
• Use UPX if the binary file is big (AWS Lambda limit is 250 MB)
• Use Quarkus, Micronaut, etc.

Now let's summarize our tutorial. As you can see, GraalVM solves the Java cold start issue with its amazing performance capabilities. However, we can achieve great results only with additional explicit configuration. GraalVM also comes with several limitations. For example, it compiles with a limited list of libraries and isn't suitable for large enterprise projects.

As for the warm-up state, Aleksandr says he has sent approximately 10,000 requests to the single instance of Lambda V2 (Java optimized) and Lambda V3 (GraalVM). GraalVM has constant great performance with approximately 7ms. Java demonstrates inferior performance at the beginning and then becomes ~15 ms, which looks like the Second-tier JIT optimization wasn't applied.

Useful links

AWS Lambda battle: x86 vs ARM(Graviton2) by Aleksandr Filichkin

Repository for the tutorial on GitHub

Native Image documentation

GraalVM Slack Channel

GraalVM native images explained by Oleg Šelajev

GraalVM Native Image Support in the AWS SDK for Java 2.x

Custom AWS Lambda runtimes