Performance comparison: Spring Boot + Spring Data vs Micronaut + Gorm performance

Update 10/23/2018 12:30 pm PDT:

With the latest release of micronaut v1.0.0, it matches Spring without any changes to bombardier i.e. it's keeping the connection alive by default now.

You can find the latest test output here

Summary:

micronaut with 1.0.0 release beats Spring in startup time and performs same or slightly better in run time with a very simplistic one database entity based test. I would expect micronaut to perform better with it's strategy of providing implementations at compile time vs Spring's run time. I will perform a test with more number of enitites and with GETs and PUTs and provide a link to that article here.

Update 10/23/2018 8:40 PDT:

With the keep-alive connection header while testing with bombardier, micronaut provides the same results as Spring. So the updated comparison table is:

You can find the result output for all of the run(s) here: https://github.com/RaviH/spring-data-micronaut-gorm/tree/master/performance_run_output

If you are interested in latest run with bombardier with keep-alive that output is here

All the result outputs are here

Update 10/23/2018 7:38 PDT:

I have been working with Micronauts team and they have quickly found the reason why bombardier tests provide such different results. bombardier closes the connection after every request and hence the huge discrepancy in the result. To address this Micronaut 1.0.GA would release with Http 1.1 default of keeping the connection open. I will be updating this article as soon as I can get hands on it. I will be updating the article pretty soon here with the results of provoding Connection:keep-alive header with bombardier on Micronaut 1.0.0.RC3

Recently, I came across Micronaut framework and found it to be very interesting. It provides low memory footprint and fast bootup time. I have used Micronaut now for couple non-database backed applications and it starts up really fast and is performing really well. In my performance tests comparing Micronaut app to Spring app, Micronaut performed better (will be covered in a separate post).

The next step for me was to see if I could recommend Micronaut for database backed applications. Micronaut works with straight up hibernate and even provides GORM (a Spring Data alternative). I am already familiar with Spring Data so I am curious about GORM. I decided to give it a spin and see how it holds up compared to Spring Data app.

You can find my repository here: https://github.com/RaviH/spring-data-micronaut-gorm

Details about the test:

a. This is a very basic application with a single domain object: Person.
b. 3 records were inserted into an in-memory H2 database at startup time.
c. Only GETs are tested
d. I haven't tweaked any of the apps for better performance i.e. this is as out of the box as we can get.
e. Spring Boot v2.0.6 is used Micronaut: 1.0.0.RC3 is used.
f. OS: MacOS High Sierra, 16 GB RAM, 2.5 Ghz Intel Core I7
g. mvn and java version:

$ mvn -version
Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-03T12:39:06-07:00)
Maven home: /usr/local/Cellar/maven/3.5.0/libexec
Java version: 1.8.0_162, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"

i. Used different tools (bombardier, ab, loadtest for performance tests since each one of them offers different results.

Results

Tabular comparison: Total time to complete requests = total time

Summary:

• Micronaut has a much faster startup time than Spring.
• With bombardier: Spring data app blows Micronaut app out of the water as far as throughput is concerned.
• With ab: micronaut wins by a slight margin.
• With loadtest: it's too small a difference to decide.

Next Steps
This is probably going to be an in progress article. I will add more information as I gain it.

a. Try a different load test tool.
b. Try loading these applications on a container to take my machine out of the picture.
c. Try a more real world scenario i.e. more entities and real relationships between them, constraints on domain classes.

Please feel free to share your thoughts/comments on the article. Try this out on your machine and feel free to share your results.

Just be nice ;-)