Apache Ignite 2.9.0 performance

Question

We have setup a ignite cluster 2.9.0 with native persistence enabled in AKS. We wanted to evaluate apache ignite for our analytics scenario. For this we did performance benchmarking to simulate our prod traffic through jmeter with varying parameters for different kinds of sql queries. When we did the benchmarking - we see below graph. Y-axis is query time and X-axis is time. We want to understand - why there are spikes in the graph at regular intervals?

Configuration details -

3 nodes of 16 Gb ram and 3 cores

Default Data region max size - 8 GB per node

Cache Configuration - Partitioned, atomicity mode - atomic, queryParallelism -4, sqlOnheapCacheEnabled, 1 backup

JVM_OPTS = -Xms1g -Xmx1g -server -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC -XX:+AlwaysPreTouch

Approx table size < 1.5 gb

Jmeter requests - 3 concurrent threads continuosly for 5 minutes

Answer 1

These could be due to memory issues, server load, disk latency(w/AKS you are using network mounted disks), etc..

your VM config: JVM_OPTS = -Xms1g -Xmx1g --- you should give the servers more memory. https://ignite.apache.org/docs/latest/perf-and-troubleshooting/memory-tuning
.

You might also consider gathering gc logs and analyzing the output to spot any pauses generated by the GC. https://ignite.apache.org/docs/latest/perf-and-troubleshooting/troubleshooting#detailed-gc-logs

Try running without explicitely setting queryParallelism, as it is an optimizing parameter that needs to be set only if your existing query performance is not optimal. https://ignite.apache.org/docs/latest/SQL/sql-tuning#query-parallelism

Follow this guide to tune your sql queries: https://ignite.apache.org/docs/latest/perf-and-troubleshooting/sql-tuning

Finally, sqlOnheapCacheEnabled is only needed in specialized circumstances. Ignite is specifically designed to store data in off heap to avoid excessive garbage collection and boost performance.

See: https://ignite.apache.org/docs/latest/memory-architecture#memory-architecture