Skip to content

Commit 37d6bb2

Browse files
JaidenAshmoreJaidenAshmore
authored
Merge pull request #108 from JaidenAshmore/issue/36_basic_performance_testing
refs #36: add a module for performance/functionality comparisons
2 parents 9a297f6 + 6917903 commit 37d6bb2

File tree

13 files changed

+712
-1
lines changed

13 files changed

+712
-1
lines changed

‎README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,12 @@ mvn clean install -DskipTests
236236
(cd examples/java-dynamic-sqs-listener-core-examples && mvn exec:java)
237237
```
238238

239+
### Comparing Libraries
240+
If you want to see the difference between this library and others like the
241+
[Spring Cloud AWS Messaging](https://github.com/spring-cloud/spring-cloud-aws/tree/master/spring-cloud-aws-messaging) and
242+
[Amazon SQS Java Messaging Library](https://github.com/awslabs/amazon-sqs-java-messaging-lib), take a look at the [sqs-listener-libary-comparison](./examples/sqs-listener-library-comparison)
243+
module. This allows you to test the performance and usage of each library for different scenarios, such as heavy IO message processing, etc.
244+
239245
### Other examples
240246
See [examples](./examples) for all of the other available examples.
241247

‎examples/pom.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
<module>java-dynamic-sqs-listener-spring-starter-examples</module>
2424
<module>java-dynamic-sqs-listener-spring-aws-example</module>
2525
<module>java-dynamic-sqs-listener-spring-integration-test-example</module>
26+
<module>sqs-listener-library-comparison</module>
2627
</modules>
2728

2829
<dependencies>

‎examples/sqs-listener-library-comparison/README.md‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# SQS Listener Library Comparison
2+
This module has the Spring Cloud, JMS and Java Dynamic SQS Listener libraries all integrated into a Spring Boot application. This can be used
3+
to compare the performance of each of the types of libraries with different types of messages being processed. For example, a message that has a lot of
4+
IO or to test what happens if the time to get new messages from the queue is large.
5+
6+
## Getting Started
7+
You can run the [Application](src/main/java/com/jashmore/sqs/examples/Application.java) which will place messages onto a queue and then the corresponding
8+
listener will process them. Take a look at the [ExampleConstants](src/main/java/com/jashmore/sqs/examples/ExampleConstants.java) for the types of variables
9+
that can be altered, such as the time to process the messages, etc.
10+
11+
## Results (02/June/2019)
12+
The results that I ran on my local machine (which is not the greatest environment but gives a rough estimate) are the following:
13+
14+
### tl;dr
15+
*Ordered from fastest to slowest*
16+
17+
| Method | Concurrency | Time 1000 messages (ms) |
18+
| JMS | 30 | 4889 |
19+
| Java Dynamic SQS Listener (PrefetchingQueueListener) | 30 | 10550 |
20+
| Java Dynamic SQS Listener (PrefetchingQueueListener) | 10 | 10560ms |
21+
| JMS | 10 | 10587 |
22+
| Spring Cloud | 10 | 20284 |
23+
| Java Dynamic SQS Listener (QueueListener) | 30 | 20385 |
24+
| Java Dynamic SQS Listener (QueueListener) | 10 | 20399 |
25+
26+
### Configuration
27+
- Message latency = 100ms
28+
- Message IO time = 100ms
29+
- Commit ID = `a1bdde0d3c96d71997c3001cecc3ddba0d98c709`
30+
31+
### Raw Results
32+
#### JMS (concurrency = 10)
33+
```
34+
23:44:54.720 [DefaultMessageListenerContainer-5] INFO c.j.sqs.examples.MessageListeners - Time for processing 100 messages is 1454ms
35+
23:44:55.738 [DefaultMessageListenerContainer-5] INFO c.j.sqs.examples.MessageListeners - Time for processing 200 messages is 2472ms
36+
23:44:56.754 [DefaultMessageListenerContainer-5] INFO c.j.sqs.examples.MessageListeners - Time for processing 300 messages is 3488ms
37+
23:44:57.765 [DefaultMessageListenerContainer-6] INFO c.j.sqs.examples.MessageListeners - Time for processing 400 messages is 4499ms
38+
23:44:58.779 [DefaultMessageListenerContainer-5] INFO c.j.sqs.examples.MessageListeners - Time for processing 500 messages is 5513ms
39+
23:44:59.788 [DefaultMessageListenerContainer-5] INFO c.j.sqs.examples.MessageListeners - Time for processing 600 messages is 6522ms
40+
23:45:00.798 [DefaultMessageListenerContainer-2] INFO c.j.sqs.examples.MessageListeners - Time for processing 700 messages is 7532ms
41+
23:45:01.812 [DefaultMessageListenerContainer-4] INFO c.j.sqs.examples.MessageListeners - Time for processing 800 messages is 8546ms
42+
23:45:02.834 [DefaultMessageListenerContainer-2] INFO c.j.sqs.examples.MessageListeners - Time for processing 900 messages is 9568ms
43+
23:45:03.853 [DefaultMessageListenerContainer-6] INFO c.j.sqs.examples.MessageListeners - Time for processing 1000 messages is 10587ms
44+
```
45+
46+
#### JMS (concurrency = 30)
47+
```
48+
23:45:56.165 [DefaultMessageListenerContainer-9] INFO c.j.sqs.examples.MessageListeners - Time for processing 100 messages is 1388ms
49+
23:45:56.779 [DefaultMessageListenerContainer-10] INFO c.j.sqs.examples.MessageListeners - Time for processing 200 messages is 2002ms
50+
23:45:57.239 [DefaultMessageListenerContainer-1] INFO c.j.sqs.examples.MessageListeners - Time for processing 300 messages is 2462ms
51+
23:45:57.637 [DefaultMessageListenerContainer-3] INFO c.j.sqs.examples.MessageListeners - Time for processing 400 messages is 2860ms
52+
23:45:57.972 [DefaultMessageListenerContainer-28] INFO c.j.sqs.examples.MessageListeners - Time for processing 500 messages is 3195ms
53+
23:45:58.317 [DefaultMessageListenerContainer-15] INFO c.j.sqs.examples.MessageListeners - Time for processing 600 messages is 3540ms
54+
23:45:58.652 [DefaultMessageListenerContainer-22] INFO c.j.sqs.examples.MessageListeners - Time for processing 700 messages is 3875ms
55+
23:45:58.978 [DefaultMessageListenerContainer-28] INFO c.j.sqs.examples.MessageListeners - Time for processing 800 messages is 4201ms
56+
23:45:59.331 [DefaultMessageListenerContainer-12] INFO c.j.sqs.examples.MessageListeners - Time for processing 900 messages is 4554ms
57+
23:45:59.666 [DefaultMessageListenerContainer-21] INFO c.j.sqs.examples.MessageListeners - Time for processing 1000 messages is 4889ms
58+
```
59+
60+
#### Spring Cloud (concurrency = 10, this is the max)
61+
```
62+
23:48:11.143 [simpleMessageListenerContainer-11] INFO c.j.sqs.examples.MessageListeners - Time for processing 100 messages is 1860ms
63+
23:48:13.201 [simpleMessageListenerContainer-10] INFO c.j.sqs.examples.MessageListeners - Time for processing 200 messages is 3918ms
64+
23:48:15.265 [simpleMessageListenerContainer-4] INFO c.j.sqs.examples.MessageListeners - Time for processing 300 messages is 5982ms
65+
23:48:17.316 [simpleMessageListenerContainer-2] INFO c.j.sqs.examples.MessageListeners - Time for processing 400 messages is 8033ms
66+
23:48:19.363 [simpleMessageListenerContainer-10] INFO c.j.sqs.examples.MessageListeners - Time for processing 500 messages is 10080ms
67+
23:48:21.408 [simpleMessageListenerContainer-3] INFO c.j.sqs.examples.MessageListeners - Time for processing 600 messages is 12125ms
68+
23:48:23.452 [simpleMessageListenerContainer-4] INFO c.j.sqs.examples.MessageListeners - Time for processing 700 messages is 14169ms
69+
23:48:25.491 [simpleMessageListenerContainer-10] INFO c.j.sqs.examples.MessageListeners - Time for processing 800 messages is 16208ms
70+
23:48:27.530 [simpleMessageListenerContainer-2] INFO c.j.sqs.examples.MessageListeners - Time for processing 900 messages is 18247ms
71+
23:48:29.567 [simpleMessageListenerContainer-8] INFO c.j.sqs.examples.MessageListeners - Time for processing 1000 messages is 20284ms
72+
```
73+
74+
#### Jashmore Prefetching (concurrency = 10)
75+
```
76+
23:39:25.302 [message-listeners-prefetching-concurrency10-2] INFO c.j.sqs.examples.MessageListeners - Time for processing 100 messages is 989ms
77+
23:39:26.378 [message-listeners-prefetching-concurrency10-4] INFO c.j.sqs.examples.MessageListeners - Time for processing 200 messages is 2065ms
78+
23:39:27.446 [message-listeners-prefetching-concurrency10-4] INFO c.j.sqs.examples.MessageListeners - Time for processing 300 messages is 3133ms
79+
23:39:28.517 [message-listeners-prefetching-concurrency10-8] INFO c.j.sqs.examples.MessageListeners - Time for processing 400 messages is 4204ms
80+
23:39:29.592 [message-listeners-prefetching-concurrency10-8] INFO c.j.sqs.examples.MessageListeners - Time for processing 500 messages is 5279ms
81+
23:39:30.652 [message-listeners-prefetching-concurrency10-0] INFO c.j.sqs.examples.MessageListeners - Time for processing 600 messages is 6339ms
82+
23:39:31.712 [message-listeners-prefetching-concurrency10-7] INFO c.j.sqs.examples.MessageListeners - Time for processing 700 messages is 7399ms
83+
23:39:32.769 [message-listeners-prefetching-concurrency10-6] INFO c.j.sqs.examples.MessageListeners - Time for processing 800 messages is 8456ms
84+
23:39:33.821 [message-listeners-prefetching-concurrency10-4] INFO c.j.sqs.examples.MessageListeners - Time for processing 900 messages is 9508ms
85+
23:39:34.873 [message-listeners-prefetching-concurrency10-7] INFO c.j.sqs.examples.MessageListeners - Time for processing 1000 messages is 10560ms
86+
```
87+
88+
#### Jashmore Prefetching (concurrency = 30)
89+
```
90+
23:38:28.696 [message-listeners-prefetching-concurrency30-12] INFO c.j.sqs.examples.MessageListeners - Time for processing 100 messages is 972ms
91+
23:38:29.772 [message-listeners-prefetching-concurrency30-16] INFO c.j.sqs.examples.MessageListeners - Time for processing 200 messages is 2048ms
92+
23:38:30.847 [message-listeners-prefetching-concurrency30-15] INFO c.j.sqs.examples.MessageListeners - Time for processing 300 messages is 3123ms
93+
23:38:31.926 [message-listeners-prefetching-concurrency30-13] INFO c.j.sqs.examples.MessageListeners - Time for processing 400 messages is 4202ms
94+
23:38:32.990 [message-listeners-prefetching-concurrency30-17] INFO c.j.sqs.examples.MessageListeners - Time for processing 500 messages is 5266ms
95+
23:38:34.056 [message-listeners-prefetching-concurrency30-17] INFO c.j.sqs.examples.MessageListeners - Time for processing 600 messages is 6332ms
96+
23:38:35.119 [message-listeners-prefetching-concurrency30-16] INFO c.j.sqs.examples.MessageListeners - Time for processing 700 messages is 7395ms
97+
23:38:36.174 [message-listeners-prefetching-concurrency30-12] INFO c.j.sqs.examples.MessageListeners - Time for processing 800 messages is 8450ms
98+
23:38:37.225 [message-listeners-prefetching-concurrency30-16] INFO c.j.sqs.examples.MessageListeners - Time for processing 900 messages is 9501ms
99+
23:38:38.274 [message-listeners-prefetching-concurrency30-14] INFO c.j.sqs.examples.MessageListeners - Time for processing 1000 messages is 10550ms
100+
```
101+
102+
#### Jashmore QueueListener (concurrency = 10)
103+
```
104+
23:40:36.464 [message-listeners-queue-listener-method-concurrency10-6] INFO c.j.sqs.examples.MessageListeners - Time for processing 100 messages is 1889ms
105+
23:40:38.536 [message-listeners-queue-listener-method-concurrency10-8] INFO c.j.sqs.examples.MessageListeners - Time for processing 200 messages is 3961ms
106+
23:40:40.597 [message-listeners-queue-listener-method-concurrency10-2] INFO c.j.sqs.examples.MessageListeners - Time for processing 300 messages is 6022ms
107+
23:40:42.658 [message-listeners-queue-listener-method-concurrency10-8] INFO c.j.sqs.examples.MessageListeners - Time for processing 400 messages is 8083ms
108+
23:40:44.725 [message-listeners-queue-listener-method-concurrency10-6] INFO c.j.sqs.examples.MessageListeners - Time for processing 500 messages is 10150ms
109+
23:40:46.778 [message-listeners-queue-listener-method-concurrency10-9] INFO c.j.sqs.examples.MessageListeners - Time for processing 600 messages is 12203ms
110+
23:40:48.830 [message-listeners-queue-listener-method-concurrency10-1] INFO c.j.sqs.examples.MessageListeners - Time for processing 700 messages is 14255ms
111+
23:40:50.879 [message-listeners-queue-listener-method-concurrency10-4] INFO c.j.sqs.examples.MessageListeners - Time for processing 800 messages is 16304ms
112+
23:40:52.928 [message-listeners-queue-listener-method-concurrency10-1] INFO c.j.sqs.examples.MessageListeners - Time for processing 900 messages is 18353ms
113+
23:40:54.974 [message-listeners-queue-listener-method-concurrency10-8] INFO c.j.sqs.examples.MessageListeners - Time for processing 1000 messages is 20399ms
114+
```
115+
116+
#### Jashmore QueueListener (concurrency = 30)
117+
```
118+
23:41:39.507 [message-listeners-queue-listener-method-concurrency30-7] INFO c.j.sqs.examples.MessageListeners - Time for processing 100 messages is 1875ms
119+
23:41:41.578 [message-listeners-queue-listener-method-concurrency30-4] INFO c.j.sqs.examples.MessageListeners - Time for processing 200 messages is 3946ms
120+
23:41:43.639 [message-listeners-queue-listener-method-concurrency30-2] INFO c.j.sqs.examples.MessageListeners - Time for processing 300 messages is 6007ms
121+
23:41:45.705 [message-listeners-queue-listener-method-concurrency30-7] INFO c.j.sqs.examples.MessageListeners - Time for processing 400 messages is 8073ms
122+
23:41:47.768 [message-listeners-queue-listener-method-concurrency30-7] INFO c.j.sqs.examples.MessageListeners - Time for processing 500 messages is 10136ms
123+
23:41:49.823 [message-listeners-queue-listener-method-concurrency30-6] INFO c.j.sqs.examples.MessageListeners - Time for processing 600 messages is 12191ms
124+
23:41:51.874 [message-listeners-queue-listener-method-concurrency30-1] INFO c.j.sqs.examples.MessageListeners - Time for processing 700 messages is 14242ms
125+
23:41:53.923 [message-listeners-queue-listener-method-concurrency30-5] INFO c.j.sqs.examples.MessageListeners - Time for processing 800 messages is 16291ms
126+
23:41:55.971 [message-listeners-queue-listener-method-concurrency30-8] INFO c.j.sqs.examples.MessageListeners - Time for processing 900 messages is 18339ms
127+
23:41:58.017 [message-listeners-queue-listener-method-concurrency30-2] INFO c.j.sqs.examples.MessageListeners - Time for processing 1000 messages is 20385ms
128+
```

‎examples/sqs-listener-library-comparison/pom.xml‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
3+
<parent>
4+
<artifactId>examples</artifactId>
5+
<groupId>com.jashmore</groupId>
6+
<version>2.3.0-SNAPSHOT</version>
7+
</parent>
8+
<modelVersion>4.0.0</modelVersion>
9+
10+
<artifactId>sqs-listener-library-comparison</artifactId>
11+
12+
<name>Java Dynamic SQS Listener - Spring Starter - Examples</name>
13+
<description>Contains examples for using the Spring Starter implementation of the framework.</description>
14+
15+
<properties>
16+
<findbugs.config.location>../../configuration/findbugs/bugsExcludeFilter.xml</findbugs.config.location>
17+
</properties>
18+
19+
<dependencies>
20+
<dependency>
21+
<groupId>org.springframework.boot</groupId>
22+
<artifactId>spring-boot-starter</artifactId>
23+
<version>${spring.boot.version}</version>
24+
</dependency>
25+
26+
<dependency>
27+
<groupId>org.springframework.boot</groupId>
28+
<artifactId>spring-boot-starter-web</artifactId>
29+
<version>${spring.boot.version}</version>
30+
</dependency>
31+
32+
<dependency>
33+
<groupId>org.projectlombok</groupId>
34+
<artifactId>lombok</artifactId>
35+
<scope>provided</scope>
36+
</dependency>
37+
38+
<dependency>
39+
<groupId>com.jashmore</groupId>
40+
<artifactId>java-dynamic-sqs-listener-spring-starter</artifactId>
41+
<version>${project.version}</version>
42+
</dependency>
43+
44+
<dependency>
45+
<groupId>org.elasticmq</groupId>
46+
<artifactId>elasticmq-rest-sqs_2.11</artifactId>
47+
<version>0.13.9</version>
48+
</dependency>
49+
50+
<dependency>
51+
<groupId>com.jashmore</groupId>
52+
<artifactId>local-amazon-sqs</artifactId>
53+
<version>${project.version}</version>
54+
</dependency>
55+
56+
<dependency>
57+
<groupId>org.springframework.cloud</groupId>
58+
<artifactId>spring-cloud-aws-messaging</artifactId>
59+
<version>2.1.0.RELEASE</version>
60+
</dependency>
61+
62+
<dependency>
63+
<groupId>org.springframework</groupId>
64+
<artifactId>spring-jms</artifactId>
65+
</dependency>
66+
67+
<dependency>
68+
<groupId>com.amazonaws</groupId>
69+
<artifactId>amazon-sqs-java-messaging-lib</artifactId>
70+
<version>1.0.4</version>
71+
<type>jar</type>
72+
</dependency>
73+
</dependencies>
74+
75+
<dependencyManagement>
76+
<dependencies>
77+
<dependency>
78+
<groupId>org.springframework.boot</groupId>
79+
<artifactId>spring-boot-dependencies</artifactId>
80+
<version>${spring.boot.version}</version>
81+
<type>pom</type>
82+
<scope>import</scope>
83+
</dependency>
84+
</dependencies>
85+
</dependencyManagement>
86+
87+
<build>
88+
<resources>
89+
<resource>
90+
<directory>${basedir}/src/main/resources</directory>
91+
<filtering>true</filtering>
92+
<includes>
93+
<include>**/*.yml</include>
94+
<include>**/application*.properties</include>
95+
</includes>
96+
</resource>
97+
<resource>
98+
<directory>${basedir}/src/main/resources</directory>
99+
<excludes>
100+
<exclude>**/*.yml</exclude>
101+
<exclude>**/application*.properties</exclude>
102+
</excludes>
103+
</resource>
104+
</resources>
105+
106+
<pluginManagement>
107+
<plugins>
108+
<plugin>
109+
<groupId>org.springframework.boot</groupId>
110+
<artifactId>spring-boot-maven-plugin</artifactId>
111+
<configuration>
112+
<mainClass>com.jashmore.sqs.examples.Application</mainClass>
113+
</configuration>
114+
<executions>
115+
<execution>
116+
<goals>
117+
<goal>repackage</goal>
118+
</goals>
119+
</execution>
120+
</executions>
121+
</plugin>
122+
<plugin>
123+
<groupId>org.apache.maven.plugins</groupId>
124+
<artifactId>maven-resources-plugin</artifactId>
125+
<configuration>
126+
<delimiters>
127+
<delimiter>${resource.delimiter}</delimiter>
128+
</delimiters>
129+
<useDefaultDelimiters>false</useDefaultDelimiters>
130+
</configuration>
131+
</plugin>
132+
133+
<plugin>
134+
<groupId>org.apache.maven.plugins</groupId>
135+
<artifactId>maven-compiler-plugin</artifactId>
136+
<configuration>
137+
<source>${java.version}</source>
138+
<target>${java.version}</target>
139+
</configuration>
140+
</plugin>
141+
</plugins>
142+
</pluginManagement>
143+
</build>
144+
145+
</project>

‎examples/sqs-listener-library-comparison/src/main/java/com/jashmore/sqs/examples/Application.java‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
package com.jashmore.sqs.examples;
2+
3+
import lombok.extern.slf4j.Slf4j;
4+
import org.springframework.boot.SpringApplication;
5+
import org.springframework.boot.autoconfigure.SpringBootApplication;
6+
7+
@SpringBootApplication
8+
@Slf4j
9+
public class Application {
10+
public static void main(String[] args) {
11+
SpringApplication.run(Application.class);
12+
}
13+
}

‎examples/sqs-listener-library-comparison/src/main/java/com/jashmore/sqs/examples/ExampleConstants.java‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
package com.jashmore.sqs.examples;
2+
3+
import static com.jashmore.sqs.examples.Queues.PREFETCHING_30_QUEUE_NAME;
4+
5+
import lombok.experimental.UtilityClass;
6+
7+
@UtilityClass
8+
public class ExampleConstants {
9+
/**
10+
* The number of messages to place onto the queue before beginning to process the messages.
11+
*/
12+
public static final int NUMBER_OF_MESSAGES = 1_000;
13+
14+
/**
15+
* The amount of time that the thread will be slept while processing the message. This will represent IO.
16+
*/
17+
public static final long MESSAGE_IO_TIME_IN_MS = 500;
18+
19+
/**
20+
* The amount of time it takes to get a message from the remote SQS queue.
21+
*/
22+
public static final long MESSAGE_RETRIEVAL_LATENCY_IN_MS = 200;
23+
24+
/**
25+
* The actual queue that messages will be placed onto during the test.
26+
*/
27+
public static final String QUEUE_TO_TEST = PREFETCHING_30_QUEUE_NAME;
28+
}

0 commit comments

Comments
 (0)