Skip to content

Commit b6505bf

Browse files
committed
feat: use "project.action." prefix for Kafka events
1 parent 67f97a8 commit b6505bf

File tree

5 files changed

+48
-48
lines changed

5 files changed

+48
-48
lines changed

ReadMe.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
## Dependencies
44

55
- nodejs https://nodejs.org/en/ (v8)
6-
- Kafka
6+
- Kafka
77
- Informix
88
- Postgres
99
- Docker, Docker Compose
@@ -21,9 +21,9 @@ The following parameters can be set in config files or in env variables:
2121
if not provided, then SSL connection is not used, direct insecure connection is used;
2222
if provided, it can be either path to private key file or private key content
2323
- KAFKA_GROUP_ID: the Kafka group id, default value is 'legacy-project-processor'
24-
- CREATE_PROJECT_TOPIC: create project Kafka topic, default value is 'project.notification.create'
25-
- UPDATE_PROJECT_TOPIC: update project Kafka topic, default value is 'project.notification.update'
26-
- DELETE_PROJECT_TOPIC: delete project member Kafka topic, default value is 'project.notification.delete'
24+
- CREATE_PROJECT_TOPIC: create project Kafka topic, default value is 'project.action.create'
25+
- UPDATE_PROJECT_TOPIC: update project Kafka topic, default value is 'project.action.update'
26+
- DELETE_PROJECT_TOPIC: delete project member Kafka topic, default value is 'project.action.delete'
2727
- INFORMIX: Informix database configuration parameters, refer `config/default.js` for more information
2828
- POSTGRES: Postgres database configuration parameters, refer `config/default.js` for more information
2929

@@ -47,20 +47,20 @@ Configuration for the tests is at `config/test.js`, only add such new configurat
4747
`bin/kafka-server-start.sh config/server.properties`
4848
- note that the zookeeper server is at localhost:2181, and Kafka server is at localhost:9092
4949
- use another terminal, go to same directory, create the needed topics:
50-
`bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic project.notification.create`
50+
`bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic project.action.create`
5151

52-
`bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic project.notification.update`
52+
`bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic project.action.update`
5353

54-
`bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic project.notification.delete`
54+
`bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic project.action.delete`
5555

5656
- verify that the topics are created:
5757
`bin/kafka-topics.sh --list --zookeeper localhost:2181`,
5858
it should list out the created topics
59-
- run the producer and then write some message into the console to send to the `project.notification.create` topic:
59+
- run the producer and then write some message into the console to send to the `project.action.create` topic:
6060
in the console, write message, one message per line:
61-
`{ "topic": "project.notification.create", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project", "id": 1000, "name": "Develop website", "description": "<h>Test</h><p>This is description</p>", "type": "Develop website", "directProjectId": null, "billingAccountId": 70015983, "type": "Web Application", "createdBy": 8547899 } }`
61+
`{ "topic": "project.action.create", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project", "id": 1000, "name": "Develop website", "description": "<h>Test</h><p>This is description</p>", "type": "Develop website", "directProjectId": null, "billingAccountId": 70015983, "type": "Web Application", "createdBy": 8547899 } }`
6262
- optionally, use another terminal, go to same directory, start a consumer to view the messages:
63-
`bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic project.notification.create --from-beginning`
63+
`bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic project.action.create --from-beginning`
6464
- writing/reading messages to/from other topics are similar
6565

6666
## Topcoder Informix Database Setup

Verification.md

Lines changed: 19 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -15,45 +15,45 @@ npm run test-data
1515
```
1616

1717
## Verification
18-
1. start kafka-console-producer to write messages to `project.notification.create` topic:
19-
`bin/kafka-console-producer.sh --broker-list localhost:9092 --topic project.notification.create`
18+
1. start kafka-console-producer to write messages to `project.action.create` topic:
19+
`bin/kafka-console-producer.sh --broker-list localhost:9092 --topic project.action.create`
2020
2. write message:
21-
`{ "topic": "project.notification.create", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project", "id": 1000, "name": "Develop website", "description": "<h>Test</h><p>This is description</p>", "directProjectId": null, "billingAccountId": 70015983, "type": "Web Application", "createdBy": 132458 } }`
21+
`{ "topic": "project.action.create", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project", "id": 1000, "name": "Develop website", "description": "<h>Test</h><p>This is description</p>", "directProjectId": null, "billingAccountId": 70015983, "type": "Web Application", "createdBy": 132458 } }`
2222
3. check the app console to verify message has been properly handled.
2323
4. Again, write another message(directProjectId is provided at this time):
24-
`{ "topic": "project.notification.create", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project", "id": 1001, "name": "<h1>Test Project</h1>", "description": "<h>Test</h><p>This is description</p>", "directProjectId": 500, "billingAccountId": null, "type": "Web", "createdBy": 132458 } }`
24+
`{ "topic": "project.action.create", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project", "id": 1001, "name": "<h1>Test Project</h1>", "description": "<h>Test</h><p>This is description</p>", "directProjectId": 500, "billingAccountId": null, "type": "Web", "createdBy": 132458 } }`
2525
5. check the app console to verify message has been properly handled.
2626
6. Try to write an invalid message:
27-
`{ "topic": "project.notification.create", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project", "id": 1001, "name": "<h1>Test Project</h1>", "description": "<h>Test</h><p>This is description</p>", "directProjectId": 500, "billingAccountId": 100, "type": "Web", "createdBy": 132458 } }`
27+
`{ "topic": "project.action.create", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project", "id": 1001, "name": "<h1>Test Project</h1>", "description": "<h>Test</h><p>This is description</p>", "directProjectId": 500, "billingAccountId": 100, "type": "Web", "createdBy": 132458 } }`
2828
7. You will see error message in the app console.
29-
8. start kafka-console-producer to write messages to `project.notification.update` topic:
30-
`bin/kafka-console-producer.sh --broker-list localhost:9092 --topic project.notification.update`
29+
8. start kafka-console-producer to write messages to `project.action.update` topic:
30+
`bin/kafka-console-producer.sh --broker-list localhost:9092 --topic project.action.update`
3131
9. write message:
32-
`{ "topic": "project.notification.update", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project", "id": 1001, "directProjectId": 500, "billingAccountId": 70015984, "updatedBy": 132458 } }`
32+
`{ "topic": "project.action.update", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project", "id": 1001, "directProjectId": 500, "billingAccountId": 70015984, "updatedBy": 132458 } }`
3333
10. check the app console to verify message has been properly handled.
3434
11. Try to write an invalid message:
35-
`{ "topic": "project.notification.update", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project", "id": 1001, "directProjectId": 500, "billingAccountId": 1, "updatedBy": 132458 } }`
35+
`{ "topic": "project.action.update", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project", "id": 1001, "directProjectId": 500, "billingAccountId": 1, "updatedBy": 132458 } }`
3636
12. You will see error message in the app console.
37-
13. start kafka-console-producer to write messages to `project.notification.update` topic:
38-
`bin/kafka-console-producer.sh --broker-list localhost:9092 --topic project.notification.create`
37+
13. start kafka-console-producer to write messages to `project.action.update` topic:
38+
`bin/kafka-console-producer.sh --broker-list localhost:9092 --topic project.action.create`
3939
14. write messages:
40-
`{ "topic": "project.notification.create", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project.member", "projectId": 1001, "userId": 132457, "role": "copilot", "createdBy": 132458 } }`
40+
`{ "topic": "project.action.create", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project.member", "projectId": 1001, "userId": 132457, "role": "copilot", "createdBy": 132458 } }`
4141

42-
`{ "topic": "project.notification.create", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project.member", "projectId": 1001, "userId": 124835, "role": "manager", "createdBy": 132458 } }`
42+
`{ "topic": "project.action.create", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project.member", "projectId": 1001, "userId": 124835, "role": "manager", "createdBy": 132458 } }`
4343

44-
`{ "topic": "project.notification.create", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project.member", "projectId": 1001, "userId": 124836, "role": "account_manager", "createdBy": 132458 } }`
44+
`{ "topic": "project.action.create", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project.member", "projectId": 1001, "userId": 124836, "role": "account_manager", "createdBy": 132458 } }`
4545

4646
15. check the app console to verify messages has been properly handled.
4747
16. Repeat step 14 again.
4848
17. You will see error messages in the app console.
49-
18. start kafka-console-producer to write messages to `project.notification.update` topic:
50-
`bin/kafka-console-producer.sh --broker-list localhost:9092 --topic project.notification.delete`
49+
18. start kafka-console-producer to write messages to `project.action.update` topic:
50+
`bin/kafka-console-producer.sh --broker-list localhost:9092 --topic project.action.delete`
5151
19. write messages:
52-
`{ "topic": "project.notification.delete", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project.member", "projectId": 1001, "userId": 132457, "role": "copilot", "deletedBy": 132458 } }`
52+
`{ "topic": "project.action.delete", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project.member", "projectId": 1001, "userId": 132457, "role": "copilot", "deletedBy": 132458 } }`
5353

54-
`{ "topic": "project.notification.delete", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project.member", "projectId": 1001, "userId": 124835, "role": "manager", "deletedBy": 132458 } }`
54+
`{ "topic": "project.action.delete", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project.member", "projectId": 1001, "userId": 124835, "role": "manager", "deletedBy": 132458 } }`
5555

56-
`{ "topic": "project.notification.delete", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project.member", "projectId": 1001, "userId": 124836, "role": "account_manager", "deletedBy": 132458 } }`
56+
`{ "topic": "project.action.delete", "originator": "project-api", "timestamp": "2018-07-02T00:00:00", "mime-type": "application/json", "payload": { "resource": "project.member", "projectId": 1001, "userId": 124836, "role": "account_manager", "deletedBy": 132458 } }`
5757

5858
20. check the app console to verify messages has been properly handled.
5959
21. Repeat step 14 again.

config/default.js

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,9 @@ module.exports = {
1414
// Kafka group id
1515
KAFKA_GROUP_ID: process.env.KAFKA_GROUP_ID || 'legacy-project-processor',
1616

17-
CREATE_PROJECT_TOPIC: process.env.CREATE_PROJECT_TOPIC || 'project.notification.create',
18-
UPDATE_PROJECT_TOPIC: process.env.UPDATE_PROJECT_TOPIC || 'project.notification.update',
19-
DELETE_PROJECT_TOPIC: process.env.DELETE_PROJECT_TOPIC || 'project.notification.delete',
17+
CREATE_PROJECT_TOPIC: process.env.CREATE_PROJECT_TOPIC || 'project.action.create',
18+
UPDATE_PROJECT_TOPIC: process.env.UPDATE_PROJECT_TOPIC || 'project.action.update',
19+
DELETE_PROJECT_TOPIC: process.env.DELETE_PROJECT_TOPIC || 'project.action.delete',
2020

2121
// informix database configuration
2222
INFORMIX: {

test/processor.test.js

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -190,7 +190,7 @@ describe('Topcoder - Legacy Project Processor E2E Test', () => {
190190
}
191191
})
192192
await waitJob()
193-
should.equal(errorLogs[0], 'The message topic invalid doesn\'t match the Kafka topic project.notification.create.')
193+
should.equal(errorLogs[0], 'The message topic invalid doesn\'t match the Kafka topic project.action.create.')
194194
})
195195

196196
it('failure - processor create project with nonexistent billing account ', async () => {

test/testData.js

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ const testTopics = {
1010
integerFields: ['payload.id', 'payload.directProjectId', 'payload.billingAccountId', 'payload.createdBy'],
1111
testMessages: [
1212
{
13-
topic: 'project.notification.create',
13+
topic: 'project.action.create',
1414
originator: 'project-api',
1515
timestamp: '2018-07-02T00:00:00',
1616
'mime-type': 'application/json',
@@ -26,7 +26,7 @@ const testTopics = {
2626
}
2727
},
2828
{
29-
topic: 'project.notification.create',
29+
topic: 'project.action.create',
3030
originator: 'project-api',
3131
timestamp: '2018-07-02T00:00:00',
3232
'mime-type': 'application/json',
@@ -42,7 +42,7 @@ const testTopics = {
4242
}
4343
},
4444
{
45-
topic: 'project.notification.create',
45+
topic: 'project.action.create',
4646
originator: 'project-api',
4747
timestamp: '2018-07-02T00:00:00',
4848
'mime-type': 'application/json',
@@ -66,7 +66,7 @@ const testTopics = {
6666
integerFields: ['payload.id', 'payload.directProjectId', 'payload.billingAccountId'],
6767
testMessages: [
6868
{
69-
topic: 'project.notification.update',
69+
topic: 'project.action.update',
7070
originator: 'project-api',
7171
timestamp: '2018-07-02T00:00:00',
7272
'mime-type': 'application/json',
@@ -79,7 +79,7 @@ const testTopics = {
7979
}
8080
},
8181
{
82-
topic: 'project.notification.update',
82+
topic: 'project.action.update',
8383
originator: 'project-api',
8484
timestamp: '2018-07-02T00:00:00',
8585
'mime-type': 'application/json',
@@ -92,7 +92,7 @@ const testTopics = {
9292
}
9393
},
9494
{
95-
topic: 'project.notification.update',
95+
topic: 'project.action.update',
9696
originator: 'project-api',
9797
timestamp: '2018-07-02T00:00:00',
9898
'mime-type': 'application/json',
@@ -105,7 +105,7 @@ const testTopics = {
105105
}
106106
},
107107
{
108-
topic: 'project.notification.update',
108+
topic: 'project.action.update',
109109
originator: 'project-api',
110110
timestamp: '2018-07-02T00:00:00',
111111
'mime-type': 'application/json',
@@ -125,7 +125,7 @@ const testTopics = {
125125
integerFields: ['payload.projectId', 'payload.userId', 'payload.createdBy'],
126126
testMessages: [
127127
{
128-
topic: 'project.notification.create',
128+
topic: 'project.action.create',
129129
originator: 'project-api',
130130
timestamp: '2018-07-02T00:00:00',
131131
'mime-type': 'application/json',
@@ -138,7 +138,7 @@ const testTopics = {
138138
}
139139
},
140140
{
141-
topic: 'project.notification.create',
141+
topic: 'project.action.create',
142142
originator: 'project-api',
143143
timestamp: '2018-07-02T00:00:00',
144144
'mime-type': 'application/json',
@@ -151,7 +151,7 @@ const testTopics = {
151151
}
152152
},
153153
{
154-
topic: 'project.notification.create',
154+
topic: 'project.action.create',
155155
originator: 'project-api',
156156
timestamp: '2018-07-02T00:00:00',
157157
'mime-type': 'application/json',
@@ -164,7 +164,7 @@ const testTopics = {
164164
}
165165
},
166166
{
167-
topic: 'project.notification.create',
167+
topic: 'project.action.create',
168168
originator: 'project-api',
169169
timestamp: '2018-07-02T00:00:00',
170170
'mime-type': 'application/json',
@@ -177,7 +177,7 @@ const testTopics = {
177177
}
178178
},
179179
{
180-
topic: 'project.notification.create',
180+
topic: 'project.action.create',
181181
originator: 'project-api',
182182
timestamp: '2018-07-02T00:00:00',
183183
'mime-type': 'application/json',
@@ -198,7 +198,7 @@ const testTopics = {
198198
integerFields: ['payload.projectId', 'payload.userId', 'payload.deletedBy'],
199199
testMessages: [
200200
{
201-
topic: 'project.notification.delete',
201+
topic: 'project.action.delete',
202202
originator: 'project-api',
203203
timestamp: '2018-07-02T00:00:00',
204204
'mime-type': 'application/json',
@@ -211,7 +211,7 @@ const testTopics = {
211211
}
212212
},
213213
{
214-
topic: 'project.notification.delete',
214+
topic: 'project.action.delete',
215215
originator: 'project-api',
216216
timestamp: '2018-07-02T00:00:00',
217217
'mime-type': 'application/json',
@@ -224,7 +224,7 @@ const testTopics = {
224224
}
225225
},
226226
{
227-
topic: 'project.notification.delete',
227+
topic: 'project.action.delete',
228228
originator: 'project-api',
229229
timestamp: '2018-07-02T00:00:00',
230230
'mime-type': 'application/json',

0 commit comments

Comments
 (0)