Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PIN 5392: Eservice event consumer and tenant event consumer implementation #78

Draft
wants to merge 45 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
a0614f6
Added eservice-consumer, update packages
eduwardo Oct 23, 2024
70f1598
Fix imports
eduwardo Oct 23, 2024
a4e45b7
Removed dismissing microservices
eduwardo Oct 23, 2024
765bae9
Fix lint
eduwardo Oct 23, 2024
bfedc97
Update turbo config cache
eduwardo Oct 24, 2024
284d70c
integration tenant-event-consumer
Oct 24, 2024
37ef501
fix tenantSchema
Oct 24, 2024
bbe4155
integration saveTenant and deleteTenant api in eservice-operation, ch…
Oct 24, 2024
eb1d4de
change test folder tenant event cosumer with removal of generateId fu…
Oct 24, 2024
bbd6bdf
integration delete eservice in to eserviceService
Oct 25, 2024
c173e90
integration packages kafka producer
Oct 25, 2024
d1b2bcd
integration delete eservice and integration kafka producer
Oct 25, 2024
5698732
docker compose
Oct 25, 2024
b01c1a8
Add saveTenant service.
Oct 25, 2024
6676bc7
Edit saveTenant API for pass tenantId inside a param.
Oct 28, 2024
f3ef5cf
integration test for delete eservice
Oct 28, 2024
c67046a
added tenantId as a parameter
diego-lon Oct 28, 2024
576c4bc
get tenant id from params
diego-lon Oct 28, 2024
f1505fe
removed tenantId on payload save tenant
diego-lon Oct 28, 2024
7a5433a
fix unit test
diego-lon Oct 28, 2024
7eb84f6
fix v1 test
diego-lon Oct 28, 2024
2a70aa4
clean env
diego-lon Oct 28, 2024
dc44c31
lint and format
diego-lon Oct 28, 2024
3a8e5b3
fix format check
diego-lon Oct 28, 2024
876a6f1
clean purpose event
diego-lon Oct 28, 2024
ffb6d79
Update eslint consumers
eduwardo Oct 28, 2024
92ceade
Fix consumers Dockerfile
eduwardo Oct 28, 2024
fd3c074
Update eslint-plugin-prettier version
eduwardo Oct 28, 2024
ca59968
Alligned typescript versions
eduwardo Oct 28, 2024
4b295f9
Update packages license
eduwardo Oct 28, 2024
aa3c67a
Removed unused check command
eduwardo Oct 28, 2024
060566d
Update operations envs
eduwardo Oct 28, 2024
110cede
Update root package
eduwardo Oct 29, 2024
083faab
Separated tenant router and service
eduwardo Oct 29, 2024
aa8967a
Removed duplicated saveEservice
eduwardo Oct 29, 2024
2bdac77
Removed duplicated RESTS
eduwardo Oct 29, 2024
3d1800e
Update eserviceRouter
eduwardo Oct 29, 2024
f9f582c
Update tenantRouter
eduwardo Oct 29, 2024
81d400e
Fix deleteEservice
eduwardo Oct 29, 2024
1891100
Added saveTenant, deleteTenant
eduwardo Oct 29, 2024
4ac8d2d
Update tenantQueryBuilder
eduwardo Oct 29, 2024
a6f689a
Update scripts
eduwardo Oct 29, 2024
9107d67
Added tenantNotFound to httpErrorMapper
eduwardo Oct 29, 2024
00e8dbb
added pk on tenants
diego-lon Oct 29, 2024
0baf90a
pk on tenants
diego-lon Oct 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions docker/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
version: "3"
name: pagopa-interop-probing
services:
zookeeper:
image: quay.io/debezium/zookeeper:2.3

kafka:
image: quay.io/debezium/kafka:2.3
ports:
- 9092:9092
- 29092:29092
depends_on:
- zookeeper
environment:
ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_LISTENERS: LISTENER_DOCKER://kafka:29092,LISTENER_HOST://kafka:9092
KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER://kafka:29092,LISTENER_HOST://localhost:9092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER:PLAINTEXT,LISTENER_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_DOCKER
KAFKA_NUM_PARTITIONS: 3
KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"

kafka-ui:
container_name: kafka-ui
image: provectuslabs/kafka-ui:53a6553765a806eda9905c43bfcfe09da6812035
ports:
- 8084:8080
depends_on:
- kafka
environment:
KAFKA_CLUSTERS_0_NAME: local
KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:29092

networks:
develop-network:
name: interop-probing-develop-network
driver: bridge
11 changes: 8 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@
"start:probing-api": "turbo start --filter pagopa-interop-probing-api",
"start:probing-caller": "turbo start --filter pagopa-interop-probing-caller",
"start:probing-eservice-operations": "turbo start --filter pagopa-interop-probing-eservice-operations",
"start:probing-eservice-registry-reader": "turbo start --filter pagopa-interop-probing-eservice-registry-reader",
"start:probing-eservice-registry-updater": "turbo start --filter pagopa-interop-probing-eservice-registry-updater",
"start:probing-response-updater": "turbo start --filter pagopa-interop-probing-response-updater",
"start:probing-scheduler": "turbo start --filter pagopa-interop-probing-scheduler",
"start:probing-statistics-api": "turbo start --filter pagopa-interop-probing-statistics-api",
"start:probing-telemetry-writer": "turbo start --filter pagopa-interop-probing-telemetry-writer",
"start:event:eservice": "turbo start --filter pagopa-interop-probing-eservice-event-consumer",
"start:event:tenant": "turbo start --filter pagopa-interop-probing-tenant-event-consumer",
"test": "turbo test",
"test:it": "turbo test:it",
"build": "turbo build",
Expand All @@ -22,7 +22,12 @@
"format:check": "turbo format:check",
"format:write": "turbo format:write",
"generate-model": "turbo generate-model",
"generate-client": "turbo generate-client"
"generate-client": "turbo generate-client",
"infra:start": "./scripts/infra-start.sh",
"infra:stop": "./scripts/infra-stop.sh",
"infra:destroy": "./scripts/infra-destroy.sh",
"prune-dist": "find . -name 'dist' -type d -exec rm -rf {} +",
"prune-modules": "find . -name 'node_modules' -type d -exec rm -rf {} +"
},
"devDependencies": {
"@tsconfig/node-lts": "20.1.1",
Expand Down
4 changes: 2 additions & 2 deletions packages/clients/eservice-operations/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,6 @@
"@zodios/express": "10.6.1",
"openapi-zod-client": "1.13.4",
"ts-pattern": "5.0.6",
"zod": "3.22.4"
"zod": "3.23.8"
}
}
}
95 changes: 83 additions & 12 deletions packages/clients/eservice-operations/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,18 +45,6 @@ export type ApiUpdateEserviceFrequencyResponse = ZodiosResponseByPath<
"/eservices/:eserviceId/versions/:versionId/updateFrequency"
>;

export type ApiSaveEservicePayload = ZodiosBodyByPath<
Api,
"put",
"/eservices/:eserviceId/versions/:versionId/saveEservice"
>;

export type ApiSaveEserviceResponse = ZodiosResponseByPath<
Api,
"put",
"/eservices/:eserviceId/versions/:versionId/saveEservice"
>;

export type ApiUpdateResponseReceivedPayload = ZodiosBodyByPath<
Api,
"post",
Expand Down Expand Up @@ -158,3 +146,86 @@ export type ApiEserviceMainDataResponse = ZodiosResponseByPath<
"get",
"/eservices/mainData/:eserviceRecordId"
>;

export type ApiSaveEserviceHeaders = ZodiosHeaderParamsByPath<
Api,
"post",
"/eservices/:eserviceId/versions/:versionId/saveEservice"
>;

export type ApiSaveEservicePayload = ZodiosBodyByPath<
Api,
"post",
"/eservices/:eserviceId/versions/:versionId/saveEservice"
>;

export type ApiSaveEserviceParams = ZodiosPathParamsByPath<
Api,
"post",
"/eservices/:eserviceId/versions/:versionId/saveEservice"
>;

export type ApiSaveEserviceResponse = ZodiosResponseByPath<
Api,
"post",
"/eservices/:eserviceId/versions/:versionId/saveEservice"
>;

export type ApiDeleteEserviceHeaders = ZodiosHeaderParamsByPath<
Api,
"delete",
"/eservices/:eserviceId/deleteEservice"
>;

export type ApiDeleteEserviceParams = ZodiosPathParamsByPath<
Api,
"delete",
"/eservices/:eserviceId/deleteEservice"
>;

export type ApiDeleteEserviceResponse = ZodiosResponseByPath<
Api,
"delete",
"/eservices/:eserviceId/deleteEservice"
>;

export type ApiSaveTenantParams = ZodiosPathParamsByPath<
Api,
"post",
"/tenants/:tenantId/saveTenant"
>;
export type ApiSaveTenantHeaders = ZodiosHeaderParamsByPath<
Api,
"post",
"/tenants/:tenantId/saveTenant"
>;

export type ApiSaveTenantPayload = ZodiosBodyByPath<
Api,
"post",
"/tenants/:tenantId/saveTenant"
>;

export type ApiSaveTenantResponse = ZodiosResponseByPath<
Api,
"post",
"/tenants/:tenantId/saveTenant"
>;

export type ApiDeleteTenantHeaders = ZodiosHeaderParamsByPath<
Api,
"delete",
"/tenants/:tenantId/deleteTenant"
>;

export type ApiDeleteTenantParams = ZodiosPathParamsByPath<
Api,
"delete",
"/tenants/:tenantId/deleteTenant"
>;

export type ApiDeleteTenantResponse = ZodiosResponseByPath<
Api,
"delete",
"/tenants/:tenantId/deleteTenant"
>;
6 changes: 3 additions & 3 deletions packages/commons/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
"license": "Apache-2.0",
"dependencies": {
"@aws-sdk/client-sqs": "3.507.0",
"@types/uuid": "9.0.7",
"@types/uuid": "9.0.8",
"@zodios/core": "10.9.6",
"@zodios/express": "10.6.1",
"express-winston": "4.2.0",
Expand All @@ -28,7 +28,7 @@
"ts-pattern": "5.0.6",
"uuid": "10.0.0",
"winston": "3.11.0",
"zod": "3.22.4",
"zod": "3.23.8",
"zod-validation-error": "3.3.0"
},
"devDependencies": {
Expand All @@ -42,4 +42,4 @@
"prettier": "3.2.5",
"typescript": "5.2.2"
}
}
}
6 changes: 6 additions & 0 deletions packages/commons/src/logging/logger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@ export type LoggerMetadata = {
serviceName?: string;
correlationId?: string | null;
messageId?: string | null;
eventType?: string;
eventVersion?: number;
streamId?: string;
version?: number;
eserviceId?: string;
purposeId?: string;
};

const parsedLoggerConfig = LoggerConfig.safeParse(process.env);
Expand Down
12 changes: 12 additions & 0 deletions packages/eservice-event-consumer/.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
APPLICATION_NAME="interop-probing-eservice-event-consumer"
LOG_LEVEL=debug

API_OPERATIONS_BASEURL="http://localhost:3000"

KAFKA_CLIENT_ID="key"
KAFKA_GROUP_ID="2"
KAFKA_BROKERS="localhost:9092"
KAFKA_DISABLE_AWS_IAM_AUTH="true"
KAFKA_TOPIC="eservice"

AWS_REGION="eu-central-1"
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM node:20.11.1-slim@sha256:357deca6eb61149534d32faaf5e4b2e4fa3549c2be610ee1019bf340ea8c51ec as build
FROM node:20.11.1-slim as build

RUN corepack enable

Expand All @@ -7,24 +7,26 @@ COPY package.json /app/
COPY pnpm-lock.yaml /app/
COPY pnpm-workspace.yaml /app/

COPY ./packages/probing-eservice-registry-updater/package.json /app/packages/probing-eservice-registry-updater/package.json
COPY ./packages/eservice-event-consumer/package.json /app/packages/eservice-event-consumer/package.json
COPY ./packages/commons/package.json /app/packages/commons/package.json
COPY ./packages/models/package.json /app/packages/models/package.json
COPY ./packages/clients/eservice-operations/package.json /app/packages/clients/eservice-operations/package.json
COPY ./packages/kafka-connector/package.json /app/packages/kafka-connector/package.json

RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --frozen-lockfile

COPY tsconfig.json /app/
COPY turbo.json /app/
COPY ./packages/probing-eservice-registry-updater /app/packages/probing-eservice-registry-updater
COPY ./packages/probing-eservice-operations/open-api /app/packages/probing-eservice-operations/open-api
COPY ./packages/eservice-event-consumer /app/packages/eservice-event-consumer
COPY ./packages/operations/open-api /app/packages/operations/open-api
COPY ./packages/commons /app/packages/commons
COPY ./packages/kafka-connector /app/packages/kafka-connector
COPY ./packages/models /app/packages/models
COPY ./packages/clients/eservice-operations /app/packages/clients/eservice-operations

RUN pnpm build
RUN rm -rf /app/packages/probing-eservice-registry-updater/dist/test
RUN rm -rf /app/packages/eservice-event-consumer/dist/test

WORKDIR /app/packages/probing-eservice-registry-updater
WORKDIR /app/packages/eservice-event-consumer

CMD [ "node", "." ]
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "pagopa-interop-probing-eservice-registry-reader",
"name": "pagopa-interop-probing-eservice-event-consumer",
"version": "1.0.0",
"description": "PagoPA probing eservice registry reader",
"description": "PagoPA probing eservice event consumer",
"main": "dist",
"type": "module",
"scripts": {
Expand All @@ -17,9 +17,25 @@
"keywords": [],
"author": "",
"license": "Apache-2.0",
"dependencies": {
"@pagopa/interop-outbound-models": "1.0.4",
"@tsconfig/node-lts": "20.1.1",
"@zodios/core": "10.9.6",
"axios": "1.7.4",
"dotenv-flow": "3.3.0",
"kafka-connector": "workspace:*",
"kafkajs": "2.2.4",
"pagopa-interop-probing-commons": "workspace:*",
"pagopa-interop-probing-models": "workspace:*",
"pagopa-interop-probing-eservice-operations-client": "workspace:*",
"ts-pattern": "5.0.6",
"uuid": "10.0.0",
"zod": "3.23.8"
},
"devDependencies": {
"@types/dotenv-flow": "3.3.3",
"@types/node": "20.10.4",
"@types/uuid": "9.0.8",
"@typescript-eslint/eslint-plugin": "6.21.0",
"@typescript-eslint/parser": "6.21.0",
"eslint": "8.56.0",
Expand All @@ -29,16 +45,5 @@
"ts-node": "10.9.2",
"typescript": "5.2.2",
"vitest": "0.33.0"
},
"dependencies": {
"@aws-sdk/client-s3": "3.549.0",
"@tsconfig/node-lts": "20.1.1",
"@types/uuid": "9.0.7",
"dotenv-flow": "3.3.0",
"pagopa-interop-probing-commons": "workspace:*",
"pagopa-interop-probing-models": "workspace:*",
"ts-pattern": "5.0.6",
"uuid": "10.0.0",
"zod": "3.22.4"
}
}
2 changes: 2 additions & 0 deletions packages/eservice-event-consumer/src/handlers/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export * from "./messageHandlerV1.js";
export * from "./messageHandlerV2.js";
Loading