Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sample config to run with opensearch #4512

Merged
merged 6 commits into from
Sep 27, 2021
Merged

Conversation

longquanzheng
Copy link
Contributor

@longquanzheng longquanzheng commented Sep 26, 2021

What changed?
Add sample config to run with opensearch

docker config is from https://hub.docker.com/r/opensearchproject/opensearch

Why?
For users who want to use opensearch as a replacement of ElasticSearch

How did you test it?
Local test with visibility sampel:

docker-compose -f docker/dev/cassandra-opensearch-kafka.yml

then install schemas:

make install-schema
make install-schema-es-opensearch

then start server:

./cadence-server --zone es_opensearch start

and register a domain:

cadence --do samples-domain d re

Then start visibility sample workflow worker

./bin/searchattributes -m worker &

Then start sample workflow:

./bin/searchattributes
2021-09-26T10:57:45.639-0700	INFO	common/sample_helper.go:109	Logger created.
2021-09-26T10:57:45.639-0700	DEBUG	common/factory.go:151	Creating RPC dispatcher outbound	{"ServiceName": "cadence-frontend", "HostPort": "127.0.0.1:7933"}
2021-09-26T10:57:45.644-0700	INFO	common/sample_helper.go:161	Domain successfully registered.	{"Domain": "samples-domain"}
2021-09-26T10:57:45.697-0700	INFO	common/sample_helper.go:195	Started Workflow	{"WorkflowID": "searchAttributes_8e0e8988-f3af-4012-bd0f-29af1259ff86", "RunID": "5386a7e7-13bb-4e65-a7a6-e5dacf63a026"}
qlong@~/indeed/cadence-samples:
(qlong-selector-signal-counter-wrong) $2021-09-26T10:57:45.767-0700	INFO	searchattributes/searchattributes_workflow.go:45	SearchAttributes workflow started	{"Domain": "samples-domain", "TaskList": "searchAttributesGroup", "WorkerID": "62351@IT-USA-25920@searchAttributesGroup", "WorkflowType": "main.searchAttributesWorkflow", "WorkflowID": "searchAttributes_8e0e8988-f3af-4012-bd0f-29af1259ff86", "RunID": "5386a7e7-13bb-4e65-a7a6-e5dacf63a026"}
2021-09-26T10:57:45.768-0700	INFO	searchattributes/searchattributes_workflow.go:56	Current Search Attributes: 	{"Domain": "samples-domain", "TaskList": "searchAttributesGroup", "WorkerID": "62351@IT-USA-25920@searchAttributesGroup", "WorkflowType": "main.searchAttributesWorkflow", "WorkflowID": "searchAttributes_8e0e8988-f3af-4012-bd0f-29af1259ff86", "RunID": "5386a7e7-13bb-4e65-a7a6-e5dacf63a026", "CustomIntField": "1"}
2021-09-26T10:57:45.768-0700	INFO	searchattributes/searchattributes_workflow.go:122	Current Search Attributes:
CustomDatetimeField=2019-01-01T00:00:00-08:00
CustomStringField=String field is for text. When query, it will be tokenized for partial match. StringTypeField cannot be used in Order By
CustomIntField=2
CustomKeywordField=Update1
CustomBoolField=true
CustomDoubleField=3.14
	{"Domain": "samples-domain", "TaskList": "searchAttributesGroup", "WorkerID": "62351@IT-USA-25920@searchAttributesGroup", "WorkflowType": "main.searchAttributesWorkflow", "WorkflowID": "searchAttributes_8e0e8988-f3af-4012-bd0f-29af1259ff86", "RunID": "5386a7e7-13bb-4e65-a7a6-e5dacf63a026"}
2021-09-26T10:57:45.768-0700	INFO	searchattributes/searchattributes_workflow.go:122	Current Search Attributes:
CustomIntField=2
CustomKeywordField=Update2
CustomBoolField=true
CustomDoubleField=3.14
CustomDatetimeField=2019-01-01T00:00:00-08:00
CustomStringField=String field is for text. When query, it will be tokenized for partial match. StringTypeField cannot be used in Order By
	{"Domain": "samples-domain", "TaskList": "searchAttributesGroup", "WorkerID": "62351@IT-USA-25920@searchAttributesGroup", "WorkflowType": "main.searchAttributesWorkflow", "WorkflowID": "searchAttributes_8e0e8988-f3af-4012-bd0f-29af1259ff86", "RunID": "5386a7e7-13bb-4e65-a7a6-e5dacf63a026"}
2021-09-26T10:57:45.768-0700	DEBUG	internal/internal_event_handlers.go:571	NewTimer	{"Domain": "samples-domain", "TaskList": "searchAttributesGroup", "WorkerID": "62351@IT-USA-25920@searchAttributesGroup", "WorkflowType": "main.searchAttributesWorkflow", "WorkflowID": "searchAttributes_8e0e8988-f3af-4012-bd0f-29af1259ff86", "RunID": "5386a7e7-13bb-4e65-a7a6-e5dacf63a026", "TimerID": "2", "Duration": "2s"}
2021-09-26T10:57:47.831-0700	DEBUG	internal/internal_event_handlers.go:489	ExecuteActivity	{"Domain": "samples-domain", "TaskList": "searchAttributesGroup", "WorkerID": "62351@IT-USA-25920@searchAttributesGroup", "WorkflowType": "main.searchAttributesWorkflow", "WorkflowID": "searchAttributes_8e0e8988-f3af-4012-bd0f-29af1259ff86", "RunID": "5386a7e7-13bb-4e65-a7a6-e5dacf63a026", "ActivityID": "3", "ActivityType": "main.listExecutions"}
2021-09-26T10:57:47.869-0700	INFO	searchattributes/searchattributes_workflow.go:128	List executions.	{"Domain": "samples-domain", "TaskList": "searchAttributesGroup", "WorkerID": "62351@IT-USA-25920@searchAttributesGroup", "ActivityID": "3", "ActivityType": "main.listExecutions", "WorkflowType": "main.searchAttributesWorkflow", "WorkflowID": "searchAttributes_8e0e8988-f3af-4012-bd0f-29af1259ff86", "RunID": "5386a7e7-13bb-4e65-a7a6-e5dacf63a026", "Query": "CustomIntField=2 and CustomKeywordField='Update2' order by CustomDatetimeField DESC"}
2021-09-26T10:57:48.927-0700	INFO	searchattributes/searchattributes_workflow.go:106	Workflow completed.	{"Domain": "samples-domain", "TaskList": "searchAttributesGroup", "WorkerID": "62351@IT-USA-25920@searchAttributesGroup", "WorkflowType": "main.searchAttributesWorkflow", "WorkflowID": "searchAttributes_8e0e8988-f3af-4012-bd0f-29af1259ff86", "RunID": "5386a7e7-13bb-4e65-a7a6-e5dacf63a026", "Execution": "WorkflowExecutionInfo{Execution: WorkflowExecution{WorkflowId: searchAttributes_8e0e8988-f3af-4012-bd0f-29af1259ff86, RunId: 5386a7e7-13bb-4e65-a7a6-e5dacf63a026}, Type: WorkflowType{Name: main.searchAttributesWorkflow}, StartTime: 1632679065649512000, HistoryLength: 0, ExecutionTime: 1632679065649512000, Memo: Memo{}, SearchAttributes: SearchAttributes{IndexedFields: map[BinaryChecksums:[91 34 51 50 51 56 48 54 51 52 102 54 98 50 48 56 48 52 50 100 50 99 50 97 53 57 52 53 53 54 55 100 56 53 34 93] CustomBoolField:[116 114 117 101] CustomDatetimeField:[34 50 48 49 57 45 48 49 45 48 49 84 48 48 58 48 48 58 48 48 45 48 56 58 48 48 34] CustomDoubleField:[51 46 49 52] CustomIntField:[50] CustomKeywordField:[34 85 112 100 97 116 101 50 34] CustomStringField:[34 83 116 114 105 110 103 32 102 105 101 108 100 32 105 115 32 102 111 114 32 116 101 120 116 46 32 87 104 101 110 32 113 117 101 114 121 44 32 105 116 32 119 105 108 108 32 98 101 32 116 111 107 101 110 105 122 101 100 32 102 111 114 32 112 97 114 116 105 97 108 32 109 97 116 99 104 46 32 83 116 114 105 110 103 84 121 112 101 70 105 101 108 100 32 99 97 110 110 111 116 32 98 101 32 117 115 101 100 32 105 110 32 79 114 100 101 114 32 66 121 34]]}, TaskList: searchAttributesGroup, IsCron: false}"}

Potential risks

Release notes

Documentation Changes

@coveralls
Copy link

coveralls commented Sep 26, 2021

Pull Request Test Coverage Report for Build d01c9cc1-34f7-43fb-a691-bdd9fca1d01f

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 54 unchanged lines in 9 files lost coverage.
  • Overall coverage increased (+0.05%) to 56.48%

Files with Coverage Reduction New Missed Lines %
common/task/weightedRoundRobinTaskScheduler.go 1 89.64%
service/matching/matcher.go 2 91.46%
common/cache/lru.go 3 90.73%
common/persistence/dataManagerInterfaces.go 3 60.91%
common/persistence/nosql/nosqlplugin/cassandra/workflow.go 3 50.23%
service/history/queue/timer_queue_processor_base.go 4 78.83%
service/history/execution/mutable_state_task_refresher.go 7 71.61%
common/persistence/nosql/nosqlplugin/cassandra/workflowUtils.go 12 77.74%
common/persistence/nosql/nosqlplugin/cassandra/workflowParsingUtils.go 19 87.01%
Totals Coverage Status
Change from base Build 595980a1-7fb5-4955-af65-abddbc21db1a: 0.05%
Covered Lines: 80243
Relevant Lines: 142073

💛 - Coveralls

@longquanzheng longquanzheng merged commit 3ab4843 into master Sep 27, 2021
@longquanzheng longquanzheng deleted the qlong-open-search branch September 27, 2021 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants