Skip to content

Scribe ms #578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 32 commits into
base: main
Choose a base branch
from
Draft

Scribe ms #578

wants to merge 32 commits into from

Conversation

@claudiomansillab
Copy link
Contributor

@claudiomansillab claudiomansillab commented May 23, 2025

No description provided.

claudiomansillab and others added 30 commits May 8, 2025 14:50
@claudiomansillab
correction and tests working
564240a
@claudiomansillab
formatted all files in correction ms and changes workflow to add inte…
9448e66 
…gration test to ci/cd
@claudiomansillab
added config yaml
e4fe761
@claudiomansillab
fixed input_path in config yaml
a5982bf
@claudiomansillab
fixed come variables
7c41b8e
@claudiomansillab
Merge branch 'main' of https://github.com/alercebroker/pipeline into …
082b058 
…feat/correction-ms-ztf-step
@claudiomansillab
added initial files for scribe parser
bf37630
@claudiomansillab
done scribe_parser and added test_correction to test the parser
31982fa
@claudiomansillab
added tests for scribe_parser
eab5c17
@kaymedinam
removed pre prod, added a parser in utils to fulfill function
6b728ed
@claudiomansillab
new config for scribe parser, added test por the scribe parser
122b368
@claudiomansillab
changed schema_path in scribe producer config to relative
58c9cfa
@claudiomansillab
fixed scribe_parser calls in test_db_operations and post_execute in step
79f7347
@claudiomansillab
produce_scribe function fixed
4c3204d
@ignacioreyes
pull from main
9c633d7
@claudiomansillab
step writing messages, step and run_step minor changes
c739d26
@claudiomansillab
Merge branch 'scribe-ms' of https://github.com/alercebroker/pipeline …
03e90cc 
…into scribe-ms
@claudiomansillab
changed path on run_step
6c04d33
@claudiomansillab
formatted with black
7f49e52
@claudiomansillab
added measurement_id in scribe
cd04828
@claudiomansillab
Merge branch 'main' of https://github.com/alercebroker/pipeline into …
34c54bb 
…scribe-ms
@claudiomansillab
added 3 new functions to reuse in parser_sql
6ead22c
@claudiomansillab
Passed the main functions to parser_utils to have parser_sql the smal…
74c72be 
…l posibble. Changes of expected output in test_db_operation. Small changes in json and ztf_messages data
@claudiomansillab
added a drop_db in test, deleted a print in parser_sql and passed scr…
c4062bd 
…ipts thorugh black
@claudiomansillab
deleted drop_db from test_db and added prints to debug in _connection.py
549004d
@claudiomansillab
Merge branch 'main' of https://github.com/alercebroker/pipeline into …
6f7508f 
…scribe-ms
@claudiomansillab
changed name of pk in models ztf_ss and ztf_ps1 for a conflict name
7cc2c85
@claudiomansillab
refactored execute function from step (just order) new functions adde…
70dce2e 
…d to step_utils, added test to do the step refactor, test utils and jsons to run the test.
@claudiomansillab
pulled changes from main
b8e1ee3
@claudiomansillab
changed a yaml path
a23ddb6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@claudiomansillab @kaymedinam @ignacioreyes