Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nsq_to_nsq: multiple topics support #945

Merged
merged 2 commits into from
Sep 25, 2017

Conversation

ploxiln
Copy link
Member

@ploxiln ploxiln commented Sep 25, 2017

my proposed changes to #912

I squashed #912 down to one commit, then added my proposed changes, mostly code-style. See my commit messages. If we agree on these changes, I'll squash most of my changes (all but the last commit) into @jlr52's commit (which will keep his authorship).

The biggest change is to re-inline commandLineValidation() and initConsumerAndHandler() into main() - this is admittedly subjective, but the reasoning is that it reduces the diff against master to about half of what #912 was.

I'll do my own quick manual tests now (but I'm confident :). Take a quick look and let me know what you think @jlr52 @mreiferson

@ploxiln
Copy link
Member Author

ploxiln commented Sep 25, 2017

Tested locally with a few nsqd, topics, and made-up messages, both with and without --destination-topic. Seems to work :)

@mreiferson
Copy link
Member

LGTM, thanks

@mreiferson
Copy link
Member

FYI, GitHub added a feature where maintainers can push to PR branches, see https://github.com/blog/2247-improving-collaboration-with-forks.

I've been using this to cleanup commits for recently merged PRs from contributors and I think it would have worked for you on #912 @ploxiln.

@ploxiln
Copy link
Member Author

ploxiln commented Sep 25, 2017

Ah, I didn't think of that, will try that next time

jlr52 and others added 2 commits September 25, 2017 14:14
Multiple --topic flags can be passed.

If --destination-topic is specified, all consumed topics will be
published to that single destination-topic name at the destination.
If --destination-topic is *not* specified, all consumed topics will
be published to the corresponding topic name topic at the destination.
and rename "publisherHandlerRef" to "publisher"
@mreiferson
Copy link
Member

@ploxiln interested in applying the same multi-topic changes to nsq_to_http? It might benefit from the Go http client changes too.

@ploxiln
Copy link
Member Author

ploxiln commented Sep 26, 2017

Not anytime soon, I just want to finish what I've started (I haven't forgotten the "contrib modules" thing :)

nsq_to_http uses http_api.NewDeadlineTransport() so it'll get the http client transport updates when they're merged. I don't think we make custom Transports anywhere besides that helper function.

@jlr52
Copy link
Contributor

jlr52 commented Oct 4, 2017

@ploxiln Thanks for cleaning up the diff, I definitely forgot to change that back. I will make sure I remember not to increase the diff next time.

@mreiferson @ploxiln Can I take a stab on supporting multi topic for nsq_to_http ?

@mreiferson
Copy link
Member

@jlr52 go for it!

@ploxiln ploxiln deleted the nsq_to_nsq_multi branch March 6, 2018 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants