Skip to content

Enable SCRAM-SHA-256 and SCRAM-SHA-512 for sasl - #1918

Merged
dpkp merged 46 commits into
dpkp:masterfrom
real-digital:enable-sasl
Dec 29, 2019
Merged

dpkp merged 46 commits into
dpkp:masterfrom
real-digital:enable-sasl

Conversation

@swenzel

@swenzel swenzel commented Oct 2, 2019 •

Copy link
Copy Markdown
Contributor

closes #1882


This change is Reviewable

@swenzel

swenzel commented Oct 2, 2019

Copy link
Copy Markdown
Contributor Author

Documentation is still missing, I'll add something on Friday.

Swen Wenzel added 29 commits October 16, 2019 15:10
@swenzel

swenzel commented Oct 17, 2019

Copy link
Copy Markdown
Contributor Author

@dpkp @jeffwidman ready to be reviewed now.
That one test that is still failing seems to only fail some of the times. My best guess is that the broker itself hasn't completely fetched the data yet and that a small sleep might fix this.
But I think I have already fixed more than I should 😅
Looking forward to reading your feedback :)

@ofek

ofek commented Nov 25, 2019

Copy link
Copy Markdown
Contributor

@dpkp @jeffwidman We are extremely interested in this as well. Could you please take a look?

@israelglar

Copy link
Copy Markdown

Would be great to have this feature. Is there a timeframe for when we can merge it?

Thanks!

@dpkp dpkp left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much for this PR and the excellent work on test fixtures!

Comment thread .travis.yml
directories:
- $HOME/.cache/pip
- servers/
- servers/dist

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch!

Comment thread kafka/client_async.py
return found

return None
return found

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drive-by!

Comment thread test/__init__.py
pass

logging.getLogger(__name__).addHandler(NullHandler())
logging.basicConfig(level=logging.INFO)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's your thinking on the logging changes here?

Comment thread kafka/conn.py
AUTHENTICATING = '<authenticating>'


class ScramClient:

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts on moving this to its own file ?

Comment thread kafka/conn.py
close = False
else:
try:
client_first = scram_client.first_message().encode()

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we specify an encoding explicitly? is 'utf-8' always the default? does it matter?

Comment thread kafka/conn.py
self._send_bytes_blocking(size + client_first)

(data_len,) = struct.unpack('>i', self._recv_bytes_blocking(4))
server_first = self._recv_bytes_blocking(data_len).decode()

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similarly re: codec. would prefer to specific 'utf-8' explicitly

Comment thread kafka/conn.py
server_first = self._recv_bytes_blocking(data_len).decode()
scram_client.process_server_first_message(server_first)

client_final = scram_client.final_message().encode()

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and here - utf-8

Comment thread kafka/conn.py
self._send_bytes_blocking(size + client_final)

(data_len,) = struct.unpack('>i', self._recv_bytes_blocking(4))
server_final = self._recv_bytes_blocking(data_len).decode()

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and finally also here

Comment thread test/fixtures.py
if sasl_mechanism is not None:
self.sasl_mechanism = sasl_mechanism.upper()
else:
self.sasl_mechanism = None

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the else clause seems redundant?

Comment thread test/fixtures.py
# wait and try again
# on travis the brokers sometimes take a while to find themselves
time.sleep(0.5)
self._create_topic_via_admin_api(topic_name, num_partitions, replication_factor, timeout_ms)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might prefer an outer loop to enable as many retries as needed until timeout

This was referenced Dec 29, 2019
@dpkp
dpkp merged commit ee1c4a4 into dpkp:master Dec 29, 2019
@ofek

ofek commented Jan 9, 2020

Copy link
Copy Markdown
Contributor

@dpkp Hello! Will this be released soon?

@ofek

ofek commented Jan 16, 2020

Copy link
Copy Markdown
Contributor

@dpkp @jeffwidman Any update?

@ofek

ofek commented Jan 16, 2020

Copy link
Copy Markdown
Contributor

Alternatively, can I please do something to help?

@jeffwidman

jeffwidman commented Jan 16, 2020 •

Copy link
Copy Markdown
Contributor

Hey @ofek I don't have the rights to cut a new release, but I just pinged @dpkp and he said on IM that he's currently traveling w/o his laptop for the next week so can't look at it until he returns. Sorry about that.

If you weren't shipping to end-users and just using internally then it'd be trivial to pin to a specific commit hash rather than needing a release... but alas, you are shipping to end users.

@ofek

ofek commented Jan 16, 2020

Copy link
Copy Markdown
Contributor

No worries, thanks!

@ofek

ofek commented Jan 28, 2020

Copy link
Copy Markdown
Contributor

Hello again! Any update?

@jfpatenaude

Copy link
Copy Markdown

I'm also waiting on this feature, @dpkp can you give us an update on the expected timeline?

@swenzel
swenzel deleted the enable-sasl branch March 2, 2020 15:18
@terence-bigtt

Copy link
Copy Markdown

looking forward for that too !

@ofek

ofek commented May 19, 2020

Copy link
Copy Markdown
Contributor

@terence-bigtt This is released.

@Neustradamus

Copy link
Copy Markdown

@swenzel, @dpkp: Thanks a lot for all about SCRAM.

Can you look, a guy has found a bug?

Linked to:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for sasl mechanism: SCRAM-SHA-256

8 participants