sasl configuration - refresh callback for runtime rediscovery

[pimpelsang]

Hi,

This PR proposes to allow sasl credentials configuration parameter to be callback that returns same sasl options object.

The callback will be called every time a new connection is established sasl authentication is executed, so it allows credentials to be changed during runtime and kafkajs will still reconnect.

Same background as with #854

Example:

const refreshCredentials = async() => {
  const credentials = await myCustomCredentialsService()

  return { 
      username: credentials.username,
      password: credentials.secret,
  }
}

const kafka = new Kafka({
  clientId: 'my-app',
  sasl: {
    mechanism: 'scram-sha-256',
    refresh: refreshCredentials
  }
})

[Nevon]

I had actually completely forgot that I was involved in a similar effort a year back.

I believe a good place to start is by reading https://cwiki.apache.org/confluence/display/KAFKA/KIP-368%3A+Allow+SASL+Connections+to+Periodically+Re-Authenticate, which was implemented in #496. Essentially, when we authenticate we get back a session lifetime in ms. If that amount of time (minus some threshold) has passed, Broker.isConnected will return false, which will trigger a re-authentication.

In parallel, #445 tried to implement pretty much the same thing as this PR. In one of my comments from almost exactly a year ago, I mention the need to separate "getting the current credentials" and "renewing the credentials": #445 (comment) This is similar to how it's handled in the Java client (or in Java in general) through JAAS, where a LoginModule is responsible for creating a credential class which can optionally implement the Refreshable interface, which supports the methods boolean isCurrent() and void refresh().

The comment goes into the details of it, and I think it still stands. The reason being that with the current implementation, the credentials are only renewed when a new connection is created. If the connection already exists, the credentials will not be renewed. This means that it will not play nicely with KIP-368.

So rather than evaluating sasl once when we create the connection, the connection should maintain a reference to the original sasl provider function so that we can renew the credentials when (re-)authenticating.

[pimpelsang]

Hi again, sorry for long delay.

I've updated code so now the sasl update is evaluated in SASLAuthenticator.authenticate() method instead of only once inside connectionBuilder. I didn't go with the SASLProvider interface as having just refresh() callback function looked cleaner

[pimpelsang]

I see that the SASLOptions structure changes per authorization method... how should the rediscover API look like? Is it relevant for all methods or just scram/plain?

[pimpelsang]

I think lets close it for inactivity. Workaround is that you can keep reference to configuration .sasl data object and modify it on runtime.