Authentication and authorization of pub/sub

[kleunen]

I have working:

• username/password login
• anonymous login
• auth sub topics
• auth pub topics
• password hashing [ hash(sha256 : salt : password) ]
• unit tests working (they use the default auth configuration: anonymous user allowed to pub/sub #)
  https://github.com/redboltz/mqtt_cpp/pull/896/files#diff-44ba049abad6813622f466f2db254566065420da4d9a478215ec06979243b7eaR130-R147
• certificate based auth

Still to do:

• Performance optimization?

[kleunen]

I closed the PR because it seems the build / checks seem to get triggered on your account as well.
And the build takes a long time now it seems ..

[codecov]

Codecov Report

Merging #896 (9da0a15) into master (f1ebfbf) will increase coverage by 0.17%.
The diff coverage is 85.76%.

@@            Coverage Diff             @@
##           master     #896      +/-   ##
==========================================
+ Coverage   83.05%   83.23%   +0.17%     
==========================================
  Files          63       64       +1     
  Lines        9929    10310     +381     
==========================================
+ Hits         8247     8582     +335     
- Misses       1682     1728      +46     

[kleunen]

@redboltz Progress on the authentication/authorization is quite good. I would like to implement also the certificate based authentication. But for this, i would need access to the asio ssl socket to enable the certificate callback on a specific connection (set_verify_mode / set_verify_callback).

How can I configure this callback for a specific connection ?

[redboltz]

@redboltz Progress on the authentication/authorization is quite good.

Sounds good!

I would like to implement also the certificate based authentication. But for this, i would need access to the asio ssl socket to enable the certificate callback on a specific connection (set_verify_mode / set_verify_callback).

How can I configure this callback for a specific connection ?

I think that "for a specific connection" is difficult.
Here is an example code for server that supports client_cert authentication.

mqtt_cpp/example/tls_both_client_cert.cpp

Lines 347 to 376 in bed6d65

	ctx.set_verify_mode(MQTT_NS::tls::verify_peer);
	ctx.load_verify_file(base + "cacert.pem");
	ctx.set_verify_callback(
	[]
	(bool preverified,
	boost::asio::ssl::verify_context& ctx) {
	std::cout << "[clicrt] preverified:" << std::boolalpha << preverified << std::endl;
	if (!preverified) return false;
	int depth = X509_STORE_CTX_get_error_depth(ctx.native_handle());
	std::cout << "[clicrt] depth:" << depth << std::endl;
	if (depth > 0) return true;
	X509* cert = X509_STORE_CTX_get_current_cert(ctx.native_handle());
	X509_NAME* name = X509_get_subject_name(cert);
	std::string cname;
	cname.resize(0xffff);
	auto size = X509_NAME_get_text_by_NID(name, NID_commonName, &cname[0], static_cast<int>(cname.size()));
	cname.resize(static_cast<std::size_t>(size));
	std::cout << "[clicrt] CNAME:" << cname << std::endl;
	return true;
	}
	);
	boost::asio::io_context iocs;
	auto s = MQTT_NS::server_tls<>(
	boost::asio::ip::tcp::endpoint(
	boost::asio::ip::tcp::v4(),
	port
	),
	std::move(ctx),
	iocs
	);

The server applies client_cert authentication for all incoming connections.
I think that there are some choices.

1. Client two more servers for mqtts_client_cert_auth and wss_client_cert_auth. It requires two more ports to listen.
2. First, appliy client_cert authenticasion for all connections. And then, the authentication returns a specific error (e.g. no client_cert set), then move to the normal (MQTT username, password) authentication.

I think that 2 is better. But I'm not sure it is possible.
The 2 is enabled only if the broker option like enable_client_cert_auth is set.

[kleunen]

You can also set the verify_callback on a specific stream:
https://www.boost.org/doc/libs/1_77_0/doc/html/boost_asio/reference/ssl__stream/set_verify_callback.html

That way you can use the certificate on the specific stream. I know this is possible, i have done this in another (not opensource) project. Users would login based on their client client certificate. However I need access to ssl_stream after the server has accepted the new connection.

[redboltz]

I'm not sure the following getter can be used for the purpose:

mqtt_cpp/include/mqtt/server.hpp

Lines 351 to 365 in 6c76ccb

	/**
	* @brief Get boost asio ssl context.
	* @return ssl context
	*/
	tls::context& get_ssl_context() {
	return ctx_;
	}
	/**
	* @brief Get boost asio ssl context.
	* @return ssl context
	*/
	tls::context const& get_ssl_context() const {
	return ctx_;
	}

If it is possible, how to decide? Server has no information about client_cert or not.

[kleunen]

If it is possible, how to decide? Server has no information about client_cert or not.

After the accept of the connection has occurred:
https://github.com/redboltz/mqtt_cpp/blob/master/example/broker.cpp#L95-L97

The set_verify_callback needs to be called on the tls stream of the specific connection.
https://www.boost.org/doc/libs/1_77_0/doc/html/boost_asio/reference/ssl__stream/set_verify_callback.html

This should be done before the handshake is performed on the connection:

mqtt_cpp/include/mqtt/server.hpp

Lines 399 to 411 in abd28e4

	ps->async_handshake(
	tls::stream_base::server,
	[this, socket = force_move(socket), tim, underlying_finished, &ioc_con]
	(error_code ec) mutable {
	*underlying_finished = true;
	tim->cancel();
	if (ec) {
	return;
	}
	auto sp = std::make_shared<endpoint_t>(ioc_con, force_move(socket), version_);
	if (h_accept_) h_accept_(force_move(sp));
	}
	);

So I guess I need to add in server.hpp, i may understand now.

[redboltz]

I don't understand what you mean yet.
As I mentioned the example, server can client_cert auth that way for all connections. Do you agree with that?
Or the example way is not enough. Do you mean for all client cert authentication, the example way doesn't work?
I think that for all client_cert authentication, you only need to add set_verify_file at

mqtt_cpp/example/broker.cpp

Line 19 in bed6d65

boost::asio::ssl::context init_ctx()

Am I missing something?

---

I think that you want to implement mixed TLS connections.
One is TLS only for server certification and encryption, not client_cert authentication, then on MQTT CONNECT phase, UserName and Password are used for authentication.
The other is client_cert authentication.

How to mix the two authentication ?? My question is that.
client_cert authentication is done at the earlier phase than MQTT connection. So at the client_cert phase.

[redboltz]

Maybe I understand what is the reason of confusion.

I guess you think that load_verify_file should be set for each incoming connections.
I don't think so. It is enough to set the server (acceptor) because the server pass the copy of context.
The parameter ctx is store as ctx_ by server and ctx_ is passed for each incoming socket at

mqtt_cpp/include/mqtt/server.hpp

Line 358 in bed6d65

auto socket = std::make_shared<socket_t>(ioc_con_, ctx_);

https://github.com/redboltz/mqtt_cpp/blob/bed6d658ad3502caebc532c01ac397553f78ef7e/example/tls_both_client_cert.cpp
This is the example I mentioned. It is not only for the first connection but also 2nd 3rd... all connections.

[kleunen]

The actual client socket is part of boost::asio::ssl::verify_context in the callback of the server?

[kleunen]

How to mix the two authentication ?? My question is that.
client_cert authentication is done at the earlier phase than MQTT connection. So at the client_cert phase.

My idea was to pass on the CNAME as the username to the mqtt login.
But indeed, i have no idea how to do this yet ...

[redboltz]

The actual client socket is part of boost::asio::ssl::verify_context in the callback of the server?

It's https://www.boost.org/doc/libs/1_77_0/doc/html/boost_asio/reference/ssl__stream.html
Constructed by https://www.boost.org/doc/libs/1_77_0/doc/html/boost_asio/reference/ssl__stream/stream/overload1.html
So the stream has ctx (or reference of ctx). I read the code. It has native_handle() of the ctx but it is implementation detail.
And it is used by https://www.boost.org/doc/libs/1_77_0/doc/html/boost_asio/reference/ssl__stream/set_verify_callback.html

[redboltz]

How to mix the two authentication ?? My question is that.
client_cert authentication is done at the earlier phase than MQTT connection. So at the client_cert phase.

My idea was to pass on the CNAME as the username to the mqtt login. But indeed, i have no idea how to do this yet ...

Yes some of customizable field (default could be CNAME) is used as UserName in the broker.
And the UserName is regard to be authorized. That is OK.
If client don't want to user client_cert authentication but use UserName and Password authentication on TLS connection, then the client doesn't set ClientCert file (information).
I guess that there is some way to know that. The worst scenario, we can not distinguish invalid client certicfiate and no client certificate.
Hopefully, only on no client certificate, then continue UserName and Password authentication.
If client certificate is invalid, just reject the connection.

If there is no way to distinguish them, the solution is always continue UserName and Password authentication.
I think that it is acceptable compromisation.

[kleunen]

How to mix the two authentication ?? My question is that.
client_cert authentication is done at the earlier phase than MQTT connection. So at the client_cert phase.

My idea was to pass on the CNAME as the username to the mqtt login. But indeed, i have no idea how to do this yet ...

Yes some of customizable field (default could be CNAME) is used as UserName in the broker. And the UserName is regard to be authorized. That is OK. If client don't want to user client_cert authentication but use UserName and Password authentication on TLS connection, then the client doesn't set ClientCert file (information). I guess that there is some way to know that. The worst scenario, we can not distinguish invalid client certicfiate and no client certificate. Hopefully, only on no client certificate, then continue UserName and Password authentication. If client certificate is invalid, just reject the connection.

If there is no way to distinguish them, the solution is always continue UserName and Password authentication. I think that it is acceptable compromisation.

But how to get the CNAME into broker, such that is used as default value for login ?
https://github.com/redboltz/mqtt_cpp/pull/896/files#diff-8438e6574a9cbca06fc27190e2dab4239f23e5e53b736428ad7b1d37dec831e7R1052-R1056

Using map with socket as key? (socket -> cname)

[redboltz]

Now, I understand about the difficulty.
Server certification and encryption are in common process for all clients.
But client certification and getting CNAME from the connection is different.
At the context set_verify_callback(), we can access only this (boost::asio::ssl::context), preverified, and ctx (boost::asio::ssl::verify_context).

boost::asio::ssl::verify_context provides ony native_handle() that is implementation defined.
https://www.boost.org/doc/libs/1_77_0/doc/html/boost_asio/reference/ssl__verify_context.html

boost::asio::ssl::context has more member functions but I couldn't find the key candidate.
https://www.boost.org/doc/libs/1_77_0/doc/html/boost_asio/reference/ssl__context.html

I think that we need to add something to server_tls and server_tls_ws.

They stores context.

mqtt_cpp/include/mqtt/server.hpp

Line 427 in 7ac3e0c

tls::context ctx_;

class server_tls {
public:
    using verify_callback_t = 
        std::function<void(
            bool,
            boost::asio::ssl::verify_context&,
            socket_t // additional parameter
        >;
    void set_verify_callback(verify_callback_t cb) {
        verify_cb_ = force_move(cb); // just store the callback
    }

private:
    verify_callback_t verify_cb_;
}

And at

mqtt_cpp/include/mqtt/server.hpp

Line 371 in 7ac3e0c

auto socket = std::make_shared<socket_t>(ioc_con, ctx_);

        auto socket = std::make_shared<socket_t>(ioc_con, ctx_);
        ctx_.set_verify_callback(
            [this, socket] // copy capture socket shared_ptr
            (bool preverified, boost::asio::ssl::verify_context& ctx) {
                if (verify_cb_) {
                    return verify_cb_(preverified, ctx, force_move(socket));
                }
                else {
                    return false;
                }
            }
        );
    }

Now, broker can access to the socket (shared_ptr).

I think that this is similar approach you mentioned (I'm not 100% sure)

[redboltz]

I noticed that even if we provide set_verify_callback wrapper (socket parameter version), the callback setter (broker) needs to find corresponding endpoint by the socket. Maybe map is needed. It is not efficient.

I have a question.
The following code there are ctx and vctx (I renamed).

    boost::asio::ssl::context  ctx(boost::asio::ssl::context::tlsv12);
    // ...
    ctx.set_verify_callback(
        []
        (bool preverified,
         boost::asio::ssl::verify_context& vctx) {
            std::cout << "[clicrt] preverified:" << std::boolalpha <<  preverified << std::endl;
            if (!preverified) return false;
            int depth = X509_STORE_CTX_get_error_depth(vctx.native_handle());
            std::cout << "[clicrt] depth:" << depth << std::endl;
            if (depth > 0) return true;
            X509* cert = X509_STORE_CTX_get_current_cert(vctx.native_handle());
            X509_NAME* name = X509_get_subject_name(cert);
            std::string cname;
            cname.resize(0xffff);
            auto size = X509_NAME_get_text_by_NID(name, NID_commonName, &cname[0], static_cast<int>(cname.size()));
            cname.resize(static_cast<std::size_t>(size));
            std::cout << "[clicrt] CNAME:" << cname << std::endl;
            return true;
        }
    );

We can extract CNAME from vctx in the varify callback.
It this a only change to do that?
Or after verified, can we extract CNAME from ctx ?

[kleunen]

I noticed that even if we provide set_verify_callback wrapper (socket parameter version), the callback setter (broker) needs to find corresponding endpoint by the socket. Maybe map is needed. It is not efficient.

No, map is not a very nice solution. Other possibility is to store it as metadata with the socket. From a socket you can obtain the certificate information. For TCP socket and TLS socket without certificate, this information is not available.

[kleunen]

Or after verified, can we extract CNAME from ctx ?

i am not sure, but i don't think that is possible

[redboltz]

How about this ?

At

mqtt_cpp/include/mqtt/server.hpp

Line 371 in 7ac3e0c

auto socket = std::make_shared<socket_t>(ioc_con, ctx_);

        auto socket = std::make_shared<socket_t>(ioc_con, ctx_);
        auto username = std::make_shared<std::string>(); // shared_ptr for username
        ctx_.set_verify_callback(
            [this, username] // copy capture socket shared_ptr
            (bool preverified, boost::asio::ssl::verify_context& ctx) {
                if (verify_cb_) {
                    return verify_cb_(preverified, ctx, force_move(username)); // user can set username in the callback
                }
                else {
                    return false;
                }
            }
        );
    }

Capture username to

mqtt_cpp/include/mqtt/server.hpp

Line 408 in 7ac3e0c

auto sp = std::make_shared<endpoint_t>(ioc_con, force_move(socket), version_);

   sp->set_preauthed_user_name(*username);

And add endpoint to

public:
    void set_preauthed_user_name(std::string user_name) {
        preauthed_user_name_.emplace(force_move(user_name));
    }
    optional<std::string> get_preauthed_user_name() const {
        return preauthed_user_name_;
    }
private:
    optional<std::string> preauthed_user_name_;

And from the broker,

mqtt_cpp/include/mqtt/broker/broker.hpp

Lines 1003 to 1014 in 7ac3e0c

	bool connect_handler(
	con_sp_t spep,
	buffer client_id,
	optional<buffer> /*username*/,
	optional<buffer> /*password*/,
	optional<will> will,
	bool clean_start,
	std::uint16_t /*keep_alive*/,
	v5::properties props
	) {
	auto& ep = *spep;

    if (auto preauthed_user_name_opt = ep.get_preauthed_user_name()) {
        // Got already authenticated UserName
    }

[kleunen]

How about this ?

At

mqtt_cpp/include/mqtt/server.hpp

Line 371 in 7ac3e0c

auto socket = std::make_shared<socket_t>(ioc_con, ctx_);

        auto socket = std::make_shared<socket_t>(ioc_con, ctx_);
        auto username = std::make_shared<std::string>(); // shared_ptr for username
        ctx_.set_verify_callback(
            [this, username] // copy capture socket shared_ptr
            (bool preverified, boost::asio::ssl::verify_context& ctx) {
                if (verify_cb_) {
                    return verify_cb_(preverified, ctx, force_move(username)); // user can set username in the callback
                }
                else {
                    return false;
                }
            }
        );
    }

Capture username to

mqtt_cpp/include/mqtt/server.hpp

Line 408 in 7ac3e0c

auto sp = std::make_shared<endpoint_t>(ioc_con, force_move(socket), version_);

   sp->set_preauthed_user_name(*username);

And add endpoint to

public:
    void set_preauthed_user_name(std::string user_name) {
        preauthed_user_name_.emplace(force_move(user_name));
    }
    optional<std::string> get_preauthed_user_name() const {
        return preauthed_user_name_;
    }
private:
    optional<std::string> preauthed_user_name_;

And from the broker,

mqtt_cpp/include/mqtt/broker/broker.hpp

Lines 1003 to 1014 in 7ac3e0c

	bool connect_handler(
	con_sp_t spep,
	buffer client_id,
	optional<buffer> /*username*/,
	optional<buffer> /*password*/,
	optional<will> will,
	bool clean_start,
	std::uint16_t /*keep_alive*/,
	v5::properties props
	) {
	auto& ep = *spep;

    if (auto preauthed_user_name_opt = ep.get_preauthed_user_name()) {
        // Got already authenticated UserName
    }

Yes, i do think that may be the best option.

[redboltz]

Ok, please go forward on this way.

[kleunen]

Ok, please go forward on this way.

I have it working, only thing is, when CNAME does not match any username in the auth file:

• Invalid login -> disconnect (i would say yes), or:
• Go on and try anonymous or username/password login

[redboltz]

Ok, please go forward on this way.

I have it working, only thing is, when CNAME does not match any username in the auth file:

• Invalid login -> disconnect (i would say yes), or:
• Go on and try anonymous or username/password login

Thanks.

Invalid login is good because the client already indicated client_cert authentication.
If the client want to anonymous authentication on mqtts or wss, then simply remove client certificate file setting.

It is important that if client doesn't set client cert, then anonymous or username and password authentication should work.
That means the broker support both client_cert authentication and anonymous or username and password authentication on mqtts and wss.

[redboltz]

In order to do that, the broker needs to distinguish something like the following:

1. Client doesn't set client_cert
2. Client sets client_cert but it is invalid on the TLS layer e.g. the certificate is not signed by valid CA.
3. Client sets client_cert and it is valid on the TLS layer but the CNAME(customizable filed) value doesn't in the authentication user list.

However, I don't know it can be.
The ideal scenario is only the case 1, continue to username and password authentication.
2 and 3 are rejected.

But if asio cannot distinguish 1 and 2, then , continue to username and password authentication. It is acceptable compromization.

[kleunen]

In order to do that, the broker needs to distinguish something like the following:

1. Client doesn't set client_cert
2. Client sets client_cert but it is invalid on the TLS layer e.g. the certificate is not signed by valid CA.
3. Client sets client_cert and it is valid on the TLS layer but the CNAME(customizable filed) value doesn't in the authentication user list.

However, I don't know it can be. The ideal scenario is only the case 1, continue to username and password authentication. 2 and 3 are rejected.

But if asio cannot distinguish 1 and 2, then , continue to username and password authentication. It is acceptable compromization.

I think this has to do with the preverified boolean, but not sure how this works.

I think this is behaviour that is happening now:

But if asio cannot distinguish 1 and 2, then , continue to username and password authentication. It is acceptable compromization.

[redboltz]

https://www.boost.org/doc/libs/1_77_0/doc/html/boost_asio/reference/ssl__context/set_verify_callback/overload1.html

  bool preverified, // True if the certificate passed pre-verification.

I understand as follows:

client_cert	preverified	broker behavior
not set	false	continue to password authentication
invalid signed	false	continue to password authentication (compromization)
valid singed no exist user	true	rejected
valid singed exist user	true	accepted and set_preverified_user_name

So the first two can't be distinguished. It is OK.
I think that the 3rd valid singed no exist user case, there are some design choice.
One is checking authenticate user list in verify_callback and reject.
The other is in verify_callback do the same process as 4th. So call set_preverified_user_name.
And at the connect_handler, check the authentication user list. If precerified_user_name doesn't exist, then sent CONNACK with error code.

I think that the latter is better. Because the error reporting is unified to password authentication and keep simple implementation on server_tls and server_tls_ws.

[kleunen]

Can you try again ? I think this is because the peer was getting validated, but tls_client has no certificate.

[redboltz]

Can you try again ? I think this is because the peer was getting validated, but tls_client has no certificate.

I got the same result. Still tls_client cannot connect to broker but can connect tls_server.

[redboltz]

When I remove auth.json but config set auth.json (by default) then, I got infinity loop at the following point:

│       43      while(true) {                                                                                                                                                                                                                                             │
│       44          char c;                                                                                                                                                                                                                                               │
│       45          if(input.get(c).eof()) break;                                                                                                                                                                                                                         │
│       46                                                                                                                                                                                                                                                                │
│       47          if (!inside_double_quote && !inside_single_quote && c == '#') inside_comment = true;                                                                                                                                                                  │
│       48          if (!inside_double_quote && c == '\'') inside_single_quote = !inside_single_quote;                                                                                                                                                                    │
│       49          if (!inside_single_quote && c == '"') inside_double_quote = !inside_double_quote;                                                                                                                                                                     │
│       50          if (!inside_double_quote && c == '\n') inside_comment = false;                                                                                                                                                                                        │
│       51                                                                                                                                                                                                                                                                │
│  >    52          if(!inside_comment) result << c;                                                                                                                                                                                                                      │
│       53      }      

[redboltz]

That was the reason of the error

$ ./tls_client 127.0.0.1 12345 cacert.pem                                                                                                                                                                
terminate called after throwing an instance of 'boost::wrapexcept<boost::system::system_error>'
  what():  handshake: sslv3 alert handshake failure (SSL routines, ssl3_read_bytes) [asio.ssl:336151568]
[1]    8237 IOT instruction (core dumped)  ./tls_client 127.0.0.1 12345 cacert.pem

I place auth.json. Then I got the same error.

When I execute tls_client again, then I got

CLIENT_RANDOM 92dceb71bbf2671f7d93b30a1d0bb02dbaa9b82692cf7cf8ccd117af028624dd f9969119c83883593343310f4701eca673519cb462f054e864b5d10329e33e3a27f2a1255b87c38960ced7e7b4ad4134
Connack handler called
Session Present: false
Connack Return Code: not_authorized
error: End of file

It seems that it is expected result. I run tls_client again and again, then I got expected result.
However, when I launch broker again, then I got the error.
It seems that it happens the first connection of the broker.

[redboltz]

I just create #920. It adds client certificate related options to client_cli. It is helpful to test this PR.
I will merge #920 soon, then please rebase the merged master. I think that no conflict could happen.

command option example:

using client certificate

$ ./client_cli --protocol mqtts --port 12345 --verify_file cacert.pem --certificate client.crt.pem --private_key client.key.pem --verbose 5 

using username and password

$ ./client_cli --username u1 --password hoge --protocol mqtts --port 12345 --verify_file cacert.pem --verbose 5 

[redboltz]

#920 merged

[redboltz]

client_cli and tls_client can get the same result.
The rest problem is connection failed due to handshake: sslv3 alert handshake failure (SSL routines, ssl3_read_bytes) .
It happens only when the first time connection just after broker launch.

[kleunen]

That is strange. Maybe related to asio tls handling?

[kleunen]

When I remove auth.json but config set auth.json (by default) then, I got infinity loop at the following point:

│       43      while(true) {                                                                                                                                                                                                                                             │
│       44          char c;                                                                                                                                                                                                                                               │
│       45          if(input.get(c).eof()) break;                                                                                                                                                                                                                         │
│       46                                                                                                                                                                                                                                                                │
│       47          if (!inside_double_quote && !inside_single_quote && c == '#') inside_comment = true;                                                                                                                                                                  │
│       48          if (!inside_double_quote && c == '\'') inside_single_quote = !inside_single_quote;                                                                                                                                                                    │
│       49          if (!inside_single_quote && c == '"') inside_double_quote = !inside_double_quote;                                                                                                                                                                     │
│       50          if (!inside_double_quote && c == '\n') inside_comment = false;                                                                                                                                                                                        │
│       51                                                                                                                                                                                                                                                                │
│  >    52          if(!inside_comment) result << c;                                                                                                                                                                                                                      │
│       53      }      

I was not able to reproduce this, but I added an error message in the broker when the auth file was not opened/loaded.

[kleunen]

andshake: sslv3 alert handshake failure (SSL routines, ssl3_read_bytes)

maybe this is related ?
https://stackoverflow.com/questions/59224873/boost-beast-handshake-sslv3-alert-handshake-failure-error

https://github.com/boostorg/beast/blob/bfd4378c133b2eb35277be8b635adb3f1fdaf09d/example/http/client/sync-ssl/http_client_sync_ssl.cpp#L66-L71

[redboltz]

andshake: sslv3 alert handshake failure (SSL routines, ssl3_read_bytes)

maybe this is related ? https://stackoverflow.com/questions/59224873/boost-beast-handshake-sslv3-alert-handshake-failure-error

https://github.com/boostorg/beast/blob/bfd4378c133b2eb35277be8b635adb3f1fdaf09d/example/http/client/sync-ssl/http_client_sync_ssl.cpp#L66-L71

Hmm, I'm not sure.

On 29a8519 , the problem still happens.
It never happens on tls_server but happens broker. It happens when I don't user client certificate.

	tls_server(first time)	tls_server(second time and later)	broker (first time)	broker(second time and later)
client_cli	ok	ok	ng	ok
tls_client	ok	ok	ng	ok

Only broker first time is ng. So I guess that the difference between tls_server and broker has a hint for analize.

Is this reproduced on your environment?

[kleunen]

Yes, I see the same behaviour under windows / msvc

[redboltz]

My environment is
Linux archboltz 5.16.16-arch1-1 #1 SMP PREEMPT Mon, 21 Mar 2022 22:59:40 +0000 x86_64 GNU/Linux
clang version 13.0.1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

I guess that it is not an environmental problem.

[kleunen]

I think the issue is in tls_client. If I use mqttbox as client, it works first time. Maybe try mosquitto client also.

[kleunen]

Also on linux (ubuntu) I don't see this issue. It connects the first time ..

[redboltz]

Maybe I found the problem on the broker.cpp. Now re-checking.

[kleunen]

Something uninitialised?

[redboltz]

Fixed.
See e81997f

The first listen() is called before ctx is completely initialized. So only the first connection is failed.
My commit above is very rough to demostrate the concept.
Could you refine the logic based on the concept ?

[kleunen]

I will have a look

[kleunen]

I made a separate listen for all server types, to be consistent. This fix seems to work.

[redboltz]

Confirmed. Thanks.

[redboltz]

Confirmed. I think that the PR is ready to merge except cording style fix.
The PR is pretty big. So I fix coding style and minor issues. Please check #921 (last commit).