Feature/rdk 4115/windows port

[vdm-dev]

Greetings from RoboDK Team.
We've done some work here and now your library builds and runs on Windows (including RTDE protocol).
We added the 3rdparty folder with two sub-projects:

1. endian.h for converting between BIG ENDIAN and LITTLE ENDIAN,
2. pthread-win32 for POSIX threads on Windows (added as git sub-module).

[urfeex]

Thank you very much for the input. I am not completely sure whether the licensing model with the 3rdparty projects would work in that constellation. I'll come back to that.

There might be the chance of removing the pthread calls, though / replace them with Windwos threads where needed. E.g. by using SetThreadPriority on windows and sched_setscheduler on POSIX. What would you think about that?

[vdm-dev]

pthread-win32 is licensed under two licenses: Apache 2.0 and GNU LGPL.
endian.h is licensed as public domain.

But anyway, I agree with you and think it's wasteful to use a POSIX Thread Layer for the sake of a couple calls.
Give me some time to do some in-depth research on the possibility of doing away with pthreads on Windows.

[vdm-dev]

It's done. pthreads are no longer needed on Windows. I left endian.h because I don't want to make more #ifdef's.
Here is the license for the endian.h:

// https://gist.github.com/panzi/6856583
//
// I, Mathias Panzenböck, place this file hereby into the public domain. Use
// it at your own risk for whatever you like. In case there are
// jurisdictions that don't support putting things in the public domain you
// can also consider it to be "dual licensed" under the BSD, MIT and Apache
// licenses, if you want to. This code is trivial anyway. Consider it an
// example on how to get the endian conversion functions on different
// platforms.

[urrsk]

Thanks for the PR. Really interesting.
Is there any reason why you are not suggesting the updated one at https://gist.github.com/panzi/6856583
It seems that there has been added something that relates to windows and QNX

[vdm-dev]

Thank you.
The reason is that version uses the htons/htonl/htonll methods from WinSock2.h, while my version uses the _byteswap_ushort/_byteswap_ulong/_byteswap_uint64 methods from stdlib.h within MSVC (https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/byteswap-uint64-byteswap-ulong-byteswap-ushort?view=msvc-170).
WinSock2.h is too big to include it for the sake of three functions. It slows down compilation.
Besides, calls to htons are guaranteed to result in a full method call from ws2_32.dll, while methods from the Standard C library can be inline or translated into platform-dependent processor instructions (e.g. bswap: https://c9x.me/x86/html/file_module_x86_id_21.html).

[urfeex]

Could we combine this PR with a Windows CI? I am not experienced with compiling things on Windows, so if you could help us out there, that would be great :-)

[vdm-dev]

That would be great, but unfortunately I've been pretty busy with work lately.
It is possible that I will prepare scripts for CI in the future, but I don't know when yet.

[urfeex]

That would be great, but unfortunately I've been pretty busy with work lately. It is possible that I will prepare scripts for CI in the future, but I don't know when yet.

Thank you for the reply. I'll see whether I can draft something up quickly, myself, then.

[urrsk]

@panzi will you be that we include the 3rdparty/endian/endian.h into this project under Apache 2.0 or BSD3 license?
Just want to be completely sure as the statement in the header is only referring to apache and bsd.

The header seems very useful. Thanks for the great work.

[panzi]

You may include it under whatever license, even without referencing where you got it from. As I said, I put it under the public domain. Anyone can do anything with it. If that is too vague for you, you may consider it to be Apache 2.0, as is your project, to make things simpler. But again, I don't care. It's trivial. And I don't think I would do it like that nowadays anyway. You can make endian independent macros that should compile away if the endianess matches your machine. Like:

/** @param uint8_t* PTR */
#define READ_U32_LE(PTR) (uint32_t)((uint32_t)(PTR)[0] | ((uint32_t)(PTR)[1] << 8) | ((uint32_t)(PTR)[2] << 16) | ((uint32_t)(PTR)[3] << 24))

Example that shows it has the same assembly for gcc -O2: https://godbolt.org/z/5v61vz7ad

[urfeex]

Thank you again @vdm-dev! I've whipped together a windows build workflow in https://github.com/urfeex/Universal_Robots_Client_Library/blob/win-build/.github/workflows/win-build.yml based on your changes and the ones from #244.

It builds, but running the tests seems not to be successful for some tests. Did you experience the same things? As I currently don't have a Windows machine available, I cannot verify this locally.

[vdm-dev]

I'll check back later this week.

[vdm-dev]

I have investigated the problem and added fixes to both the tests and the tcp_server.cpp file.

• T18, T24: The errors in the TCPServerTest.socket_creation and TCPServerTest.check_address_already_in_use tests were related to the SO_REUSEADDR option, which behaves a bit differently on Windows than on Linux. In particular, on Windows it allows you to take control away from a previously created socket, even if it is OK and involved in communication.
• T108: The TCPSocketTest.test_read_on_socket_abruptly_closed error occurred because the socket was closed using the close() method, which is also available in Windows but cannot be used to close sockets. It works only with files and mimics the POSIX subsystem from the Windows 95 days. On Windows, a socket descriptor is represented by a void pointer, not an int type. That's why I introduced the ur_close() helper function for it in the socket_t.h file. Calling the correct function solved the problem.
• T106, T115, T117, T122: Errors of the *.connect_non_connected_robot type occur because of a false assumption that the blocking connect() method will return an error twice if it fails to connect within a second and a half. Even test files contain remarks about it that this kind of test is incorrect. A quick solution to the problem is to increase the waiting time for test execution to 7.5 seconds. But actually, if you want to track the connection timeout, you should use non-blocking sockets and the select() method or its analog to find out the socket state.

[urfeex]

Awesome, thank you! Would you mind me pushing the Windows CI workflow to that branch?

[vdm-dev]

That would be great!

[urfeex]

See RoboDK#1. Note that this also merges in current master hence all of those unrelated changes to your branch.

[codecov]

Codecov Report

Attention: Patch coverage is 86.66667% with 10 lines in your changes missing coverage. Please review.

Project coverage is 73.62%. Comparing base (44b650f) to head (8b3104d).
Report is 6 commits behind head on master.

Files with missing lines	Patch %	Lines
src/comm/tcp_server.cpp	84.84%	1 Missing and 4 partials ⚠️
src/control/reverse_interface.cpp	66.66%	1 Missing and 2 partials ⚠️
src/control/script_command_interface.cpp	80.00%	0 Missing and 1 partial ⚠️
src/control/trajectory_point_interface.cpp	75.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #229      +/-   ##
==========================================
+ Coverage   73.53%   73.62%   +0.09%     
==========================================
  Files          80       80              
  Lines        3182     3174       -8     
  Branches      399      395       -4     
==========================================
- Hits         2340     2337       -3     
+ Misses        622      621       -1     
+ Partials      220      216       -4     

Flag	Coverage Δ
ur20-latest	72.31% <86.66%> (+0.11%)	⬆️
ur5-3.14.3	73.17% <86.66%> (+0.19%)	⬆️
ur5e-5.9.4	72.95% <86.66%> (+0.28%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[vdm-dev]

@urfeex
I've made quite a few important fixes related to the fact that a socket in Linux is int, while in Windows it is UINT_PTR (== uint64_t).
For example, I removed comparisons like (fd < 0) and replaced them with (fd == INVALID_SOCKET) wherever I saw them.
Similar changes have been made to callbacks.
I also tried to fix all the things that cause warnings in MSVC, but only those that relate to the socket subsystem.
On Windows, everything builds and fully passes the tests. Please make sure that everything is fine on Linux as well.

[urfeex]

@vdm-dev thanks again for all the efforts you've put into this already. As soon as #246 is merged, I would like to ask you to merge in master once more and format the code, best using pre-commit run -a. I'll give it a test on Linux in the meantime.

[urfeex]

@vdm-dev could you please merge in current master and format the code as explained above?

Also, we should make sure the endian file is installed and available for downstream packages. The easiest solution probably would be to place it inside include/ur_client_library/3rdparty/endian.h.

[urfeex]

@vdm-dev I've given this some manual testing and it seems to be working fine :-)

If you could merge in current master and format the code I think we would be good to merge that in.

[urfeex]

@vdm-dev I've created a PR to your branch making the merge and code format. Please consider merging this.

[vdm-dev]

Thank you. So much to do lately, I can't keep up with everything. I merged your code with mine.

[urfeex]

@vdm-dev I've spent quite some time testing this branch and there are couple of issues left, especially regarding shutdown.

However, I think those issues are not directly coming from this PR, but they might just be showing through this PR since we are on another platform. I've created one more PR on your branch as the file handling inside the UrDriver tests was not working correctly.

With that merged, I would be fine with merging this PR here and tracking the outstanding issues in separate issues.

[vdm-dev]

Done

[urfeex]

Thank you! I will merge #275, #276, make a patch release and then merge this one.

• Remove sending an idle command in quintic spline test #275
• Fix trajectory result in trajectory forward mode when no trajectory is running #276
• Release/1.7.1 #277

[urfeex]

Thank you @vdm-dev for implementing this and for sticking around :-)