Skip to content

gh-111662: Update socket module to use AC for optimizing performance #111661

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Nov 8, 2023
Merged

gh-111662: Update socket module to use AC for optimizing performance #111661

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
896060a
change calling convention of socket.htons
wrongnull Nov 2, 2023
a06b91d
📜🤖 Added by blurb_it.
blurb-it[bot] Nov 2, 2023
26ebcea
delete news
wrongnull Nov 3, 2023
cd3f611
Merge branch 'main' into meth_o-for-htons
wrongnull Nov 3, 2023
0f3ca47
clinic argument for socket.htons
wrongnull Nov 3, 2023
d92ddd3
clinic argument for socket.ntohs
wrongnull Nov 3, 2023
144f61c
clinic argument for socket.inet_aton
wrongnull Nov 3, 2023
6ccd817
clinic argument for socket.inet_ntoa
wrongnull Nov 3, 2023
a533180
clinic argument for socket.if_nametoindex
wrongnull Nov 4, 2023
b1f28f5
clinic argument for socket.ntohl
wrongnull Nov 4, 2023
387725b
clinic argument for socket.htonl
wrongnull Nov 4, 2023
6ea506f
Merge branch 'main' into meth_o-for-htons
wrongnull Nov 4, 2023
bc6733b
remove bitwise=True from clinic converter
wrongnull Nov 4, 2023
a27a86d
Revert "remove bitwise=True from clinic converter"
wrongnull Nov 6, 2023
c62d4e3
Revert "clinic argument for socket.htonl"
wrongnull Nov 6, 2023
e55aac5
Revert "clinic argument for socket.ntohl"
wrongnull Nov 6, 2023
39e68a8
Merge branch 'main' into meth_o-for-htons
wrongnull Nov 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
165 changes: 164 additions & 1 deletion Modules/clinic/socketmodule.c.h
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

127 changes: 59 additions & 68 deletions Modules/socketmodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6332,14 +6332,18 @@ AF_UNIX if defined on the platform; otherwise, the default is AF_INET.");
#endif /* HAVE_SOCKETPAIR */


/*[clinic input]
_socket.socket.ntohs
x: int
/

Convert a 16-bit unsigned integer from network to host byte order.
[clinic start generated code]*/

static PyObject *
socket_ntohs(PyObject *self, PyObject *args)
_socket_socket_ntohs_impl(PySocketSockObject *self, int x)
/*[clinic end generated code: output=a828a61a9fb205b2 input=9a79cb3a71652147]*/
{
int x;

if (!PyArg_ParseTuple(args, "i:ntohs", &x)) {
return NULL;
}
if (x < 0) {
PyErr_SetString(PyExc_OverflowError,
"ntohs: can't convert negative Python int to C "
Expand All @@ -6355,11 +6359,6 @@ socket_ntohs(PyObject *self, PyObject *args)
return PyLong_FromUnsignedLong(ntohs((unsigned short)x));
}

PyDoc_STRVAR(ntohs_doc,
"ntohs(integer) -> integer\n\
\n\
Convert a 16-bit unsigned integer from network to host byte order.");


static PyObject *
socket_ntohl(PyObject *self, PyObject *arg)
Expand Down Expand Up @@ -6395,14 +6394,18 @@ PyDoc_STRVAR(ntohl_doc,
Convert a 32-bit integer from network to host byte order.");


/*[clinic input]
_socket.socket.htons
x: int
/

Convert a 16-bit unsigned integer from host to network byte order.
[clinic start generated code]*/

static PyObject *
socket_htons(PyObject *self, PyObject *args)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

/*[clinic input]
_socket.socket.htons
    x: int
    /

Convert a 16-bit unsigned integer from host to network byte order.
[clinic start generated code]*/

Use AC instead of writing manual parsing.
I got a similar performance gain in my local environment.

AS-IS: 5000000 loops, best of 5: 60.9 nsec per loop
TO-BE: 10000000 loops, best of 5: 23.3 nsec per loop

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wrongnull
You should take a look at the https://devguide.python.org/development-tools/clinic/ if you're not familiar with Argument Clinic :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure a similar approach can be applied to all other functions with only one argument. If yes, I'll be happy to make such a commit in this branch.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@corona10 seems ready. clinic argument was also applied to other functions where it was beneficial

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if ValueError is more preferable if negative value is passed to ntohl and htonl functions

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if ValueError is more preferable if negative value is passed to ntohl and htonl functions

Please don't change the exception type; it will occur in user regression.
Can I know where the type of exception will be changed if AC is applied?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if ValueError is more preferable if negative value is passed to ntohl and htonl functions

Please don't change the exception type; it will occur in user regression. Can I know where the type of exception will be changed if AC is applied?

Exception type will change to ValueError from OverflowError in htonl and ntohl functions after applying usigned_long argument converter.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exception type will change to ValueError from OverflowError in htonl and ntohl functions after applying usigned_long argument converter.

In that case, I would like to recommend not to touch it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exception type will change to ValueError from OverflowError in htonl and ntohl functions after applying usigned_long argument converter.

In that case, I would like to recommend not to touch it.

OK, I reverted this changes

_socket_socket_htons_impl(PySocketSockObject *self, int x)
/*[clinic end generated code: output=d785ee692312da47 input=053252d8416f4337]*/
{
int x;

if (!PyArg_ParseTuple(args, "i:htons", &x)) {
return NULL;
}
if (x < 0) {
PyErr_SetString(PyExc_OverflowError,
"htons: can't convert negative Python int to C "
Expand All @@ -6418,11 +6421,6 @@ socket_htons(PyObject *self, PyObject *args)
return PyLong_FromUnsignedLong(htons((unsigned short)x));
}

PyDoc_STRVAR(htons_doc,
"htons(integer) -> integer\n\
\n\
Convert a 16-bit unsigned integer from host to network byte order.");


static PyObject *
socket_htonl(PyObject *self, PyObject *arg)
Expand Down Expand Up @@ -6459,14 +6457,17 @@ Convert a 32-bit integer from host to network byte order.");

/* socket.inet_aton() and socket.inet_ntoa() functions. */

PyDoc_STRVAR(inet_aton_doc,
"inet_aton(string) -> bytes giving packed 32-bit IP representation\n\
\n\
Convert an IP address in string format (123.45.67.89) to the 32-bit packed\n\
binary format used in low-level network functions.");
/*[clinic input]
_socket.socket.inet_aton
ip_addr: str
/

static PyObject*
socket_inet_aton(PyObject *self, PyObject *args)
Convert an IP address in string format (123.45.67.89) to the 32-bit packed binary format used in low-level network functions.
[clinic start generated code]*/

static PyObject *
_socket_socket_inet_aton_impl(PySocketSockObject *self, const char *ip_addr)
/*[clinic end generated code: output=5bfe11a255423d8c input=a120e20cb52b9488]*/
{
#ifdef HAVE_INET_ATON
struct in_addr buf;
Expand All @@ -6479,11 +6480,6 @@ socket_inet_aton(PyObject *self, PyObject *args)
/* Have to use inet_addr() instead */
unsigned int packed_addr;
#endif
const char *ip_addr;

if (!PyArg_ParseTuple(args, "s:inet_aton", &ip_addr))
return NULL;


#ifdef HAVE_INET_ATON

Expand Down Expand Up @@ -6532,30 +6528,29 @@ socket_inet_aton(PyObject *self, PyObject *args)
}

#ifdef HAVE_INET_NTOA
PyDoc_STRVAR(inet_ntoa_doc,
"inet_ntoa(packed_ip) -> ip_address_string\n\
\n\
Convert an IP address from 32-bit packed binary format to string format");
/*[clinic input]
_socket.socket.inet_ntoa
packed_ip: Py_buffer
/

static PyObject*
socket_inet_ntoa(PyObject *self, PyObject *args)
Convert an IP address from 32-bit packed binary format to string format.
[clinic start generated code]*/

static PyObject *
_socket_socket_inet_ntoa_impl(PySocketSockObject *self, Py_buffer *packed_ip)
/*[clinic end generated code: output=b671880a3f62461b input=95c2c4a1b2ee957c]*/
{
Py_buffer packed_ip;
struct in_addr packed_addr;

if (!PyArg_ParseTuple(args, "y*:inet_ntoa", &packed_ip)) {
return NULL;
}

if (packed_ip.len != sizeof(packed_addr)) {
if (packed_ip->len != sizeof(packed_addr)) {
PyErr_SetString(PyExc_OSError,
"packed IP wrong length for inet_ntoa");
PyBuffer_Release(&packed_ip);
PyBuffer_Release(packed_ip);
return NULL;
}

memcpy(&packed_addr, packed_ip.buf, packed_ip.len);
PyBuffer_Release(&packed_ip);
memcpy(&packed_addr, packed_ip->buf, packed_ip->len);
PyBuffer_Release(packed_ip);

SUPPRESS_DEPRECATED_CALL
return PyUnicode_FromString(inet_ntoa(packed_addr));
Expand Down Expand Up @@ -7049,18 +7044,23 @@ PyDoc_STRVAR(if_nameindex_doc,
\n\
Returns a list of network interface information (index, name) tuples.");

/*[clinic input]
_socket.socket.if_nametoindex
oname: object(converter="PyUnicode_FSConverter")
/

Returns the interface index corresponding to the interface name if_name.
[clinic start generated code]*/

static PyObject *
socket_if_nametoindex(PyObject *self, PyObject *args)
_socket_socket_if_nametoindex_impl(PySocketSockObject *self, PyObject *oname)
/*[clinic end generated code: output=f7fc00511a309a8e input=662688054482cd46]*/
{
PyObject *oname;
#ifdef MS_WINDOWS
NET_IFINDEX index;
#else
unsigned long index;
#endif
if (!PyArg_ParseTuple(args, "O&:if_nametoindex",
PyUnicode_FSConverter, &oname))
return NULL;

index = if_nametoindex(PyBytes_AS_STRING(oname));
Py_DECREF(oname);
Expand All @@ -7073,10 +7073,6 @@ socket_if_nametoindex(PyObject *self, PyObject *args)
return PyLong_FromUnsignedLong(index);
}

PyDoc_STRVAR(if_nametoindex_doc,
"if_nametoindex(if_name)\n\
\n\
Returns the interface index corresponding to the interface name if_name.");

static PyObject *
socket_if_indextoname(PyObject *self, PyObject *arg)
Expand Down Expand Up @@ -7215,19 +7211,15 @@ static PyMethodDef socket_methods[] = {
{"socketpair", socket_socketpair,
METH_VARARGS, socketpair_doc},
#endif
{"ntohs", socket_ntohs,
METH_VARARGS, ntohs_doc},
_SOCKET_SOCKET_NTOHS_METHODDEF
{"ntohl", socket_ntohl,
METH_O, ntohl_doc},
{"htons", socket_htons,
METH_VARARGS, htons_doc},
_SOCKET_SOCKET_HTONS_METHODDEF
{"htonl", socket_htonl,
METH_O, htonl_doc},
{"inet_aton", socket_inet_aton,
METH_VARARGS, inet_aton_doc},
_SOCKET_SOCKET_INET_ATON_METHODDEF
#ifdef HAVE_INET_NTOA
{"inet_ntoa", socket_inet_ntoa,
METH_VARARGS, inet_ntoa_doc},
_SOCKET_SOCKET_INET_NTOA_METHODDEF
#endif
#ifdef HAVE_INET_PTON
{"inet_pton", socket_inet_pton,
Expand All @@ -7250,8 +7242,7 @@ static PyMethodDef socket_methods[] = {
#if defined(HAVE_IF_NAMEINDEX) || defined(MS_WINDOWS)
{"if_nameindex", socket_if_nameindex,
METH_NOARGS, if_nameindex_doc},
{"if_nametoindex", socket_if_nametoindex,
METH_VARARGS, if_nametoindex_doc},
_SOCKET_SOCKET_IF_NAMETOINDEX_METHODDEF
{"if_indextoname", socket_if_indextoname,
METH_O, if_indextoname_doc},
#endif
Expand Down