Rewrite socket documentation entirely, leave no stone unturned

[GSPP]

The documentation for socket-based code is not in a good state. The code samples are actually great: They demonstrate all the ways you can misuse sockets 🙃

@geoffkizer has written an analysis of one particular gem. I know from experience that there are many, many other snippets of similar quality. One can only imagine how many novices have been mislead by that documentation. As an extensive Stack Overflow contributor, I have seen a constant stream of confused people who sometimes explicitly point at the official documentation as the source of their misguided patterns.

As an example, let's look at how the documentation for TcpClient.DataAvailable demonstrates its misuse very nicely. And then, let's compare it to the suffering that it has caused: Searching the web for "site:stackoverflow.com tcpclient dataavailable" returns about 1000 results. Stack Overflow is a great way to see what "grassroots" problems there are with a certain technology.

Another example: Socket.BeginConnect should not block using an event (what's the point?). I have seen people use this exact pattern, and I know where it comes from.

Reworking one or two of those samples is a drop in the bucket. The entire socket documentation needs to be inspected and revised.

I don't think it's such a good idea to put small sample code snippets in many locations (as it is done now). This just spreads the documentation out over many pages. Socket programming requires an understanding of fundamental concepts. For example, the concept of a byte stream is a general idea. This idea is concretely used in many socket APIs. The concept should be explained once and then be referenced. Another concept is that you can't know if a TCP connection is still viable. No API can exist that tells whether a socket is actually connected in a practical sense.

Most information should be laid out on a few large pages that comprehensively write up a certain topic. Individual APIs can then link to sections of those larger pages for guidance. What good is sample code for a Socket.ReadAsync call when you don't understand how sockets work in general? And we don't want to duplicate the proper usage guidance on dozens of pages for each API that directly or indirectly reads from a socket.

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

The documentation for socket-based code is not in a good state. The code samples are actually great: They demonstrate all the ways you can misuse sockets 🙃

@geoffkizer has written an analysis of one particular gem. I know from experience that there are many, many other snippets of similar quality. One can only imagine how many novices have been mislead by that documentation. As an extensive Stack Overflow contributor, I have seen a constant stream of confused people who sometimes explicitly point at the official documentation as the source of their misguided patterns.

As an example, let's look at how the documentation for TcpClient.DataAvailable demonstrates its misuse very nicely. And then, let's compare it to the suffering that it has caused: Searching the web for "site:stackoverflow.com tcpclient dataavailable" returns about 1000 results. Stack Overflow is a great way to see what "grassroots" problems there are with a certain technology.

Another example: Socket.BeginConnect should not block using an event (what's the point?). I have seen people use this exact pattern, and I know where it comes from.

Reworking one or two of those samples is a drop in the bucket. The entire socket documentation needs to be inspected and revised.

I don't think it's such a good idea to put small sample code snippets in many locations (as it is done now). This just spreads the documentation out over many pages. Socket programming requires an understanding of fundamental concepts. For example, the concept of a byte stream is a general idea. This idea is concretely used in many socket APIs. The concept should be explained once and then be referenced. Another concept is that you can't know if a TCP connection is still viable. No API can exist that tells whether a socket is actually connected in a practical sense.

Most information should be laid out on a few large pages that comprehensively write up a certain topic. Individual APIs can then link to sections of those larger pages for guidance. What good is sample code for a Socket.ReadAsync call when you don't understand how sockets work in general? And we don't want to duplicate the proper usage guidance on dozens of pages for each API that directly or indirectly reads from a socket.

Author:	GSPP
Assignees:	-
Labels:	area-System.Net.Sockets, untriaged
Milestone:	-

[alexrp]

If we're listing problems with Socket documentation, might I add exceptions and error codes to the list?

It's a real pain to figure out what exceptions or SocketErrors one should expect from various Socket APIs in different situations. As an example, how many people realize that closing a listening socket necessitates wrapping Accept/AcceptAsync in a try/catch that handles both ObjectDisposedException and SocketException with specifically SocketError.OperationAborted?

Stuff like this is not documented clearly anywhere and I've had to just figure it out by trial and error over the years. For all the pain that native socket programming can be, at the very least manual pages clearly list possible errno codes and what situations cause them.

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

The documentation for socket-based code is not in a good state. The code samples are actually great: They demonstrate all the ways you can misuse sockets 🙃

@geoffkizer has written an analysis of one particular gem. I know from experience that there are many, many other snippets of similar quality. One can only imagine how many novices have been mislead by that documentation. As an extensive Stack Overflow contributor, I have seen a constant stream of confused people who sometimes explicitly point at the official documentation as the source of their misguided patterns.

As an example, let's look at how the documentation for TcpClient.DataAvailable demonstrates its misuse very nicely. And then, let's compare it to the suffering that it has caused: Searching the web for "site:stackoverflow.com tcpclient dataavailable" returns about 1000 results. Stack Overflow is a great way to see what "grassroots" problems there are with a certain technology.

Another example: Socket.BeginConnect should not block using an event (what's the point?). I have seen people use this exact pattern, and I know where it comes from.

Reworking one or two of those samples is a drop in the bucket. The entire socket documentation needs to be inspected and revised.

I don't think it's such a good idea to put small sample code snippets in many locations (as it is done now). This just spreads the documentation out over many pages. Socket programming requires an understanding of fundamental concepts. For example, the concept of a byte stream is a general idea. This idea is concretely used in many socket APIs. The concept should be explained once and then be referenced. Another concept is that you can't know if a TCP connection is still viable. No API can exist that tells whether a socket is actually connected in a practical sense.

Most information should be laid out on a few large pages that comprehensively write up a certain topic. Individual APIs can then link to sections of those larger pages for guidance. What good is sample code for a Socket.ReadAsync call when you don't understand how sockets work in general? And we don't want to duplicate the proper usage guidance on dozens of pages for each API that directly or indirectly reads from a socket.

Author:	GSPP
Assignees:	-
Labels:	Pri3, area-System.Net.Sockets, :watch: Not Triaged
Milestone:	-

[antonfirsov]

I agree with all observations, but it's a lot of work to systematically update all docs & samples, and the review and example creation work has to be done by the networking core team which has limited throughput. I'm skeptical we can prioritize such a large update over bugfix & feature work.

I think the most realistic approach to get some improvements done in reasonable time is still to go case-by case, opening separate issues with specific suggestions, so we have smaller actionable work items to address, making it easier to also involve the docs team.

TcpClient.DataAvailable, Socket.BeginConnect and Accept/AcceptAsync exception issues are good examples. If you have spotted anything else in particular, please list them here, or even better - open separate issues.

@alexrp note that we may address the accept issue in the product itself: dotnet/runtime#63115. (But we still need the docs improvements for the old API-s.)

/cc @karelz since it's a good example for (eventually inevitable) documentation work

[antonfirsov]

I don't think it's such a good idea to put small sample code snippets in many locations (as it is done now). This just spreads the documentation out over many pages. Socket programming requires an understanding of fundamental concepts. For example, the concept of a byte stream is a general idea. This idea is concretely used in many socket APIs. The concept should be explained once and then be referenced. Another concept is that you can't know if a TCP connection is still viable. No API can exist that tells whether a socket is actually connected in a practical sense.

Ok, some of this might be hard to approach incrementally. Personally, I would be fine deleting the poor code samples, creating conceptual docs, and link the conceptual docs from API docs instead.

[GSPP]

@antonfirsov that sounds like a good plan. It means that most samples will be deleted. Here is another one that I found just now that gets me going:

What is the point of this code? 😵

---

I think there's a lot of value in having samples that show how to accomplish a realistic task. For example, https://docs.microsoft.com/en-us/dotnet/api/system.net.sockets.socket.-ctor?view=net-6.0#system-net-sockets-socket-ctor(system-net-sockets-addressfamily-system-net-sockets-sockettype-system-net-sockets-protocoltype) makes an attempt to show how to download a web page. This is very educational and could be directly executable code. (Of course, this piece of code is not the way we want it to be done, but that's beside the point.).

Some ideas for educational samples:

• Download a web page
• A tiny HTTP server that can be accessed from the browser (ignoring almost the entire HTTP specification, just hardcoded strings)
• A simple BinaryReader/Writer based network protocol
• A ping-based host scanner for the local network
• Use SslStream to connect to an actual SMTP server and send yourself an email
• Use asynchronous IO to establish 1 million TCP connections as part of a demo chat server
• Keep a GUI responsive by using await directly in event callbacks
• Low allocation code patterns
• A chat system between two hosts so that you can chat with your friend on the other side of the world

Each of these samples could be in the form of a long-page walkthrough in essentially the same format that @stephentoub uses for his epic .NET blog posts. The resulting code size should be 1-3 pages, just the bare bones needed to operate the APIs correctly. But the walkthrough would pick it apart in detail, for example explaining that DataAvailable cannot be used to detect the end of the transmission. It would be Literate programming. The entire code should be directly in the walkthrough (no downloads necessary).

A positive example for documentation is how the C# patterns feature is documented. It's a comprehensive walkthrough of that concept.

Not every concept makes sense as a sample. For example, low allocation code patterns have nothing to do with a specific use case. They can be described in an abstract way. Possibly, in the form of a performance page that contains all performance specific guidance.

---

API pages for reading and writing data would just say in the remarks:

[remarks for the specific member here]

Creating a reliable networked application requires an understanding of certain fundamental concepts.
For a general understanding of networking concepts and guidance on .NET networking APIs, see these resources:

• A list of the relevant pages
• ...

Centralize general concepts, distribute specific knowledge.

---

I wanted to try this out so I wrote this sample:

public static void DownloadWebPage()
{
    var url = "http://example.org/";

    try
    {
        var uri = new Uri(url);

        using var socket = new Socket(SocketType.Stream, ProtocolType.Tcp);

        socket.Connect(uri.Host, uri.Port);

        using var networkStream = new NetworkStream(socket);
        using var streamWriter = new StreamWriter(networkStream, Encoding.ASCII) { NewLine = "\r\n" };

        streamWriter.WriteLine($"GET {uri.PathAndQuery} HTTP/1.1");
        streamWriter.WriteLine($"Host: {uri.Host}");
        streamWriter.WriteLine();
        streamWriter.Flush();

        socket.Shutdown(SocketShutdown.Send);

        using var streamReader = new StreamReader(networkStream);

        var response = streamReader.ReadToEnd();

        Console.WriteLine("Server responded with the following HTTP response:");
        Console.WriteLine();
        Console.WriteLine(response);
    }
    catch (Exception ex)
    {
        Console.WriteLine("Exception occurred:");
        Console.WriteLine();
        Console.WriteLine(ex);
    }
}

With added comments and a walkthrough, this is what a sample could be like. Nothing superfluous. Just straightforward and correct usage to accomplish a realistic goal.