feat(local-ic): stream interaction and container logs

[Reecepbcups]

Summary

For the Spawn UI platform, we need a way to get logs from the network -> the users UI. This PR adds support to stream the logs to both the console & a unique file. Then the user can query /logs in the REST api. It will tail the last N lines, and continue to return new data on new interaction.

Further, the container logs for actuals of a container are streamed too. You can use the container ID or the node name itself. much better than docker ps; docker logs <id> -f for API users.

Based on concept here: https://github.com/Reecepbcups/golang_stream

TODO

• stream docker container logs out as well? (i.e. /container_logs?id=)
• Remove dumb comments
• Print tail X lines of logs endpoint (instead of trying to merge them)
• Select logs by chain-id vs being the container ID (some mapping between)

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
interchaintest-docs	⬜️ Ignored (Inspect)	Visit Preview		Oct 18, 2024 4:34am

[jtieri]

Had a couple nits around logs and a couple questions but otherwise the changes look good.

Although it is out of scope of this PR one thing that jumped out to me is that the StartChain function in start.go is growing to be rather large. There are a few places where logic could be broken out into separate functions.

[jtieri]

Is this necessary because certain operating systems or terminals cannot render the colors in the logs?

[Reecepbcups]

Since this is being forwarded over a REST api, it is always the case. So most users parsing that data don't care about. We may in the future want to parse out the colors which is why it is possible to opt out and see those ASNI escape sequences

[jtieri]

What was the reasoning for instantiating the slice with this specific value? Does changing this value break anything? If so a comment regarding the reason for this specific size could be helpful.

[Reecepbcups]

1024 is the standard. Due to the size of some JSON blobs sent through (relayer), it was exceeding that limit for Galen. Causing parsing issues

It was exceeding the size for some relayer JSON blobs for Galen. He was unable to parse it since it came in multiple chunks. I wanted to ensure we have enough of a buffer and 8192 felt like a good spot (21024 was still too small, I am sure 4 would have been fine but increased just to be sure since the data sent it relative small and controlled anyways)

[jtieri]

Not necessary but this is the type of decision that can be helpful to document via a comment. You never know when someone is going to come across and naively change values without understanding what the implications are.

[jtieri]

Is there a specific reason we have to use a Context as a field member of this struct vs. piping a Context into the StreamContainer method or wherever a Context is needed for the ContainerStream?

It's typically more idiomatic to pass a Context around vs. storing one inside of a struct type

One of the exceptions to this rule are typically when you have an external request that starts a background process/operation.

Edit: After reading the rest of the code it seems like that is what may be happening here when a user makes a request to /container_logs so this may be a correct usage of context in a struct but figured I would ask for clarity.

[Reecepbcups]

tldr; better Dev UX for the same relative solution

It's cleaner than having to write methods return func(w http.ResponseWriter, req *http.Request) signatures just for ctx to be passed in in a wrapped type.

Current:

r.HandleFunc("/container_logs", containerStream.StreamContainer)

Would be required change

r.HandleFunc("/container_logs", containerStream.handleStreamContainer(ctx))

func (cs *ContainerStream) handleStreamContainer(ctx context) func(w http.ResponseWriter, req *http.Request) {

   return func(w http.ResponseWriter, req *http.Request) {
      // all the same logic here, just wrapped now
    }
}

ctx is only used for timeout limits and set on startup. So any runtime changes are unaffected in either approach - and the first approach is just net easier for a new contributor to wrap their head around.