feat: Add redirected actor logs

[Pijukatel]

Description

• Add convenience methods and arguments for redirecting streamed actor log into another logger.
• This changes the default behavior in non breaking way. By default the logs will be redirected now.
• Update log endpoints calls to accept raw parameter.

Example usage:

Starting new actor run through call

3 different use-cases:

...
async with Actor:
    # Default logger
    await Actor.new_client().actor(actor_id="...").call(timeout_secs=5)

    # No logger
    await Actor.new_client().actor(actor_id="...").call(timeout_secs=5, logger = None)

    # Custom logger
    custom_logger = logging.getLogger("apify.kustom_logga")
    handler = logging.StreamHandler()
    custom_logger.addHandler(handler)
    custom_logger.propagate = False

    await Actor.new_client().actor(actor_id="SbjD4JEucMevUdQAH").call(timeout_secs=5, logger = custom_logger)

Attaching to already running actor, redirecting all logs from actor start

(This is the technique to be used when starting actor through actor.start as well)

...
async with Actor:
    ...
    # With context manager
    async with await Actor.new_client().run("some_id").get_streamed_log():
        await asyncio.sleep(5)
        # Logging will stop out of context

     # With direct call (lifecycle of logging has to be manually handled)
    streamed_log = await Actor.new_client().run("some_id").get_streamed_log()
    streamed_log.start()
    await asyncio.sleep(5)
    streamed_log.close()

Attaching to already running actor, redirecting only new logs

(This can be useful when communicating with some long-running standby actor)

...
async with Actor:
    # With context manager
    async with await Actor.new_client().run("some_id").get_streamed_log(from_start=False):
        await asyncio.sleep(5)
        # Logging will stop out of context

Example actor with all the logging options shown:

https://console.apify.com/actors/m7ZFYX42SCRFOL0JW

Issues

• Closes: Redirect logs from other actors #402

[Pijukatel]

This is a minimal draft version with only async client updated so far to have early review on the concept (sync client will be done later).

Such change can be later propagated to Actor class in sdk.

Example usage shown on the screenshot, where we call actor called logger three times with different logging arguments:

1. Default - redirect logger (will be new default behavior), user do not need to do anything and some predefined reasonable logger is used. (Logger name is f"apify.{actor}-{run_id}")
2. No redirect logger (same as current behavior, but have to be selected by explicitly passing None to logger argument)
3. Custom logger defined by the user (I have created some example logger for the sake of the example, but it can be anything.)

Redirected messages have estimated logging level, so in most cases you can figure out what level it was in the original log and set it accordingly in this log.

Example actor: https://console.apify.com/actors/m7ZFYX42SCRFOL0JW/source

Not in the scope of first version: Respecting the coloring of the original logs -> This information is not available in the logs returned from the api call, so we could try to recreate the coloring during runtime by inspecting all the messages and injecting color markers where suitable, but that could be very computational heavy, so not sure if it is worth it. It would be nice if api could return the colored original, but that would require saving the logs twice, which is also probably not worth it.

[Pijukatel]

Original log colors are now respected:

Thanks a lot @barjin for letting me know about the secret parameter of the log endpoint!

[Copilot]

Pull Request Overview

This PR introduces a new log redirection feature for Actor runs, allowing logs to be streamed and redirected to either a default or a custom logger. Key changes include:

• New synchronous and asynchronous methods (get_streamed_log) in the Run and Log clients for redirecting logs.
• Implementation of StreamedLog classes (sync and async) to handle chunked log messages and ensure proper formatting.
• Updates to the Actor client to support an optional logger parameter for redirecting logs, along with additional tests and dependency updates.

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/unit/test_logging.py	Added tests to verify correct log formatting and redirection behavior.
src/apify_client/clients/resource_clients/run.py	Introduced get_streamed_log methods for both sync and async clients.
src/apify_client/clients/resource_clients/log.py	Updated log retrieval methods with a new raw option and added StreamedLog classes for log streaming.
src/apify_client/clients/resource_clients/actor.py	Extended Actor.call method to support log redirection via an optional logger parameter.
src/apify_client/_logging.py	Added create_redirect_logger and improved RedirectLogFormatter with color support.
pyproject.toml	Added necessary dependencies for color support via colorama.

[Pijukatel]

No need to do deep review @MQ37. I think you might be using this functionality soon. Could you please take a look at the use-cases and see if it is convenient to use from your point of view and if there is missing something?
(Timeouts and status messages will be done in separate PR)

[janbuchar]

I haven't managed to read the whole thing yet, but here are some comments

[janbuchar]

By default, it is set to "default" which means that the default logger will be created and used

Um, and what does that actually mean for the caller? Let's try and come up with a better description.

[Pijukatel]

I tried to rephrase it like this:

.... Using "default" literal means that a predefined default logger will be used. ...

[janbuchar]

I'm confused with the intended usage here. Are you supposed to call this method and then pass the result into an actor.call(logger=...) call?

[Pijukatel]

No, actor.call(logger=...) is one group of use-cases that deal with you starting the other actor and then get_streamed_log is called internally.

Using get_streamed_log method directly has different use-case. It is when you want to stream log from already started actor (for example long-running actor in standby)

async with await Actor.new_client().run("some_id").get_streamed_log(from_start=False):
    # Do stuff while logs from other actor are redirceted

[MQ37]

Just checked the example usage and from UX viewpoint this looks good, thank you 👍 @Pijukatel

[vdusek]

Does the Actor call log propagation work recursively? (Actor calls an Actor who calls another Actor...)

[Pijukatel]

Does the Actor call log propagation work recursively? (Actor calls an Actor who calls another Actor...)

That is a really good point. It does if the actors were build with version that includes this change. On the other hand the line breaking does not work as expected anymore as the time marker is present for each log (so with two level of log redirection it will be there 3 times and thus there will be extra split)

See example log from this recursive actor:

Removing timestamp in redirected logs is probably also not OK as those timestamps are important especially for logs that happened before the streaming started.
That is something to figure out before merging this.

[Pijukatel]

Does the Actor call log propagation work recursively? (Actor calls an Actor who calls another Actor...)

I updated the split pattern and test data to properly respect the real data and now it works. See example of 4 level recursive actor and it's logs:

[vdusek]

Could we do it better here? Maybe pytest-mark-usefixtures.

[Pijukatel]

@janbuchar I could not find the comment, but I remember we were already discussing it some time ago in another review and you were in favor of # noqa.

I have no problem using this fixture and I am also OK with the # noqa, so if you agree with @vdusek on the preferred way in our code base, I will just use it.

[janbuchar]

I'm mostly OK with both, but usefixtures does have the drawback of not being applicable to other fixtures...

[vdusek]

alright, so it's up to you Pepa 🙂

[Pijukatel]

no change then :-)

[vdusek]

LGTM