Skip to content

[BUG]: Stage destructors not called #1114

Closed

Description

Version

23.11

Which installation method(s) does this occur on?

Source

Describe the bug.

I noticed that the destructor for a C++ stage isn't called, even when the pipeline has gone out of scope, and gc.collect() has been called.

Minimum reproducible example

import gc
import logging
import os
import shutil
import tempfile

from morpheus.utils.logger import configure_logging
from morpheus.config import Config
from morpheus.config import CppConfig
from morpheus.pipeline import LinearPipeline
from morpheus.stages.input.file_source_stage import FileSourceStage
from morpheus.stages.output.write_to_file_stage import WriteToFileStage

MORPHEUS_ROOT_DIR = os.environ['MORPHEUS_ROOT']
TEST_DF = os.path.join(MORPHEUS_ROOT_DIR, "tests", "tests_data", "filter_probs.csv")


def run_pipe(out_file: str):
    config = Config()
    pipe = LinearPipeline(config)
    pipe.set_source(FileSourceStage(config, filename=TEST_DF))
    pipe.add_stage(WriteToFileStage(config, filename=out_file, overwrite=False))
    pipe.run()


def wrapper(logger: logging.Logger):
    tmp_dir = tempfile.mkdtemp()
    out_file = os.path.join(tmp_dir, "test.csv")

    logger.debug("Running pipe")
    try:
        run_pipe(out_file)
    finally:
        logger.debug("Pipe finished, cleaning up")
        shutil.rmtree(tmp_dir)
        logger.debug("Calling gc.collect()")
        gc.collect()
        logger.debug("gc.collect() - done")


if __name__ == "__main__":
    configure_logging(log_level=logging.DEBUG)
    logger = logging.getLogger("morpheus.tests.test_destructors")
    #CppConfig.set_should_use_cpp(False)
    # gc.set_debug(gc.DEBUG_LEAK)
    wrapper(logger)

Relevant log output

No response

Full env printout

No response

Other/Misc.

No response

Code of Conduct

  • I agree to follow Morpheus' Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

  • Status

    Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions