Context/Scope of vars not working when schema/mixin imported from module

[diefans]

Bug Report

I am very confused, how the scope/context is applied resp. how _default_config is resolved.

My naive understanding was (before I moved the definitions to the module) that the scope/context is created at "init-time" and is somehow passed/"shining" from module level through the deeper schema tree... but now this issue confuses my a lot...

1. Minimal reproduce step (Required)

If I move schema/mixin/protocol definitions to a module a context var is not found: the working example is here

# INFO: this is in schemata.foo
protocol LoginProtocol:
    a_url: str
    b_url: str

mixin DefaultLoginMixin for LoginProtocol:
    # INFO: If no a_url, b_url is defined in the final schema, we apply the
    # value from _default_login
    a_url: str = a_url or _default_login.a_url
    b_url: str = b_url or _default_login.b_url

schema Config:
    mixin [
        DefaultLoginMixin
    ]

schema Stage:
    [...str]: Config
    _default_login: LoginProtocol

import schemata.foo

ci00 = foo.Stage {
    # FIXME: how is _default_login reaching the mixin, when defined in one script,
    # but not if the mixin is imported from a module?
    _default_login = foo.LoginProtocol {
        a_url = "https://default.a_url"
        b_url = "https://default.b_url"
    }
    provider_1 = foo.Config {}
    provider_2 = foo.Config {
        # INFO: override the default
        a_url = "https://custom.a_url"
        b_url = "https://custom.b_url"
    }
}

2. What did you expect to see? (Required)

ci00:
  provider_1:
    a_url: https://default.a_url
    b_url: https://default.b_url
  provider_2:
    a_url: https://custom.a_url
    b_url: https://custom.b_url

3. What did you see instead (Required)

EvaluationError
 --> /home/olli/code/bm/integration-config/schemata/foo.k:8:1
  |
8 |     a_url: str = a_url or _default_login.a_url
  |  invalid value 'UndefinedType' to load attribute 'a_url'
  |

4. What is your KCL components version? (Required)

❯ kcl --version
kcl version v0.10.8

[diefans]

Even if I define the Config schema in the same module, as where _default_config is created/instantiated/scoped, it still does not work. So this seems to defy the meaning of "mixin"...

import schemata.foo

schema Config:
    # FIXME: seems like resolution of _default_login is just looking into,
    # where the mixin is defined and not where it is applied/used/mixed-in
    mixin [
        foo.DefaultLoginMixin
    ]

schema Stage:
    [...str]: Config
    _default_login: foo.LoginProtocol

ci00 = Stage {
    # FIXME: how is _default_login reaching the mixin, when defined in one script,
    # but not if the mixin is imported from a module?
    _default_login = foo.LoginProtocol {
        a_url = "https://default.a_url"
        b_url = "https://default.b_url"
    }
    provider_1 = Config {}
    provider_2 = Config {
        # INFO: override the default
        a_url = "https://custom.a_url"
        b_url = "https://custom.b_url"
    }
}

[diefans]

@Peefy can you confirm, that my observation makes sense to you...? I also tried to find the place in the code, where I can start to fix it by myself... (resolving this would greatly improve our kcl code)

[Peefy]

cc @zong-zhe @He1pa

[He1pa]

It seems that you have some misunderstanding about the use of mixin and protocol

We have an example about Protocol and Mixin: https://www.kcl-lang.io/docs/next/reference/lang/tour#advanced

For your case, you can implement it with the following code

schema DefalutConfig:
    a_url: str
    b_url: str

protocol LoginProtocol:
    _default_login: DefalutConfig
    _custom_a_url?: str
    _custom_b_url?: str

mixin DefaultLoginMixin for LoginProtocol:
    a_url: str = _custom_a_url or _default_login.a_url
    b_url: str = _custom_b_url or _default_login.b_url

schema Config:
    mixin [
        DefaultLoginMixin
    ]
    _default_login: DefalutConfig
    _custom_a_url?: str
    _custom_b_url?: str

schema Stage:
    [...str]: Config
    _default_login: DefalutConfig

ci00 = Stage {
    _default_login = DefalutConfig {
            a_url = "https://default.a_url"
            b_url = "https://default.b_url"
        }
    provider_1 = Config {
        _default_login = _default_login
        
    }
    provider_2 = Config {
        _default_login = _default_login
        _custom_a_url = "https://custom_a_url.a_url"
        _custom_b_url = "https://custom_b_url.b_url"
    }
}

[diefans]

Thanks @He1pa for the example, and I can follow it, but (and I should have copied the working example into this ticket to outline my amazement)... I am still wondering if this is a bug or a feature, since it is working without explicitly introducing _default_login into the scope of Config {...}

protocol LoginProtocol:
    a_url: str
    b_url: str

mixin DefaultLoginMixin for LoginProtocol:
    # INFO: If no a_url, b_url is defined in the final schema, we apply the
    # value from _default_login
    a_url: str = a_url or _default_login.a_url
    b_url: str = b_url or _default_login.b_url

schema Config:
    mixin [
        DefaultLoginMixin
    ]

schema Stage:
    [...str]: Config
    _default_login: LoginProtocol

ci00 = Stage {
    # FIXME: how is _default_login reaching the mixin, when defined in one script,
    # but not if the mixin is imported from a module?
    _default_login = LoginProtocol {
        a_url = "https://default.a_url"
        b_url = "https://default.b_url"
    }
    provider_1 = Config {}
    provider_2 = Config {
        # INFO: override the default
        a_url = "https://custom.a_url"
        # the value comes from _default_login here
        # b_url = "https://custom.b_url"
    }
}
config = Config {
    # without a_url and b_url we get an error here
    a_url = "https://must.be.set.a_url"
    b_url = "https://must.be.set.b_url"
}

resulting in

ci00:
  provider_1:
    a_url: https://default.a_url
    b_url: https://default.b_url
  provider_2:
    a_url: https://custom.a_url
    b_url: https://default.b_url
config:
  a_url: https://must.be.set.a_url
  b_url: https://must.be.set.b_url

So the bug I was assuming I have found, was, that when moving the definition of DefaultLoginMixin to another module this is throwing the aforementioned error - it would be nice, that you give an explanation why this is working and if it is supposed to be in accordance to the spec.

My explanation that this works is, that Stage._default_login provides the context for the mixin attached to Config, since this is "embedded" in Stage and the mixin is just picking up the surrounding context. Defining Config outside of Stage requires to set a_url and b_url (which makes sense).

[He1pa]

This is an undesigned situation. We never considered that Protocol appears as an attribute in attr. like

schema Stage:
    [...str]: Config
    _default_login: LoginProtocol

Protocol is some assembly logic of Mixin. And Mixin is used in the way of mixin[xxx]. Maybe some checks should be added to avoid this usage.

[diefans]

@He1pa this protocol feature is only a "byproduct" of my issue... sorry

The main topic/question is: why is the example working, when everything (mainly DefaultLoginMixin definition) is in one file/module and why is it not working when I put the DefaultLoginMixin definition into different module and import it from there?

I have created a repo for this issue, so you can just check it out and test easily the difference - since I have the impression, that my issue is not really understood. I have also removed the usage of protocol-as-a-schema: https://github.com/bettermarks/kcl-issue-1777

So the issue is about the import of the mixin and why does it break.

[He1pa]

ok， I will check again

[cblkwell]

I'm running into what looks like a similar issue to me. I've been working on some prototyping for a KCL deployment, and I'm getting this error:

$ kcl my-conf.k render/render.k -D image="ghcr.io/image" -D cloud="gcp"
Error: EvaluationError
   --> /Volumes/code-repos/kcl-prototype/models/mixins/deployment.k:111:1
    |
111 |                     for _i, _s in _d.ref_secrets if _d.ref_secrets \
    |  invalid value 'UndefinedType' to load attribute 'ref_secrets'
    |

when I try to run kcl my-conf.k render/render.k -D image="ghcr.io/image" -D cloud="gcp" in the root of this repo/commit:

https://github.com/cblkwell/kcl-prototype/tree/deployment-secrets-broken

I'm not sure what is undefined in this particular case.

[cblkwell]

To update with another example, I broke application code out from module code here, and now running kcl run -D image="ghcr.io/image" -D cloud="aws" in example/my-prototype/ gets me this error:

EvaluationError
  --> /Users/cblackwell/.kcl/kpm/kcl-prototype_c570175/models/backend/app_backend.k:79:1
   |
79 |   secretProviderClasses: [secretsv1.SecretProviderClass] = sum([[serviceSecret, metadataSecret],globalRefSecrets],[])
   |  can only concatenate list (not "UndefinedType") to list
   |

This was previously working when the application code was consolidated in a single file and I was working in the module, rather than including it as a dependency.

[diefans]

@cblkwell

I'm not sure what is undefined in this particular case.

_d is undefined... I was trying to have a look into your repo, but it seems you deleted it...

[cblkwell]

@cblkwell

I'm not sure what is undefined in this particular case.

_d is undefined... I was trying to have a look into your repo, but it seems you deleted it...

Sorry about that, I had to move it -- I'll get the example up again shortly.

[cblkwell]

Okay, here's the correct reference with a tag; if you run the kcl run -D image="foo" cloud="aws" in example/my-prototype/sector/my-app-prod-aws-use1, you should see the error: https://github.com/cblkwell/kcl-prototype/tree/deployment-mixin-bug