Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the API thread-safe #377

Open
FFY00 opened this issue Oct 12, 2021 · 6 comments
Open

Make the API thread-safe #377

FFY00 opened this issue Oct 12, 2021 · 6 comments
Labels
enhancement New feature or request

Comments

@FFY00
Copy link
Member

FFY00 commented Oct 12, 2021

As per the discussion in #375.

This will require us to ditch pep517 for our own backend invocation implementation.

I will try to look into it this weekend.

@FFY00 FFY00 added the enhancement New feature or request label Oct 12, 2021
@layday
Copy link
Member

layday commented Oct 12, 2021

I don't understand why we need to ditch pep517 to make the API thread-safe, can you explain?

@FFY00
Copy link
Member Author

FFY00 commented Oct 12, 2021

Sorry, we are not required to. If we stay with pep517, we will have to at least ditch their runners and implement a custom runner that calls our own runner API.
IMO, it would make sense to take the opportunity to drop it as it would make bootstrapping much easier, and would allow us to do further optimizations, like using the same process when possible.

@layday
Copy link
Member

layday commented Oct 12, 2021

We just have to pass some env vars I thought to the runner - all we'd have to do is make sure we're wrapping the pep517/user-provided runners.

@FFY00
Copy link
Member Author

FFY00 commented Oct 12, 2021

No, we need to change the runner API as the one in pep517 does not let us set the environment, only update it.

@FFY00
Copy link
Member Author

FFY00 commented Oct 12, 2021

Actually, we can get by setting the environments we need to clear in #375 to an empty string, but it might break code that checks for the environment variable, it should be fine though.

layday added a commit to layday/build that referenced this issue Oct 24, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.

Factored out parsing `pyproject.toml`.

Fixed passing the absolute path to the srcdir validator in the project
builder.

Reworded `ProjectBuilder` docstrings for clarity.

Moved runner logic to the isolated env as prelude to fixing pypa#377.
As a consequence the runner is not replaceable when using isolation.

Removed pip version detection.  The default env updates pip in a subprocess.
layday added a commit to layday/build that referenced this issue Oct 24, 2021
layday added a commit to layday/build that referenced this issue Oct 25, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.

Factored out parsing `pyproject.toml`.

Fixed passing the absolute path to the srcdir validator in the project
builder.

Reworded `ProjectBuilder` docstrings for clarity.

Moved runner logic to the isolated env as prelude to fixing pypa#377.
As a consequence the runner is not replaceable when using isolation.

Removed pip version detection.  The default env updates pip in a subprocess.
layday added a commit to layday/build that referenced this issue Oct 25, 2021
layday added a commit to layday/build that referenced this issue Oct 25, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.

Factored out parsing `pyproject.toml`.

Fixed passing the absolute path to the srcdir validator in the project
builder.

Reworded `ProjectBuilder` docstrings for clarity.

Moved runner logic to the isolated env as prelude to fixing pypa#377.
As a consequence the runner is not replaceable when using isolation.

Removed pip version detection.  The default env updates pip in a subprocess.
layday added a commit to layday/build that referenced this issue Oct 25, 2021
layday added a commit to layday/build that referenced this issue Oct 25, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.

Reworded `ProjectBuilder` docstrings for clarity.

Moved runner logic to the isolated env as prelude to fixing pypa#377.
As a consequence the runner is not replaceable when using isolation.

Removed pip version detection.  The default env updates pip in a subprocess.
layday added a commit to layday/build that referenced this issue Oct 25, 2021
layday added a commit to layday/build that referenced this issue Oct 28, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.

Moved runner logic to the isolated env as prelude to fixing pypa#377.
As a consequence the runner is not replaceable when using isolation.

Removed pip version detection.  The default env updates pip in a subprocess.
layday added a commit to layday/build that referenced this issue Oct 28, 2021
layday added a commit to layday/build that referenced this issue Oct 28, 2021
layday added a commit to layday/build that referenced this issue Oct 28, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.

Moved runner logic to the isolated env as prelude to fixing pypa#377.
As a consequence the runner is not replaceable when using isolation.

Removed pip version detection.  The default env updates pip in a subprocess.
layday added a commit to layday/build that referenced this issue Oct 28, 2021
layday added a commit to layday/build that referenced this issue Oct 28, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.

Moved runner logic to the isolated env as prelude to fixing pypa#377.
As a consequence the runner is not replaceable when using isolation.

Removed pip version detection.  The default env updates pip in a subprocess.
layday added a commit to layday/build that referenced this issue Oct 28, 2021
layday added a commit to layday/build that referenced this issue Oct 28, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.

Moved runner logic to the isolated env as prelude to fixing pypa#377.
As a consequence the runner is not replaceable when using isolation.

Removed pip version detection.  The default env updates pip in a subprocess.
layday added a commit to layday/build that referenced this issue Oct 28, 2021
layday added a commit to layday/build that referenced this issue Oct 29, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.

Moved runner logic to the isolated env as prelude to fixing pypa#377.
As a consequence the runner is not replaceable when using isolation.

Removed pip version detection.  The default env updates pip in a subprocess.
layday added a commit to layday/build that referenced this issue Oct 29, 2021
layday added a commit to layday/build that referenced this issue Oct 29, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

Moved env mangling logic to the `IsolatedEnv` as prelude to fixing pypa#377.
The runner is, for now, not replaceable when using isolation.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.
`ProjectBuilder.script_dir` was removed. Adding the `script_dir` to the
path is the responsibility of the isolated env.

Removed pip version detection.  The default env updates pip in a subprocess.
layday added a commit to layday/build that referenced this issue Oct 29, 2021
layday added a commit to layday/build that referenced this issue Oct 29, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

Moved env mangling logic to the `IsolatedEnv` as prelude to fixing pypa#377.
The runner is, for now, not replaceable when using isolation.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.
`ProjectBuilder.script_dir` was removed. Adding the `script_dir` to the
path is the responsibility of the isolated env.
layday added a commit to layday/build that referenced this issue Oct 29, 2021
layday added a commit to layday/build that referenced this issue Oct 29, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

Moved env mangling logic to the `IsolatedEnv` as prelude to fixing pypa#377.
The runner is, for now, not replaceable when using isolation.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.
`ProjectBuilder.script_dir` was removed. Adding the `script_dir` to the
path is the responsibility of the isolated env.
layday added a commit to layday/build that referenced this issue Oct 29, 2021
layday added a commit to layday/build that referenced this issue Oct 29, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

Moved env mangling logic to the `IsolatedEnv` as prelude to fixing pypa#377.
The runner is, for now, not replaceable when using isolation.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.
`ProjectBuilder.script_dir` was removed. Adding the `script_dir` to the
path is the responsibility of the isolated env.
layday added a commit to layday/build that referenced this issue Oct 29, 2021
layday added a commit to layday/build that referenced this issue Oct 30, 2021
layday added a commit to layday/build that referenced this issue Oct 30, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

Moved env mangling logic to the `IsolatedEnv` as prelude to fixing pypa#377.
The runner is, for now, not replaceable when using isolation.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.
`ProjectBuilder.script_dir` was removed. Adding the `script_dir` to the
path is the responsibility of the isolated env.
layday added a commit to layday/build that referenced this issue Oct 30, 2021
layday added a commit to layday/build that referenced this issue Nov 2, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

Moved env mangling logic to the `IsolatedEnv` as prelude to fixing pypa#377.
The runner is, for now, not replaceable when using isolation.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.
`ProjectBuilder.script_dir` was removed. Adding the `script_dir` to the
path is the responsibility of the isolated env.
layday added a commit to layday/build that referenced this issue Nov 2, 2021
layday added a commit to layday/build that referenced this issue Nov 2, 2021
layday added a commit to layday/build that referenced this issue Nov 2, 2021
layday added a commit to layday/build that referenced this issue Nov 2, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

Moved env mangling logic to the `IsolatedEnv` as prelude to fixing pypa#377.
The runner is, for now, not replaceable when using isolation.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.
`ProjectBuilder.script_dir` was removed. Adding the `script_dir` to the
path is the responsibility of the isolated env.
layday added a commit to layday/build that referenced this issue Nov 2, 2021
layday added a commit to layday/build that referenced this issue Nov 3, 2021
layday added a commit to layday/build that referenced this issue Nov 5, 2021
The `IsolatedEnv` has been reworked to subsume env creation
logic.

`IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

The `IsolatedEnv` takes IsolatedEnvManager` as an argument
instead of passing the manager's attributes to the env piecemeal.

`ProjectBuilder.from_isolated_env` has been added to auto-fill
init args from an isolated env.

Moved env mangling logic to the `IsolatedEnv` as prelude to fixing pypa#377.
The runner is, for now, not replaceable when using isolation.

The `ProjectBuilder`'s properties were made read-only. Mutating
the builder is not necessary with the addition of `from_isolated_env`.
`ProjectBuilder.script_dir` was removed. Adding the `script_dir` to the
path is the responsibility of the isolated env.
layday added a commit to layday/build that referenced this issue Nov 5, 2021
layday added a commit to layday/build that referenced this issue Nov 6, 2021
* `IsolatedEnv` reworked to subsume env creation logic.

* `IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

* `ProjectBuilder.from_isolated_env` added to auto-fill init args from
  an isolated env.

* Made the `IsolatedEnv` responsible for customising the environ as a prelude
  to fixing pypa#377.  Removed `ProjectBuilder.scripts_dir`.

* `ProjectBuilder`'s @properties made read-only. Mutating the builder is not
  necessary with the addition of `ProjectBuilder.from_isolated_env`.
layday added a commit to layday/build that referenced this issue Nov 6, 2021
layday added a commit to layday/build that referenced this issue Nov 6, 2021
* `IsolatedEnv` reworked to subsume env creation logic.

* `IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

* `ProjectBuilder.from_isolated_env` added to auto-fill init args from
  an isolated env.

* Made `IsolatedEnv` responsible for customising the environ as a
  prelude to fixing pypa#377.  Removed `ProjectBuilder.scripts_dir`.

* `ProjectBuilder`'s @properties made read-only. Mutating the builder
  is not necessary with the addition of
  `ProjectBuilder.from_isolated_env`.
layday added a commit to layday/build that referenced this issue Nov 6, 2021
layday added a commit to layday/build that referenced this issue Jan 7, 2022
* `IsolatedEnv` reworked to subsume env creation logic.

* `IsolatedEnvBuilder` renamed to `IsolatedEnvManager`.

* `ProjectBuilder.from_isolated_env` added to auto-fill init args from
  an isolated env.

* Made `IsolatedEnv` responsible for customising the environ as a
  prelude to fixing pypa#377.  Removed `ProjectBuilder.scripts_dir`.

* `ProjectBuilder`'s @properties made read-only. Mutating the builder
  is not necessary with the addition of
  `ProjectBuilder.from_isolated_env`.
@konstin
Copy link

konstin commented Dec 27, 2022

What's the status on this? After 1611da0 ProjectBuilder(src_dir).metadata_path(metadata_dir) seems to work for me in parallel in a thread pool, but would like to avoid deploying something that might have a race condition hidden

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants