Skip to content

Conversation

vsnever
Copy link
Contributor

@vsnever vsnever commented Jan 29, 2020

This is a draft pull request for fast emitters and integrators that work with regular grids, It was originally proposed for Cherab (issue #204) but, as @CnlPepper suggested, these emitters and integrators are more suitable for Raysect.

I tried to address all the issues raised in the discussion.

The emitters work as follows:

  • The emission can be provided as a 4D array, 2D array or scipy sparse matrix along with array of wavelengths for which this emission is specified. The units are: W / (m^3 str nm) for continuous spectrum and W / (m^3 str) for discrete spectrum (spectral lines). The user specifies the type of spectrum with continuous argument, which is True by default. RegularGridEmitter stores the emission as a compressed sparse column-major matrix (scipy.sparse.csc_matrix) for fast column access required for resampling.
  • Similar to SpectralFunction, the RegularGridEmitter automatically rebuilds the cache when the existing cache is not match the ray spectral properties. The cache validity is checked each time the emission_function() or RegularGridIntegrator.integrate() is called. The cache is stored as a compressed sparse row-major matrix (scipy.sparse.csr_matrix) for fast row access required for volume integration. If continuous=True, the resampling is performed the same way as in InterpoatedSF, but with optional extrapolation. If continuous=False, the spectrum is not interpolated, instead the emission at each spectral bin sums the emissions at all spectral lines that fall into that bin, divided by the bin's width. Thus, the cache always contain the emission in W / (m^3 str nm).
  • Since RegularGridEmitter is designed to work with very large grids, it has a few memory-saving options.
    1. If the emission is provided in float32, the cache will be also stored in float32. Also, the user has an option to store the cache in float32 even if the emission is in float64.
    2. In multi-process rendering, each process creates it's own cache. This looks like a correct behaviour, but can easily "eat" all available memory. To prevent this from happening, the user can preset the cache with cache_build(min_wavelength, max_wavelength, bins) function before camera.observe() is called. All processes will use this cache as long as it remains valid (of course building the cache in advance is useless in dispersive rendering).
    3. The user has an option to override the cache with cache_override(new_cache, min_wavelength, max_wavelength), so if there is not enough memory to store both the original emission and the cache, the user can provide the cache only by initialising the emitter with an empty csc_matrix and then calling cache_override(). Again, this will not work in case of dispersive rendering.
  • The cache is directly accessible from C-interface through read-only memoryviews. All functions that control the cache are accessible in both Python and C.

Currently, building the cache is slow, and it's noticeable for large grids. I'll try to optimise it in the future.

This draft pull request contains 4 demos. The first one is a kind of tutorial.
The code has built-in documentation but I didn't update the .rst files yet. Please look through the code, and if it's OK in general, I'll update the documentation and make this PR ready for review.

Also, I have a question to @CnlPepper. The SpectralFunction has custom __getstate__(), __setstate__(), __reduce__() methods. Do I need them in RegularGridEmitter too?

@CnlPepper CnlPepper removed this from the Release v0.7 milestone Nov 7, 2020
@vsnever vsnever marked this pull request as ready for review March 3, 2021 23:01
@vsnever
Copy link
Contributor Author

vsnever commented Mar 3, 2021

I updated the documentation for the API and demonstrations. Regarding the __getstate__() and __setstate__() methods, I think these are not necessary in the material class.

vsnever and others added 6 commits February 10, 2022 14:28
Cython 3.0 release introduces breaking changes. Pin to earlier Cython
until the code has been updated for compatibility.

Make a new post1 release of this so downstream packages will pick up
the working version of 0.8.1.
jacklovell and others added 6 commits December 11, 2023 14:37
When installing with pip or PyPA's `build` utility, , the `-j` option
for parallel builds is not passed by default. This increases the build
time on systems where wheels need to be built, e.g. for releases or
on systems where no pre-built wheel is available.

This commit introduces a modified `build_ext` command which by default
parallelises over all available CPUs, with an optional override using
the `RAYSECT_BUILD_JOBS` environment variable if for any reason the
number of build processes needs to be controlled manually.
Wheels for Python 3.7 to 3.9 can be built using manylinux2010, as
there are binary wheels of oldest-supported-numpy available for these
versions. For 3.10 onwards manylinux2014 is required for numpy binary
wheels. Building numpy from source in the manylinux containers is not
viable.

There are now two scripts, one for each manylinux version, which will
build wheels for all upstream-supported Python versions. These can be
run as-is: info is in the first few lines of these files. Also, the
dev notes on building wheels wtih manylinux have been updated to
reference these files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants