JDK-8217472: 3D-API - Attenuation-coefficients for PointLight

[ProggerFox]

Good evening folks,

while playing on a 3d scene, i noticed that the PointLight (very specific, maybe other use-cases for it might come up too) lacked a way to set the radius of how far the light is emitted from the light-source.

Regarding this i was hoping there would be a operation like the "setInfluencingBounds"-method (hence the title) as in the predecessor Java3D available, since the PointLights-emission procedure seems to also rely on its BoundTransforms.
Sadly there is no way of modifying this bit and therefore the PointLight seems rather limited because of it, because not every lamp or similiar light-entity emits light in the same manner.

Maybe i overlooked certain steps and someone might take some time to elaborate on how to take on this issue, but else i would assume it to be a missing feature.

Thanks in advance!

[nlisker]

What you are looking for are attenuation factors. These need to be added at the native graphics pipelines level, which is a good amount of work. I can help with the D3D pipeline, but someone else has to do the others.

As a workaround, you can find all the nodes a certain distance from your source and add them to the light's scope. It's not perfect, but it's something.

Lighting and materials in general can use a lot of work, like adding directional lights and spotlights, an emissive component etc.

[ProggerFox]

Thanks for your reply,

it would be very appreciated if you could help out with the D3D pipeline.
Luckily i also dealed with OpenGL a (longer) while back, so i would gladly help out with the ES2 pipeline.
I never heard about the SW/(Pisces?) pipeline though and cant find references quickly at the moment, so hopefully someone will be able to deal with this pipeline as well.

The idea with adding the nodes to the scope came up to me as well, though the core problem of the light-emission still remains, with only the nodes in the scope being affected then of course.
As closest to the result i could get was via a self-illuminated object (by using a Sphere with a illumination-texture in a material), though it was more of a hack than a full-fledged solution.
Working on a native solution instead seems to be a better time spent in the end though.

Well, i would gladly try my luck to work on these missing components then, since it is also in my interest to enchance the capabilities of my favourite framework :)

[nlisker]

SW is the software pipeline and it doesn't support 3D. Note that Apple are phasing out OpenGL support in favor of Metal.

We'll need:

• A concrete proposal of what we want to get done. I suggest we start from something simple like attenuation factors for the current point light implementation and see how that goes (see caveat below). When we have that I'll open a JBS issue.
• To gather all the interested people who can help; a mail on the discussion mailing list (openjfx-discuss@openjdk.java.net) is a good place to start as this has been brought up before in some form or another.
• A guarantee that this isn't a drive-by feature. That means that the contributors will provide support for this feature in the longer run (bug fixes etc.).
• Approval from the co-leads - Johan or Kevin - to implement it.

One thing that caught my eye while I was looking at the D3D implementation a while back was that the point light implementation is a custom one and not the point light given by the D3D API. I don't know why that is, maybe to unify the behavior across platforms. I'll re-ask about releasing Prism internal documentation.

In any case, if you intend to work on OpenJFX you'll need to sign the OCA if you haven't done so already.

[nlisker]

I asked for the Prism documentations, but it's not guaranteed we'll get any. Meanwhile, I'll do some experimentation in d3d with the point light and see what kind of mayhem it causes. If you can try to meddle with the OpenGL pipline we'll get an idea of what we are up against.

[ProggerFox]

Evening,

sorry for my long absence, job and building up the working environment.
I prepared most of the steps you mentioned, this i will shortly open a new thread to ask for more help.
About the guarantee you asked for, could you elaborate more on that? And what would be the terms, because of course it is in my main interest to improve and stabilize this framework (since i also use it mainly for my projects), but i can not guarantee that i will always be available.

Also i sent the OCA to Oracle, approval stil needs to be given, after that i will public something here so you can check out my findings.
Regarding this i took some researchs into the ES2 pipeline and it seems that there might be a need for additional shaders or at least slights modifications to existings shaders to implement the attenuation factors.
There are even Lights-functions in OpenGL directly use the underlying PointLight (also Directional/Spot-Lights too), it might be enough to just use these and if not, then the old way of plainly adding and modifying values it is then.
No big deal in the end, the bridge for Prism-JavaFX might need more work, but definitely doable.
I will start with this after i finished the mentioned steps and got the approval.

Else i am working on this case, just to let you know.

Cheers!

[ProggerFox]

Regarding the Apple's pipeline-change, we should try to settle this as fast as well, since it will surely affect how we should tackle our own matters.
This we should further discuss in the corresponding issue, though.

To add to that i would tend to Vulkan + MoltenVK, simply for the reason that by going this route we ensure a higher portability.
Also they provide a specific solution to enable the Vulkan-pipeline on Macs, via converting from Vulkan to Metal shaders, thus i doubt that there will be such a big performance hit in the end.
This is at least from my perspective, in the end we should of course try to go with the route the community decides.

[nlisker]

About the guarantee you asked for, could you elaborate more on that?

Not really. It's not a formal agreement. One of the things the co-leads evaluate when we ask for permission to implement this is if someone will be able to maintain it. Otherwise, people will add their code and leave, leaving technical debt. If you plan to stay around (I certainly do) I guess it will be fine, but they will decide.

Also i sent the OCA to Oracle

Good. It will take some time to be approved, but we can work meanwhile. It just stops you from submitting a PR/patch.

Regarding this i took some researchs into the ES2 pipeline and it seems that there might be a need for additional shaders or at least slights modifications to existings shaders to implement the attenuation factors.
There are even Lights-functions in OpenGL directly use the underlying PointLight (also Directional/Spot-Lights too), it might be enough to just use these and if not, then the old way of plainly adding and modifying values it is then.

If I understand you correctly, the ES2 pipeline defines a custom point light instead of using the OpenGL functions? That's the case for D3D. The thing is, the material (I guess what you call shader) and lighting are coupled, so if I use the D3D PointLight implementation it won't affect the material. Is this what you mean?
So I think we have options:

1. Mimic the D3D/OpenGL lights by expanding the current implementation.
2. Replace the current implementation with the libraries implementation.

Not sure which is worse :) The first one is more flexible and we might avoid inconsistencies between the pipelines, the second one has ready-to-use implementations but seems more ambitious.

Regarding the Apple's pipeline-change, we should try to settle this as fast as well, since it will surely affect how we should tackle our own matters.
This we should further discuss in the corresponding issue, though.
...

Yeah, the graphics discussion has been brought up several times in the past, it will probably be brought up again when we start this discussion, and I'm guessing it will be brought up again after the work is done...

#193
http://mail.openjdk.java.net/pipermail/openjfx-discuss/2018-October/000025.html
http://mail.openjdk.java.net/pipermail/openjfx-discuss/2018-October/000038.html
http://mail.openjdk.java.net/pipermail/openjfx-discuss/2018-October/000001.html
http://mail.openjdk.java.net/pipermail/openjfx-dev/2018-June/021965.html
More discussion from February if you want to dig...

No big deal in the end, the bridge for Prism-JavaFX might need more work, but definitely doable.
I will start with this after i finished the mentioned steps and got the approval.

That's great. This week might be loaded for me but I'll try to squeeze in a research at an adequate level to get a better idea of what needs to be done on D3D.

[ProggerFox]

Not really. It's not a formal agreement. One of the things the co-leads evaluate when we ask for permission to implement this is if someone will be able to maintain it. …

Then this wont be much of a problem.
It is not in my intention to leave this issue unfinished, so there is my guarantee ;)

If I understand you correctly, the ES2 pipeline defines a custom point light instead of using the OpenGL functions?

ES2 surely uses the regular OpenGL light-sources, there should be no custom implementation of any lightings, as far as i know.
At the moment there are Light-structs created in the shaders, which simply contain color- and location-vectors, which are further on used in the calculation for the final result (overall illumination of (a point of) the target)
These values will be used later on in creating said lightings in the renderer.

But i proposed, that we could utilize the said "native OpenGL"-lightings via the shader (i recall it was possible to create light-sources in that way too) and thus "just" use the native lightings in OpenGL.
So instead of implementing own mathematics and calculations, we do a kind of "executive"-approach, by telling at which point which kind of lighting should be created/used.
Since OpenGL already is able to produce up to 8 different kind of lightings (Spot, Directional, Point, Ambient, etc..) by setting certain parameters, we might shift a little of the work for different types of lightings in ES2 at least.
But i dont know if this is truly applicable with the overall structure of JFX, that i will have to test yet (also i never did it that way before too).
Here is the proposed light-function in OpenGL by the way:
https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glLight.xml

The thing is, the material (I guess what you call shader) and lighting are coupled, so if I use the D3D PointLight implementation it won't affect the material. Is this what you mean?

The materials are generally affected by the lights in a 3d world, if i am not mistaken.
In the case of lights it is mostly the brightness and sometimes also the color value that affect the resulting material in overall.
But how is this realized in D3D then, if it doesnt affect the material?
In OpenGL the lights add to the resulting color of a material and depending on the light, this is at a certain spot only.

And a shader is basically a "little program" that is given to the GPU to execute (graphical) effects/results.
In fact, it is just a plain C-file (except it normally has the extensions .frag, .vert, etc... depending on what kind of shader it is) where one basically modify values, mostly by doing heavy mathematics, to achieve the corresponding values to produce said results.
To say the shader is not specifically the material, but it is the "producer" of said materials/ lightings, etc...
In a OpenGL-app (or in subsets) one has generally a set of different shaders to execute specific effects (or a combination of said shaders)
You can find the said shaders for ES2-JFX here, if you want to take a look into the existing shaders for yourself:
https://github.com/javafxports/openjdk-jfx/tree/develop/modules/javafx.graphics/src/main/resources/com/sun/prism/es2/glsl

And what i meant was that ultimately the shaders respsonsible for the lightning lack the attenuation factors in the calculation for the final output, so they either have to be added to the corresponding shaders or we add new shaders that extends said functionalities (unlikely and unfavoured for the moment)
I would tend towards the first option though, simply because all lights will make use of attenuation factors, thus it would make sense to implement it as a base functionality for all light-processings.
But then, as previously stated, i suggested the approach of using the native lights, if possible.
So tinkering with the mathematics in the shaders would be the solution after the first one would fail, thats my plan at least.

So I think we have options:
Mimic the D3D/OpenGL lights by expanding the current implementation.
Replace the current implementation with the libraries implementation.
Not sure which is worse :) The first one is more flexible and we might avoid inconsistencies between the pipelines, the second one has ready-to-use implementations but seems more ambitious

I cannot speak for D3D, but for ES2 it should be enough to be able to retrieve all the necessary datas to set up its attenuations factors, the rest of it will be done via the said shaders.
Not to speak of it as a simple matter, but i suspect a "simple bridge" to be able to set attenuation factor values via JFX and a functionality for retrieval from Prism will suffice for ES2 in the end.
So yes, if you mean by mimic the pipelines that we will use the pipeline-structures as reference to further expand the Prism/JavaFX-implementations, then yes, this is exactly what i was hoping for how it would turn out :)
Many of the functionalities, that seems to be lacking in JFX, already have industry- standard implementations that can be found everywhere around in the web (in fact i found an industry-standard for implementing attenuation factors in OpenGL already)
So hopefully the biggest load of work will be maintaining the Prism-layer in the end, and for it to mainly function as a data-provider for all of the pipelines, which in turn can be set on the JFX-side.

How exactly would you go on about working on your said options though?

Yeah, the graphics discussion has been brought up several times in the past, it will probably be brought up again when we start this discussion, and I'm guessing it will be brought up again after the work is done

The suggested route of the switch to ANGLE seems quite interesting... (http://mail.openjdk.java.net/pipermail/openjfx-dev/2018-June/021973.html)
We could also look into how to switch to another render-pipeline that is already maintained and provides a lot of functionality many of the JFX-devs seem to need.
This wont change our work though, and for the moment we should continue with Prism until a clear state is reached.
An issue regarding that should be opened then.

That's great. This week might be loaded for me but I'll try to squeeze in a research at an adequate level to get a better idea of what needs to be done on D3D

Same here, time is not benefical to me either.
But i already found somebody helping me out with working on the ES2-pipeline, so it wont be much of a problem.
Looking forward to your findings.

[nlisker]

Just updating that after I discussed this with Ambarish (one of the Oracle devs), he gave me some reading material, so I'm doing the reading now.

[nlisker]

Iv'e managed to implement attenuation. It's very preliminary, but as a POC it works. The pictures show a box's face with a light source in front, and I move the box farther away each time.

Did you have any luck on the OpenGL side?

[ProggerFox]

Happy new year!
And wow, there i decide to check this forum out after a long while and one gets a very pleasant surprise for the new year too ;)
Great job!
How did you accomplish your solution in D3D?

On my side, I worked out some modifications for the shaders, which was nothing great to be honest.
The Light-struct was simply extended with further parameters, a calculation to get the light intensity over a distance pre-returning the light was added to the corresponding fragment-shaders and missing uniforms were added to the render-instance.
Else most of the functionality was already provided, but apparently not utilized.
Though i had a slight hangup on the general "Bridge" for Prism at one point, for which i wanted to come up with a common solution with you before continuing further.
Sadly i recently have very little time on my hand, due to other projects and private matters, so progress wont happen as fast as expected.

Anyway, if possible i would like to modify my solution to be compatible with yours, if needed.
It is great that at least one of us is able to provide a display of progress, this may encourage others to use JavaFX for 3d-applications even more already :)

Also could you provide me the said reading material you got as well? Thanks!

[nlisker]

How did you accomplish your solution in D3D?

Not sure on what level you are talking about. On the shader level, I passed the world coordinates from the vertex shader to the pixel shader. Then I mocked a light position and calculated the distance between them and divided the diffuse color value by the distance (up to a constant). I didn't do anything outside of the Prism 3D3 graphics pipeline yet as I was working towards a POC only.

The solution requires to upgrade the shader models from 2.0 to 3.0, which is parallel to upgrading DirectX from 9.0 to 9.0c. This will require authorization by Kevin/Johan as it's a backwards compatibility issue.

Though i had a slight hangup on the general "Bridge" for Prism at one point, for which i wanted to come up with a common solution with you before continuing further.

Definitely. We need to think about the end result first. Attenuation is used only by some light types (https://docs.microsoft.com/en-us/windows/desktop/direct3d9/light-types):

• Ambient
• Directional
• Point
• Spot

So only they should have them. We will need 3 attenuation parameters (https://docs.microsoft.com/en-us/windows/desktop/direct3d9/attenuation-and-spotlight-factor) and a range parameter which is a hard cutoff for attenuation calculation. These will be DoublePropertys.

The mathematical behavior for each light should be (in pseudocode):

float attenuation;
float dist = distance(pixelPos, lightPos);
if (dist > range) {
    continue to next light; // same as attenuation = 0: light does not participate
} else {
        attenuation = 1/( a0 + a1 * dist  + a2 * dist ²);
}
float[3] diffuseColor = diffuseColor * attenuation;
float[3] specularColor = specularColor * attenuation;

Where the last 2 lines mean that the attenuation factor multiplies whatever the calculation was before. Optimizations not included.

As for the Java side, I can take care of it, or at least most of it. Last time I looked I didn't foresee any problem there.

Also could you provide me the said reading material you got as well? Thanks!

It's a collection of some very general to specific HLSL related links:

• https://stackoverflow.com/questions/38938498/how-do-i-convert-gl-fragcoord-to-a-world-space-point-in-a-fragment-shader
• http://developer.download.nvidia.com/CgTutorial/cg_tutorial_chapter01.html
• https://docs.microsoft.com/en-us/windows/desktop/direct3dhlsl/dx-graphics-hlsl-writing-shaders-9#uniform-shader-inputs
• https://docs.microsoft.com/en-us/windows/desktop/direct3dhlsl/dx-graphics-hlsl-semantics#migration-from-direct3d-9-to-direct3d-10-and-later
• https://docs.microsoft.com/en-us/windows/desktop/direct3d9/texture-coordinates

I'm going to start a discussion in openjfx-discuss@openjdk.java.net, please follow it and chime in when needed.

[nlisker]

Some experimentation with SpotLight implementation:

The red dot is the light source location. The light direction is +z (directed at the surface directly). The inner angle is 30 deg, below which the surface receives maximum intensity; the outer angle is 60 deg, above which the surface receives no light; the part between them is governed by a quadratic falloff using the formula given by Direct3D.

Here the light direction is (1, 0, 1) (45 deg between +x and +z).

Distance attenuation was not included for simplicity.