-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FlexibleConfig #2668
Comments
I need to find a little more time to look this over, but my first thought is that it probably relates to the CoreRegistry->Context switch (config in different contexts), adding more config at the world level, being able to modify module config after creating a world, etc. Not that we need to do all the things before we can complete this, just something to keep in mind, even for myself when I get a chance to think more about it :-) |
Understood. Ok, let me know if there is any change I should make to the proposed architecture. Meanwhile I have realized that if the subscribe/unsubscribe functionality is available through the Setting instance we also need functionality to mark a Setting as stale, meaning it is no longer associated with a config. I'm guessing we'll have to move the unsubscribe/subscribe functionality back into the FlexibleConfig itself, to keep things simple. |
Any update on this? I'd like to take up this issue. |
@Cervator: I just re-read your last comment. In the case of the RenderingConfig I'd normally implement it as a subclass of FlexibleConfig and I'd store its only instance in the main Context. Nodes can then publish their settings on it. Alternatively, as I can imagine it only as a singleton, I'd use a static method to retrieve it with I thought briefly of storing the settings straight into a "RenderingContext" but a context fundamentally is a <Class<?>, instance> map while a config would be more constrained: a <ResourceURN, Setting> map. Can you let us know if this fits with what you mentioned? @eviltak: thank you for volounteering. If cervator has no objections I'm happy for you to take care of this. I'll be in standby to review the PR when it becomes available. |
Well, I haven't thought it through but in general it is better to get a instance of a class from the context than from a singleton. As you then have the option in future to change it to be not a singleton if we need to. |
Looks like a good idea to me - it ties in nicely with the work done on tabbed UIs (thanks @jellysnake!) as well as making the rendering API more accessible to modules. @eviltak - feel free to work on this (and ping me if you want it added as a GCI task!) |
@flo: as you wish. Question: could we have a Context.getMainContext() static method? Or should we always get it through injection? Or in another way? It is sometimes unclear to me where to get the context from. |
@emanuele3d the context should be obtained from whatever created the instance of the class. If available via injection. In cases of static method it would have to be passed in by argument. |
Bump - a related config issue for world gen just came up and this seemed the most sensible existing issue to mention it on, tho we may split it out into its own thing and get an Epic going already with all this config stuff soon. The world gen tutorial uses an inner class implementing Maybe that system could be updated to use a more configuration-specific approach like suggested here, dropping Some other more or less related issues: #2509, #2068 / #2074, #1843. #1770, #1709 (this last one is yours btw @emanuele3d - or would it in theory be complete by now?) |
Does this let modules access arbitrary configs? Would be nice if Sucks that configs are currently limited to engine subsystems. |
@Cervator: I've implemented #1709 only in the RenderingConfig and RenderingDebugConfig. There are other Configs in Terasology and I have not touched those. So, it is complete from my (rendering) perspective but not from the perspective of the whole project. This issue about FlexibleConfigs might superseed #1709 though. In the description above it is only implied but I should probably make it absolutely clear that the vision is for implementations of FlexibleConfigs to make all settings automatically observable. |
@SoniEx2: yes. Both engine and modules would be able to store and retrieve arbitrary settings from a FlexibleConfig implementation. |
@emanuele3d So how's this supposed to work with HTTP? Any module can add stuff to their HTTP whitelist, literally making the whitelist useless? Why isn't this module-restricted? |
How are the HTTP whitelist and Configs related? |
@emanuele3d The way #2618 works is that there's a whitelist for TCP sockets, and that whitelist cannot be accessed/modified by modules. However, if there was an HTTP module (read: not subsystem) then its TCP whitelist would be basically available to any module using it (albeit restricted to the HTTP protocol only), unless it also implemented a whitelist. For HTTP to implement a whitelist, it needs to, well, have a config. That config needs to be accessible by it, but BY NO OTHER MODULE. Then, when a module requests a connection, it looks through the whitelist and checks if the specific module can access the specific address, and if it can then everything's good and stuff. You don't want modules to freely add or remove things to the HTTP whitelist. You want the server admin to control the HTTP whitelist. Having an HTTP module would also be just the start. We'd also have a database module (with the same whitelist mechanism as stated above), maybe an IRC module (not entirely sure), and so on. I don't think you want all those modules to be in the engine. (The database module itself would have to be massive, maybe bigger than the engine itself, due to how many database protocols there are out there.) |
Also you might wanna take a look at my changes to |
Thank you for clarifying this for me @SoniEx2. FlexibleConfig at this stage is just an idea for a way to handle existing configs and for modules to publish their settings. This will allow both users and code to access them in a more uniform fashion. That covers most needs at this stage. If something, i.e. a module's setting, needs to remain entirely or selectively secure, it doesn't have to published via an object implementing FlexibleConfig. You can use your own config class, use a non-public FlexibleConfig or, if it makes sense, you could extend FlexibleConfigs and implement a SecureConfig. Does that answer your question? |
Uh, alright. I might be able to do it a different way that doesn't require any configs, with the minor drawback that one won't be able to restrict the protocols a module uses... But thanks anyway! |
To clarify, this issue is just for the implementation of rendering config, isn't it? Usages of |
The needs of the rendering code were the inspiration for this issue, but this issue is really only about writing/implementing the FlexibleConfig described above and any other related interface/class so that engine and modules can take advantage of it to store and retrieve settings. Once that's done one separate, follow-up task emerges: a UI needs to be developed for the user to see and modify the content of a FlexibleConfig instance. I'm counting on @rzats to either chip-in on this or mentor somebody else on this. Once the UI is available existing configs can be migrated to instances implementing the FlexibleConfig interface, among them the RenderingConfig and the RenderingDebugConfig. So, for this issue I'm really only expecting a FlexibleConfig interface, a FlexibleConfigImpl class and any small satellite interface/class that is required, i.e. Setting/SettingImpl: no need to hook existing configs to it just yet. Does this clarify this issue? |
@emanuele3d yes, that makes everything clear. I'd love to work on other Also, do I need to create tests for |
Yes, especially in this case tests are a good idea, exactly for that reason. Looking forward to the PR! |
@emanuele3d I'm deciding against using a I'm also going for a simpler unserializable event dispatching system that may or may not use the existing Also, does |
Regarding not using an interface: I'm not sure when interfaces became incompatible with composition, but I'm sure I'll understand what you mean when I see the code. 😉 Regarding using/not using PropertyChangeListener: I took advantage of it in the rendering configs due to the discussion I had with msteiger and immortius in #1709. I guess you have to keep in mind the risks while attempting to re-invent the wheel, but if in the process you invent a wing, or an anti-gravity plate, why not? 😄 Regarding Setting, when I wrote about ranges I was thinking more about aids to the UI than validity checks when storing a new value. But it sounds like a good idea to have validity checks. And I guess each Setting subclass, i.e. a FloatSetting, can implement the checks that make sense for it. The only thing I'd suggest in this context would be to keep as much as possible optional. I.e. sometimes it make sense for a setting of type float to have a range, but sometimes it make sense to leave it unbounded or only partially bounded. Something to keep in mind. |
@emanuele3d is it alright if the values returned by |
Let's think for a moment how we would like an engine or module developer interact with a flexible config. Out of the top of my head, I envision it this way:
So, as you can see I'm assuming whoever wants to retrieve a Setting knows what kind of setting she is getting. So the cast is not when one retrieves the value but when one retrieves the setting. The UI on the other hand will have to find what each setting's type is. But it appears getClass() returns the subclass rather than the superclass and this should be useful to have an <Handler, SettingSubClass> map so that each Setting instance retrieved from a FlexibleConfig object can be processed by the right handler. Does this help? |
@emanuele3d To make the |
Ok, please take the piece of fictional code above and show me how it would look like with your architecture. |
It would look similar to the following:
|
And where do you plan to add functionality such as ranges for numerical settings? |
Each |
Understood. As the UI will eventually go through the list of setting stored in the configs in bulk, how will it know how to deal with an instance? I.e. let's assume a float setting with a range. How will it find the retrieved setting contains a float and how will it retrieve its range, if any, to provide the appropriate widget for it? |
Just as a side question, but any thought given to using a library like config as it seems to handle all this already while being extendable as well? |
@emanuele3d the same way it would know that if the object is a |
@OvermindDL1: the library you suggest would make sense IF:
Both @eviltak and I have put some time into this. Constructive feedback is appreciated, but I'm not sure advising such a change in direction qualifies, no matter how well intentioned and even reasonable your suggestion is. I am sure technically it would work. Ultimately given time and effort we could integrate and use any library. Whether it would save us time for this particular task, that's a much harder call to make and for the time being I see too feeble of a reason to change the course. @eviltak: above you provided an example in which the nature of a setting was already known. Now, let's assume you are the UI builder code, you are iterating through the instances of Setting in a flexible config and an instance is now available to you in the variable |
(@OvermindDL1 if you're not ok with this I'll delete) |
@SoniEx2 as far as splitting implementation from interface and other major version bumping needs then I have no issue with that as long as it follows good timing and has justifiable design + implementers. I foresee another round of backwards incompatible changes coming with GSOC 2017 (if we get in) and finishing the extraction and adoption of Probably some of that work including any related GSOC proposals would hang out in some longer term branches until they're ready for the big version bump. Maybe a super nice and fancy config could be done that way. I think your PR #2618 could get merged to engine sooner with its simple approach of just doing its Socket or HTTP thing from the engine for a few months, while the BIG config/sandboxing thing might be a phase two that could eventually lead to better config and segregation in modules. The "small" config thing discussed here (small in comparison to rewriting all the things anyway) likewise might be a thing we can merge in the short term, unless somehow we should go straight for the big option. But this is something @emanuele3d is eager to put into use the first moment we can get good working code. Ultimately decisions get based more on who is available to implement what, when, and how constructively we can discuss it all. We have the Socket PR, which is fine, but connecting it to any major rewriting that has yet to happen will block it. Likewise @emanuele3d and @eviltak here are already working on implementation details for the original purpose of this issue. As far as typesafe's config goes I believe @immortius looked into it at one point and found some deficiencies. Maybe there are ways around that and with your and @OvermindDL1's help maybe we can get beyond all that and write the bestest config system ever. But code talks louder than IRC logs. One step at a time - we should probably treat your PR and the original issue here as one phase and major redesigns as a later one. How does that sound? |
@Cervator I was talking about using an external config library (aka what @OvermindDL1 talked about). The best option is to use a good external config library. But such a config library needs split interfaces and implementation. Since none exist, converting one would require it to get a rewrite and major version bump. |
A few thoughts/comments. The current design of modules puts them in a sandbox, which restricts what classes and functionality they have access to from the engine and its dependencies (that is, java and other libraries it uses). However, it isn't designed to sandbox modules from each other - the architecture is more of a core-feature providing engine with game logic and asset providing modules. So the idea of a module introducing features to be secured for use by other modules is not supported as such. I get the feeling that, @SoniEx2, you were envisioning modules as a more fundamental extension system with the ability to introduce engine features as well. So this conflict is probably the source of most of your issues. On the other hand I feel for your desire to introduce these features without having to touch engine code. I guess I would propose some system for loading jars or possibly modules as part of the engine itself, outside the sandbox. These would have to be manually added to terasology's config, and would not be subject to automatic downloads. Basically, these would be true extensions to the engine and have full access to all engine and java features. Including the @API annotation, so these modules would be able to expose methods to the sandbox, with permission requirements. Then you would be able to make full use of the permission system - although I understand this hasn't been integrated with Terasology yet either in terms of user prompts and permission rejections. Certainly agree with the frustration with the core and other libraries that don't split interfaces and implementation well (there is a lot of rubbish in the core API from the early java days too). The envisioned solution for these issues was for the engine to provide safe encapsulations around these things. |
@immortius I was talking about using an external config library in the engine, like how you currently use an external JSON library (GSON) in the engine. |
@emanuele3d That's the obstacle that I'm stumbling upon. There's no way to know what type a Since the above problem is going to be prevalent no matter what method we use, there is no other option but to have loads of |
Instinctively I find the idea of a method returning a widget to be tempting. I don't like how responsibilities would be getting tangled though. The Setting class as originally envisioned is fairly "dumb": it provides accessors, un/subscribing functionality and you make a good point recommending validity checks. By adding UI functionality right within this object (even though it could be provided by some kind of UIBuilder instance held within it) we would be blurring the line between data and presentation. It makes sense for responsibilities to be clearly defined: a Setting instance provides valid data and a way to monitor it, while the UI is responsible for displaying the setting to the user in whichever way is appropriate. And even just the ability to quickly describe this in plain English has a value on its own, i.e. when writing documentation. So. The setting instance needs to provide an easily retrievable indication of its type so that a simple map ui-side can easily associate settings to handlers. As you prefer not to have specialized Setting subclasses, i.e. FloatSetting, I'd suggest to store the type in a ResourceURN provided on construction, i.e.:
and have it accessible via a getType method or something similar. I am concerned about this otherwise typeless approach though. But let's see where we get from here. |
There is another option, which is how Minecraft does blocks. However, in the Minecraft case, those things are immutable. But the basic idea is to have typed keys.
In this case, settings are stored in a Map instead of fields with get methods. And since the key is a full object, nothing stops us from adding an |
@emanuele3d The approach isn't typeless; I was proposing to let the Consider that we have a Another problem with this method is highlighted at the end of this comment. @SoniEx2 I'm already following a similar but better method. The concept of putting validation in the keys doesn't seem intuitive to me.
The The only pitfall of this method and using a UI-side map to build Making the |
PR #2757 has been created for this issue. |
For note, the reason why I asked about |
@eviltak: switching to the PR discussion - no point in continuing the discussion here at this stage. |
Introduction
The current rendering config is a relatively straightforward (if long) Java class with hardcoded video settings and their handling. This includes accessors (get*(), set*(), is*()), but also functionality to subscribe to the config to be notified of changes - to specific values or to the whole config.
This works, but is inflexible. Ideally engine and modules should be able to publish their configuration settings, as needed. Here is a couple of simple scenarios to ponder about.
Scenario 1 - ShadowMapNode
The ShadowMapNode is responsible for generating a ShadowMap used to cast shadows from the main light (sun/moon). This node wishes to publish two settings to the rendering config: a) a boolean defining its enabled/disabled state b) an integer defining the size of the (square) shadow map, in pixels.
Sceneario 2 - RealCameras module
The RealCameras module (so far a fictional one) introduces a number of realistic effects for the camera. Among them are lens flares. These are generated by a specialized node injected into the render graph on module startup. The node renders up to 20 lens flare elements, each having a set of parameters to play with, such as the texture asset used, the distance along the flare axis and the maximum opacity of the element. All these settings need to be published at runtime on the rendering config, for the user to play with.
FlexibleConfig and Setting classes
Enter the FlexibleConfig class, barely more than a <ResourceURN, Setting> map:
void add(Setting)
- doesn't allow overwriting a stored setting object with a new one using the same ResourceUrnboolean remove(ResourceUrn)
- return true if the setting has been removed from the config. This happens only if the setting no longer has subscribers.Setting get(ResourceUrn)
boolean has(ResourceUrn)
Individual Setting instances would contain:
Setting
is largely immutable and contains the information from the FlexibleConfigs Features section. But it would have a setter methods for the stored value. It also provides subscribe()/unsubscribe() methods and aboolean hasSubscribers()
method.A few clarifications about the ui-path feature. A ui path would provide a hint to the ui on how to group settings together. I.e. all settings having a path /engine/shadows/shadowmap could be displayed together when the "shadows" list item is selected in the "engine" tab of the rendering config. It could be meaningful however for the same setting to be also be displayed in a separate tab/menu, hence the possibility for multiple ui paths. Ultimately however, it is up to the UI code to decide how to take advantage of them. I.e. a simpler UI might not provide "tabs" and might group settings between horizontal separators instead, potentially ignoring the deeper sections of a path.
How does this all sound? And notice that this issue specifically avoids dealing with the UI needed to display the content of a FlexibleConfig instance. That will be for a separate issue.
The text was updated successfully, but these errors were encountered: