Lumen a json based effect server

[Liquidize]

Hello everyone,

I've been working on a little project inspired by Dave's NightDriverServer application. It is called "Lumen" and it is fairly similar in principle. Instead of having to define locations in code, they are defined in JSON files and can be hot loaded in. There are some other differences, such as a REST api for queuing effects and modifying effect settings (WIP) as well as loading in third party libraries for additional effects to make sharing custom effects a bit easier. It is still a WIP but I hope to expand on it quite a bit, more information can be found on the project page and project wiki here: https://github.com/Liquidize/Lumen

Please feel free to take a look if you'd like, im open to all feedback, if you do end up running it, I'd run it out of docker for the time being to make it easier to add/modify json files. The fastest way to get started would be to just clone the repo, and modify the "Desk.json" location file and run Lumen in debug or release mode.

[davepl]

Hi Daniel, I’m working on a WinForms app that is intended to improve and replace the current server app; the stuff under the Server folder stays the same, largely, and there is a new top level WinForms app that uses it. I’m just about at the point where I need to be able load, save, and edit a list of “Location” objects and “LigjhtStrip” objects. Before I run off an do something new, maybe I can just leverage what you have done? Can you tell me a bit about what you’ve done for serialization and persistence? Thanks! Dave

…

On Dec 26, 2023, at 3:43 PM, Daniel Deptula ***@***.***> wrote: Hello everyone, I've been working on a little project inspired by Dave's NightDriverServer application. It is called "Lumen" and it is fairly similar in principle. Instead of having to define locations in code, they are defined in JSON files and can be hot loaded in. There are some other differences, such as a REST api for queuing effects and modifying effect settings (WIP) as well as loading in third party libraries for additional effects to make sharing custom effects a bit easier. It is still a WIP but I hope to expand on it quite a bit, more information can be found on the project page and project wiki here: https://github.com/Liquidize/Lumen Please feel free to take a look if you'd like, im open to all feedback, if you do end up running it, I'd run it out of docker for the time being to make it easier to add/modify json files. The fastest way to get started would be to just clone the repo, and modify the "Desk.json" location file and run Lumen in debug or release mode. — Reply to this email directly, view it on GitHub <#578>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA4HCF2DU6IWBNDS4LELM4TYLNOKTAVCNFSM6AAAAABBDW3XA6VHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZWGAYDCMJRGQ>. You are receiving this because you are subscribed to this thread.

[Liquidize]

Hi Dave, in my case I am doing the following

For "Locations" and the "Lightstrip" objects, I have a "LocationsRegistry" singleton which on app starts looks in the "locations" folder and loads location json objects, storing them by their name for hot-reloading purposes, meaning each location needs a unique name. I opted to abstract the "GraphicsBase" drawing functionality from the "Location" object itself, and create "Canvas" objects, which has their types stored in a "CanvasRegistry". The "Location" objects now just have a "CanvasType" property along with all the other ones (e.g: Width, Height, etc). When deserialized at startup the "StartWorkerThread" method is called, and it accesses the canvas registry, and instantiates a new Canvas object for the drawing of that location. The "LightStrip" object array of the location is serialized as is, other than some properties having the "[JsonIgnore]" tag as they do not need to be serialized (e.g: Current Queue depth). I'm not serializing the exact state of a location after all. I am also not serializing complete "LedEffect" classes from the Scheduled Effects.

In the case of "LedEffects" I have an "EffectsRegistry" when the app is started it loads all of the effect types in the loaded assemblies and creates factory functions to instantiate them, storing them in the registry by their Name property. I've opted to do this so I am not serializing full LedEffect objects, and instead for the Locations "ScheduledEffects" objects, I store the name of the effect and a dictionary of objects that act as the effects settings. When the effect needs to be ran, the registry creates a new instance of the effect and then the dictionary is passed to a "SetEffectSettings" method, that sets the properties of the effect.

Here is an example of a location's JSON

{
  "name": "Desk",
  "isApiEnabled": true,
  "controllers": [
    {
      "host": "73.73.10.40",
      "name": "Desk",
      "useCompression": true,
      "channel": 0,
      "swapRedGreen": false,
      "batchSize": 20,
      "framesPerBuffer": 500,
      "reversed": false,
      "offset": 0,
      "width": 316,
      "height": 1
    }
  ],
  "scheduledEffects": [
    {
      "effectName": "BreathingEffect",
      "daysOfWeek": 7,
      "startHour": 0,
      "startMinute": 0,
      "endHour": 24,
      "endMinute": 60,
      "effectSettings": {
        "color": {
          "$type": "Lumen.Api.Graphics.LedColor, Lumen.Api",
          "r": 255,
          "g": 0,
          "b": 60
        },
        "lifetime": 0,
        "frequency": 5
      }
    }
  ],
  "framesPerSecond": 30,
  "canvasType": "Canvas1D",
  "width": 316,
  "height": 1
}

I opted to do it this way because I am working on a REST API to interact with the server, and having end users known the exact type names for effects so that the deserialization works properly seemed a bit much. I still feel like its a bit bad enough that you need to specify an effect parameter is a palette or color because they're just being deserialized as generic objects,

I am also not exposing the "Location" or "ControllerChannel" (LightStrip) objects to the API for use in third party DLL's. I'm not quite sure what the benefit of having child object types of these classes would be,. The drawing functionality of a location is now exposed via canvas objects instead. For the "LightStrip" object since the ESP module just needs to be sent a 1d byte array of color data, I can't think of any reason why different types would be needed.

There are probably a lot better ways to handle it all, but this is what I came up with in a few hours myself.

Thanks,
Daniel

[davepl]

Here’s an example of one of mine… I’ll be checking this code into NightDriverDesktop imminently, so it seems we’re both kind of working on the same thing! My best advice would be continue on the path you’re on, but don’t think too hard about serialization (make it work, but keep it abstract so we can merge later). I’m doing the load/save/delete/edit/new menu and verbs in the app, and they should serve the console side as well. The other issue is that I’ve copied/forked the NightDriverServer code to be a sub-protect of NightDriverDesktop, so the code in Server will be orphaned at some point. I’d say you’d be better off working against the Desktop tree, but it’s pretty early, so a lot of thrashing. - Dave { "FramesPerBuffer": 312, "PercentBufferUse": 0.75, "Reversed": false, "HostName": "192.168.8.77", "FriendlyName": "Bonsai Tree", "CompressData": true, "Channel": 0, "Brightness": 1, "Connects": 0, "Watts": 0, "RedGreenSwap": false, "BatchSize": 10, "BatchTimeout": 1, "SiteName": "Tree", "Width": 144, "Height": 1 },

…

On Dec 27, 2023, at 7:41 AM, Daniel Deptula ***@***.***> wrote: Hi Dave, in my case I am doing the following For "Locations" and the "Lightstrip" objects, I have a "LocationsRegistry" singleton which on app starts looks in the "locations" folder and loads location json objects, storing them by their name for hot-reloading purposes, meaning each location needs a unique name. I opted to abstract the "GraphicsBase" drawing functionality from the "Location" object itself, and create "Canvas" objects, which has their types stored in a "CanvasRegistry". The "Location" objects now just have a "CanvasType" property along with all the other ones (e.g: Width, Height, etc). When deserialized at startup the "StartWorkerThread" method is called, and it accesses the canvas registry, and instantiates a new Canvas object for the drawing of that location. The "LightStrip" object array of the location is serialized as is, other than some properties having the "[JsonIgnore]" tag as they do not need to be serialized (e.g: Current Queue depth). I'm not serializing the exact state of a location after all. I am also not serializing complete "LedEffect" classes from the Scheduled Effects. In the case of "LedEffects" I have an "EffectsRegistry" when the app is started it loads all of the effect types in the loaded assemblies and creates factory functions to instantiate them, storing them in the registry by their Name property. I've opted to do this so I am not serializing full LedEffect objects, and instead for the Locations "ScheduledEffects" objects, I store the name of the effect and a dictionary of objects that act as the effects settings. When the effect needs to be ran, the registry creates a new instance of the effect and then the dictionary is passed to a "SetEffectSettings" method, that sets the properties of the effect. Here is an example of a location's JSON { "name": "Desk", "isApiEnabled": true, "controllers": [ { "host": "73.73.10.40", "name": "Desk", "useCompression": true, "channel": 0, "swapRedGreen": false, "batchSize": 20, "framesPerBuffer": 500, "reversed": false, "offset": 0, "width": 316, "height": 1 } ], "scheduledEffects": [ { "effectName": "BreathingEffect", "daysOfWeek": 7, "startHour": 0, "startMinute": 0, "endHour": 24, "endMinute": 60, "effectSettings": { "color": { "$type": "Lumen.Api.Graphics.LedColor, Lumen.Api", "r": 255, "g": 0, "b": 60 }, "lifetime": 0, "frequency": 5 } } ], "framesPerSecond": 30, "canvasType": "Canvas1D", "width": 316, "height": 1 } I opted to do it this way because I am working on a REST API to interact with the server, and having end users known the exact type names for effects so that the deserialization works properly seemed a bit much. I still feel like its a bit bad enough that you need to specify an effect parameter is a palette or color because they're just being deserialized as generic objects, I am also not exposing the "Location" or "ControllerChannel" (LightStrip) objects to the API for use in third party DLL's. I'm not quite sure what the benefit of having child object types of these classes would be,. The drawing functionality of a location is now exposed via canvas objects instead. For the "LightStrip" object since the ESP module just needs to be sent a 1d byte array of color data, I can't think of any reason why different types would be needed. There are probably a lot better ways to handle it all, but this is what I came up with in a few hours myself. Thanks, Daniel — Reply to this email directly, view it on GitHub <#578 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA4HCFY6Y5Z3TMEP6MLR6SLYLQ6RRAVCNFSM6AAAAABBDW3XA6VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TSNJXG43DE>. You are receiving this because you commented.

[Liquidize]

I'll keep that in mind! In regards to orphaning the Server code, have you considered moving the main bits of it all into a library project instead? Might be easier to maintain across different use cases for the classes.

[davepl]

That’s roughly what I’m doing - but not as a lib per se. The Server code is compiled and the Desktop project references it… but if there’s a more formal version of a lib that would be more appropriate, let me know! =- Dave

…

On Dec 27, 2023, at 9:01 AM, Daniel Deptula ***@***.***> wrote: I'll keep that in mind! In regards to orphaning the Server code, have you considered moving the main bits of it all into a library project instead? Might be easier to maintain across different use cases for the classes. — Reply to this email directly, view it on GitHub <#578 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA4HCFZF5M2X5NQOOHRJV7DYLRH5DAVCNFSM6AAAAABBDW3XA6VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TSNJYGM4DM>. You are receiving this because you commented.

[Liquidize]

Oh no, that would do it. I just had thought when I took a look the other day at NightDriverDesktop it looked like it just had the files copied into the project, but maybe I was seeing things haha. On Wed, Dec 27, 2023 at 12:47 PM David W Plummer ***@***.***> wrote:

…

That’s roughly what I’m doing - but not as a lib per se. The Server code is compiled and the Desktop project references it… but if there’s a more formal version of a lib that would be more appropriate, let me know! =- Dave > On Dec 27, 2023, at 9:01 AM, Daniel Deptula ***@***.***> wrote: > > > I'll keep that in mind! In regards to orphaning the Server code, have you considered moving the main bits of it all into a library project instead? Might be easier to maintain across different use cases for the classes. > > — > Reply to this email directly, view it on GitHub < #578 (reply in thread)>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AA4HCFZF5M2X5NQOOHRJV7DYLRH5DAVCNFSM6AAAAABBDW3XA6VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TSNJYGM4DM>. > You are receiving this because you commented. > — Reply to this email directly, view it on GitHub <#578 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABCJQ7LW2MTX357VQUWTUP3YLRNK7AVCNFSM6AAAAABBDW3XA6VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TSNJYGY4DA> . You are receiving this because you authored the thread.Message ID: <PlummersSoftwareLLC/NightDriverStrip/repo-discussions/578/comments/7958680 @github.com>

[rbergen]

@Liquidize I noticed the part in your repo's README about the license, mentioning that you thought the license that applies to NightDriverServer is GPLv2. The applicable license is actually GPLv3, which is the license that applies to all NightDriver projects/repos. I've added COPYING.txt files documenting this to all repos that didn't have one yet, including NightDriverServer. You may want to update the documentation in your repo accordingly.

[Liquidize]

Sure thing, I'll get that updated. Thanks.