Make ZM able to use Postgresql

[parvit]

Hi, this PR responds to issue #355, it integrates the SOCI library to allow support for two database backends:

• MySQL (MariaDB)
• Postgresql

I have done my best to try to not be too invasive but it is still quite the change in respect to the current code.

I have tested it in small part (mainly monitors and capturing) but i do not have the setup to test every possible feature of zm so take this with a bit of salt. Help is needed to test if there is any regression on parts i have not been able to test.

[parvit]

@Carbenium Any progress on the review?

[GreasyMonkee]

Hi Parvit,

I am going to show my "Noob" status here :-)

I have not used ZoneMinder before, what do I need to do to implement ZoneMinder, with Postgresql on Fedora 36 or later?

Is there any "build" of an RPM from here required? Any guidance or direction would be appreciated.

[connortechnology]

This is a work in progress. ZoneMinder is NOT available for Postgres at this time. If you want it, you will have to throw money at @parvit by means of the bounty.

[GreasyMonkee]

This is a work in progress. ZoneMinder is NOT available for Postgres at this time. If you want it, you will have to throw money at @parvit by means of the bounty.

Yes, understood, and have already contributed to the bounty - it was me who had requested the FC36, and am very grateful to Parvit and the team for the prompt work on that aspect - I was not sure how far the integration had gone, or what else was required - again showing my "Noob" status on Git and other development stuff.

[parvit]

Hi, the integration (minus the new conflicts from the recent changes) is done and i'm waiting for feedback on the PR.

My suggestion is that before attempting to try this branch you should setup the normal environment from source first and then add the new dependencies.

[connortechnology]

This work is great, and looks very good. The problem is that the db creation scripts have not been updated, the perl code has not been updated, and the php code has not been updated to deal with the differences in sql syntax between mysql and postgresql. So this only gets us half way there.

Perhaps if I merge it someone will work on the rest.

[parvit]

@connortechnology I could help with the php and sql, however not with the perl part.

[ghost]

Do we want to push this through? If there is interest I might be able to help with what remains.

[parvit]

Do we want to push this through? I might be able to help with what remains.

If you want to make a test i can certainly align the implementation and then we can follow through. I have not abandoned this.

[ghost]

Make a test? Can you give me a little more details?

[parvit]

Make a test? Can you give me a little more details?

Test in the sense of building this version and run it to check that the various features did not break with the changes i've introduced.

Mostly with MySQL backend as the postgres needs more work outside of the C part to work.

[ghost]

Given it's been so long do you want to rebase against upstream? I am new to zm so I will probably not be great at regression testing though. If it passes the current test suite maybe we can get it merged and fix whatever comes up?

[parvit]

Given it's been so long do you want to rebase against upstream?

I'm importing the changes that have happened in the meantime in master but i also need to change the queries as there's no corresponding update for my branch.

[ghost]

Perhaps if I merge it someone will work on the rest.

I am willing to put in some cycles to help push this forward.

[ghost]

I'm importing the changes that have happened in the meantime in master but i also need to change the queries as there's no corresponding update for my branch.

Please let me know when done and I will fork your repo then.

[ghost]

I know we are to wrap a dependency like SOCI but why not use it directly? I am thinking that once this is merged we should consider adding SQLite3 support too so that would be another set of adapters. Did you run and test the postgresql version? How? The initial schema appears to be an templatized mysqldump. The schema changes are all mysql specific so have you though about how to manage that? Do we want to select the database at install-time (i.e. build separate a core and per-database packages)? Is there a way to testing all these platforms before submitting a PR to have CI run it? How would I help you with this PR? Do I clone your repo and submit PRs to your fork?

Some minor review comments: there are a number of lines with trailing spaces before newline that I suggest you remove. Remove the commented out freebsd-14-0-snap line?

You put a lot of work into this already. I think we should get it merged as soon as possible to get more eyes on any mysql regressions then work on getting whatever remains for pq up to snuff.

[parvit]

Hi @allanwind, my answers:

1. That's just one way to go about the change and has the advantage of hiding the many details of the SOCI backends.
2. Certainly it can be done, however the issue is that the various backends have differing opinions on how to manage the necessary features: eg. auto increments, how to convert to and from the various native types, so it probably won't be as straight-forward as you might expect
3. I've run it locally but my setup is in a VM so i did just manage to create and use a simple webcam, so my manual tests were not extensive in any way
4. Yes i did dump the mysql and adapt the postgres as the task required only a change in the C portion, so i have given no thought to how to upgrade / handle the mysql schema changes
5. The database backend is selectable dynamically through the configuration, you only need the various dependencies at compile time and when starting if not installed it will just stop with error
6. I would not know if it is possible / feasible to run the tests with the different backends in your CI environments, i've concentrated on getting it to build