TKL 16.1 Odoo issues when TKLBAM restore run.

[l-arnold]

I had an issue that had this included in December 2021 but closed as I had too many issues.

Just doing a site migration with TKLBAM and again all rendering gets lost in a TKLBAM restore.

I worked through this manually back then but need to see if I can do so again.

[JedMeister]

So is this a restore of a v16.1 Odoo backup to a v16.1 server? Or a restore from a different version?

If it's the first, then I can only assume that there is something that your site needs, that isn't being included in the backup (either you're storing stuff in places that ideally you shouldn't, or we're excluding a path from the backup that we shouldn't). There is also a possibility that you need to refresh your caches (I'm not sure how you do that with Odoo, but I'm sure it's possible).

If it's the second (i.e. your backup comes from an earlier TurnKey release, then that's a pain, but my guess is mismatching Odoo versions. Unfortunately, there isn't really a great way around that (other than us not changing Odoo versions or you restoring to the older version). Having said that, once you've updated everything, if you create a new backup of the new (working) Odoo instance, then that should restore cleanly and "just work" when restoring to v16.1.

[l-arnold]

I have had this happen both on the Hub and on Hub to Linode, and Linode to Linode migrations. I am not sure what is not properly migrated.
Yesterday I tried, after TKLBAM to then backup the moved database. Delete the moved database, and then move it in. That did not work for me.
What did work was deleting the moved database, then moving a manually backed up and restored database from server to server (this was Linode to Linode). This was followed by a 2 cycle restart I recall. It did render in the end.

[l-arnold]

if one goes to (DOMAIN)/web/database/manager you can normally do a backup of the database to your local machine in one of 2 formats... then with the new server, do the same.
The risk is, and what I was not able to complete, is if the old machine goes south, it is not possible, at least for me so far, to use only the backed up Database. This is important to work through.
This is a 16.1 to 16.1 migration.

[l-arnold]

Easiest way to test is to Build an Odoo Server. Add a second database to it. Run Backup. Then start a new Odoo Server, run TKLBAM restore and see if the data comes in. This is essentially what I did except that I had several months of data in the new database (had deleted the sample database from my original server so of course that does not migrate). I even deleted the sample database from the second database to be sure that was not somehow tripping things (try 2 or 3)

[JedMeister]

This is a 16.1 to 16.1 migration.

Great, thanks for that vital info.

And I assume that you are using the same version of Odoo on both systems? I.e. have you updated Odoo itself between the servers?

if one goes to (DOMAIN)/web/database/manager you can normally do a backup of the database to your local machine in one of 2 formats... then with the new server, do the same.
The risk is, and what I was not able to complete, is if the old machine goes south, it is not possible, at least for me so far, to use only the backed up Database. This is important to work through.

So it seems that the DB restore part is the problem? Is there any error message in the restore log(s)? Can you please post the restore log from a machine that experiences this behaviour? Alternatively, do a manual restore from the commandline like this:

tklbam-restore BACKUP_ID | tee ~/tklbam-restore.log

(Where BACKUP_ID is the backup ID of your backup).

Easiest way to test is to Build an Odoo Server. Add a second database to it. Run Backup. Then start a new Odoo Server, run TKLBAM restore and see if the data comes in. This is essentially what I did except that I had several months of data in the new database (had deleted the sample database from my original server so of course that does not migrate). I even deleted the sample database from the second database to be sure that was not somehow tripping things (try 2 or 3)

Ok, thanks for this. I'm not sure when I'll get to it, but that's useful to know how to recreate the issue.

[JedMeister]

Hang on, Odoo uses a postgresql DB - so I think the problem you're experiencing might be this issue: #1640

Can you please have a look at that and try the workaround noted there?

We need to do some major work on TKLBAM and I need to complete v17.0 before I'll have the spare cycles to start that work. But that (this?) bug will need to be addressed.

[l-arnold]

16.1 to 16.1 TKL (Which is Odoo 14).
Have had the issue both with a "un updated" Odoo 14 and related attempted Restore
and
an "updated" Odoo (apt-update) + (apt-upgrade) Odoo system. (Which I did run on the new system as well and with the manual DB it seemed to help)
I think one (of a few) issue also has to do with how Odoo Upgrades the Database..

The header on that Issue (1640) sounds similar perhaps, but realize that I have had this happen also on "very virgin systems". It even happened on a very New Hub system back when I first reported this. (November or December 2021).
Your description of "dropping" to allow the Restore is more or less what I was doing manually. It could be done programmatically. Again though, upgrading the database may also be needed.

I AM happy to report that through this I made the mistake of "restoring to my source" which was a VERY BAD Idea. AND tklbam-restore-rollback worked VERY Well! Yay.
I had been trying to take a fresh backup and put to a new system but my Shell screens got confused and I restored there.
Fixing TKLBAM so you CANNOT restore to the Source SYSTEM seems very a good idea. That rollback command is not well publicized.

[JedMeister]

I have had this happen also on "very virgin systems"

The only requirement of this bug is that the DB schemas don't match. IIRC more specifically, the bug occurs if the new DB has changes to existing table config and/or additional tables. So in the case of Odoo, I imagine that doing anything (of any value) would change the DB schema. I'd need to test further though to be sure.

Fixing TKLBAM so you CANNOT restore to the Source SYSTEM seems very a good idea.

Currently, that is the only scenario that should be (almost) guaranteed to work 100%! So I'm not really sure how to best approach that. Regarding the restore rollback, doesn't TKLBAM explicitly mention the option of rolling back the restore once the restore is completed? IIRC it's one of the last lines of the logging output?! (If it's not, then we should add that!) I'm not really sure how much more than that we could do to ensure that users are aware of it? Although I'm open to ideas if you have suggestions.

FWIW, the expectations of TKLBAM restores is noted on the TKLBAM doc page:

Assuming the exact same server type (e.g. LAMP backup restored to LAMP server) and a pre-tested backup (to ensure that it includes all the required files):
A backup should always restore cleanly on the same server.
A backup should usually restore cleanly on a server of the exact same version (e.g. v16.1 -> v16.1).
A backup should most likely restore cleanly on a server of the same major version (e.g. v16.0 -> v16.1).
A backup might restore ok on a server of a single newer major version (e.g. v15.2 -> v16.1) although it may require some tweaking.
A backup will almost certainly need some manual intervention if you restore to a server of a (much) newer version (e.g. v14.2 -> v16.1).

[l-arnold]

This comes from an old note I had re updating the database.. I believe the process is looped in many places in Odoo but not in TKLBAM... perhaps it could be applied once we know what it actually does

`Odoo Postgress -update=all

Upgrade the database. For linux, start the odoo server with the –update=all parameter. For windows, as an administrator open the command prompt (cmd). Change to the odoo server directory and start the server with odoo-server.exe –update=all to update the modules. This process will take a short while (1-2minutes max) and the database won’t be accessible by the client`

[JedMeister]

That's good info, thanks Landis.

My suspicion is that running that command will only make changes if it needs to. So running that on restore shouldn't do any harm. Can you please double check?

If that's the case, we can make a tklbam restore hook script that runs that after restore.

[l-arnold]

Ok. Will do soon.

My thought is that for "in place" restores, the Database itself be "renamed" and left in place, then the Restored Database be written in to the system. It may be however that because of Postgres there also be "full backup" and "full restore"... I'll see how an in-place restore goes to a secondary server after "renaming" the database on it first.

[l-arnold]

Well, I spent a little time today. I have an active Odoo install and an Odoo that I created a few months ago and brought forward (described above).
Today I tried a simple "tklbam-restore (backup#)"
Two times it ran through the process (did not work the first time). It appears that it is updating the database, but it is not.

In order to bring the system up to date one needs to run the Odoo Database Manager on the Running System... Do a Backup, Then go to the Restore To System, Delete the database there, then Restore the newly backed up database.

I cannot get it to restore in any other way.
Maybe I am missing something but i have done this before.

I had thought that since I had done this process a few month's ago the database could just be updated. Perhaps there is some encrypted id for the Database that does not match the one I am trying to restore to. Seems strange though. Everything should be identical.

All for now.

[l-arnold]

Ok. I have likely noted some of this earlier but want to sumarize.

If a new TKL Odoo 16.1 App is installed, then TKL-BAM restore run to populate it next one must

• Delete the Demo Database
• perhaps delete the restored database (ref the "loses rendering" part of this issues original description)
• manually upload a database (saved by Odoo) on original server. (this is basically a Dump file but works)

If in a previously restored and now working (by following the above) server which you want to bring "up to date" with a subsequent TKLBAM restore, the following occurs:

• Files are updated up to current
• TKBAM restore of the database (same name) over the running database (same name) will cause many many errors visible during the restore process in SSH/Webshell etc. This includes module set "constraints" visible in the screenshot attached.
• the restored database (if it was restored over and not excluded by a tklbam switch) will need to be deleted.
• (last line in outline above again) manually upload a database (saved by Odoo) on original server. (this is basically a Dump file but works)

Don't crash your server.
Perhaps the rendering issue stated initially and above works itself out over time. Seems that it did one time recently.

OK. working through this in parallel
Screenshot below from the end of tklbam-restore scroll.

as I write this.