Skip to content

Invoking oracle clone using NetBackup APIs. #90

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 94 additions & 16 deletions recipes/python/backup-restore/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,15 +141,15 @@ Execution flow of group VM backup and restore script:

This mssql_db_backup_restore.py script demonstrates how to Protect a MSSQL Database or Instance using a protection plan, and perform a alternate recovery of a single database or all user databases using NetBackup APIs.

`python -W ignore recipes/python/backup-restore/mssql_db_backup_restore.py --master_server <master_server> --master_server_port 1556 --master_username <master_username> --master_password <master_password> --mssql_instance <mssql_instance_name> --mssql_database <mssql_database_name> --mssql_server_name <mssql_server_name> --mssql_use_localcreds 0 --mssql_domain <mssql_domain> --mssql_username <mssql_sysadmin_user> --mssql_password <mssql_sysadmin_pwd> --stu_name <storage_unit_used_in_protection_plan> --protection_plan_name <protection_plan_name> --asset_type <mssql_asset_type> --restore_db_prefix <mssql_restore_database_name_prefix> --restore_db_path <mssql_restore_database_path> --recover_all_user_dbs <0|1> --recover_from_copy <1|2> --copy_stu_name <storage_unit_used_for_copy>`
`python -W ignore recipes/python/backup-restore/mssql_db_backup_restore.py --primary_server <primary_server> --primary_server_port 1556 --primary_username <primary_username> --primary_password <primary_password> --mssql_instance <mssql_instance_name> --mssql_database <mssql_database_name> --mssql_server_name <mssql_server_name> --mssql_use_localcreds 0 --mssql_domain <mssql_domain> --mssql_username <mssql_sysadmin_user> --mssql_password <mssql_sysadmin_pwd> --stu_name <storage_unit_used_in_protection_plan> --protection_plan_name <protection_plan_name> --asset_type <mssql_asset_type> --restore_db_prefix <mssql_restore_database_name_prefix> --restore_db_path <mssql_restore_database_path> --recover_all_user_dbs <0|1> --recover_from_copy <1|2> --copy_stu_name <storage_unit_used_for_copy>`

All parameters can also be passed as command line arguments.
- `python mssql_db_backup_restore.py -h`
```
usage: mssql_db_backup_restore.py [-h] [--master_server MASTER_SERVER]
[--master_server_port MASTER_SERVER_PORT]
[--master_username MASTER_USERNAME]
[--master_password MASTER_PASSWORD]
usage: mssql_db_backup_restore.py [-h] [--primary_server PRIMARY_SERVER]
[--primary_server_port PRIMARY_SERVER_PORT]
[--primary_username PRIMARY_USERNAME]
[--primary_password PRIMARY_PASSWORD]
[--mssql_instance MSSQL_INSTANCE]
[--mssql_database MSSQL_DATABASE]
[--mssql_server_name MSSQL_SERVER_NAME]
Expand All @@ -169,14 +169,14 @@ Mssql backup and alternate database recovery scenario

Arguments:
-h, --help show this help message and exit
--master_server MASTER_SERVER
NetBackup master server name
--master_server_port MASTER_SERVER_PORT
NetBackup master server port
--master_username MASTER_USERNAME
NetBackup master server username
--master_password MASTER_PASSWORD
NetBackup master server password
--primary_server PRIMARY_SERVER
NetBackup primary server name
--primary_server_port PRIMARY_SERVER_PORT
NetBackup primary server port
--primary_username PRIMARY_USERNAME
NetBackup primary server username
--primary_password PRIMARY_PASSWORD
NetBackup primary server password
--mssql_instance MSSQL_INSTANCE
MSSQL Instance name
--mssql_database MSSQL_DATABASE
Expand Down Expand Up @@ -208,14 +208,92 @@ Arguments:
Storage Unit name for the copy

Execution flow of a Single MSSQL database protection and alternate database recovery workflow:
- Login to Master Server get authorization token for API use
- Login to Primary Server get authorization token for API use
- Add Credential with Credential Management API
- Create a MSSQL Instance Asset and associate Credential
- Asset API to find the MSSQL Instance asset id for subscription in a Protection Plan
- Create MSSQL Protection Plan and configure MSSQL database policy attribute to SkipOffline databases
- Subscribe the MSSQL Instance Assset in Protection Plan
- Subscribe the MSSQL Instance Asset in Protection Plan
- Fetch Asset id for database for alternate recovery
- Get recoverypoint for the database asset using its asset id
- Perform alternate database recovery of the database and report recovery job id or Perfrom alternate recovery of all user databases, if recover_alluserdbs is specified.
- Perform alternate database recovery of the database and report recovery job id or Perform alternate recovery of all user databases, if recover_alluserdbs is specified.
- Cleanup by removing subscription of Instance in Protection Plan, Remove Protection Plan and remove Mssql Credential

### - Oracle Clone workflow

This oracle_db_clone.py script demonstrates how to perform a complete database or a pluggable database clone for a previous Oracle database backup using NetBackup APIs.

`py -W ignore recipes\python\backup-restore\oracle_db_clone.py --primary_server <primary_server> --primary_server_port 1556 --primary_username <primary_username> --primary_password <primary_password> --source_oracle_db <source_backup_db> --source_database_id <source_oracle_dbid> --target_oracle_server <target_server_for_clone> --os_oracle_domain <os_domain> --os_oracle_username <os_username> --os_oracle_password <os_password> --clone_db_file_path <db_file_path> --oracle_home <target_oracle_home> --oracle_base_config <target_oracle_baseconfig> --force_clone 1`

`py -W ignore recipes\python\backup-restore\oracle_db_clone.py --primary_server <primary_server> --primary_server_port 1556 --primary_username <primary_username> --primary_password <primary_password> --source_oracle_db <source_backup_db> --source_database_id <source_oracle_dbid> --target_oracle_server <target_server_for_clone> --credential_id <credential_id> --clone_db_file_path <db_file_path> --oracle_home <target_oracle_home> --oracle_base_config <target_oracle_baseconfig> --pdb_clone 1 --avail_oracle_sid <avail_oracle_sid> --clone_aux_file_path <aux_file_path> --force_clone 1`

All parameters can also be passed as command line arguments.
- `python oracle_db_clone.py -h`
```
usage: oracle_db_clone.py [-h] [--primary_server PRIMARY_SERVER]
[--primary_server_port PRIMARY_SERVER_PORT]
[--primary_username PRIMARY_USERNAME]
[--primary_password PRIMARY_PASSWORD]
[--source_oracle_db SOURCE_ORACLE_DB]
[--source_database_id SOURCE_DATABASE_ID]
[--target_oracle_server TARGET_ORACLE_SERVER]
[--os_oracle_domain OS_DOMAIN]
[--os_oracle_username OS_USERNAME]
[--os_oracle_password OS_PASSWORD]
[--clone_db_file_path DB_FILE_PATH]
[--oracle_home ORACLE_HOME]
[--oracle_base_config ORACLE_BASE_CONFIG]
[--pdb_clone PDB_CLONE]
[--avail_oracle_sid EXISTING_ORACLE_SID]
[--clone_aux_file_path AUXILIARY_FILE_PATH]
[--credential_id CREDENTIAL_ID]
[--force_clone FORCE_CLONE]

Arguments:
-h, --help show this help message and exit
--primary_server PRIMARY_SERVER
NetBackup primary server name
--primary_server_port PRIMARY_SERVER_PORT
NetBackup primary server port
--primary_username PRIMARY_USERNAME
NetBackup primary server username
--primary_password PRIMARY_PASSWORD
NetBackup primary server password
--source_oracle_db SOURCE_ORACLE_DB
Source Oracle database name or pluggable database name, for which there is a previous backup available
--source_database_id SOURCE_DATABASE_ID
Source Oracle container database ID
--target_oracle_server TARGET_ORACLE_SERVER
Target Oracle server for clone
--os_oracle_domain OS_DOMAIN
Target operating system domain
--os_oracle_username OS_USERNAME
Target operating system username
--os_oracle_password OS_PASSWORD
Target operating system user password
--clone_db_file_path DB_FILE_PATH
File path for the cloned database
--oracle_home ORACLE_HOME
Oracle home directory on the target server
--oracle_base_config ORACLE_BASE_CONFIG
Oracle base config directory on the target server
--pdb_clone PDB_CLONE
Perform a CDB clone(0) or PDB clone(1); default: 0
--avail_oracle_sid EXISTING_ORACLE_SID
Existing container for a PDB clone
--clone_aux_file_path AUXILIARY_FILE_PATH
File path for auxiliary instance
--credential_id CREDENTIAL_ID
Credential ID for the instance credential in Credential Management Service
--force_clone FORCE_CLONE
Force a clone operation even if pre-recovery check fails; default: 0

Flow of an Oracle database clone workflow:
- Pre-requisite: A backup of an Oracle database must be available
- Log in to Primary Server to get authorization token for API use
- Get asset ID for the Oracle database backup with the Asset Service API
- Get the recovery point for the specific asset ID via Recovery Point Service API
- Perform a prerecovery check for the database or pluggable database clone request
- Perform a complete database clone or a pluggable database clone and report the job ID

See the Swagger documentation on your primary server for full API documentation https://localhost:1556/api-docs/index.html
14 changes: 7 additions & 7 deletions recipes/python/backup-restore/mssql_db_backup_restore.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@
import workload_mssql

PARSER = argparse.ArgumentParser(description="MSSQL Instance backup and Database restore scenario")
PARSER.add_argument("--master_server", type=str, help="NetBackup master server name")
PARSER.add_argument("--master_server_port", type=int, help="NetBackup master server port", required=False)
PARSER.add_argument("--master_username", type=str, help="NetBackup master server username")
PARSER.add_argument("--master_password", type=str, help="NetBackup master server password")
PARSER.add_argument("--primary_server", type=str, help="NetBackup primary server name")
PARSER.add_argument("--primary_server_port", type=int, help="NetBackup primary server port", required=False)
PARSER.add_argument("--primary_username", type=str, help="NetBackup primary server username")
PARSER.add_argument("--primary_password", type=str, help="NetBackup primary server password")
PARSER.add_argument("--mssql_instance", type=str, help="MSSQL Instance name")
PARSER.add_argument("--mssql_database", type=str, help="MSSQL Database name")
PARSER.add_argument("--mssql_server_name", type=str, help="MSSQL server name")
Expand All @@ -40,13 +40,13 @@
ASSET_TYPE = ARGS.asset_type if ARGS.asset_type else 'instance'
ALT_DB = ARGS.restore_db_prefix

BASEURL = common.get_nbu_base_url(ARGS.master_server, ARGS.master_server_port)
TOKEN = common.get_authenticate_token(BASEURL, ARGS.master_username, ARGS.master_password)
BASEURL = common.get_nbu_base_url(ARGS.primary_server, ARGS.primary_server_port)
TOKEN = common.get_authenticate_token(BASEURL, ARGS.primary_username, ARGS.primary_password)
INSTANCE_NAME = ARGS.mssql_instance
DATABASE_NAME = ARGS.mssql_database
ALT_DB_PATH = ARGS.restore_db_path
ALLDATABASES=[]
print(f"User authentication completed for master server:[{ARGS.master_server}]")
print(f"User authentication completed for primary server:[{ARGS.primary_server}]")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use some async-await methods here to ensure the message is printed only if the common.get_authenticate_token process is done and successfully generates a token?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is mssql scripts so i am not changing the functionality - this change is only to remove "master" usage


try:
print(f"Setup the environment for Mssql Server:[{ARGS.mssql_server_name}]")
Expand Down
88 changes: 88 additions & 0 deletions recipes/python/backup-restore/oracle_db_clone.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
""" This script execute the Oracle Database backup and clone scenario. """

## The script can be run with Python 3.6 or higher version.

## The script requires 'requests' library to make the API calls.
## The library can be installed using the command: pip install requests.

import argparse
import common
import workload_oracle

PARSER = argparse.ArgumentParser(description="Oracle Database clone scenario")
PARSER.add_argument("--primary_server", type=str, help="NetBackup primary server name")
PARSER.add_argument("--primary_server_port", type=int, help="NetBackup primary server port", required=False)
PARSER.add_argument("--primary_username", type=str, help="NetBackup primary server username")
PARSER.add_argument("--primary_password", type=str, help="NetBackup primary server password")
PARSER.add_argument("--source_oracle_db", type=str, help="Source client oracle database unique or pluggable database display name")
PARSER.add_argument("--source_database_id", type=str, help="Source client oracle container database id")
PARSER.add_argument("--target_oracle_server", type=str, help="Target client name")
PARSER.add_argument("--os_oracle_domain", type=str, help="Target oracle server domain")
PARSER.add_argument("--os_oracle_username", type=str, help="Target oracle username")
PARSER.add_argument("--os_oracle_password", type=str, help="Target oracle user password")
PARSER.add_argument("--clone_db_file_path", type=str, help="Clone database path", required=True)
PARSER.add_argument("--oracle_home", type=str, help="Oracle home", required=True)
PARSER.add_argument("--oracle_base_config", type=str, help="Oracle base config", required=True)
PARSER.add_argument("--pdb_clone", type=int, help="Complete CDB clone (0) or a PDB clone(1)", default=0)
PARSER.add_argument("--avail_oracle_sid", type=str, help="Target existing container for pdb clone")
PARSER.add_argument("--clone_aux_file_path", type=str, help="Auxiliary file paths")
PARSER.add_argument("--credential_id", type=str, help="Credential Id", required=False)
PARSER.add_argument("--force_clone", type=int, help="Force a clone request", default=0)

ARGS = PARSER.parse_args()

if __name__ == '__main__':
WORKLOAD_TYPE = 'oracle'
ASSET_TYPE = "PDB" if (ARGS.pdb_clone) else "DATABASE"

BASEURL = common.get_nbu_base_url(ARGS.primary_server, ARGS.primary_server_port)
TOKEN = common.get_authenticate_token(BASEURL, ARGS.primary_username, ARGS.primary_password)
print(f"User authentication completed for primary server:[{ARGS.primary_server}]")

try:
# run asset-service to get the asset id
RECOVERY_ASSET_ID = workload_oracle.get_oracle_asset_info(BASEURL, TOKEN, ASSET_TYPE, ARGS.source_oracle_db, ARGS.source_database_id)
print(f"Oracle Asset Id:[{RECOVERY_ASSET_ID}]")
# run recovery-point-service to get recovery-point-id
RECOVERY_POINT_ID = common.get_recovery_points(BASEURL, TOKEN, WORKLOAD_TYPE, RECOVERY_ASSET_ID)
print(f"Oracle Recovery Point Id:[{RECOVERY_POINT_ID}]")

if (ARGS.pdb_clone):
preCheckURL = f"{BASEURL}recovery/workloads/oracle/scenarios/pdb-complete-clone/pre-recovery-check"
cloneURL = f"{BASEURL}recovery/workloads/oracle/scenarios/pdb-complete-clone/recover"
recovery_input_json = "post_oracle_pdb_clone.json"
data = workload_oracle.create_pdb_clone_request_payload(recovery_input_json, RECOVERY_POINT_ID, ARGS.credential_id, ARGS.target_oracle_server,
ARGS.clone_db_file_path, ARGS.oracle_home, ARGS.oracle_base_config, ARGS.avail_oracle_sid, ARGS.clone_aux_file_path)
else:
preCheckURL = f"{BASEURL}recovery/workloads/oracle/scenarios/database-complete-clone/pre-recovery-check"
cloneURL = f"{BASEURL}recovery/workloads/oracle/scenarios/database-complete-clone/recover"
recovery_input_json = "post_oracle_cdb_clone.json"
data = workload_oracle.create_cdb_clone_request_payload(recovery_input_json, RECOVERY_POINT_ID, ARGS.os_oracle_domain, ARGS.os_oracle_username,
ARGS.os_oracle_password, ARGS.target_oracle_server, ARGS.clone_db_file_path, ARGS.oracle_home, ARGS.oracle_base_config)

# run a pre-recovery-check
print("Perform a pre-recovery-check")
status_code, response_text = workload_oracle.create_oracle_recovery_request(preCheckURL, TOKEN, data)

common.validate_response(status_code, 200, response_text)
print(f"Pre-recovery-check response :{response_text['data']}")
preCheckFail = 0
for _result in response_text['data']:
if (_result['attributes']['result'] != 'Passed'):
print(f"FAIL REASON: {_result['attributes']['description']}")
preCheckFail += 1

if ((not preCheckFail) or (ARGS.force_clone)):
# run a clone
print("Perform a database clone")

status_code, response_text = workload_oracle.create_oracle_recovery_request(cloneURL, TOKEN, data)

common.validate_response(status_code, 201, response_text)
CLONE_JOBID = response_text['data']['id']
print(f"Clone initiated , follow Job # :[{CLONE_JOBID}]")
else:
print("Pre-recovery check failed. Force kick off clone using --force_clone")

finally:
print("To cleanup the cloned instance, run dbca. Add the instance to /etc/oratab to be discovered by oracle for cleanup.")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this script mainly created for Linux users? If not, should we add cleanup suggestions for Windows users?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, for windows they only need to run dbca. so thats the first half and the second part in case the instance is not discovered and doesn't appear on dbca

Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
{
"data": {
"type":"recoveryRequest",
"attributes": {
"recoveryPoint": "recoverypointid-uuid",
"recoveryObject": {
"instanceCredentials": {
"credentialType": "CREDENTIAL_DETAILS",
"domain": "domain",
"username": "administrator",
"password": "password"
}
},
"alternateRecoveryOptions": {
"client": "alternateClientName",
"oracleSID": "clonedInstanceName",
"oracleHome": "C:\\oracleHome",
"oracleBaseConfig" : "C:\\oracleBase",
"databaseName": "clonedDb",
"controlFilePaths": {
"renameAllFilesToSameLocation": {
"destination" : "c:\\temp\\"
}
},
"databaseFilePaths": {
"renameAllFilesToSameLocation": {
"destination" : "c:\\temp\\"
}
},
"tempFilePaths": {
"renameAllFilesToSameLocation": {
"destination" : "c:\\temp\\"
}
},
"redoFilePaths": {
"renameAllFilesToSameLocation" : {
"destination": "c:\\temp\\"
}
}
},
"recoveryOptions": {
"streams": 4,
"createDirectories": true
}
}
}
}
Loading