Skip to content

Owls83995 - Sample scripts to shutdown and start a specific managed server/cluster/domain #2002

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
Nov 5, 2020
Merged

Owls83995 - Sample scripts to shutdown and start a specific managed server/cluster/domain #2002

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
4d6de23
owls-83995 - Scripts to start/stop a managed server/cluster/domain
ankedia Oct 20, 2020
e20b630
fix method comments
ankedia Oct 20, 2020
7a2b1e3
Minor changes
ankedia Oct 20, 2020
cf096af
Address review comments and fix script comments/usages.
ankedia Oct 21, 2020
302a597
Added integration tests, made few doc changes based on review comment…
ankedia Oct 22, 2020
a134689
Clarify script usage, updated README file and minor changes.
ankedia Oct 22, 2020
97b7fcd
Changes to add script usage details
ankedia Oct 22, 2020
fde5cb8
Address PR review comments
ankedia Oct 22, 2020
2927762
Review comment and cleanup.
ankedia Oct 22, 2020
01e6307
Documentation changes based on PR review comments.
ankedia Oct 23, 2020
4eba072
Fully qualified replica value as per review comments
ankedia Oct 23, 2020
eadad48
edit docs
rosemarymarano Oct 23, 2020
5845883
edit README
rosemarymarano Oct 23, 2020
1fda64a
Address PR review comments
ankedia Oct 23, 2020
71922eb
Changes to address PR review comments and removed ItLifecycleSampleSc…
ankedia Oct 23, 2020
d66c1a8
fix indentation
ankedia Oct 23, 2020
05c5a3f
fix comment and typo
ankedia Oct 23, 2020
0ba9537
Added validation as per review comment.
ankedia Oct 24, 2020
0ab8bd0
changes to address review comment and minor cleanup
ankedia Oct 25, 2020
3950815
PR review comment - changes to assume default policy is IF_NEEDED if …
ankedia Oct 26, 2020
2859f08
changes for new algorithm as documented on http://aseng-wiki.us.oracl…
ankedia Oct 30, 2020
81e0486
More changes for new algorithm.
ankedia Oct 31, 2020
c6c287b
code refactoring and minor doc update.
ankedia Nov 1, 2020
3456c68
Minor change for dynamic server name validation
ankedia Nov 2, 2020
df3c33f
Changes to address review comments.
ankedia Nov 2, 2020
3e028fb
More review comment changes and cleanup.
ankedia Nov 3, 2020
10feb99
Unset policy to start independent (stadalone) maanged server instead …
ankedia Nov 3, 2020
15c725a
Latest review comment changes.
ankedia Nov 3, 2020
e5b1044
More changes based on review comments.
ankedia Nov 4, 2020
b17410a
Chnages for latest review comments.
ankedia Nov 4, 2020
38d442d
Remove unused action variable and assignments.
ankedia Nov 4, 2020
e4bf493
Fix the logic to display error when config map not found and return p…
ankedia Nov 4, 2020
b775210
Changes for latest review comments.
ankedia Nov 4, 2020
c1f109d
Changes for latest round of review comments.
ankedia Nov 4, 2020
2c5ae36
use printError instead of echo
ankedia Nov 4, 2020
899073a
Changes to remove integration tests and doc review comments.
ankedia Nov 5, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
More changes for new algorithm.
  • Loading branch information
@ankedia
ankedia committed Oct 31, 2020
commit 81e04869b8bb243f182ac6b22999043e465e210f
189 changes: 109 additions & 80 deletions kubernetes/samples/scripts/domain-lifecycle/helper.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,22 +171,28 @@ function getSortedListOfServers {
done
fi
# Get servers with ALWAYS policy
for localServerName in "${sortedServers[@]}"; do
getEffectivePolicy "${domainJson}" "${localServerName}" "${clusterName}" policy
#if unset policy is true and server is current server
if [[ "${unsetPolicy}" == "true" && "${serverName}" == "${localServerName}" ]]; then
policy=UNSET
fi
if [ "${policy}" == "ALWAYS" ]; then
sortedByAlwaysServers+=(${localServerName})
else
otherServers+=(${localServerName})
fi
done
sortedServersSize=${#sortedServers[@]}
if [ "${sortedServersSize}" -gt 0 ]; then
for localServerName in "${sortedServers[@]}"; do
getEffectivePolicy "${domainJson}" "${localServerName}" "${clusterName}" policy
#if unset policy is true and server is current server
if [[ "${unsetPolicy}" == "true" && "${serverName}" == "${localServerName}" ]]; then
policy=UNSET
fi
if [ "${policy}" == "ALWAYS" ]; then
sortedByAlwaysServers+=(${localServerName})
else
otherServers+=(${localServerName})
fi
done
fi

for otherServer in "${otherServers[@]}"; do
sortedByAlwaysServers+=($otherServer)
done
otherServersSize=${#otherServers[@]}
if [ "${otherServersSize}" -gt 0 ]; then
for otherServer in "${otherServers[@]}"; do
sortedByAlwaysServers+=($otherServer)
done
fi
}

function getReplicaCount {
Expand Down Expand Up @@ -219,19 +225,22 @@ function checkServersStoppedByUnsetPolicyWhenAlways {
echo "started servers before are -> ${startedServers[@]}"
fi
replicaCount=$((replicaCount-1))
for localServerName in "${sortedByAlwaysServers[@]}"; do
# Get effictive server policy
getEffectivePolicy "${domainJson}" "${localServerName}" "${clusterName}" policy
if [ "${serverName}" == "${localServerName}" ]; then
policy=UNSET
fi
# check if server should be started based on policy and replica count
shouldStart ${currentReplicas} ${policy} ${replicaCount} result
if [ "${result}" == 'True' ]; then
currentReplicas=$((currentReplicas+1))
startedServers+=(${localServerName})
fi
done
sortedByAlwaysSize=${#sortedByAlwaysServers[@]}
if [ "${sortedByAlwaysSize}" -gt 0 ]; then
for localServerName in "${sortedByAlwaysServers[@]}"; do
# Get effictive server policy
getEffectivePolicy "${domainJson}" "${localServerName}" "${clusterName}" policy
if [ "${serverName}" == "${localServerName}" ]; then
policy=UNSET
fi
# check if server should be started based on policy and replica count
shouldStart ${currentReplicas} ${policy} ${replicaCount} result
if [ "${result}" == 'True' ]; then
currentReplicas=$((currentReplicas+1))
startedServers+=(${localServerName})
fi
done
fi

startedSize=${#startedServers[@]}
if [ ${startedSize} -gt 0 ]; then
Expand Down Expand Up @@ -261,19 +270,22 @@ function checkServersStoppedByDecreasingReplicasAndUnsetPolicy {
getReplicaCount "${domainJson}" "${clusterName}" replicaCount
startedSize=${#startedServers[@]}
replicaCount=$((replicaCount-1))
for localServerName in "${sortedByAlwaysServers[@]}"; do
# Get effictive server policy
getEffectivePolicy "${domainJson}" "${localServerName}" "${clusterName}" policy
# check if server should be started based on policy and replica count
if [ "${serverName}" == "${localServerName}" ]; then
policy=UNSET
fi
shouldStart ${currentReplicas} ${policy} ${replicaCount} result
if [ "${result}" == 'True' ]; then
currentReplicas=$((currentReplicas+1))
startedServers+=(${localServerName})
fi
done
sortedByAlwaysSize=${#sortedByAlwaysServers[@]}
if [ "${sortedByAlwaysSize}" -gt 0 ]; then
for localServerName in "${sortedByAlwaysServers[@]}"; do
# Get effictive server policy
getEffectivePolicy "${domainJson}" "${localServerName}" "${clusterName}" policy
# check if server should be started based on policy and replica count
if [ "${serverName}" == "${localServerName}" ]; then
policy=UNSET
fi
shouldStart ${currentReplicas} ${policy} ${replicaCount} result
if [ "${result}" == 'True' ]; then
currentReplicas=$((currentReplicas+1))
startedServers+=(${localServerName})
fi
done
fi

startedSize=${#startedServers[@]}
if [ ${startedSize} -gt 0 ]; then
Expand Down Expand Up @@ -303,16 +315,19 @@ function checkServersStartedByCurrentReplicasAndPolicy {
unsetPolicy="false"
getSortedListOfServers "${domainJson}" "${serverName}" "${clusterName}" "${unsetPolicy}"
getReplicaCount "${domainJson}" "${clusterName}" replicaCount
for localServerName in "${sortedByAlwaysServers[@]}"; do
# Get effictive server policy
getEffectivePolicy "${domainJson}" "${localServerName}" "${clusterName}" policy
# check if server should be started based on policy and replica count
shouldStart ${currentReplicas} ${policy} ${replicaCount} result
if [ "${result}" == 'True' ]; then
currentReplicas=$((currentReplicas+1))
startedServers+=(${localServerName})
fi
done
sortedByAlwaysSize=${#sortedByAlwaysServers[@]}
if [ "${sortedByAlwaysSize}" -gt 0 ]; then
for localServerName in "${sortedByAlwaysServers[@]}"; do
# Get effictive server policy
getEffectivePolicy "${domainJson}" "${localServerName}" "${clusterName}" policy
# check if server should be started based on policy and replica count
shouldStart ${currentReplicas} ${policy} ${replicaCount} result
if [ "${result}" == 'True' ]; then
currentReplicas=$((currentReplicas+1))
startedServers+=(${localServerName})
fi
done
fi
startedSize=${#startedServers[@]}
if [ ${startedSize} -gt 0 ]; then
if checkStringInArray ${serverName} ${startedServers[@]}; then
Expand All @@ -323,11 +338,12 @@ function checkServersStartedByCurrentReplicasAndPolicy {
eval $__started="false"
}

function checkServerStartByIncreasingReplicasAndUnsetPolicy {
function checkServerStartByUnsetPolicy {
local domainJson=$1
local serverName=$2
local clusterName=$3
local __started=$4
local withReplicas=$4
local __started=$5
local localServerName=""
local policy=""
local replicaCount=0
Expand All @@ -338,20 +354,25 @@ function checkServerStartByIncreasingReplicasAndUnsetPolicy {
unsetPolicy="true"
getSortedListOfServers "${domainJson}" "${serverName}" "${clusterName}" "${unsetPolicy}"
getReplicaCount "${domainJson}" "${clusterName}" replicaCount
replicaCount=$((replicaCount+1))
for localServerName in "${sortedByAlwaysServers[@]}"; do
# Get effictive server policy
getEffectivePolicy "${domainJson}" "${localServerName}" "${clusterName}" policy
# check if server should be started based on policy and replica count
if [ "${serverName}" == "${localServerName}" ]; then
policy=UNSET
fi
shouldStart "${currentReplicas}" "${policy}" "${replicaCount}" result
if [ "${result}" == 'True' ]; then
currentReplicas=$((currentReplicas+1))
startedServers+=(${localServerName})
fi
done
if [ "${withReplicas}" == "INCREASED" ]; then
replicaCount=$((replicaCount+1))
fi
sortedByAlwaysSize=${#sortedByAlwaysServers[@]}
if [ "${sortedByAlwaysSize}" -gt 0 ]; then
Copy link

@tbarnes-us tbarnes-us Nov 3, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a minor point: in my experimentation changing to the following:

    # note that the lack of quotes is deliberate
    for localServerName in ${sortedByAlwaysServers[@]:-}; do
    done

is nearly equivalent to:

  sortedByAlwaysSize=${#sortedByAlwaysServers[@]}
  if [ "${sortedByAlwaysSize}" -gt 0 ]; then
    for localServerName in "${sortedByAlwaysServers[@]}"; do
    done
  fi

The drawback is that it (a) assumes server names won't have special chars like quotes in them - which is probably a reasonable assumption, and (b) it defeats the 'set -u' var name check so a bad var name would short circuit the loop...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to use new syntax to avoid size check for empty arrays.

for localServerName in "${sortedByAlwaysServers[@]}"; do
# Get effictive server policy
getEffectivePolicy "${domainJson}" "${localServerName}" "${clusterName}" policy
# check if server should be started based on policy and replica count
if [ "${serverName}" == "${localServerName}" ]; then
policy=UNSET
fi
shouldStart "${currentReplicas}" "${policy}" "${replicaCount}" result
if [ "${result}" == 'True' ]; then
currentReplicas=$((currentReplicas+1))
startedServers+=(${localServerName})
fi
done
fi
startedSize=${#startedServers[@]}
if [ ${startedSize} -gt 0 ]; then
if checkStringInArray ${serverName} ${startedServers[@]}; then
Expand Down Expand Up @@ -458,20 +479,28 @@ Please make sure server name is correct."
namePrefixSize=". | {name: .name, prefix:.dynamicServersConfig.serverNamePrefix, \
max:.dynamicServersConfig.maxDynamicClusterSize}"
dynamicClusters=($(echo $jsonTopology | jq "${dynamicClause}" | jq -cr "${namePrefixSize}"))
for dynaClusterNamePrefix in "${dynamicClusters[@]}"; do
prefix=$(echo ${dynaClusterNamePrefix} | jq -r .prefix)
if [[ "${serverName}" == "${prefix}"* ]]; then
serverCount=$(echo "${serverName: -1}")
maxSize=$(echo ${dynaClusterNamePrefix} | jq -r .max)
if [ "${serverCount}" -gt "${maxSize}" ]; then
printError "${errorMessage}"
exit 1
dynamicClustersSize=${#dynamicClusters[@]}
if [ "${dynamicClustersSize}" -gt 0 ]; then
for dynaClusterNamePrefix in "${dynamicClusters[@]}"; do
prefix=$(echo ${dynaClusterNamePrefix} | jq -r .prefix)
if [[ "${serverName}" == "${prefix}"* ]]; then
serverCount=$(echo "${serverName: -1}")
maxSize=$(echo ${dynaClusterNamePrefix} | jq -r .max)
number='^[0-9]+$'
if ! [[ $serverCount =~ $number ]] ; then
echo "error: Server name is not valid for dynamic cluster."
exit 1
fi
if [ "${serverCount}" -gt "${maxSize}" ]; then
printError "${errorMessage}"
exit 1
fi
eval $__clusterName="'$(echo ${dynaClusterNamePrefix} | jq -r .name)'"
eval $__isValidServer=true
break
fi
eval $__clusterName="'$(echo ${dynaClusterNamePrefix} | jq -r .name)'"
eval $__isValidServer=true
break
fi
done
done
fi
staticClause=".domain.configuredClusters[] | select (.dynamicServersConfig == null)"
nameCmd=" . | {name: .name, serverName: .servers[].name}"
configuredClusters=($(echo $jsonTopology | jq "${staticClause}" | jq -cr "${nameCmd}"))
Expand Down
45 changes: 30 additions & 15 deletions kubernetes/samples/scripts/domain-lifecycle/startServer.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,9 @@ keepReplicaConstant=false
verboseMode=false
managedServerPolicy=""
action=""
isValidServer=""
withRelicas=""
patchJson=""

while getopts "vkd:n:m:s:h" opt; do
case $opt in
Expand Down Expand Up @@ -120,29 +123,37 @@ if [ "${domainPolicy}" == 'NEVER' ]; then
fi

getEffectivePolicy "${domainJson}" "${serverName}" "${clusterName}" effectivePolicy
checkServersStartedByCurrentReplicasAndPolicy "${domainJson}" "${serverName}" "${clusterName}" started
if [[ ${effectivePolicy} == "IF_NEEDED" && ${started} == "true" ]]; then
echo "[INFO] The server should be already started or it's starting. The start policy for server ${serverName} is ${effectivePolicy} and server is chosen to be started based on current replica count."
exit 0
elif [[ "${effectivePolicy}" == "ALWAYS" && ${started} == "true" ]]; then
echo "[INFO] The server should be already started or it's starting. The start policy for server ${serverName} is ${effectivePolicy}."
exit 0
if [ -n "${clusterName}" ]; then
checkServersStartedByCurrentReplicasAndPolicy "${domainJson}" "${serverName}" "${clusterName}" started
if [[ ${effectivePolicy} == "IF_NEEDED" && ${started} == "true" ]]; then
echo "[INFO] The server should be already started or it's starting. The start policy for server ${serverName} is ${effectivePolicy} and server is chosen to be started based on current replica count."
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe let's have a zoom and word-smith the script's messages a little together?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, sure. That would be helpful.

exit 0
elif [[ "${effectivePolicy}" == "ALWAYS" && ${started} == "true" ]]; then
echo "[INFO] The server should be already started or it's starting. The start policy for server ${serverName} is ${effectivePolicy}."
exit 0
fi
else
if [ "${effectivePolicy}" == "ALWAYS" ]; then
echo "[INFO] The server should be already started or it's starting. The start policy for server ${serverName} is ${effectivePolicy}."
exit 0
fi
fi

# Get current server start policy of the server
getCurrentPolicy "${domainJson}" "${serverName}" managedServerPolicy
targetPolicy="ALWAYS"
createServerStartPolicyPatch "${domainJson}" "${serverName}" "${targetPolicy}" alwaysStartPolicyPatch

# if server is part of a cluster and replica count needs to be updated, patch the replica count and server start policy
if [[ -n ${clusterName} && "${keepReplicaConstant}" != 'true' ]]; then
#check if server can be started by increasing replicas and unsetting policy
checkServerStartByIncreasingReplicasAndUnsetPolicy "${domainJson}" "${serverName}" "${clusterName}" started
withRelicas="INCREASED"
checkServerStartByUnsetPolicy "${domainJson}" "${serverName}" "${clusterName}" "${withRelicas}" started
operation="INCREMENT"
createReplicaPatch "${domainJson}" "${clusterName}" "${operation}" incrementReplicaPatch replicaCount
if [ "${incrementReplicaPatch}" == "MAX_REPLICA_COUNT_EXCEEDED" ]; then
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the replica count is already maxed out, why not just continue and start the server by unsetting it (e.g. implicitly bringing it to IF_NEEDED)?

Copy link
Member Author

@ankedia ankedia Nov 3, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code path for the default case where script increases the replica count. I'm not sure if unsetting policy (bringing it to IF_NEEDED) will start the server. The user can use '-k' to keep replica constant and start the server by unsetting policy. For e.g. if server1 to server4 are running and max replica count is 4. If user wants to start server5 and policy is NEVER, unsetting policy will not start server.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed, the assumption is that this case will be caught by previous validation check. I have removed the check.

exit 1
fi
targetPolicy="ALWAYS"
createServerStartPolicyPatch "${domainJson}" "${serverName}" "${targetPolicy}" alwaysStartPolicyPatch
if [[ -n ${managedServerPolicy} && ${started} == "true" ]]; then
# Server starts by increasing replicas and policy unset, increment and unset
echo "[INFO] Unsetting the current start policy '${managedServerPolicy}' for '${serverName}' and incrementing replica count."
Expand All @@ -155,26 +166,30 @@ if [[ -n ${clusterName} && "${keepReplicaConstant}" != 'true' ]]; then
action="PATCH_REPLICA"
else
# Patch server policy to always and increment replicas
patchJson="{\"spec\": {\"clusters\": "${incrementReplicaPatch}",\"managedServers\": "${alwaysStartPolicyPatch}"}}"
action="PATCH_REPLICA_AND_POLICY"
echo "[INFO] Patching start policy of server '${serverName}' from '${effectivePolicy}' to 'ALWAYS' and \
incrementing replica count for cluster '${clusterName}'."
patchJson="{\"spec\": {\"clusters\": "${incrementReplicaPatch}",\"managedServers\": "${alwaysStartPolicyPatch}"}}"
action="PATCH_REPLICA_AND_POLICY"
fi
elif [[ -n ${clusterName} && "${keepReplicaConstant}" == 'true' ]]; then
# if replica count needs to stay constant, only patch server start policy
# Replica count needs to stay constant
# check if server can be started by unsetting policy
withRelicas="CONSTANT"
checkServerStartByUnsetPolicy "${domainJson}" "${serverName}" "${clusterName}" "${withRelicas}" started
if [[ "${effectivePolicy}" == "NEVER" && ${started} == "true" ]]; then
# Server starts by unsetting policy, unset policy
echo "[INFO] Unsetting the current start policy '${effectivePolicy}' for '${serverName}'."
createPatchJsonToUnsetPolicy "${domainJson}" "${serverName}" patchJson
action="UNSET_POLICY"
else
# Patch server policy to always
echo "[INFO] Patching start policy for '${serverName}' to '${targetPolicy}'."
patchJson="{\"spec\": {\"managedServers\": "${alwaysStartPolicyPatch}"}}"
action="PATCH_POLICY"
fi
else
# if server is an independent managed server, only patch server start policy
patchJson="{\"spec\": {\"managedServers\": "${serverStartPolicyPatch}"}}"
# if server is an independent managed server, only patch server start policy to ALWAYS
patchJson="{\"spec\": {\"managedServers\": "${alwaysStartPolicyPatch}"}}"
action="PATCH_POLICY"
fi

Expand Down
17 changes: 13 additions & 4 deletions kubernetes/samples/scripts/domain-lifecycle/stopServer.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ started=""
action=""
effectivePolicy=""
managedServerPolicy=""
stoppedWhenAlwaysPolicyReset=""

while getopts "vks:m:n:d:h" opt; do
case $opt in
Expand Down Expand Up @@ -102,16 +103,24 @@ clusterName=$(${kubernetesCli} get pod ${domainUid}-${serverName} -n ${domainNam
domainJson=$(${kubernetesCli} get domain ${domainUid} -n ${domainNamespace} -o json)

getEffectivePolicy "${domainJson}" "${serverName}" "${clusterName}" effectivePolicy
checkServersStartedByCurrentReplicasAndPolicy "${domainJson}" "${serverName}" "${clusterName}" started
if [[ "${effectivePolicy}" == "NEVER" || "${started}" != "true" ]]; then
echo "[INFO] Server should be already stopping or stopped. This is either because of the sever start policy or server is chosen to be stopped based on current replica count."
exit 0
if [ -n "${clusterName}" ]; then
checkServersStartedByCurrentReplicasAndPolicy "${domainJson}" "${serverName}" "${clusterName}" started
if [[ "${effectivePolicy}" == "NEVER" || "${started}" != "true" ]]; then
echo "[INFO] Server should be already stopping or stopped. This is either because of the sever start policy or server is chosen to be stopped based on current replica count."
exit 0
fi
else
if [ "${effectivePolicy}" == "NEVER" ]; then
echo "[INFO] Server should be already stopping or stopped because sever start policy is 'NEVER'."
exit 0
fi
fi

# Create server start policy patch with NEVER value
createServerStartPolicyPatch "${domainJson}" "${serverName}" "${serverStartPolicy}" neverStartPolicyPatch
getCurrentPolicy "${domainJson}" "${serverName}" managedServerPolicy


if [[ -n "${clusterName}" && "${keepReplicaConstant}" != 'true' ]]; then
# server is part of a cluster and replica count needs to be updated
checkServersStoppedByDecreasingReplicasAndUnsetPolicy "${domainJson}" "${serverName}" "${clusterName}" stoppedWhenRelicaReducedAndPolicyReset
Expand Down