Skip to content

Commit 376b6df

Browse files
committed
init
1 parent 75193e9 commit 376b6df

File tree

135 files changed

+1383
-11
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

135 files changed

+1383
-11
lines changed

devops-for-datastage.html-e

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
PDF export is disabled (set environment variable ENABLE_PDF_EXPORT to 1 to enable)

docs/.pages

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
11
nav:
2-
- index.md
32
- overview
4-
- blog
5-
- about
6-
- help
3+
- flow-analysis
4+
- testing
5+
- patterns
6+
- pipelines
7+
- mettleci-cli
8+
- troubleshooting
79
- tags.md

docs/blog/index.md

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,2 @@
1-
---
2-
classification: confidential #Remove this line if it's not IBM Confidential.
3-
status: draft #Status can be draft, reviewed or published.
4-
owner: John McKeever
5-
---
6-
# Blog
1+
# Blog
2+

docs/flow-analysis/.pages

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
title: Flow Analysis
2+
nav:
3+
- flow-analysis.md

docs/flow-analysis/flow-analysis.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
---
2+
classification: confidential #Remove this line if it's not IBM Confidential.
3+
status: draft #Status can be draft, reviewed or published.
4+
owner: John McKeever
5+
---
6+
# Introduction
7+
8+
!!! note "The latest version can always be found here: https://pages.github.ibm.com/datamigrators/devops-for-datastage/"
9+
10+
[Download the latest (PDF)](pdf/devops-for-datastage.pdf){ .md-button } [Download the latest (DOCX)](out/devops-for-datastage.docx){ .md-button }
11+
12+
<div class="grid cards" markdown>
13+
14+
- :material-clock-fast:{ .lg .middle } __Set up in 5 minutes__
15+
16+
---
17+
18+
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
19+
20+
- :material-scale-balance:{ .lg .middle } __Open Source, MIT__
21+
22+
---
23+
24+
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
25+
26+
</div>
27+

docs/mettleci-cli/.pages

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
title: MettleCI CLI
2+
nav:
3+
- command-shell.md

docs/mettleci-cli/command-shell.md

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
---
2+
status: reviewed
3+
owner: John McKeever
4+
tags:
5+
- DATASTAGE
6+
- TESTING
7+
---
8+
# MettleCI Command Shell
9+
10+
The MettleCI Command Line Interface is available from either your Windows command line or Unix shell. It provides one way of accessing MettleCI's build and deployment functions, and supports two different modes of operation: console or command line. MettleCI commands accept various parameters which can optionally be sourced from a command file.
11+
12+
## Console Mode
13+
14+
To enter console mode start the MettleCI Command Line Interface by entering `mettleci` (UNIX) or `mettleci.cmd` (Windows).
15+
16+
```bat
17+
C:\> mettleci.cmd
18+
MettleCI Command Line
19+
(C)2019 Data Migrators Pty Ltd
20+
Enter [namespace] [command] {options}
21+
or 'help', 'exit', or 'quit'.
22+
mettleci>
23+
```
24+
25+
In console mode MettleCI prints a command prompt and waits for a command. Each command is processed without exiting MettleCI. You may need to provide authentication options for a commands which invoke functionality in third party systems. You can enter help to get assistance, or exit the console mode by entering exit, or quit at the prompt.
26+
27+
## Command mode
28+
29+
In command mode you can enter commands one at a time at your operating system's command line. Start each command (omitting the quotes) with `mettleci` (unix) or `mettleci.cmd` (Windows) followed by a namespace and command, then the parameters.
30+
31+
Open image2019-10-25_22-51-19.png
32+
33+
Some of the available commands (listed below) use IBM DataStage client components, and so are platform specific. For example, job compilation requires access to Windows-only components, and so will not be supported on Unix environments. The MettleCI command is followed by a namespace, which groups a collection of build and deployment operations. Each of these commands accepts a number of mandatory and/or optional parameters.
34+
35+
36+
```
37+
mettleci [namespace] [command] {commandoptions}
38+
```
39+
40+
Here's an example of the MettleCI Command Line being used to compile an entire DataStage project (using the ['compile' command provided by the 'datastage' namespace](link.md)) :
41+
42+
```shell
43+
# Note that the example below uses the line continuation charactere ('\' on Unix or '^' on Windows)
44+
# to aid readability, but your mettleci command line can all be on a single line if you prefer
45+
$> mettleci datastage compile \
46+
-domain test1-svcs.datamigrators.io:59445 \
47+
-server test1-engn.datamigrators.io -project dstage1 \
48+
-username isadmin -password isadminpwd
49+
Analyzing assets to compile
50+
Compilation folder location = C:\Apps\command-shell\log\compiliation
51+
Attempting to compile with 4 working threads.
52+
Compiling DataStage jobs...
53+
* Compile 'test2-engn.datamigrators.io/dstage1/Jobs/Load/EX_Account.pjb' - COMPLETED
54+
[SNIP]
55+
* Compile 'test2-engn.datamigrators.io/dstage1/Jobs/Load/TX_StockHolding.pjb' - COMPLETED
56+
Compilation complete
57+
$>
58+
```
59+
60+
Note that MettleCI Command Line namespaces, commands, and options are all case sensitive.
61+
62+
## Use a password containing special characters
63+
64+
If the password contains special characters, you will need to wrap it with single or double quote or by using escape characters.
65+
66+
| Password contains | Windows-based | Unix-based |
67+
|-------------------|---------------|------------|
68+
| ! (exclamation) | Use password without modification. For example: MyPassword! | Wrap password with single quote. For example: 'MyPassword!' |
69+
| “ (double quote) | Use escape character \. For example: My\”Password | Wrap password with single quote. For example: 'My"Password' |
70+
| ' (single quote) | Wrap password with double quote. For example: “My'Password” | Wrap password with single quote and use escape character \. For example: 'My'\''Password' |
71+
| * (asterisk) | Use password without modification. For example: My*Password | Wrap password with single quote. For example: 'My*Password' |
72+
| <space> | Wrap password with double quote. For example: “My Password” | Wrap password with single quote. For example: 'My Password' |
73+
74+
## Using external command files with the MettleCI CLI
75+
76+
MettleCI allows you to define a MettleCI command in a text-based ‘command file’ and pass the file as parameter to the MettleCI command. This is accomplished using the '@' command syntax:
77+
78+
```shell
79+
# Here's a typical command file
80+
$> cat file mycommand.txt
81+
datastage
82+
compile
83+
-domain
84+
test1-svcs.datamigrators.io:59445
85+
-username
86+
isadmin
87+
-password
88+
isadminpwd
89+
-server
90+
test1-engn.datamigrators.io
91+
-project
92+
dstage1
93+
# ... and here's how to invoke it
94+
$> mettleci @mycommand.txt
95+
```
96+
97+
Note:
98+
99+
* Each element of a command file needs to be on an individual line (i.e. separated by your operating system’s newline ASCII character combination)
100+
* A command file can only contain the definition of a single MettleCI command
101+
* You can run the MettleCI Command Line with multiple commands by invoking it with individual command files from a shell script with one command per line. E.g.
102+
103+
```bash
104+
#!/usr/bin/env bash
105+
mettleci @mycommand1.txt
106+
mettleci @mycommand2.txt
107+
mettleci @mycommand3.txt
108+
# etc.
109+
```

docs/overview/index.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
---
2-
classification: confidential #Remove this line if it's not IBM Confidential.
2+
# classification: confidential #Remove this line if it's not IBM Confidential.
33
status: draft #Status can be draft, reviewed or published.
44
owner: John McKeever
55
---
66
# Overview
77

8+
Start here.

docs/patterns/.pages

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
title: Testing Patterns
2+
nav:
3+
- testing-datastage-containers.md
4+
- testing-datastage-surrogate-key-generators.md
5+
- testing-datastage-flow-rejects.md
6+
- testing-datastage-stored-procedures.md
7+
- testing-flows-using-datetime-references.md
41.6 KB
Loading

0 commit comments

Comments
 (0)