Skip to content

Commit 1d22d61

Browse files
Update parquet-encryption.md
1 parent 8c0d518 commit 1d22d61

File tree

1 file changed

+28
-1
lines changed

1 file changed

+28
-1
lines changed

parquet-encryption.md

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
copyright:
44
years: 2017, 2020
5-
lastupdated: "2020-05-27"
5+
lastupdated: "2020-07-02"
66

77
subcollection: AnalyticsEngine
88

@@ -73,6 +73,32 @@ To enable Parquet encryption in {{site.data.keyword.iae_full_notm}}, set the fol
7373
spark.executor.extraClassPath=/home/common/lib/parquetEncryption/ibm-parquet-<latestversion>-jar-with-dependencies.jar:/home/common/lib/parquetEncryption/parquet-format-<latestversion>.jar:/home/common/lib/parquetEncryption/parquet-hadoop-<latestversion>.jar
7474
```
7575
76+
## Mandatory parameters
77+
78+
The following parameters are required for writing encrypted data:
79+
80+
- List of columns to encrypt, with the master encryption keys:
81+
```
82+
parameter name: "encryption.column.keys"
83+
parameter value: "<master key ID>:<column>,<column>;<master key ID>:<column>,.."
84+
```
85+
- The footer key:
86+
```
87+
parameter name: "encryption.footer.key"
88+
parameter value: "<master key ID>"
89+
```
90+
For example:
91+
```
92+
dataFrame.write
93+
.option("encryption.footer.key" , "k1")
94+
.option("encryption.column.keys" , "k2:SSN,Address;k3:CreditCard")
95+
.parquet("<path to encrypted files>")
96+
```
97+
98+
**Important**:
99+
- `"<path to encrypted files>"` must contain the string `.encrypted` in the URL, for example `/path/to/my_table.parquet.encrypted`.
100+
- If neither the `"encryption.column.keys"` parameter nor the `"encryption.footer.key"` parameter is set, the file will not be encrypted. If only one of these parameters is set, an exception is thrown, because these parameters are mandatory for encrypted files.
101+
76102
## Optional parameters
77103
78104
The following optional parameters can be used when writing encrypted data:
@@ -94,6 +120,7 @@ The following optional parameters can be used when writing encrypted data:
94120
parameter name: "encryption.plaintext.footer"
95121
parameter value: "true"
96122
```
123+
**Important**: The `"encryption.footer.key"` parameter must also be specified in the plain text footer mode. Although the footer is not encrypted, the key is used to sign the footer content, which means that new readers could verify its integrity. Legacy readers are not affected by the addition of the footer signature.
97124
98125
## Usage examples
99126
{: #usage-examples-parquet-encryption}

0 commit comments

Comments
 (0)