Skip to content

Commit 86df330

Browse files
Sebastian Molendapubnub-release-bot
andauthored
Crypto module (#169)
* Crypto module * Disable acceptance tests * Remove TypedDict for py3.7 compatibility * Fix compatibility with py3.7... again * Remove randbytes for 3.7 compatibility. sigh * Post review fixes * Integrate crypto_module with pubnub * Update test matrix - drop support for py3.7 as it has reached end of life * Fix type, add missing params, add example * Add tests * Fix bug with always riv encrypting files * reenable acceptance tests for crypto module * Fix miss of encrypting files * Update license * PubNub SDK v7.3.0 release. --------- Co-authored-by: PubNub Release Bot <120067856+pubnub-release-bot@users.noreply.github.com>
1 parent 701ece7 commit 86df330

24 files changed

+1328
-585
lines changed

.github/workflows/run-tests.yml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ jobs:
2626
strategy:
2727
fail-fast: true
2828
matrix:
29-
python: [3.7.17, 3.8.17, 3.9.17, 3.10.12, 3.11.4]
29+
python: [3.8.18, 3.9.18, 3.10.13, 3.11.6]
3030
steps:
3131
- name: Checkout repository
3232
uses: actions/checkout@v3
@@ -78,9 +78,13 @@ jobs:
7878
cp sdk-specifications/features/access/authorization-failure-reporting.feature tests/acceptance/pam
7979
cp sdk-specifications/features/access/grant-token.feature tests/acceptance/pam
8080
cp sdk-specifications/features/access/revoke-token.feature tests/acceptance/pam
81+
cp sdk-specifications/features/encryption/cryptor-module.feature tests/acceptance/encryption
82+
mkdir tests/acceptance/encryption/assets/
83+
cp sdk-specifications/features/encryption/assets/* tests/acceptance/encryption/assets/
8184
8285
sudo pip3 install -r requirements-dev.txt
8386
behave --junit tests/acceptance/pam
87+
behave --junit tests/acceptance/encryption/cryptor-module.feature -t=~na=python -k
8488
- name: Expose acceptance tests reports
8589
uses: actions/upload-artifact@v3
8690
if: always()

.pubnub.yml

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
name: python
2-
version: 7.2.0
2+
version: 7.3.0
33
schema: 1
44
scm: github.com/pubnub/python
55
sdks:
@@ -18,7 +18,7 @@ sdks:
1818
distributions:
1919
- distribution-type: library
2020
distribution-repository: package
21-
package-name: pubnub-7.2.0
21+
package-name: pubnub-7.3.0
2222
location: https://pypi.org/project/pubnub/
2323
supported-platforms:
2424
supported-operating-systems:
@@ -97,8 +97,8 @@ sdks:
9797
-
9898
distribution-type: library
9999
distribution-repository: git release
100-
package-name: pubnub-7.2.0
101-
location: https://github.com/pubnub/python/releases/download/7.2.0/pubnub-7.2.0.tar.gz
100+
package-name: pubnub-7.3.0
101+
location: https://github.com/pubnub/python/releases/download/v7.3.0/pubnub-7.3.0.tar.gz
102102
supported-platforms:
103103
supported-operating-systems:
104104
Linux:
@@ -169,6 +169,13 @@ sdks:
169169
license-url: https://github.com/aio-libs/aiohttp/blob/master/LICENSE.txt
170170
is-required: Required
171171
changelog:
172+
- date: 2023-10-16
173+
version: v7.3.0
174+
changes:
175+
- type: feature
176+
text: "Add crypto module that allows configure SDK to encrypt and decrypt messages."
177+
- type: bug
178+
text: "Improved security of crypto implementation by adding enhanced AES-CBC cryptor."
172179
- date: 2023-07-06
173180
version: 7.2.0
174181
changes:

CHANGELOG.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,12 @@
1+
## v7.3.0
2+
October 16 2023
3+
4+
#### Added
5+
- Add crypto module that allows configure SDK to encrypt and decrypt messages.
6+
7+
#### Fixed
8+
- Improved security of crypto implementation by adding enhanced AES-CBC cryptor.
9+
110
## 7.2.0
211
July 06 2023
312

LICENSE

Lines changed: 25 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,29 @@
1-
PubNub Real-time Cloud-Hosted Push API and Push Notification Client Frameworks
2-
Copyright (c) 2013 PubNub Inc.
3-
http://www.pubnub.com/
4-
http://www.pubnub.com/terms
1+
PubNub Software Development Kit License Agreement
2+
Copyright © 2023 PubNub Inc. All rights reserved.
53

6-
Permission is hereby granted, free of charge, to any person obtaining a copy
7-
of this software and associated documentation files (the "Software"), to deal
8-
in the Software without restriction, including without limitation the rights
9-
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
10-
copies of the Software, and to permit persons to whom the Software is
11-
furnished to do so, subject to the following conditions:
4+
Subject to the terms and conditions of the license, you are hereby granted
5+
a non-exclusive, worldwide, royalty-free license to (a) copy and modify
6+
the software in source code or binary form for use with the software services
7+
and interfaces provided by PubNub, and (b) redistribute unmodified copies
8+
of the software to third parties. The software may not be incorporated in
9+
or used to provide any product or service competitive with the products
10+
and services of PubNub.
1211

13-
The above copyright notice and this permission notice shall be included in
14-
all copies or substantial portions of the Software.
12+
The above copyright notice and this license shall be included
13+
in or with all copies or substantial portions of the software.
1514

16-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
17-
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
18-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
19-
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
20-
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
21-
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
22-
THE SOFTWARE.
15+
This license does not grant you permission to use the trade names, trademarks,
16+
service marks, or product names of PubNub, except as required for reasonable
17+
and customary use in describing the origin of the software and reproducing
18+
the content of this license.
2319

24-
PubNub Real-time Cloud-Hosted Push API and Push Notification Client Frameworks
25-
Copyright (c) 2013 PubNub Inc.
26-
http://www.pubnub.com/
27-
http://www.pubnub.com/terms
20+
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF
21+
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
22+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
23+
EVENT SHALL PUBNUB OR THE AUTHORS OR COPYRIGHT HOLDERS OF THE SOFTWARE BE
24+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF
25+
CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
26+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
27+
28+
https://www.pubnub.com/
29+
https://www.pubnub.com/terms

examples/crypto_module.py

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
from pubnub.pnconfiguration import PNConfiguration
2+
from pubnub.pubnub import PubNub
3+
from pubnub.crypto import AesCbcCryptoModule
4+
from Cryptodome.Cipher import AES
5+
6+
my_cipher_key = 'myCipherKey'
7+
my_message = 'myMessage'
8+
9+
# by default no configuration changes is needed
10+
config = PNConfiguration()
11+
config.uuid = 'myUUID'
12+
config.cipher_key = my_cipher_key
13+
pubnub = PubNub(config)
14+
15+
# message will be encrypted the same way it was encrypted previously
16+
cbc_message = pubnub.crypto.encrypt(my_message) # new way of using cryptographic module from pubnub
17+
decrypted = config.crypto.decrypt(my_cipher_key, cbc_message)
18+
assert decrypted == my_message
19+
20+
# also no configuration changes is needed if you previously updated the cipher_mode to GCM
21+
config = PNConfiguration()
22+
config.uuid = 'myUUID'
23+
config.cipher_key = my_cipher_key
24+
config.cipher_mode = AES.MODE_GCM
25+
config.fallback_cipher_mode = AES.MODE_CBC
26+
pubnub = PubNub(config)
27+
28+
# message will be encrypted the same way it was encrypted previously
29+
gcm_message = pubnub.crypto.encrypt(my_message) # new way of using cryptographic module from pubnub
30+
decrypted = config.crypto.decrypt(my_cipher_key, gcm_message)
31+
assert decrypted == my_message
32+
33+
# opt in to use crypto module with headers and improved entropy
34+
config = PNConfiguration()
35+
config.uuid = 'myUUID'
36+
config.cipher_key = my_cipher_key
37+
config.cipher_mode = AES.MODE_GCM
38+
config.fallback_cipher_mode = AES.MODE_CBC
39+
module = AesCbcCryptoModule(config)
40+
config.crypto_module = module
41+
pubnub = PubNub(config)
42+
message = pubnub.crypto.encrypt(my_message)
43+
# this encryption method is not compatible with previous crypto methods
44+
try:
45+
decoded = config.crypto.decrypt(my_cipher_key, message)
46+
except Exception:
47+
pass
48+
# but can be decrypted with new crypto module
49+
decrypted = pubnub.crypto.decrypt(message)
50+
assert decrypted == my_message

pubnub/crypto.py

Lines changed: 172 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,16 @@
11
import hashlib
22
import json
33
import random
4-
from base64 import decodebytes, encodebytes
4+
import logging
55

6-
from pubnub.crypto_core import PubNubCrypto
6+
7+
from base64 import decodebytes, encodebytes, b64decode, b64encode
78
from Cryptodome.Cipher import AES
89
from Cryptodome.Util.Padding import pad, unpad
10+
from pubnub.crypto_core import PubNubCrypto, PubNubCryptor, PubNubLegacyCryptor, PubNubAesCbcCryptor, CryptoHeader, \
11+
CryptorPayload
12+
from pubnub.exceptions import PubNubException
13+
from typing import Union, Dict
914

1015

1116
Initial16bytes = '0123456789012345'
@@ -80,9 +85,10 @@ def get_secret(self, key):
8085

8186

8287
class PubNubFileCrypto(PubNubCryptodome):
83-
def encrypt(self, key, file):
88+
def encrypt(self, key, file, use_random_iv=True):
89+
8490
secret = self.get_secret(key)
85-
initialization_vector = self.get_initialization_vector(use_random_iv=True)
91+
initialization_vector = self.get_initialization_vector(use_random_iv)
8692
cipher = AES.new(bytes(secret[0:32], "utf-8"), self.mode, bytes(initialization_vector, 'utf-8'))
8793
initialization_vector = bytes(initialization_vector, 'utf-8')
8894

@@ -92,9 +98,9 @@ def encrypt(self, key, file):
9298
initialization_vector=initialization_vector
9399
)
94100

95-
def decrypt(self, key, file):
101+
def decrypt(self, key, file, use_random_iv=True):
96102
secret = self.get_secret(key)
97-
initialization_vector, extracted_file = self.extract_random_iv(file, use_random_iv=True)
103+
initialization_vector, extracted_file = self.extract_random_iv(file, use_random_iv)
98104
try:
99105
cipher = AES.new(bytes(secret[0:32], "utf-8"), self.mode, initialization_vector)
100106
result = unpad(cipher.decrypt(extracted_file), 16)
@@ -103,3 +109,163 @@ def decrypt(self, key, file):
103109
result = unpad(cipher.decrypt(extracted_file), 16)
104110

105111
return result
112+
113+
114+
class PubNubCryptoModule(PubNubCrypto):
115+
FALLBACK_CRYPTOR_ID: str = '0000'
116+
cryptor_map = {}
117+
default_cryptor_id: str
118+
119+
def __init__(self, cryptor_map: Dict[str, PubNubCryptor], default_cryptor: PubNubCryptor):
120+
self.cryptor_map = cryptor_map
121+
self.default_cryptor_id = default_cryptor.CRYPTOR_ID
122+
123+
def _validate_cryptor_id(self, cryptor_id: str) -> str:
124+
cryptor_id = cryptor_id or self.default_cryptor_id
125+
126+
if len(cryptor_id) != 4:
127+
logging.error(f'Malformed cryptor id: {cryptor_id}')
128+
raise PubNubException('Malformed cryptor id')
129+
130+
if cryptor_id not in self.cryptor_map.keys():
131+
logging.error(f'Unsupported cryptor: {cryptor_id}')
132+
raise PubNubException('unknown cryptor error')
133+
return cryptor_id
134+
135+
def _get_cryptor(self, cryptor_id):
136+
if not cryptor_id or cryptor_id not in self.cryptor_map:
137+
raise PubNubException('unknown cryptor error')
138+
return self.cryptor_map[cryptor_id]
139+
140+
# encrypt string
141+
def encrypt(self, message: str, cryptor_id: str = None) -> str:
142+
if not len(message):
143+
raise PubNubException('encryption error')
144+
cryptor_id = self._validate_cryptor_id(cryptor_id)
145+
data = message.encode('utf-8')
146+
crypto_payload = self.cryptor_map[cryptor_id].encrypt(data)
147+
header = self.encode_header(cryptor_id=cryptor_id, cryptor_data=crypto_payload['cryptor_data'])
148+
return b64encode(header + crypto_payload['data']).decode()
149+
150+
def decrypt(self, message):
151+
data = b64decode(message)
152+
header = self.decode_header(data)
153+
if header:
154+
cryptor_id = header['cryptor_id']
155+
payload = CryptorPayload(data=data[header['length']:], cryptor_data=header['cryptor_data'])
156+
if not header:
157+
cryptor_id = self.FALLBACK_CRYPTOR_ID
158+
payload = CryptorPayload(data=data)
159+
160+
if not len(payload['data']):
161+
raise PubNubException('decryption error')
162+
163+
if cryptor_id not in self.cryptor_map.keys():
164+
raise PubNubException('unknown cryptor error')
165+
166+
message = self._get_cryptor(cryptor_id).decrypt(payload)
167+
try:
168+
return json.loads(message)
169+
except Exception:
170+
return message
171+
172+
def encrypt_file(self, file_data, cryptor_id: str = None):
173+
if not len(file_data):
174+
raise PubNubException('encryption error')
175+
cryptor_id = self._validate_cryptor_id(cryptor_id)
176+
crypto_payload = self.cryptor_map[cryptor_id].encrypt(file_data)
177+
header = self.encode_header(cryptor_id=cryptor_id, cryptor_data=crypto_payload['cryptor_data'])
178+
return header + crypto_payload['data']
179+
180+
def decrypt_file(self, file_data):
181+
header = self.decode_header(file_data)
182+
if header:
183+
cryptor_id = header['cryptor_id']
184+
payload = CryptorPayload(data=file_data[header['length']:], cryptor_data=header['cryptor_data'])
185+
else:
186+
cryptor_id = self.FALLBACK_CRYPTOR_ID
187+
payload = CryptorPayload(data=file_data)
188+
189+
if not len(payload['data']):
190+
raise PubNubException('decryption error')
191+
192+
if cryptor_id not in self.cryptor_map.keys():
193+
raise PubNubException('unknown cryptor error')
194+
195+
return self._get_cryptor(cryptor_id).decrypt(payload, binary_mode=True)
196+
197+
def encode_header(self, cryptor_id: str = None, cryptor_data: any = None) -> str:
198+
if cryptor_id == self.FALLBACK_CRYPTOR_ID:
199+
return b''
200+
if cryptor_data and len(cryptor_data) > 65535:
201+
raise PubNubException('Cryptor data is too long')
202+
cryptor_id = self._validate_cryptor_id(cryptor_id)
203+
204+
sentinel = b'PNED'
205+
version = CryptoHeader.header_ver.to_bytes(1, byteorder='big')
206+
crid = bytes(cryptor_id, 'utf-8')
207+
208+
if cryptor_data:
209+
crd = cryptor_data
210+
cryptor_data_len = len(cryptor_data)
211+
else:
212+
crd = b''
213+
cryptor_data_len = 0
214+
215+
if cryptor_data_len < 255:
216+
crlen = cryptor_data_len.to_bytes(1, byteorder='big')
217+
else:
218+
crlen = b'\xff' + cryptor_data_len.to_bytes(2, byteorder='big')
219+
return sentinel + version + crid + crlen + crd
220+
221+
def decode_header(self, header: bytes) -> Union[None, CryptoHeader]:
222+
try:
223+
sentinel = header[:4]
224+
if sentinel != b'PNED':
225+
return False
226+
except ValueError:
227+
return False
228+
229+
try:
230+
header_version = header[4]
231+
if header_version > CryptoHeader.header_ver:
232+
raise PubNubException('unknown cryptor error')
233+
234+
cryptor_id = header[5:9].decode()
235+
crlen = header[9]
236+
if crlen < 255:
237+
cryptor_data = header[10: 10 + crlen]
238+
hlen = 10 + crlen
239+
else:
240+
crlen = int(header[10:12].hex(), 16)
241+
cryptor_data = header[12:12 + crlen]
242+
hlen = 12 + crlen
243+
244+
return CryptoHeader(sentinel=sentinel, header_ver=header_version, cryptor_id=cryptor_id,
245+
cryptor_data=cryptor_data, length=hlen)
246+
except IndexError:
247+
raise PubNubException('decryption error')
248+
249+
250+
class LegacyCryptoModule(PubNubCryptoModule):
251+
def __init__(self, config) -> None:
252+
cryptor_map = {
253+
PubNubLegacyCryptor.CRYPTOR_ID: PubNubLegacyCryptor(config.cipher_key,
254+
config.use_random_initialization_vector,
255+
config.cipher_mode,
256+
config.fallback_cipher_mode),
257+
PubNubAesCbcCryptor.CRYPTOR_ID: PubNubAesCbcCryptor(config.cipher_key),
258+
}
259+
super().__init__(cryptor_map, PubNubLegacyCryptor)
260+
261+
262+
class AesCbcCryptoModule(PubNubCryptoModule):
263+
def __init__(self, config) -> None:
264+
cryptor_map = {
265+
PubNubLegacyCryptor.CRYPTOR_ID: PubNubLegacyCryptor(config.cipher_key,
266+
config.use_random_initialization_vector,
267+
config.cipher_mode,
268+
config.fallback_cipher_mode),
269+
PubNubAesCbcCryptor.CRYPTOR_ID: PubNubAesCbcCryptor(config.cipher_key),
270+
}
271+
super().__init__(cryptor_map, PubNubAesCbcCryptor)

0 commit comments

Comments
 (0)