Skip to content

Commit 57fcaea

Browse files
matajohmatajoh
authored
Adding CI, fixing large arrays (#33)
Improvements: - Increased CHUNK size as per miniz instructions - Added tests for very large arrays in NPZ files - Added some CI tests to catch issues across platforms - Removed the internal IO streams in favor of just using stringstream - NPZs can now be read from and written to memory Bugfixes: - Fixed an issue where very large arrays in NPZ files would throw an error - Fixed a bug with mac builds due to deprecated APIs Signed-off-by: Matthew Johnson <matthew@matthewajohnson.org>
1 parent 9329797 commit 57fcaea

36 files changed

+2448
-2813
lines changed

‎.clang_format

+108
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
---
2+
Language: Cpp
3+
# BasedOnStyle: LLVM
4+
AccessModifierOffset: -2
5+
AlignAfterOpenBracket: AlwaysBreak
6+
AlignConsecutiveAssignments: false
7+
AlignConsecutiveDeclarations: false
8+
AlignEscapedNewlines: DontAlign
9+
AlignOperands: false
10+
AlignTrailingComments: false
11+
AllowAllParametersOfDeclarationOnNextLine: true
12+
AllowShortBlocksOnASingleLine: false
13+
AllowShortCaseLabelsOnASingleLine: false
14+
AllowShortFunctionsOnASingleLine: Empty
15+
AllowShortIfStatementsOnASingleLine: false
16+
AllowShortLoopsOnASingleLine: false
17+
AlwaysBreakAfterDefinitionReturnType: None
18+
AlwaysBreakAfterReturnType: None
19+
AlwaysBreakBeforeMultilineStrings: true
20+
AlwaysBreakTemplateDeclarations: true
21+
BinPackArguments: false
22+
BinPackParameters: false
23+
BraceWrapping:
24+
AfterClass: true
25+
AfterControlStatement: true
26+
AfterEnum: true
27+
AfterFunction: true
28+
AfterNamespace: true
29+
AfterObjCDeclaration: true
30+
AfterStruct: true
31+
AfterUnion: true
32+
AfterExternBlock: true
33+
BeforeCatch: true
34+
BeforeElse: true
35+
IndentBraces: false
36+
SplitEmptyFunction: false
37+
SplitEmptyRecord: false
38+
SplitEmptyNamespace: false
39+
BreakBeforeBinaryOperators: None
40+
BreakBeforeBraces: Custom
41+
BreakBeforeInheritanceComma: false
42+
BreakBeforeTernaryOperators: false
43+
BreakConstructorInitializersBeforeComma: false
44+
BreakConstructorInitializers: AfterColon
45+
BreakAfterJavaFieldAnnotations: false
46+
BreakStringLiterals: true
47+
ColumnLimit: 80
48+
CommentPragmas: "^ IWYU pragma:"
49+
CompactNamespaces: false
50+
ConstructorInitializerAllOnOneLineOrOnePerLine: true
51+
ConstructorInitializerIndentWidth: 2
52+
ContinuationIndentWidth: 2
53+
Cpp11BracedListStyle: true
54+
DerivePointerAlignment: false
55+
DisableFormat: false
56+
ExperimentalAutoDetectBinPacking: false
57+
FixNamespaceComments: false
58+
ForEachMacros:
59+
- FOREACH
60+
IncludeBlocks: Regroup
61+
IncludeCategories:
62+
- Regex: '^"(llvm|llvm-c|clang|clang-c)/'
63+
Priority: 2
64+
- Regex: '^(<|"(gtest|gmock|isl|json)/)'
65+
Priority: 3
66+
- Regex: ".*"
67+
Priority: 1
68+
IncludeIsMainRegex: "(Test)?$"
69+
IndentCaseLabels: true
70+
IndentPPDirectives: None
71+
IndentWidth: 2
72+
IndentWrappedFunctionNames: false
73+
JavaScriptQuotes: Leave
74+
JavaScriptWrapImports: true
75+
KeepEmptyLinesAtTheStartOfBlocks: false
76+
MacroBlockBegin: ""
77+
MacroBlockEnd: ""
78+
MaxEmptyLinesToKeep: 1
79+
NamespaceIndentation: All
80+
ObjCBlockIndentWidth: 2
81+
ObjCSpaceAfterProperty: false
82+
ObjCSpaceBeforeProtocolList: true
83+
PenaltyBreakAssignment: 2
84+
PenaltyBreakBeforeFirstCallParameter: 19
85+
PenaltyBreakComment: 300
86+
PenaltyBreakFirstLessLess: 120
87+
PenaltyBreakString: 1000
88+
PenaltyExcessCharacter: 1000000
89+
PenaltyReturnTypeOnItsOwnLine: 600
90+
PointerAlignment: Left
91+
ReflowComments: true
92+
SortIncludes: true
93+
SortUsingDeclarations: true
94+
SpaceAfterCStyleCast: false
95+
SpaceAfterTemplateKeyword: true
96+
SpaceBeforeAssignmentOperators: true
97+
SpaceBeforeParens: ControlStatements
98+
SpaceInEmptyParentheses: false
99+
SpacesBeforeTrailingComments: 1
100+
SpacesInAngles: false
101+
SpacesInContainerLiterals: false
102+
SpacesInCStyleCastParentheses: false
103+
SpacesInParentheses: false
104+
SpacesInSquareBrackets: false
105+
Standard: Cpp11
106+
TabWidth: 2
107+
UseTab: Never
108+
---

‎.github/workflows/prgate.yaml

+156
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
name: PR Gate
2+
3+
on:
4+
pull_request:
5+
branches: ["main"]
6+
workflow_dispatch:
7+
8+
jobs:
9+
cpp-format:
10+
runs-on: ubuntu-latest
11+
strategy:
12+
matrix:
13+
path:
14+
- check: src
15+
exclude: (/miniz/)
16+
- check: test
17+
exclude: ''
18+
steps:
19+
- name: Checkout
20+
uses: actions/checkout@v4
21+
22+
- name: Run clang-format style check
23+
uses: jidicula/clang-format-action@v4.8.0
24+
with:
25+
clang-format-version: '18'
26+
check-path: ${{matrix.path['check']}}
27+
exclude-regex: ${{matrix.path['exclude']}}
28+
29+
linux:
30+
runs-on: ubuntu-latest
31+
32+
steps:
33+
- name: Checkout
34+
uses: actions/checkout@v4
35+
36+
- name: Get dependencies
37+
run: |
38+
sudo apt-get install ninja-build
39+
40+
- name: CMake config
41+
run: cmake -B ${{github.workspace}}/build --preset release-clang
42+
43+
- name: CMake build
44+
working-directory: ${{github.workspace}}/build
45+
run: ninja
46+
47+
- name: Use Python 3.11
48+
uses: actions/setup-python@v4
49+
with:
50+
python-version: "3.11"
51+
52+
- name: Generate large NPZ files
53+
working-directory: ${{github.workspace}}/
54+
run: |
55+
pip install numpy
56+
python ${{github.workspace}}/test/generate_large_test.py
57+
58+
- name: CMake test
59+
working-directory: ${{github.workspace}}/build
60+
run: ctest -V --build-config Release --timeout 120 --output-on-failure -T Test
61+
62+
linux-asan:
63+
runs-on: ubuntu-latest
64+
65+
steps:
66+
- name: Checkout
67+
uses: actions/checkout@v4
68+
69+
- name: Get dependencies
70+
run: |
71+
sudo apt-get install ninja-build
72+
73+
- name: CMake config
74+
run: cmake -B ${{github.workspace}}/build --preset release-clang -DLIBNPY_SANITIZE=address
75+
76+
- name: CMake build
77+
working-directory: ${{github.workspace}}/build
78+
run: ninja
79+
80+
- name: Use Python 3.11
81+
uses: actions/setup-python@v4
82+
with:
83+
python-version: "3.11"
84+
85+
- name: Generate large NPZ files
86+
working-directory: ${{github.workspace}}/
87+
run: |
88+
pip install numpy
89+
python ${{github.workspace}}/test/generate_large_test.py
90+
91+
- name: CMake test
92+
working-directory: ${{github.workspace}}/build
93+
run: ctest -V --build-config Release --timeout 120 --output-on-failure -T Test
94+
95+
windows:
96+
runs-on: windows-latest
97+
98+
steps:
99+
- name: Checkout
100+
uses: actions/checkout@v4
101+
102+
- name: CMake config
103+
run: |
104+
cmake -B ${{github.workspace}}/build --preset release
105+
106+
- name: CMake build
107+
working-directory: ${{github.workspace}}/build
108+
run: cmake --build . --config Release
109+
110+
- name: Use Python 3.11
111+
uses: actions/setup-python@v4
112+
with:
113+
python-version: "3.11"
114+
115+
- name: Generate large NPZ files
116+
working-directory: ${{github.workspace}}/
117+
run: |
118+
pip install numpy
119+
python ${{github.workspace}}/test/generate_large_test.py
120+
121+
- name: CMake test
122+
working-directory: ${{github.workspace}}/build
123+
run: ctest -V --build-config Release --timeout 120 --output-on-failure -T Test
124+
125+
macos:
126+
runs-on: macos-latest
127+
128+
steps:
129+
- name: Checkout
130+
uses: actions/checkout@v4
131+
132+
- name: Get dependencies
133+
run: |
134+
brew update && brew install ninja
135+
136+
- name: CMake config
137+
run: cmake -B ${{github.workspace}}/build --preset release-clang
138+
139+
- name: CMake build
140+
working-directory: ${{github.workspace}}/build
141+
run: ninja
142+
143+
- name: Use Python 3.11
144+
uses: actions/setup-python@v4
145+
with:
146+
python-version: "3.11"
147+
148+
- name: Generate large NPZ files
149+
working-directory: ${{github.workspace}}/
150+
run: |
151+
pip install numpy
152+
python ${{github.workspace}}/test/generate_large_test.py
153+
154+
- name: CMake test
155+
working-directory: ${{github.workspace}}/build
156+
run: ctest -V --build-config Release --timeout 120 --output-on-failure -T Test

‎.gitignore

+4-1
Original file line numberDiff line numberDiff line change
@@ -33,4 +33,7 @@
3333

3434
# Misc
3535
.vscode
36-
build*
36+
build*
37+
assets/test/*_large*.npz
38+
.cache
39+
.env

‎.travis.yml

-22
This file was deleted.

‎CHANGELOG.md

+13
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,18 @@
11
# Changelog
22

3+
## [2024-11-01 - Version 1.5.3](https://github.com/matajoh/libnpy/releases/tag/v1.5.3)
4+
5+
Improvements:
6+
- Increased CHUNK size as per miniz instructions
7+
- Added tests for very large arrays in NPZ files
8+
- Added some CI tests to catch issues across platforms
9+
- Removed the internal IO streams in favor of just using stringstream
10+
- NPZs can now be read from and written to memory
11+
12+
Bugfixes:
13+
- Fixed an issue where very large arrays in NPZ files would throw an error
14+
- Fixed a bug with mac builds due to deprecated APIs
15+
316
## [2021-10-05 - Version 1.5.2](https://github.com/matajoh/libnpy/releases/tag/v1.5.2)
417

518
Removing `using namespace std` to simplify library use

0 commit comments

Comments
 (0)