Skip to content

Create a GitHub workflow for checking the add_primary_keys and create_views scripts #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
frankslin merged 9 commits into from
Oct 31, 2025
Merged

Create a GitHub workflow for checking the add_primary_keys and create_views scripts #10

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
5346bb8
Add GitHub workflow to validate the DB processing scripts.
frankslin Oct 31, 2025
86a930f
Update workflow to run on all branches
frankslin Oct 31, 2025
d82ef2b
Fix LFS file checkout in workflow
frankslin Oct 31, 2025
a566d86
Update the sanity check logic to query one view at a time
frankslin Oct 31, 2025
c304c72
Disable mmap_size when querying the views
frankslin Oct 31, 2025
6598492
Upload the db even upon failure
frankslin Oct 31, 2025
e419185
Use official SQLite 3.50.4 to fix Ubuntu's segfault bug
frankslin Oct 31, 2025
7468e10
Fix SQLite file path after extraction
frankslin Oct 31, 2025
1fff849
Fix deprecated sqlite3.version attribute check
frankslin Oct 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update the sanity check logic to query one view at a time
  • Loading branch information
@frankslin
frankslin committed Oct 31, 2025
commit a566d863210d37f6e8a1b81b6ecf1de7e29a31f0
58 changes: 38 additions & 20 deletions scripts/create_views.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -995,23 +995,41 @@ echo "Finished view View_StatusData."
echo "Created CBDB views in '$DB_PATH'."

echo "Running sanity counts on views..."
sqlite3 "$DB_PATH" <<'SQL'
SELECT 'View_AltnameData' AS view_name, COUNT(*) AS row_count FROM View_AltnameData;
SELECT 'View_Association' AS view_name, COUNT(*) AS row_count FROM View_Association;
SELECT 'View_BiogAddrData' AS view_name, COUNT(*) AS row_count FROM View_BiogAddrData;
SELECT 'View_BiogInstAddrData' AS view_name, COUNT(*) AS row_count FROM View_BiogInstAddrData;
SELECT 'View_BiogInstData' AS view_name, COUNT(*) AS row_count FROM View_BiogInstData;
SELECT 'View_BiogSourceData' AS view_name, COUNT(*) AS row_count FROM View_BiogSourceData;
SELECT 'View_BiogTextData' AS view_name, COUNT(*) AS row_count FROM View_BiogTextData;
SELECT 'View_Entry' AS view_name, COUNT(*) AS row_count FROM View_Entry;
SELECT 'View_EventAddr' AS view_name, COUNT(*) AS row_count FROM View_EventAddr;
SELECT 'View_EventData' AS view_name, COUNT(*) AS row_count FROM View_EventData;
SELECT 'View_KinAddr' AS view_name, COUNT(*) AS row_count FROM View_KinAddr;
SELECT 'View_People' AS view_name, COUNT(*) AS row_count FROM View_People;
SELECT 'View_PeopleAddr' AS view_name, COUNT(*) AS row_count FROM View_PeopleAddr;
SELECT 'View_Possessions' AS view_name, COUNT(*) AS row_count FROM View_Possessions;
SELECT 'View_PossessionsAddr' AS view_name, COUNT(*) AS row_count FROM View_PossessionsAddr;
SELECT 'View_PostingAddr' AS view_name, COUNT(*) AS row_count FROM View_PostingAddr;
SELECT 'View_PostingOffice' AS view_name, COUNT(*) AS row_count FROM View_PostingOffice;
SELECT 'View_StatusData' AS view_name, COUNT(*) AS row_count FROM View_StatusData;
SQL

# List of views to check
VIEWS=(
"View_AltnameData"
"View_Association"
"View_BiogAddrData"
"View_BiogInstAddrData"
"View_BiogInstData"
"View_BiogSourceData"
"View_BiogTextData"
"View_Entry"
"View_EventAddr"
"View_EventData"
"View_KinAddr"
"View_People"
"View_PeopleAddr"
"View_Possessions"
"View_PossessionsAddr"
"View_PostingAddr"
"View_PostingOffice"
"View_StatusData"
)

# Check each view individually to identify which one causes issues
for view in "${VIEWS[@]}"; do
echo "Checking view: $view..."
if sqlite3 "$DB_PATH" "SELECT '$view' AS view_name, COUNT(*) AS row_count FROM $view;" 2>&1; then
echo " ✓ $view completed successfully"
else
EXIT_CODE=$?
echo " ✗ ERROR: $view failed with exit code $EXIT_CODE"
echo " Memory info:"
free -h 2>/dev/null || true
exit $EXIT_CODE
fi
done

echo "All sanity checks passed!"