Skip to content

Commit 0cb451f

Browse files
LELLIS87LELLIS87
authored
Add files via upload
1 parent 050b37e commit 0cb451f

38 files changed

+12299
-0
lines changed

‎.gitignore‎

Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
2+
# Avoid committing any data. If you need to make an exception for one file you can do that separately
3+
*.csv
4+
*.xlsx
5+
*.xls
6+
7+
# Linting files:
8+
.pylint.d/
9+
10+
# Data:
11+
*.csv
12+
*.sas7bdat
13+
14+
# Notebooks:
15+
*.ipynb
16+
*.ipynb_checkpoints
17+
18+
# Working file
19+
working.py
20+
temp.py
21+
22+
# Casched files
23+
cached_dataframes/
24+
*.ft
25+
26+
# System generated
27+
desktop.ini
28+
29+
# Cached files
30+
*.ft
31+
cached_dataframes/
32+
33+
####################################################################
34+
# GENERIC GITIGNORE FILE FOR PYTHON
35+
# Copied from https://github.com/github/gitignore/blob/master/Python.gitignore
36+
# This is a useful thing to just copy and paste since it should work for most use cases.
37+
# If you have additional things to exclude from the .gitignore file you should add it above
38+
# this section.
39+
#####################################################################
40+
41+
# Byte-compiled / optimized / DLL files
42+
__pycache__/
43+
*.py[cod]
44+
*$py.class
45+
46+
# C extensions
47+
*.so
48+
49+
# Distribution / packaging
50+
.Python
51+
build/
52+
develop-eggs/
53+
dist/
54+
downloads/
55+
eggs/
56+
.eggs/
57+
lib/
58+
lib64/
59+
parts/
60+
sdist/
61+
var/
62+
wheels/
63+
share/python-wheels/
64+
*.egg-info/
65+
.installed.cfg
66+
*.egg
67+
MANIFEST
68+
69+
# PyInstaller
70+
# Usually these files are written by a python script from a template
71+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
72+
*.manifest
73+
*.spec
74+
75+
# Installer logs
76+
pip-log.txt
77+
pip-delete-this-directory.txt
78+
79+
# Unit test / coverage reports
80+
htmlcov/
81+
.tox/
82+
.nox/
83+
.coverage
84+
.coverage.*
85+
.cache
86+
nosetests.xml
87+
coverage.xml
88+
*.cover
89+
*.py,cover
90+
.hypothesis/
91+
.pytest_cache/
92+
cover/
93+
94+
# Translations
95+
*.mo
96+
*.pot
97+
98+
# Django stuff:
99+
*.log
100+
local_settings.py
101+
db.sqlite3
102+
db.sqlite3-journal
103+
104+
# Flask stuff:
105+
instance/
106+
.webassets-cache
107+
108+
# Scrapy stuff:
109+
.scrapy
110+
111+
# Sphinx documentation
112+
docs/_build/
113+
114+
# PyBuilder
115+
.pybuilder/
116+
target/
117+
118+
# Jupyter Notebook
119+
.ipynb_checkpoints
120+
121+
# IPython
122+
profile_default/
123+
ipython_config.py
124+
125+
# pyenv
126+
# For a library or package, you might want to ignore these files since the code is
127+
# intended to run in multiple environments; otherwise, check them in:
128+
# .python-version
129+
130+
# pipenv
131+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
132+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
133+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
134+
# install all needed dependencies.
135+
#Pipfile.lock
136+
137+
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
138+
__pypackages__/
139+
140+
# Celery stuff
141+
celerybeat-schedule
142+
celerybeat.pid
143+
144+
# SageMath parsed files
145+
*.sage.py
146+
147+
# Environments
148+
.env
149+
.venv
150+
env/
151+
venv/
152+
ENV/
153+
env.bak/
154+
venv.bak/
155+
156+
# Spyder project settings
157+
.spyderproject
158+
.spyproject
159+
160+
# Rope project settings
161+
.ropeproject
162+
163+
# mkdocs documentation
164+
/site
165+
166+
# mypy
167+
.mypy_cache/
168+
.dmypy.json
169+
dmypy.json
170+
171+
# Pyre type checker
172+
.pyre/
173+
174+
# pytype static type analyzer
175+
.pytype/
176+
177+
# Cython debug symbols
178+
cython_debug/

‎README.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
Warning - this repository is a snapshot of a repository internal to NHS England. This means that links to videos and some URLs may not work.***
2+
3+
Repository owner: Analytical Services: Population Health, Clinical Audit and Specialist Care
4+
5+
Email: lifestyles@nhs.net
6+
7+
To contact us raise an issue on Github or via email and we will respond promptly.
8+
9+
# Background
10+
11+
This project produces the required publication outputs for the Sexual and
12+
Reproductive Health Services (Contraception) publication: Data tables, charts,
13+
and map data.
14+
15+
Data is sourced from the Sexual and Reproductive Health Activity Dataset (SRHAD),
16+
NHS hospital admissions data, NHS Business Services Authority (NHSBA)
17+
prescription cost analysis data, and the NHS corporate reference datasets.
18+
19+
# Initial package set up
20+
21+
Set up is done using the requirements.txt file
22+
23+
Run the following command in Terminal to set up the package
24+
Note that before running the below, it is advised to delete the folder with currently
25+
stored user packages: C:\Users\YOUR_SHORTCODE\AppData\Roaming\Python\Python39
26+
This will reset your packages to the default install versions before updating
27+
for this project.
28+
```
29+
pip install --user --no-warn-script-location -r requirements.txt
30+
```
31+
32+
33+
# Directory structure:
34+
```
35+
srh-services-rap
36+
│ README.md
37+
│ requirements.txt - Used to install the python dependencies
38+
│
39+
├───srh_code - This is the main code directory for this project
40+
│ │ create_publication.py - This script runs the entire publication
41+
│ │ parameters.py - Contains parameters that define the how the publication will run
42+
│ │
43+
│ └───sql_code - This folder contains all the SQL queries used in the import data stage
44+
│ │ query_ahas.sql
45+
│ │ query_asset_reporting.sql
46+
│ │ query_asset.sql
47+
│ │ query_imd_decile.sql
48+
│ │ query_imd_lsoa.sql
49+
│ │ query_la_ref.sql
50+
│ │ query_lsoa_ref.sql
51+
│ │ query_org_daily.sql
52+
│ │ query_org_sites.sql
53+
│ │ query_population.sql
54+
│ │
55+
│ utilities - This folder contains all the main modules used to create the publication
56+
│ │ charts.py - Defines the arguments needed to create and export chart outputs
57+
│ │ data_connections.py - Defines the df_from_sql function, used when importing SQL data
58+
│ │ field_definitions.py - Defines any derived fields added during processing.
59+
│ │ filter_definitions.py - Defines pe-set pipeline filters.
60+
│ │ helpers.py - Contains generalised functions used within the project
61+
│ │ load.py - Contains functions for reading in the required data
62+
│ │ logger_config.py - The configuration functions for the publication logger
63+
│ │ pre-processing.py - Contains the core pre-processing functions
64+
│ │ publication_files.py - Contains functions used to create publication ready outputs and save in relevant folders
65+
│ │ tables.py - Defines the arguments needed to create and export Excel table outputs
66+
│ │
67+
│ └───processing
68+
│ │ processing_publication.py - Contains the core functions used to produce publication outputs
69+
│ write - This folder contains all the main modules used to write the outputs to external files
70+
│ write_data.py - Contains functions for writing in the data to external files
71+
│ write_format.py - Contains functions for formatting the external files
72+
└───tests
73+
└────unittests - Unit tests for Python functions
74+
│ test_field_definitions.py
75+
│ test_filter_definitions.py
76+
│ test_helpers.py
77+
│ test_pre_processing.py
78+
│ test_processing_publication.py
79+
80+
```
81+
# Running the pipeline:
82+
83+
There are two main files that users running the process will need to interact with:
84+
85+
* The `parameters.py`
86+
* The `create_publication.py`
87+
88+
The file parameters.py contains all of the things that we expect to change from one publication
89+
to the next. Indeed, if the methodology has not changed, then this should be the only file you need
90+
to modify. A few elements require updating each year (e.g. the reporting year), but most
91+
are likely to only require occassional updates (e.g. file paths, default codes).
92+
It also allows the user to control which parts of the publication they want the pipeline to produce.
93+
94+
The publication process is run using the top-level script, create_publication.py.
95+
This script imports and runs all the required functions from the sub-modules.
96+
97+
# Link to publication
98+
https://digital.nhs.uk/data-and-information/publications/statistical/sexual-and-reproductive-health-services
99+
100+
# Licence
101+
The NHS England Sexual and Reproductive Health Services (Contraception) National Statistics publication codebase is released under the MIT License.
102+
103+
Copyright © 2023, NHS England
104+
105+
You may re-use this document/publication (not including logos) free of charge in any format or medium, under the terms of the Open Government Licence v3.0. Information Policy Team, The National Archives, Kew, Richmond, Surrey, TW9 4DU; email: psi@nationalarchives.gsi.gov.uk

‎requirements.txt‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
2+
-e .
3+
4+
# Libraries used for running the SRH services pipeline
5+
# Package versions frozen as of 13DEC2022
6+
# Based on Python version below
7+
# Python version = 3.9.12
8+
9+
# Data manipulation
10+
numpy==1.21.5
11+
pandas==1.4.2
12+
sidetable==0.9.0
13+
14+
# Excel output
15+
xlwings==0.24.9
16+
openpyxl==3.0.09
17+
pywin32==303
18+
XlsxWriter==3.0.3
19+
20+
# Word outputs (if needed)
21+
# python-docx==0.8.11
22+
# docx-mailmerge==0.5.0
23+
24+
# SQL
25+
sqlalchemy==1.4.32
26+
pyodbc==4.0.32
27+
28+
# Testing
29+
pytest==7.1.3
30+
pytest-html==3.1.1
31+
32+
# Additional dependencies of the above packages
33+
importlib-resources==5.4.0
34+
pathlib==1.0.1
35+
simplegeneric==0.8.1
36+
tzlocal==4.1
37+
pyarrow==10.0.1

‎srh_code/__init__.py‎

Whitespace-only changes.

0 commit comments

Comments
 (0)