Skip to content

Commit 0fed409

Browse files
Update Readme.md
1 parent 828a0ac commit 0fed409

File tree

1 file changed

+87
-1
lines changed

1 file changed

+87
-1
lines changed

README.md

Lines changed: 87 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,88 @@
11
# Weather-data-visualization
2-
Interactive SQL queries on BigQuery with Streamlit
2+
Interactive SQL queries on BigQuery with Streamlit. This project demonstrates running SQL queries on [BigQuery](https://cloud.google.com/bigquery/) public datasets and rendering results into an interactive [Streamlit](https://streamlit.io) app.
3+
4+
Live version: http://34.76.236.119/
5+
6+
![alt text](record.gif)
7+
8+
# Architecture
9+
![alt text](architecture.jpg)
10+
11+
12+
# Running on local machine
13+
14+
To run the application, build the docker image and run it using the following commands
15+
16+
```
17+
docker build -t app .
18+
```
19+
20+
```
21+
docker run --rm -p 8080:8080 app:latest
22+
```
23+
24+
The application will be available on http://0.0.0.0:8080.
25+
26+
**Note:**
27+
In order to use BigQuery API in python, we need to create a credential file (JSON file) and declare its path as an environment variable `GOOGLE_APPLICATION_CREDENTIALS` (by default defined in `modules/data_query.py`).
28+
To create the credential file follow steps explained here: https://cloud.google.com/bigquery/docs/quickstarts/quickstart-client-libraries
29+
30+
31+
# Running on cloud
32+
33+
- Build image and push it to a PRIVATE Container Registry (don't forget about your credentials) .
34+
35+
From a GCP project, we will use Google Shell to build the image and push it to GCR (container registry).
36+
37+
```
38+
# Get name of project
39+
# For illustration we will call it PROJECT-ID
40+
41+
gcloud config get-value project
42+
```
43+
44+
Create the following Dockerfile in your CloudShell session .
45+
46+
```
47+
FROM ahmedbendebba/weather-visualisation:latest
48+
49+
# Google Run uses env variable PORT
50+
51+
CMD gunicorn --bind :$PORT wsgi:app
52+
```
53+
54+
Finally, we can build and submit the image to GCR.
55+
56+
```
57+
gcloud builds submit --tag gcr.io/PROJECT-ID/weather-visualisation
58+
```
59+
60+
- Create an instance on Google Compute Engine
61+
-
62+
- Deploy on Google Compute Engine by running this image gcr.io/PROJECT-ID/weather-visualisation
63+
64+
```
65+
export GCR_TAG=gcr.io/PROJECT-ID/weather-visualization1
66+
67+
docker run -p 80:8080 -e PORT=8080 $GCR_TAG
68+
```
69+
70+
# Code
71+
72+
- `app.py` : Streamlit script to define the UI.
73+
74+
Modules:
75+
76+
- `modules/data_query.py`: Define Data fetcher from BigQuery (takes an SQL command and return results in Pandas DataFrame).
77+
78+
- `modules/sql_queries.py`: Pre-defined parametrized SQL commands.
79+
80+
- `modules/utils.py`: Contains a a function to compute size of data anluzed by a given query.
81+
82+
- `modules/plot.py`: Plot functions.
83+
84+
85+
---
86+
By Ahmed
87+
88+

0 commit comments

Comments
 (0)