code/Lab4_UI_part1.ipynb

{"cells":[{"cell_type":"markdown","source":["** See congested Bus Trips! **\n\nThe app predicts congestion in bus trips, uploads the result to our Elastic DB, and displays the journeys (journeyPatternId) with congestion on the map below.\nMissing data and intregration of external info is done automatically!\n\nAll you need to do is specifiy the data path (or IP if streaming), select the filters you want, and press the update button! \n\n*More detailed explanation below."],"metadata":{}},{"cell_type":"code","source":["from IPython.display import Image\nImage(url=\"https://drive.google.com/uc?id=1S13K-zCiK2nLgY9pdSSjgntxyOl-nm-x\",  width=150)"],"metadata":{},"outputs":[{"output_type":"display_data","metadata":{},"output_type":"display_data","data":{"text/html":["<img src=\"https://drive.google.com/uc?id=1S13K-zCiK2nLgY9pdSSjgntxyOl-nm-x\" width=\"150\"/>"]}}],"execution_count":2},{"cell_type":"code","source":["!pip install libify\nimport libify\nlab4_functions = libify.importer(globals(), '/Users/shubi@campus.technion.ac.il/Lab4_functions')\n\ndbutils.widgets.dropdown(\"Is Weekend?\", \"True\", ['True', 'False'])\ndbutils.widgets.dropdown(\"Passes a Popular Location?\", \"True\", ['True', 'False'])\ndbutils.widgets.dropdown(\"Is Event Date?\", \"True\", ['True', 'False'])\ndbutils.widgets.dropdown(\"Part of Day (Hours)\", \"4-7\", ['0-3',  '4-7', '8-11', '12-15', '16-19', '20-23'])\ndbutils.widgets.dropdown(\"Stream?\", \"False\", ['True', 'False'])\ndbutils.widgets.text(\"IP / Path\", '')\n\nisWeekend = eval(dbutils.widgets.get(\"Is Weekend?\"))\nisPassedLocation = eval(dbutils.widgets.get(\"Passes a Popular Location?\"))\nisEventDate = eval(dbutils.widgets.get(\"Is Event Date?\"))\nhourGroup = dbutils.widgets.get(\"Part of Day (Hours)\")\nisStream = eval(dbutils.widgets.get(\"Stream?\"))\npath_ip = dbutils.widgets.get(\"IP / Path\")\n\nparams={\"isWeekend\":isWeekend, \"isEventDate\":isEventDate, \"isPassedLocation\":isPassedLocation, \"hourGroup\":hourGroup}\n# print(params, isStream, path_ip) # todo delete\n# Our check\n# 10.0.0.30:9091\n# /mnt/dacoursedatabricksstg/dacoursedatabricksdata/busFile \ntest_predictions = lab4_functions.raw_data_to_predictions(streaming=isStream, USE_TAKE=False, min_date=None, max_date=None, filter_params=params, path_ip=path_ip)\n\nif test_predictions == 'problem':\n    print('\\n\\n\\n\\n')\n    print('Unexpected error in reading file.\\n'\n          'Please make sure the file exists, ' \n          'is of the correct format(json) and with the right schema.')\nelse:\n    journey_patterns = test_predictions.where(test_predictions.prediction==1).select('journeyPatternId')\n    lab4_functions.write_to_elastic(journey_patterns, 'journey-patterns-with-congestion', lab4_functions.elastic_settings_mappings_basic, overwrite=True, streaming=isStream)\n\ndisplayHTML(\"\"\"<iframe src=\"https://da2020w-0001.eastus.cloudapp.azure.com:5601/goto/b956cced0cd51ab36979d7f2852d2f93\" height=\"450\" width=\"1024\"></iframe>\"\"\")"],"metadata":{},"outputs":[{"output_type":"display_data","metadata":{},"output_type":"display_data","data":{"text/html":["<iframe src=\"https://da2020w-0001.eastus.cloudapp.azure.com:5601/goto/b956cced0cd51ab36979d7f2852d2f93\" height=\"450\" width=\"1024\"></iframe>"]}}],"execution_count":3},{"cell_type":"markdown","source":["* Choose the data to run on by adding ip number if it is a streaming data, and a file path if it is a batch data. and select the right value in the 'Stram?'' bullet. \n* Select what to filter the bus trips on using the following bullets:\n  * Is Event Date? - if True predict congestion on bus trips accured in a day with big event (e.g. day with football game).\n  * Passes a Popular Location? - if True predict congestion on bus trips passed near popular location\\locations (like Temple Bar, Dublin, Ireland).\n  * Is Weekend? - if True predict congestion on bus trips accured in the weekend.\n  * Part of Day - predict congestion on bus trips accured in the chosen hours of the day.\n* Run by pressing on the \"update\" button that appears at the top right of the app. (It might pop-up a massage, please press 'yes')\n* -> The map automaticaly will be updated! On the map you will see plots - each point color represent some Journy Pattern Id, and by hoover on the point you can see the Journy Pattern number.\n* Notes: \n  1. If the map is empty there are no Journy Pattern Ids with congestion with the chosen filters or you might entered a wrong ip number.\n  2. The running might take a while.\n  3. If you see a pop-up massage \"Run beyond time out\" press it to continue.\n  4. if you are getting error \"Failed to construct kafka consumer\" - you might entered a wrong ip number."],"metadata":{}},{"cell_type":"code","source":[""],"metadata":{},"outputs":[],"execution_count":5}],"metadata":{"name":"Lab4_UI_part1","notebookId":1325942436209506},"nbformat":4,"nbformat_minor":0}