diff --git a/Dockerfile b/Dockerfile index be5f60a65..29970d83f 100755 --- a/Dockerfile +++ b/Dockerfile @@ -22,12 +22,13 @@ USER developer WORKDIR $HOME -RUN pip install --user flask-restful==0.3.9 pandas==0.24.2 flask_cors==3.0.10 +RUN pip install --user flask-restful==0.3.9 pandas==0.24.2 flask_cors==3.0.10 requests==2.27.1 RUN mkdir models && \ mkdir doc ENV PYTHONPATH $PYTHONPATH:$HOME +ENV BOPTEST_DASHBOARD_SERVER https://api.boptest.net:8081/ CMD python restapi.py && bash diff --git a/README.md b/README.md index 0e4345cd3..03f31eeb0 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,9 @@ Building Optimization Performance Tests This repository contains code for the Building Optimization Performance Test framework (BOPTEST) -that is being developed as part of the IBPSA Project 1 (https://ibpsa.github.io/project1/). +that is being developed as part of the [IBPSA Project 1](https://ibpsa.github.io/project1/). + +Visit the [BOPTEST Home Page](https://ibpsa.github.io/project1-boptest/) for more information about the project, software, and documentation. ## Structure - ``/testcases`` contains test cases, including docs, models, and configuration settings. @@ -21,11 +23,11 @@ that is being developed as part of the IBPSA Project 1 (https://ibpsa.github.io/ 1) Download this repository. 2) Install [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/). 3) Build and deploy a test case using the following commands executed in the root directory of this repository and where is the name of the test case subdirectory located in [/testcases](https://github.com/ibpsa/project1-boptest/tree/master/testcases): - - * Linux or macOS: ``$ TESTCASE= docker-compose up`` + + * Linux or macOS: ``$ TESTCASE= docker-compose up`` * Windows PowerShell: ``> ($env:TESTCASE="") -and (docker-compose up)`` * A couple notes: - * The first time this command is run, the image ``boptest_base`` will be built. This takes about a minute. Subsequent usage will use the already-built image and deploy much faster. + * The first time this command is run, the image ``boptest_base`` will be built. This takes about a minute. Subsequent usage will use the already-built image and deploy much faster. * If you update your BOPTEST repository, use the command ``docker rmi boptest_base`` to remove the image so it can be re-built with the updated repository upon next deployment. * ``TESTCASE`` is simply an environment variable. Consistent with use of docker-compose, you may also edit the value of this variable in the ``.env`` file and then use ``docker-compose up``. @@ -52,6 +54,7 @@ that is being developed as part of the IBPSA Project 1 (https://ibpsa.github.io/ ## Test Case RESTful API - To interact with a deployed test case, use the API defined in the table below by sending RESTful requests to: ``http://127.0.0.1:5000/`` +- The API will return a JSON in the form ``{"status":, "message":, "payload":}``. Status codes in ``"status"`` are integers: ``200`` for successful with or without warning, ``400`` for bad input error, or ``500`` for internal error. Data returned in ``"payload"`` is the data of interest relvant to the specific API request, while the string in ``"message"`` will report any warnings or error messages to help debug encountered problems. Example RESTful interaction: @@ -76,6 +79,8 @@ Example RESTful interaction: | Receive current test scenario. | GET ``scenario`` | | Set test scenario. Setting the argument ``time_period`` performs an initialization with predefined start time and warmup period and will only simulate for predefined duration. | PUT ``scenario`` with optional arguments ``electricity_price=``, ``time_period=``. See README in [/testcases](https://github.com/ibpsa/project1-boptest/tree/master/testcases) for options and test case documentation for details.| | Receive BOPTEST version. | GET ``version`` | +| Submit KPIs, other test information, and optional string tags (up to 10) to online dashboard. Requires a formal test scenario to be completed, initialized using the PUT ``scenario`` API. | POST ``submit`` with required argument ``api_key=`` and optional arguments ``tag#=`` where # is an integer between 1 and 10. The API key can be obtained from the user account registered wth the online dashboard at (url coming soon).| + ## Development This repository uses pre-commit to ensure that the files meet standard formatting conventions (such as line spacing, layout, etc). @@ -106,7 +111,7 @@ D. Blum, J. Arroyo, S. Huang, J. Drgona, F. Jorissen, H.T. Walnum, Y. Chen, K. B J. Arroyo, F. Spiessens, and L. Helsen. (2022). ["Comparison of Optimal Control Techniques for Building Energy Management."](https://doi.org/10.3389/fbuil.2022.849754) *Frontiers in Built Environment* 8. T. Marzullo, S. Dey, N. Long, J. L. Vilaplana, and G. Henze. (2022). ["A high-fidelity building performance simulation test bed for the development and evaluation of advanced controls"](https://doi.org/10.1080/19401493.2022.2058091) *Journal of Building Performance Simulation*, 15(3), 379-397. - + J. Arroyo, C. Manna, F. Spiessens, and L. Helsen. (2022). ["Reinforced model predictive control (RL-MPC) for building energy management."](https://doi.org/10.1016/j.apenergy.2021.118346) *Applied Energy* 309: 118346. J. Arroyo, C. Manna, F. Spiessens, and L. Helsen. (2021). [“An OpenAI-Gym Environment for the Building Optimization Testing (BOPTEST) Framework.”](https://www.researchgate.net/profile/Javier-Arroyo/publication/354386346_An_OpenAI-Gym_environment_for_the_Building_Optimization_Testing_BOPTEST_framework/links/613616690360302a0082ffc1/An-OpenAI-Gym-environment-for-the-Building-Optimization-Testing-BOPTEST-framework.pdf) In *Proceedings of the 17th IBPSA Conference*, Sep 1 - 3. Bruges, Belgium. diff --git a/examples/javascript/testcase1.html b/examples/javascript/testcase1.html index 4ca1e5957..9d49c85d3 100644 --- a/examples/javascript/testcase1.html +++ b/examples/javascript/testcase1.html @@ -128,13 +128,14 @@ } else { console.log( "SUCCESS in getting results" ); - result.value = xhr_get_result.response; + json = JSON.parse(xhr_get_result.responseText)["payload"]; + result.value = JSON.stringify(json); } } }; - var results = {}; var results_arg='{"point_name":"TRooAir_y","start_time":0,"final_time":'+length.toString()+'}'; + console.log(results_arg) xhr_get_result.send(results_arg); xhr_get_kpi = new XMLHttpRequest(); @@ -146,7 +147,8 @@ } else { console.log( "SUCCESS in getting kpi" ); - kpi.value = xhr_get_kpi.response; + json = JSON.parse(xhr_get_kpi.responseText)["payload"]; + kpi.value = JSON.stringify(json); } } }; @@ -202,8 +204,9 @@ xhr.setRequestHeader("Content-type", "application/json"); xhr.onreadystatechange = function () { if (xhr.readyState == 4 && xhr.status == 200) { - output.value = xhr.response; - json = JSON.parse(xhr.responseText); + json = JSON.parse(xhr.responseText)["payload"]; + console.log(json) + output.value = JSON.stringify(json); if (count ==1) { for (key in json) { @@ -231,8 +234,6 @@ point = e.target.value; }) } - - xhr.send(data); } }; diff --git a/examples/javascript/testcase2.html b/examples/javascript/testcase2.html index 60d00a363..ad39594f6 100644 --- a/examples/javascript/testcase2.html +++ b/examples/javascript/testcase2.html @@ -129,7 +129,8 @@ } else { console.log( "SUCCESS in getting results" ); - result.value = xhr_get_result.response; + json = JSON.parse(xhr_get_result.responseText)["payload"]; + result.value = JSON.stringify(json); } } }; @@ -147,7 +148,8 @@ } else { console.log( "SUCCESS in getting kpi" ); - kpi.value = xhr_get_kpi.response; + json = JSON.parse(xhr_get_kpi.responseText)["payload"]; + kpi.value = JSON.stringify(json); } } }; @@ -188,7 +190,7 @@ data = '{"oveTSetRooHea_u":295.15,"oveTSetRooHea_activate":1, "oveTSetRooCoo_u":296.15,"oveTSetRooCoo_activate":1}' }; - console.log(data); + xhr = new XMLHttpRequest(); @@ -196,8 +198,9 @@ xhr.setRequestHeader("Content-type", "application/json"); xhr.onreadystatechange = function () { if (xhr.readyState == 4 && xhr.status == 200) { - output.value = xhr.response; - json = JSON.parse(xhr.responseText); + json = JSON.parse(xhr.responseText)["payload"]; + console.log(json) + output.value = JSON.stringify(json); if (count ==1) { for (key in json) { diff --git a/examples/julia/testcase1.jl b/examples/julia/testcase1.jl index f000f99d1..0c6d13b30 100644 --- a/examples/julia/testcase1.jl +++ b/examples/julia/testcase1.jl @@ -6,7 +6,7 @@ # GENERAL PACKAGE IMPORT # ---------------------- -using HTTP, JSON, CSV, DataFrames +using HTTP, JSON, CSV, DataFrames, Dates # TEST CONTROLLER IMPORT # ---------------------- @@ -25,30 +25,55 @@ step = 300 # -------------------- println("TEST CASE INFORMATION ------------- \n") # Test case name -name = JSON.parse(String(HTTP.get("$url/name").body)) -println("Name:\t\t\t$name") + +res = HTTP.get("$url/name") +name_status = res.status +name = JSON.parse(String(res.body))["payload"] +if name_status == 200 + println("Name:\t\t\t$name['name']") +end # Inputs available -inputs = JSON.parse(String(HTTP.get("$url/inputs").body)) -println("Control Inputs:\t\t\t$inputs") +res = HTTP.get("$url/inputs") +inputs_status = res.status +inputs = JSON.parse(String(res.body))["payload"] +if inputs_status == 200 + println("Control Inputs:\t\t\t$inputs") +end # Measurements available -measurements = JSON.parse(String(HTTP.get("$url/measurements").body)) -println("Measurements:\t\t\t$measurements") +res = HTTP.get("$url/measurements") +measurements_status = res.status +measurements = JSON.parse(String(res.body))["payload"] +if measurements_status == 200 + println("Measurements:\t\t\t$measurements") +end + # Default simulation step -step_def = JSON.parse(String(HTTP.get("$url/step").body)) -println("Default Simulation Step:\t$step_def") +res = HTTP.get("$url/measurements") +step_def_status = res.status +step_def = JSON.parse(String(res.body))["payload"] +if step_def_status == 200 + println("Default Simulation Step:\t$step_def") +end # RUN TEST CASE #---------- -println("Initializing test case simulation.") +start = Dates.now() +# Initialize test case simulation res = HTTP.put("$url/initialize",["Content-Type" => "application/json"], JSON.json(Dict("start_time" => 0,"warmup_period" => 0))) -initialize_result=JSON.parse(String(res.body)) -if !isnothing(initialize_result) +initialize_status = res.status +initialize_result=JSON.parse(String(res.body))["payload"] +if initialize_status == 200 println("Successfully initialized the simulation") end + # Set simulation step -println("Setting simulation step to $step") + res = HTTP.put("$url/step",["Content-Type" => "application/json"], JSON.json(Dict("step" => step))) +if res.status == 200 + println("Setting simulation step to $step") +end + println("Running test case ...") @@ -62,15 +87,21 @@ for i = 1:convert(Int, floor(length/step)) u = PID.compute_control(y) end # Advance in simulation - global y = JSON.parse(String(HTTP.post("$url/advance", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(u);retry_non_idempotent=true).body)) - + res = HTTP.post("$url/advance", ["Content-Type" => "application/json"], JSON.json(u);retry_non_idempotent=true) + global y = JSON.parse(String(res.body))["payload"] + if res.status == 200 + println("Successfully advanced the simulation") + end end println("Test case complete.") # VIEW RESULTS # ------------ # Report KPIs -kpi = JSON.parse(String(HTTP.get("$url/kpi").body)) +res = HTTP.get("$url/kpi") +if res.status == 200 + kpi = JSON.parse(String(res.body))["payload"] +end println("KPI RESULTS \n-----------") for key in keys(kpi) if isnothing(kpi[key]) @@ -84,16 +115,16 @@ end # POST PROCESS RESULTS # -------------------- # Get result data -res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "TRooAir_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body)) +res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "TRooAir_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body))["payload"] time = [x/3600 for x in res["time"]] # convert s --> hr TZone = [x-273.15 for x in res["TRooAir_y"]] # convert K --> C -res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "CO2RooAir_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body)) +res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "CO2RooAir_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body))["payload"] CO2Zone = [x for x in res["CO2RooAir_y"]] -res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "PHea_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body)) +res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "PHea_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body))["payload"] PHeat = res["PHea_y"] -res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "oveAct_u","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body)) +res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "oveAct_u","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body))["payload"] QHeat = res["oveAct_u"] -res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "oveAct_activate","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body)) +res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "oveAct_activate","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body))["payload"] uAct = res["oveAct_activate"] tab_res=DataFrame([time,TZone,CO2Zone,PHeat,QHeat,uAct],[:time,:TRooAir_y,:CO2RooAir_y,:PHea_y,:oveAct_u,:oveAct_activate]) CSV.write("result_testcase1.csv",tab_res) diff --git a/examples/julia/testcase2.jl b/examples/julia/testcase2.jl index c91c78a61..f79abeb84 100644 --- a/examples/julia/testcase2.jl +++ b/examples/julia/testcase2.jl @@ -24,34 +24,59 @@ step = 3600 # -------------------- println("TEST CASE INFORMATION ------------- \n") # Test case name -name = JSON.parse(String(HTTP.get("$url/name").body)) -println("Name:\t\t\t$name") + +res = HTTP.get("$url/name") +name_status = res.status +name = JSON.parse(String(res.body))["payload"] +if name_status == 200 + println("Name:\t\t\t$name['name']") +end # Inputs available -inputs = JSON.parse(String(HTTP.get("$url/inputs").body)) -println("Control Inputs:\t\t\t$inputs") +res = HTTP.get("$url/inputs") +inputs_status = res.status +inputs = JSON.parse(String(res.body))["payload"] +if inputs_status == 200 + println("Control Inputs:\t\t\t$inputs") +end # Measurements available -measurements = JSON.parse(String(HTTP.get("$url/measurements").body)) -println("Measurements:\t\t\t$measurements") +res = HTTP.get("$url/measurements") +measurements_status = res.status +measurements = JSON.parse(String(res.body))["payload"] +if measurements_status == 200 + println("Measurements:\t\t\t$measurements") +end + # Default simulation step -step_def = JSON.parse(String(HTTP.get("$url/step").body)) -println("Default Simulation Step:\t$step_def") +res = HTTP.get("$url/measurements") +step_def_status = res.status +step_def = JSON.parse(String(res.body))["payload"] +if step_def_status == 200 + println("Default Simulation Step:\t$step_def") +end # RUN TEST CASE #---------- start = Dates.now() # Initialize test case simulation res = HTTP.put("$url/initialize",["Content-Type" => "application/json"], JSON.json(Dict("start_time" => 0,"warmup_period" => 0))) -initialize_result=JSON.parse(String(res.body)) -if !isnothing(initialize_result) +initialize_status = res.status +initialize_result=JSON.parse(String(res.body))["payload"] +if initialize_status == 200 println("Successfully initialized the simulation") end + # Set simulation step -println("Setting simulation step to $step") + res = HTTP.put("$url/step",["Content-Type" => "application/json"], JSON.json(Dict("step" => step))) +if res.status == 200 + println("Setting simulation step to $step") +end + println("Running test case ...") + # simulation loop for i = 1:convert(Int, floor(length/step)) if i<2 @@ -62,18 +87,25 @@ for i = 1:convert(Int, floor(length/step)) u = sup.compute_control(y) end # Advance in simulation - res=HTTP.post("$url/advance", ["Content-Type" => "application/json"], JSON.json(u);retry_non_idempotent=true).body - global y = JSON.parse(String(res)) + res = HTTP.post("$url/advance", ["Content-Type" => "application/json"], JSON.json(u);retry_non_idempotent=true) + global y = JSON.parse(String(res.body))["payload"] + if res.status == 200 + println("Successfully advanced the simulation") + end end println("Test case complete.") time=(now()-start).value/1000. -println("Elapsed time of test was $time seconds.") +println("Elapsed time of test + was $time seconds.") # VIEW RESULTS # ------------ # Report KPIs -kpi = JSON.parse(String(HTTP.get("$url/kpi").body)) +res = HTTP.get("$url/kpi") +if res.status == 200 + kpi = JSON.parse(String(res.body))["payload"] +end println("KPI RESULTS \n-----------") for key in keys(kpi) if isnothing(kpi[key]) @@ -87,22 +119,22 @@ end # POST PROCESS RESULTS # -------------------- # Get result data -res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "TRooAir_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body)) +res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "TRooAir_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body))["payload"] time = [x/3600 for x in res["time"]] # convert s --> hr TRooAir = [x-273.15 for x in res["TRooAir_y"]] # convert K --> C -res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "CO2RooAir_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body)) +res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "CO2RooAir_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body))["payload"] CO2RooAir = [x for x in res["CO2RooAir_y"]] -res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "oveTSetRooHea_u","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body)) +res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "oveTSetRooHea_u","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body))["payload"] TSetRooHea = [x-273.15 for x in res["oveTSetRooHea_u"]] # convert K --> C -res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "oveTSetRooCoo_u","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body)) +res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "oveTSetRooCoo_u","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body))["payload"] TSetRooCoo = [x-273.15 for x in res["oveTSetRooCoo_u"]] # convert K --> C -res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "PFan_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body)) +res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "PFan_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body))["payload"] PFan = res["PFan_y"] -res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "PCoo_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body)) +res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "PCoo_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body))["payload"] PCoo = res["PCoo_y"] -res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "PHea_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body)) +res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "PHea_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body))["payload"] PHea = res["PHea_y"] -res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "PPum_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body)) +res = JSON.parse(String(HTTP.put("$url/results", ["Content-Type" => "application/json","connecttimeout"=>30.0], JSON.json(Dict("point_name" => "PPum_y","start_time" => 0, "final_time" => length));retry_non_idempotent=true).body))["payload"] PPum = res["PPum_y"] tab=DataFrame([time,TRooAir,CO2RooAir,TSetRooHea,TSetRooCoo,PFan,PCoo,PHea,PPum],[:time,:TRooAir,:CO2RooAir,:TSetRooHea,:TSetRooCoo,:PFan,:PCoo,:PHea,:PPum]) CSV.write("result_testcase2.csv",tab) diff --git a/examples/python/controllers/controller.py b/examples/python/controllers/controller.py index bcf95dad3..d5c010b85 100644 --- a/examples/python/controllers/controller.py +++ b/examples/python/controllers/controller.py @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- """ This module contains a generic controller class that is used to instantiate -concrete controller methods, found in pid.py, pidTwoZones.py, and sup.py. +concrete controller, found in pid.py, pidTwoZones.py, and sup.py. """ @@ -24,6 +24,7 @@ def __init__(self, module, use_forecast=False): """ try: + # instantiate the concrete controller specified in the configuration controller = importlib.import_module(module) except ModuleNotFoundError: print("Cannot find specified controller: {}".format(module)) diff --git a/examples/python/interface.py b/examples/python/interface.py index 6c8115dbb..fba5d1527 100644 --- a/examples/python/interface.py +++ b/examples/python/interface.py @@ -10,8 +10,10 @@ # GENERAL PACKAGE IMPORT # ---------------------- import requests +import sys import time import numpy as np +import requests from examples.python.custom_kpi.custom_kpi_calculator import CustomKPI from examples.python.controllers.controller import Controller import json @@ -19,8 +21,33 @@ import pandas as pd +def check_response(response): + """Check status code from restful API and return result if no error + + Parameters + ---------- + + response: obj, response object + + Returns + ------- + result : dict, result from call to restful API + + """ + if isinstance(response, requests.Response): + status = response.status_code + if status == 200: + response = response.json()['payload'] + return response + print("Unexpected error: {}".format(response.text)) + print("Exiting!") + sys.exit() + + def control_test(control_module='', start_time=0, warmup_period=0, length=24*3600, scenario=None, step=300, customized_kpi_config=None, use_forecast=False): - """Run test case. + """ + Main interface that executes communication between testcase (controller) and the restufl API communicating with + the model (FMU) running in docker. Parameters ---------- @@ -71,23 +98,23 @@ def control_test(control_module='', start_time=0, warmup_period=0, length=24*360 # ------------------------------------------------------------------------- # Set URL for testcase url = 'http://127.0.0.1:5000' - # Instantiate controller + # Instantiate concrete controller (pid, pidTwoZones, sup, etc.) controller = Controller(control_module, use_forecast) # GET TEST INFORMATION # ------------------------------------------------------------------------- print('\nTEST CASE INFORMATION\n---------------------') - # Test case name - name = requests.get('{0}/name'.format(url)).json() + # Retrieve testcase name from REST API + name = check_response(requests.get('{0}/name'.format(url))) print('Name:\t\t\t\t{0}'.format(name)) - # Inputs available - inputs = requests.get('{0}/inputs'.format(url)).json() + # Retrieve a list of inputs (controllable points) for the model from REST API + inputs = check_response(requests.get('{0}/inputs'.format(url))) print('Control Inputs:\t\t\t{0}'.format(inputs)) - # Measurements available - measurements = requests.get('{0}/measurements'.format(url)).json() + # Retrieve a list of measurements (outputs) for the model from REST API + measurements = check_response(requests.get('{0}/measurements'.format(url))) print('Measurements:\t\t\t{0}'.format(measurements)) - # Default control step - step_def = requests.get('{0}/step'.format(url)).json() + # Get the default simulation timestep for the model for simulation run + step_def = check_response(requests.get('{0}/step'.format(url))) print('Default Control Step:\t{0}'.format(step_def)) # IF ANY CUSTOM KPI CALCULATION, DEFINE STRUCTURES @@ -109,51 +136,54 @@ def control_test(control_module='', start_time=0, warmup_period=0, length=24*360 # Initialize test case print('Initializing test case simulation.') if scenario is not None: - # Intialize test with a scenario time period - res = requests.put('{0}/scenario'.format(url), data=scenario).json()['time_period'] + # Initialize test with a scenario time period + res = check_response(requests.put('{0}/scenario'.format(url), data=scenario))['time_period'] + print(res) # Record test simulation start time - start_time = res['time'] + start_time = int(res['time']) # Set final time and total time steps to be very large since scenario defines length final_time = np.inf total_time_steps = int((365 * 24 * 3600)/step) else: - # Intialize test with a specified start time and warmup period - res = requests.put('{0}/initialize'.format(url), data={'start_time': start_time, 'warmup_period': warmup_period}).json() - # Set final time and total time steps according to specified length + # Initialize test with a specified start time and warmup period + res = check_response(requests.put('{0}/initialize'.format(url), data={'start_time': start_time, 'warmup_period': warmup_period})) + print("RESULT: {}".format(res)) + # Set final time and total time steps according to specified length (seconds) final_time = start_time + length total_time_steps = int(length / step) # calculate number of timesteps if res: print('Successfully initialized the simulation') print('\nRunning test case...') - # Set control step - res = requests.put('{0}/step'.format(url), data={'step': step}) - # Initialize u from controller + # Set simulation time step + res = check_response(requests.put('{0}/step'.format(url), data={'step': step})) + # Initialize input to simulation from controller u = controller.initialize() # Initialize forecast storage structure forecasts = None + print(requests.get('{0}/scenario'.format(url)).json()) # Simulation Loop for t in range(total_time_steps): # Advance simulation with control input value(s) - y = requests.post('{0}/advance'.format(url), data=u).json() - # If reach end of scenario, stop + y = check_response(requests.post('{0}/advance'.format(url), data=u)) + # If simulation is complete break simulation loop if not y: break - # If custom KPIs, compute them + # If custom KPIs are configured, compute the KPIs for kpi in custom_kpis: kpi.processing_data(y) # Process data as needed for custom KPI custom_kpi_value = kpi.calculation() # Calculate custom KPI value custom_kpi_result[kpi.name].append(round(custom_kpi_value, 2)) # Track custom KPI value print('KPI:\t{0}:\t{1}'.format(kpi.name, round(custom_kpi_value, 2))) # Print custom KPI value custom_kpi_result['time'].append(y['time']) # Track custom KPI calculation time - # If controller needs a forecast, get the forecast data and update it with controller + # If controller needs a forecast, get the forecast data and provide the forecast to the controller if controller.use_forecast: - # Get forecast from BOPTEST - forecast_data = requests.get('{0}/forecast'.format(url)).json() - # Use BOPTEST forecast data to update controller-specific forecast data + # Retrieve forecast from restful API + forecast_data = check_response(requests.get('{0}/forecast'.format(url))) + # Use forecast data to update controller-specific forecast data forecasts = controller.update_forecasts(forecast_data, forecasts) else: forecasts = None - # Compute control signal for next step + # Compute control signal input to simulation for the next timestep u = controller.compute_control(y, forecasts) print('\nTest case complete.') print('Elapsed time of test was {0} seconds.'.format(time.time()-start)) @@ -161,7 +191,7 @@ def control_test(control_module='', start_time=0, warmup_period=0, length=24*360 # VIEW RESULTS # ------------------------------------------------------------------------- # Report KPIs - kpi = requests.get('{0}/kpi'.format(url)).json() + kpi = check_response(requests.get('{0}/kpi'.format(url))) print('\nKPI RESULTS \n-----------') for key in kpi.keys(): if key == 'ener_tot': @@ -192,7 +222,7 @@ def control_test(control_module='', start_time=0, warmup_period=0, length=24*360 points = list(measurements.keys()) + list(inputs.keys()) df_res = pd.DataFrame() for point in points: - res = requests.put('{0}/results'.format(url), data={'point_name': point, 'start_time': start_time, 'final_time': final_time}).json() + res = check_response(requests.put('{0}/results'.format(url), data={'point_name': point, 'start_time': start_time, 'final_time': final_time})) df_res = pd.concat((df_res, pd.DataFrame(data=res[point], index=res['time'], columns=[point])), axis=1) df_res.index.name = 'time' diff --git a/examples/python/testcase1.py b/examples/python/testcase1.py index 1c6147b2e..1d1ad39a2 100644 --- a/examples/python/testcase1.py +++ b/examples/python/testcase1.py @@ -38,7 +38,7 @@ def run(plot=False): """ # CONFIGURATION FOR THE CONTROL TEST - # ---------------------------------- + # --------------------------------------- control_module = 'examples.python.controllers.pid' start_time = 0 warmup_period = 0 @@ -46,6 +46,7 @@ def run(plot=False): step = 300 customized_kpi_dir_path = os.path.dirname(os.path.realpath(__file__)) customized_kpi_config = os.path.join(customized_kpi_dir_path, 'custom_kpi', 'custom_kpis_example.config') + # --------------------------------------- # RUN THE CONTROL TEST # -------------------- @@ -54,7 +55,7 @@ def run(plot=False): warmup_period=warmup_period, length=length, step=step, - customized_kpi_config= customized_kpi_config) + customized_kpi_config=customized_kpi_config) # POST-PROCESS RESULTS # -------------------- time = df_res.index.values/3600 # convert s --> hr diff --git a/examples/python/testcase1_scenario.py b/examples/python/testcase1_scenario.py index 64ee00bca..4cb46d521 100644 --- a/examples/python/testcase1_scenario.py +++ b/examples/python/testcase1_scenario.py @@ -3,6 +3,13 @@ This script demonstrates a minimalistic example of testing a feedback controller using the scenario options with the prototype test case called "testcase1". +""" +""" +This script demonstrates a minimalistic example of testing a feedback controller +with the prototype test case called "testcase1". It uses the testing +interface implemented in interface.py and the concrete controller implemented +in controllers/pid.py. + """ # GENERAL PACKAGE IMPORT @@ -39,12 +46,13 @@ def run(plot=False): control_module = 'examples.python.controllers.pid' scenario = {'time_period': 'test_day', 'electricity_price': 'dynamic'} step = 300 + # --------------------------------------- # RUN THE CONTROL TEST # -------------------- kpi, df_res, custom_kpi_result, forecasts = control_test(control_module, - scenario=scenario, - step=step) + scenario=scenario, + step=step) # POST-PROCESS RESULTS # -------------------- @@ -70,4 +78,4 @@ def run(plot=False): if __name__ == "__main__": - kpi, df_res, custom_kpi_result = run() + kpi, df_res, custom_kpi_result = run() \ No newline at end of file diff --git a/examples/python/testcase2.py b/examples/python/testcase2.py index 31ab2e65f..addcc9954 100644 --- a/examples/python/testcase2.py +++ b/examples/python/testcase2.py @@ -46,6 +46,7 @@ def run(plot=False): step = 3600 customized_kpi_dir_path = os.path.dirname(os.path.realpath(__file__)) customized_kpi_config = os.path.join(customized_kpi_dir_path, 'custom_kpi', 'custom_kpis_example.config') + # ------------------------------------- # RUN THE CONTROL TEST # -------------------- @@ -96,3 +97,4 @@ def run(plot=False): if __name__ == "__main__": kpi, df_res, custom_kpi_result = run() + print(kpi) diff --git a/examples/python/testcase3.py b/examples/python/testcase3.py index 2500f05b1..3b0ec88fd 100644 --- a/examples/python/testcase3.py +++ b/examples/python/testcase3.py @@ -45,6 +45,7 @@ def run(plot=False): length = 48*3600 step = 300 use_forecast = True + # ------------------------------------ # RUN THE CONTROL TEST # -------------------- @@ -94,4 +95,4 @@ def run(plot=False): if __name__ == "__main__": - kpi, df_res, custom_kpi_result = run() + kpi, df_res, custom_kpi_result = run() \ No newline at end of file diff --git a/kpis/kpi_calculator.py b/kpis/kpi_calculator.py index dc780c198..8eca0b741 100644 --- a/kpis/kpi_calculator.py +++ b/kpis/kpi_calculator.py @@ -693,7 +693,7 @@ def get_computational_time_ratio(self, plot=False): ''' - elapsed_control_time_ratio = self.case.get_elapsed_control_time_ratio() + elapsed_control_time_ratio = self.case._get_elapsed_control_time_ratio() time_rat = np.mean(elapsed_control_time_ratio) self.case.time_rat = time_rat diff --git a/makefile b/makefile index 9d54b73ca..3abc404b3 100644 --- a/makefile +++ b/makefile @@ -38,4 +38,4 @@ run-detached: echo WARNING: Use of make for building and running BOPTEST test cases is deprecated. Please use docker-compose as outlined in the README.md. stop: - docker stop ${IMG_NAME} + docker stop ${IMG_NAME} \ No newline at end of file diff --git a/releasenotes.md b/releasenotes.md index 77b99568a..d27cfe410 100644 --- a/releasenotes.md +++ b/releasenotes.md @@ -7,11 +7,17 @@ Released on xx/xx/xxxx. **The following changes are backwards-compatible and do not significantly change benchmark results:** - Add simulation support for test case FMUs compiled using Spawn of EnergyPlus. Does not address workflows for the compiling process for test case FMUs using Spawn. This is for [#406](https://github.com/ibpsa/project1-boptest/issues/406). +- New project home page launched at [https://ibpsa.github.io/project1-boptest/](https://ibpsa.github.io/project1-boptest/). This is for [#214](https://github.com/ibpsa/project1-boptest/issues/214). - Add file exclusion list to ``data_manager.py`` when loading data from fmu resource directory. This is for [#423](https://github.com/ibpsa/project1-boptest/issues/423). - Specify better command on ``README.md`` for specifying test case to deploy on Windows. This is for [#419](https://github.com/ibpsa/project1-boptest/issues/419). - Remove dependency of example controllers on ``pathlib`` package. This is for [#416](https://github.com/ibpsa/project1-boptest/issues/416). - Fix and clarify ``README.md`` for the ``/initialize`` and other API end points. This is for [#408](https://github.com/ibpsa/project1-boptest/issues/408). +**The following changes are not backwards-compatible but do not significantly change benchmark results:** + +- Add the POST ``submit`` API to submit test results to the online dashboard under a user's account there. This is for [#403](https://github.com/ibpsa/project1-boptest/issues/403). +- Update API to standardize return package format and information about about errors and warnings. The new return package is in the form ``{"status":, "message":, "payload":}``. Status codes are: 200, successful with or without warning; 400, bad input; 500, internal error. Users should expect the data returned in ``"payload"`` to be the same as the previous version API, which should facilitate the necessary code modifications to maintain compatibility with the new API. This is for [#73](https://github.com/ibpsa/project1-boptest/issues/73). + **The following new test cases have been added:** - ``multizone_office_simple_air``, a 5-zone building based on the U.S. DOE medium office reference building located in Chicago, IL, USA, served by a single-duct Variable Air Volume (VAV) with terminal reheat, air-cooled chiller, and air-to-water heat pump. This is for [#273](https://github.com/ibpsa/project1-boptest/issues/273). diff --git a/restapi.py b/restapi.py index 2d412f521..c241bcbaf 100644 --- a/restapi.py +++ b/restapi.py @@ -7,11 +7,24 @@ # GENERAL PACKAGE IMPORT # ---------------------- -from flask import Flask +from flask import Flask, make_response from flask_restful import Resource, Api, reqparse +import flask_restful from flask_cors import CORS +import six # ---------------------- + +# GENERAL HTTP RESPONSE +# ---------------------- +def construct(status, message, payload): + response = {'status': status, + 'message': message, + 'payload': payload} + return make_response(response, status) +# ---------------------- + + # TEST CASE IMPORT # ---------------- from testcase import TestCase @@ -19,6 +32,27 @@ # FLASK REQUIREMENTS # ------------------ + + +class CustomArgument(reqparse.Argument): + + def handle_validation_error(self, error, bundle_errors): + '''Customizes inherited class with general HTTP response ``construct``. + + Called when an error is raised while parsing. Aborts the request + with a 400 status and an error message + + :param error: the error that was raised + :param bundle_errors: do not abort when first error occurs, return a + dict with the name of the argument and the error message to be + bundled + + ''' + + error_str = six.text_type(error) + msg = 'Bad input for parameter {}. '.format(self.name) + error_str + flask_restful.abort(construct(400, msg, None)) + app = Flask(__name__) cors = CORS(app, resources={r"*": {"origins": "*"}}) api = Api(app) @@ -32,32 +66,42 @@ # DEFINE ARGUMENT PARSERS # ----------------------- # ``step`` interface -parser_step = reqparse.RequestParser() -parser_step.add_argument('step') +parser_step = reqparse.RequestParser(argument_class=CustomArgument) +parser_step.add_argument('step', required=True) + # ``initialize`` interface -parser_initialize = reqparse.RequestParser() -parser_initialize.add_argument('start_time') -parser_initialize.add_argument('warmup_period') +parser_initialize = reqparse.RequestParser(argument_class=CustomArgument) +parser_initialize.add_argument('start_time', required=True) +parser_initialize.add_argument('warmup_period', required=True) # ``advance`` interface -parser_advance = reqparse.RequestParser() +parser_advance = reqparse.RequestParser(argument_class=CustomArgument) for key in case.u.keys(): - parser_advance.add_argument(key) -#``forecast_parameters`` interface -parser_forecast_parameters = reqparse.RequestParser() -forecast_parameters = ['horizon','interval'] + if key != 'time': + parser_advance.add_argument(key) +# ``forecast_parameters`` interface +parser_forecast_parameters = reqparse.RequestParser(argument_class=CustomArgument) +forecast_parameters = ['horizon', 'interval'] for arg in forecast_parameters: - parser_forecast_parameters.add_argument(arg) + parser_forecast_parameters.add_argument(arg, required=True) # ``price_scenario`` interface -parser_scenario = reqparse.RequestParser() -parser_scenario.add_argument('electricity_price') -parser_scenario.add_argument('time_period') +parser_scenario = reqparse.RequestParser(argument_class=CustomArgument) +parser_scenario.add_argument('electricity_price', type=str) +parser_scenario.add_argument('time_period', type=str) # ``results`` interface -results_var = reqparse.RequestParser() -results_var.add_argument('point_name') -results_var.add_argument('start_time') -results_var.add_argument('final_time') +results_var = reqparse.RequestParser(argument_class=CustomArgument) +results_var.add_argument('point_name', type=str, required=True) +results_var.add_argument('start_time', required=True) +results_var.add_argument('final_time', required=True) +# ``submit`` interface +submit_var = reqparse.RequestParser(argument_class=CustomArgument) +submit_var.add_argument('api_key', type=str, required=True) + # add up to 10 tags +for i in range(10): + submit_var.add_argument('tag{0}'.format(i+1), type=str) +submit_var.add_argument('unit_test') # ----------------------- + # DEFINE REST REQUESTS # -------------------- class Advance(Resource): @@ -67,50 +111,55 @@ def post(self): '''POST request with input data to advance the simulation one step and receive current measurements.''' u = parser_advance.parse_args(strict=True) - y = case.advance(u) - return y + status, message, payload = case.advance(u) + return construct(status, message, payload) + class Initialize(Resource): '''Interface to initialize the test case simulation.''' def put(self): '''PUT request to initialize the test.''' - args = parser_initialize.parse_args(strict=True) - start_time = float(args['start_time']) - warmup_period = float(args['warmup_period']) - y = case.initialize(start_time,warmup_period) - return y + args = parser_initialize.parse_args() + start_time = args['start_time'] + warmup_period = args['warmup_period'] + status, message, payload = case.initialize(start_time, warmup_period) + return construct(status, message, payload) + class Step(Resource): '''Interface to test case simulation step size.''' def get(self): '''GET request to receive current simulation step in seconds.''' - step = case.get_step() - return step + status, message, payload = case.get_step() + return construct(status, message, payload) def put(self): '''PUT request to set simulation step in seconds.''' - args = parser_step.parse_args(strict=True) + args = parser_step.parse_args() step = args['step'] - case.set_step(step) - return step, 201 + status, message, payload = case.set_step(step) + return construct(status, message, payload) + class Inputs(Resource): '''Interface to test case inputs.''' def get(self): '''GET request to receive list of available inputs.''' - u_list = case.get_inputs() - return u_list + status, message, payload = case.get_inputs() + return construct(status, message, payload) + class Measurements(Resource): '''Interface to test case measurements.''' def get(self): '''GET request to receive list of available measurements.''' - y_list = case.get_measurements() - return y_list + status, message, payload = case.get_measurements() + return construct(status, message, payload) + class Results(Resource): '''Interface to test case result data.''' @@ -118,77 +167,99 @@ class Results(Resource): def put(self): '''PUT request to receive measurement data.''' args = results_var.parse_args(strict=True) - var = args['point_name'] - start_time = float(args['start_time']) - final_time = float(args['final_time']) - Y = case.get_results(var, start_time, final_time) - for key in Y: - Y[key] = Y[key].tolist() + var = args['point_name'] + start_time = args['start_time'] + final_time = args['final_time'] + status, message, payload = case.get_results(var, start_time, final_time) + return construct(status, message, payload) - return Y class KPI(Resource): '''Interface to test case KPIs.''' def get(self): '''GET request to receive KPI data.''' - kpi = case.get_kpis() - return kpi + status, message, payload = case.get_kpis() + return construct(status, message, payload) + class Forecast_Parameters(Resource): '''Interface to test case forecast parameters.''' def get(self): '''GET request to receive forecast parameters.''' - forecast_parameters = case.get_forecast_parameters() - return forecast_parameters + status, message, payload = case.get_forecast_parameters() + return construct(status, message, payload) def put(self): '''PUT request to set forecast horizon and interval inseconds.''' - args = parser_forecast_parameters.parse_args(strict=True) - horizon = args['horizon'] + args = parser_forecast_parameters.parse_args() + horizon = args['horizon'] interval = args['interval'] - case.set_forecast_parameters(horizon, interval) - forecast_parameters = case.get_forecast_parameters() - return forecast_parameters + status, message, payload = case.set_forecast_parameters(horizon, interval) + return construct(status, message, payload) + class Forecast(Resource): '''Interface to test case forecast data.''' def get(self): '''GET request to receive forecast data.''' - forecast = case.get_forecast() - return forecast + status, message, payload = case.get_forecast() + return construct(status, message, payload) + class Scenario(Resource): '''Interface to test case scenario.''' def get(self): '''GET request to receive current scenario.''' - scenario = case.get_scenario() - return scenario + status, message, payload = case.get_scenario() + return construct(status, message, payload) def put(self): '''PUT request to set scenario.''' scenario = parser_scenario.parse_args(strict=True) - result = case.set_scenario(scenario) - return result + status, message, payload = case.set_scenario(scenario) + return construct(status, message, payload) + class Name(Resource): '''Interface to test case name.''' def get(self): '''GET request to receive test case name.''' - name = case.get_name() - return name + status, message, payload = case.get_name() + return construct(status, message, payload) + class Version(Resource): '''Interface to BOPTEST version.''' def get(self): '''GET request to receive BOPTEST version.''' - version = case.get_version() - return version + status, message, payload = case.get_version() + return construct(status, message, payload) + +class Submit(Resource): + '''Submit results to dashboard.''' + + def post(self): + '''POST request to submit results to online dashboard.''' + args = submit_var.parse_args(strict=True) + api_key = args['api_key'] + unit_test=False + tags = [] + for key in args: + if ('tag' in key) and (args[key] is not None): + tags.append(args[key]) + if key=='unit_test': + if args[key] == "True": + unit_test = True + else: + unit_test = False + status, message, payload = case.post_results_to_dashboard(api_key, tags, unit_test) + return construct(status, message, payload) # -------------------- # ADD REQUESTS TO API WITH URL EXTENSION @@ -205,6 +276,7 @@ def get(self): api.add_resource(Scenario, '/scenario') api.add_resource(Name, '/name') api.add_resource(Version, '/version') +api.add_resource(Submit, '/submit') # -------------------------------------- if __name__ == '__main__': diff --git a/testcase.py b/testcase.py index a3ea95d24..0a255d7c4 100644 --- a/testcase.py +++ b/testcase.py @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- """ This module defines the API to the test case used by the REST requests to -perform functions such as advancing the simulation, retreiving test case +perform functions such as advancing the simulation, retrieving test case information, and calculating and reporting results. """ @@ -13,6 +13,14 @@ from data.data_manager import Data_Manager from forecast.forecaster import Forecaster from kpis.kpi_calculator import KPI_Calculator +import requests +import traceback +import logging +import pytz +from datetime import datetime +import uuid +import os +import json class TestCase(object): '''Class that implements the test case. @@ -46,6 +54,16 @@ def __init__(self, fmupath='models/wrapped.fmu'): # Load fmu self.fmu = load_fmu(self.fmupath) self.fmu.set_log_level(7) + # Configure the log, log file, and console output + name = 'boptest_{0}'.format(self.name) + fmt = '%(asctime)s UTC\t%(name)-20s%(levelname)s\t%(message)s' + datefmt = '%m/%d/%Y %I:%M:%S %p' + formatter = logging.Formatter(fmt,datefmt) + logging.basicConfig(filename='{0}.log'.format(name), filemode='w', level=10, format=fmt, datefmt=datefmt) + logger = logging.getLogger() + stream_handler = logging.StreamHandler() + stream_handler.setFormatter(formatter) + logger.addHandler(stream_handler) # Get version and check is 2.0 self.fmu_version = self.fmu.get_version() if self.fmu_version != '2.0': @@ -94,7 +112,7 @@ def __initilize_data(self): self.inputs_metadata = self._get_var_metadata(self.fmu, self.input_names, inputs=True) self.outputs_metadata = self._get_var_metadata(self.fmu, self.output_names) # Outputs data - self.y = {'time':np.array([])} + self.y = {'time': np.array([])} for key in self.output_names: # Do not store outputs that are current values of control inputs flag = False @@ -141,12 +159,12 @@ def __simulation(self,start_time,end_time,input_object=None): self.options['ncp'] = int((end_time-start_time)/30) # Simulate fmu try: - res = self.fmu.simulate(start_time = start_time, - final_time = end_time, - options=self.options, - input=input_object) - except Exception as e: - return None + res = self.fmu.simulate(start_time=start_time, + final_time=end_time, + options=self.options, + input=input_object) + except: + return traceback.format_exc() # Set internal fmu initialization self.initialize_fmu = False @@ -169,7 +187,7 @@ def __get_results(self, res, store=True, store_initial=False): Set to true if desired to store results in `self.y_store` and `self.u_store` store_initial: boolean - Set to true if desired to store initial point. + Set to true if desired to store the initial point. ''' @@ -196,26 +214,34 @@ def __get_results(self, res, store=True, store_initial=False): if store: self.u_store[key] = np.append(self.u_store[key], res[key_data][i:]) - def advance(self,u): + def advance(self, u): '''Advances the test case model simulation forward one step. Parameters ---------- u : dict Defines the control input data to be used for the step. - { : } + {_activate : bool, int, float, or str convertable to 1 or 0 + _u : int or float} Returns ------- - z : dict + status: int + Indicates whether an advance request has been completed. + If 200, simulation advance was completed. + If 400, invalid inputs (non-numeric) were identified. + If 500, a simulation error occurred. + message: str + Includes the debug information + payload: dict Contains the full state of measurement and input data at the end of the step. { : } If empty, simulation end time has been reached. - If None, a simulation error has occured. - + If None, a simulation error has occurred. ''' + status = 200 # Calculate and store the elapsed time if hasattr(self, 'tic_time'): self.tac_time = time.time() @@ -223,6 +249,7 @@ def advance(self,u): # Set final time self.final_time = self.start_time + self.step + alert_message = '' # Set control inputs if they exist and are written # Check if possible to overwrite if u.keys(): @@ -238,13 +265,50 @@ def advance(self,u): u_list = [] u_trajectory = self.start_time for key in u.keys(): + if (key not in self.input_names): + payload = None + status = 400 + message = "Unexpected input variable: {}.".format(key) + logging.error(message) + return status, message, payload if key != 'time' and u[key]: - value = float(u[key]) - # Check min/max if not activation input - if '_activate' not in key: - checked_value = self._check_value_min_max(key, value) + if '_activate' in key: + try: + if float(u[key]) == 1: + checked_value = 1 + elif float(u[key]) == 0: + checked_value = 0 + else: + payload = None + status = 400 + message = "Invalid value {} and/or type {} for input {}. Input must be a boolean, float, integer, string, or unicode able to be converted to a 1 or 0 (or 'T[t]rue' or 'F[f]alse').".format(u[key], type(u[key]), key) + logging.error(message) + return status, message, payload + except ValueError: + if (u[key] == 'True') or (u[key] == 'true'): + checked_value = 1 + elif (u[key] == 'False') or (u[key] == 'false'): + checked_value = 0 + else: + payload = None + status = 400 + message = "Invalid value {} and/or type {} for input {}. Input must be a boolean, float, integer, string, or unicode able to be converted to a 1 or 0 (or 'T[t]rue' or 'F[f]alse').".format(u[key], type(u[key]), key) + logging.error(message) + return status, message, payload else: - checked_value = value + try: + value = float(u[key]) + except: + payload = None + status = 400 + message = "Invalid value {} for input {}. Value must be a float, integer, or string able to be converted to a float, but is {}.".format(u[key], key, type(u[key])) + logging.error(message) + return status, message, payload + # Check min/max if not activation input + checked_value, message = self._check_value_min_max(key, value) + if message is not None: + logging.warning(message) + alert_message = message + ';' + alert_message u_list.append(key) u_trajectory = np.vstack((u_trajectory, checked_value)) input_object = (u_list, np.transpose(u_trajectory)) @@ -259,55 +323,108 @@ def advance(self,u): # Make sure stop at end of test if self.final_time > self.end_time: self.final_time = self.end_time - res = self.__simulation(self.start_time,self.final_time,input_object) + res = self.__simulation(self.start_time, self.final_time, input_object) # Process results - if res is not None: + if not isinstance(res, str): # Get result and store measurement and control inputs self.__get_results(res, store=True, store_initial=False) - # Advance start time - self.start_time = self.final_time # Raise the flag to compute time lapse self.tic_time = time.time() # Get full current state - z = self._get_full_current_state() - - return z + payload = self._get_full_current_state() + # Write any messages + if alert_message == '': + message = "Advanced simulation successfully from {0}s to {1}s.".format(self.start_time, self.final_time) + else: + message = alert_message + # Advance start time + self.start_time = self.final_time + # Log and return + logging.info(message) + return status, message, payload else: - # Error in simulation - return None + # Errors in the simulation + status = 500 + message = "Failed to advance simulation: {}.".format(res) + payload = res + logging.error(message) + return status, message, payload else: # Simulation at end time - return dict() + self.scenario_end = True + payload = dict() + message = "End of test case scenario time period reached." + logging.info(message) + + return status, message, payload def initialize(self, start_time, warmup_period, end_time=np.inf): '''Initialize the test simulation. Parameters ---------- - start_time: int + start_time: int or float Start time of simulation to initialize to in seconds. - warmup_period: int + warmup_period: int or float Length of time before start_time to simulate for warmup in seconds. - end_time: int, optional - Specifies a finite end time to allow simulation to continue + end_time: int or float, optional + Specifies a finite end time to allow the simulation to continue Default value is infinite. Returns ------- - z : dict + status: int + Indicates whether an initialization request has been completed. + If 200, initialization was completed successfully. + If 400, an invalid start time or warmup period (non-numeric) was identified. + If 500, an error occurred during the initialization simulation. + message: str + Includes detailed debugging information. + payload: dict Contains the full state of measurement and input data at the end - of the initialization. + of the initialization period. { : }. - If None, a simulation error has occured. + If None, an error occurred during the initialization simulation. ''' + status = 200 + payload = None # Reset fmu self.fmu.reset() # Reset simulation data storage self.__initilize_data() - self.elapsed_control_time_ratio =np.array([]) + self.elapsed_control_time_ratio = np.array([]) + # Check if the inputs are valid + try: + start_time = float(start_time) + except: + payload = None + status = 400 + message = "Invalid value {} for parameter start_time. Value must be a float, integer, or string able to be converted to a float, but is {}.".format(start_time, type(start_time)) + logging.error(message) + return status, message, payload + try: + warmup_period = float(warmup_period) + except: + payload = None + status = 400 + message = "Invalid value {} for parameter warmup_period. Value must be a float, integer, or string able to be converted to a float, but is {}.".format(warmup_period, type(warmup_period)) + logging.error(message) + return status, message, payload + if start_time < 0: + payload = None + status = 400 + message = "Invalid value {} for parameter start_time. Value must not be negative.".format(start_time) + logging.error(message) + return status, message, payload + if warmup_period < 0: + payload = None + status = 400 + message = "Invalid value {} for parameter warmup_period. Value must not be negative.".format(warmup_period) + logging.error(message) + return status, message, payload # Record initial testing time self.initial_time = start_time # Record end testing time @@ -316,46 +433,117 @@ def initialize(self, start_time, warmup_period, end_time=np.inf): self.initialize_fmu = True # Simulate fmu for warmup period. # Do not allow negative starting time to avoid confusions - res = self.__simulation(max(start_time-warmup_period,0), start_time) + res = self.__simulation(max(start_time-warmup_period, 0), start_time) # Process result - if res is not None: + if not isinstance(res, str): # Get result self.__get_results(res, store=True, store_initial=True) # Set internal start time to start_time self.start_time = start_time # Initialize KPI Calculator self.cal.initialize() + # Set scenario end flag to false + self.scenario_end = False # Get full current state - z = self._get_full_current_state() + payload = self._get_full_current_state() + message = "Test simulation initialized successfully to {0}s with warmup period of {1}s.".format(self.start_time, warmup_period) + logging.info(message) - return z + return status, message, payload else: + payload = None + status = 500 + message = "Failed to initialize test simulation: {}.".format(res) + logging.error(message) - return None + return status, message, payload def get_step(self): - '''Returns the current simulation step in seconds.''' + '''Returns the current control step in seconds. - return self.step + Parameters + ---------- + None - def set_step(self,step): - '''Sets the simulation step in seconds. + Returns + ------- + status: int + Indicates whether a request for querying the control step has been completed. + If 200, the step was successfully queried. + If 500, an internal error occurred. + message: str + Includes detailed debugging information. + payload: int + The current control step. + None if error during query. + + ''' + + status = 200 + message = "Queried the control step successfully." + payload = None + try: + payload = self.step + logging.info(message) + except: + status = 500 + message = "Failed to query the simulation step: {}".format(traceback.format_exc()) + logging.error(message) + + return status, message, payload + + def set_step(self, step): + '''Sets the control step in seconds. Parameters ---------- - step : int - Simulation step in seconds. + step: int or float + Control step in seconds. Returns ------- - None + status: int + Indicates whether a request for setting the control step has been completed. + If 200, the step was successfully set. + If 400, an invalid simulation step (non-numeric) was identified. + If 500, an internal error occurred. + message: str + Includes detailed debugging information. + payload: + None ''' - self.step = float(step) - - return None + status = 200 + message = "Control step set successfully." + payload = None + try: + step = float(step) + except: + payload = None + status = 400 + message = "Invalid value {} for parameter step. Value must be a float, integer, or string able to be converted to a float, but is {}.".format(step, type(step)) + logging.error(message) + return status, message, payload + if step < 0: + payload = None + status = 400 + message = "Invalid value {} for parameter step. Value must not be negative.".format(step) + logging.error(message) + return status, message, payload + try: + self.step = step + except: + payload = None + status = 500 + message = "Failed to set the control step: {}".format(traceback.format_exc()) + logging.error(message) + return status, message, payload + payload={'step':self.step} + logging.info(message) + + return status, message, payload def get_inputs(self): '''Returns a dictionary of control inputs and their meta-data. @@ -366,14 +554,30 @@ def get_inputs(self): Returns ------- - inputs : dict + status: int + Indicates whether a request for querying the inputs has been completed. + If 200, the inputs were successfully queried. + If 500, an internal error occurred. + message: str + Includes detailed debugging information. + payload: dict Dictionary of control inputs and their meta-data. + Returns None if error in getting inputs and meta-data. ''' - inputs = self.inputs_metadata + status = 200 + message = "Queried the inputs successfully." + payload = None + try: + payload = self.inputs_metadata + logging.info(message) + except: + status = 500 + message = "Failed to query the input list: {}".format(traceback.format_exc()) + logging.error(message) - return inputs + return status, message, payload def get_measurements(self): '''Returns a dictionary of measurements and their meta-data. @@ -384,59 +588,117 @@ def get_measurements(self): Returns ------- - measurements : dict + status: int + Indicates whether a request for querying the outputs has been completed. + If 200, the outputs were successfully queried. + If 500, an internal error occurred. + message: str + Includes detailed debugging information. + payload : dict Dictionary of measurements and their meta-data. + Returns None if error in getting measurements and meta-data. ''' - measurements = self.outputs_metadata + status = 200 + message = "Queried the measurements successfully." + payload = None + try: + payload = self.outputs_metadata + logging.info(message) + except: + status = 500 + message = "Failed to query the measurement list: {}".format(traceback.format_exc()) + logging.error(message) - return measurements + return status, message, payload - def get_results(self, var, start_time, final_time): + def get_results(self, point_name, start_time, final_time): '''Returns measurement and control input trajectories. Parameters ---------- - var : str + point_name: str Name of variable. - start_time : float + start_time : int or float Start time of data to return in seconds. - final_time : float + final_time : int or float Start time of data to return in seconds. Returns ------- - Y : dict or None + status: int + Indicates whether a request for querying the results has been completed. + If 200, the results were successfully queried. + If 400, invalid start time and/or invalid final time (non-numeric) were identified. + If 500, an internal error occured. + message: str + Includes detailed debugging information. + payload : dict Dictionary of variable trajectories with time as lists. {'time':[], 'var':[] } - Returns None if no variable can be found + Returns None if no variable can be found or a simulation error occurs. ''' - # Get correct point - if var in self.y_store.keys(): - Y = {'time':self.y_store['time'], - var:self.y_store[var] - } - elif var in self.u_store.keys(): - Y = {'time':self.u_store['time'], - var:self.u_store[var] - } - else: - Y = None - return Y - - # Get correct time - time1 = Y['time'] - for key in [var,'time']: - Y[key] = Y[key][time1>=start_time] - time2 = time1[time1>=start_time] - Y[key] = Y[key][time2<=final_time] - - return Y + status = 200 + try: + start_time = float(start_time) + except: + payload = None + status = 400 + message = "Invalid value {} for parameter start_time. Value must be a float, integer, or string able to be converted to a float, but is {}.".format(start_time, type(start_time)) + logging.error(message) + return status, message, payload + try: + final_time = float(final_time) + except: + payload = None + status = 400 + message = "Invalid value {} for parameter final_time. Value must be a float, integer, or string able to be converted to a float, but is {}.".format(final_time, type(final_time)) + logging.error(message) + return status, message, payload + payload = [] + try: + # Get correct point + if point_name in self.y_store.keys(): + payload = { + 'time': self.y_store['time'], + point_name: self.y_store[point_name] + } + elif point_name in self.u_store.keys(): + payload = { + 'time': self.u_store['time'], + point_name: self.u_store[point_name] + } + else: + status = 400 + message = "Invalid value {} for parameter point_name. Check lists of inputs and measurements.".format(point_name) + logging.error(message) + return status, message, None + + # Get correct time + if payload and 'time' in payload: + time1 = payload['time'] + for key in [point_name, 'time']: + payload[key] = payload[key][time1>=start_time] + time2 = time1[time1>=start_time] + payload[key] = payload[key][time2<=final_time] + except: + status = 500 + message = "Failed to query simulation results: {}".format(traceback.format_exc()) + logging.error(message) + return status, message, None + + if not isinstance(payload, (list, type(None))): + for key in payload: + payload[key] = payload[key].tolist() + + message = "Queried results data successfully for point with name {}.".format(point_name) + logging.info(message) + return status, message, payload def get_kpis(self): '''Returns KPI data. @@ -449,53 +711,144 @@ def get_kpis(self): Returns ------- - kpis : dict + status: int + Indicates whether a request for querying the KPIs has been completed. + If 200, the KPIs were successfully queried. + If 500, an internal error occured. + message: str + Includes detailed debugging information + payload : dict Dictionary containing KPI names and values. - {:} + {:}. + Returns None if error during calculation. ''' - # Set correct price scenario for cost - if self.scenario['electricity_price'] == 'constant': - price_scenario = 'Constant' - elif self.scenario['electricity_price'] == 'dynamic': - price_scenario = 'Dynamic' - elif self.scenario['electricity_price'] == 'highly_dynamic': - price_scenario = 'HighlyDynamic' - # Calculate the core kpis - kpis = self.cal.get_core_kpis(price_scenario=price_scenario) - - return kpis - - def set_forecast_parameters(self,horizon,interval): + status = 200 + message = "Queried KPIs successfully." + try: + # Set correct price scenario for cost + if self.scenario['electricity_price'] == 'constant': + price_scenario = 'Constant' + elif self.scenario['electricity_price'] == 'dynamic': + price_scenario = 'Dynamic' + elif self.scenario['electricity_price'] == 'highly_dynamic': + price_scenario = 'HighlyDynamic' + # Calculate the core kpis + payload = self.cal.get_core_kpis(price_scenario=price_scenario) + except: + payload = None + status = 500 + message = "Failed to query KPIs: {}".format(traceback.format_exc()) + logging.error(message) + logging.info(message) + + return status, message, payload + + def set_forecast_parameters(self, horizon, interval): '''Sets the forecast horizon and interval, both in seconds. Parameters ---------- - horizon : int + horizon: int or float Forecast horizon in seconds. - interval : int + interval: int or float Forecast interval in seconds. Returns ------- - None + status: int + Indicates whether a request for setting the forecast parameters has been completed. + If 200, the parameters were successfully set. + If 400, invalid horizon and/or interval (non-numeric) were identified. + If 500, an internal error occured. + message: str + Includes detailed debugging information. + payload : dict + Dictionary containing forecast parameters names and values: + {:}. + Returns None if error during setting. ''' - self.horizon = float(horizon) - self.interval = float(interval) - - return None + status = 200 + message = "Forecast horizon and interval were set successfully." + payload = dict() + try: + horizon = float(horizon) + except: + payload = None + status = 400 + message = "Invalid value {} for parameter horizon. Value must be a float, integer, or string able to be converted to a float, but is {}.".format(horizon, type(horizon)) + logging.error(message) + return status, message, payload + try: + interval = float(interval) + except: + payload = None + status = 400 + message = "Invalid value {} for parameter interval. Value must be a float, integer, or string able to be converted to a float, but is {}.".format(interval, type(interval)) + logging.error(message) + return status, message, payload + if horizon < 0: + payload = None + status = 400 + message = "Invalid value {} for parameter horizon. Value must not be negative.".format(horizon) + logging.error(message) + return status, message, payload + if interval < 0: + payload = None + status = 400 + message = "Invalid value {} for parameter interval. Value must not be negative.".format(interval) + logging.error(message) + return status, message, payload + try: + self.horizon = horizon + self.interval = interval + payload['horizon'] = self.horizon + payload['interval'] = self.interval + except: + status = 500 + message = "Failed to set forecast horizon and interval: {}".format(traceback.format_exc()) + logging.error(message) + logging.info(message) + + return status, message, payload def get_forecast_parameters(self): - '''Returns the current forecast horizon and interval parameters.''' + '''Returns the current forecast horizon and interval parameters. - forecast_parameters = dict() - forecast_parameters['horizon'] = self.horizon - forecast_parameters['interval'] = self.interval + Parameters + ---------- + None + + Returns + ------- + status: int + Indicates whether a request for querying the forecast parameters has been completed. + If 200, the forecast parameters were successfully queried. + If 500, an internal error occurred. + message: str + Includes detailed debugging information . + payload: dict + Dictionary containing forecast parameters names and values. + {:} + + ''' + + status = 200 + message = "Queried the forecast parameters successfully." + payload = dict() + if self.horizon is not None and self.interval is not None: + payload['horizon'] = self.horizon + payload['interval'] = self.interval + else: + status = 500 + message = "Failed to query the forecast parameters: {}".format(traceback.format_exc()) + logging.error(message) + logging.info(message) - return forecast_parameters + return status, message, payload def get_forecast(self): '''Returns the test case data forecast @@ -506,69 +859,155 @@ def get_forecast(self): Returns ------- - forecast : dict + status: int + Indicates whether a request for querying the forecast has been successfully completed. + If 200, the forecast was successfully queried. + If 500, an internal error occurred. + message: str + Includes detailed debugging information + payload: dict Dictionary with the requested forecast data {:} where is a string with the variable key and is a list with - the forecasted values. 'time' is included as a variable + the forecasted values. 'time' is included as a variable. ''' # Get the forecast - forecast = self.forecaster.get_forecast(horizon=self.horizon, - interval=self.interval) + status = 200 + message = "Queried the forecast data successfully." + try: + payload = self.forecaster.get_forecast(horizon=self.horizon, + interval=self.interval) + except: + status = 500 + message = "Failed to query the test case forecast data: {}".format(traceback.format_exc()) + payload = None + logging.error(message) + logging.info(message) - return forecast + return status, message, payload def set_scenario(self, scenario): - '''Sets the case scenario. + '''Sets the test case scenario. Parameters ---------- scenario : dict {'electricity_price': <'constant' or 'dynamic' or 'highly_dynamic'>, - 'time_period': see available keys for test case - } + 'time_period': see available keys for test case + } If any value is None, it will not change existing. Returns ------- - result : dict + status: int + Indicates whether a request for setting the scenario has been completed + If 200, the scenario was successfully set. + If 400, invalid electricity_price and/or time_period (non-numeric) were identified. + If 500, an internal error occurred. + message: str + Includes the detailed debug information + payload: dict {'electricity_price': if succeeded in changing then True, else None, 'time_period': if succeeded then initial measurements, else None - } + } ''' - result = {'electricity_price':None, - 'time_period':None} - - if not hasattr(self,'scenario'): + status = 200 + message = "Test case scenario was set successfully." + payload = { + 'electricity_price': None, + 'time_period': None + } + if not hasattr(self, 'scenario'): self.scenario = {} - # Handle electricity price - if scenario['electricity_price']: - self.scenario['electricity_price'] = scenario['electricity_price'] - result['electricity_price'] = True - # Handle timeperiod - if scenario['time_period']: - self.scenario['time_period'] = scenario['time_period'] - warmup_period = 7*24*3600 - key = self.scenario['time_period'] - start_time = self.days_json[key]*24*3600-7*24*3600 - end_time = start_time + 14*24*3600 - result['time_period'] = self.initialize(start_time, warmup_period, end_time=end_time) - - # It's needed to reset KPI Calculator when scenario is changed - self.cal.initialize() + try: + # Handle electricity price + if scenario['electricity_price']: + if scenario['electricity_price'] not in ['constant', 'dynamic', 'highly_dynamic']: + status = 400 + message = "Scenario parameter electricy_price is {}, " \ + "but should be 'constant', 'dynamic', or 'highly_dynamic'.". \ + format(scenario['electricity_price']) + logging.error(message) + return status, message, payload + self.scenario['electricity_price'] = scenario['electricity_price'] + payload['electricity_price'] = self.scenario['electricity_price'] + # Handle timeperiod + if scenario['time_period']: + if scenario['time_period'] not in self.days_json: + status = 400 + message = "Scenario parameter time_period is {}, but " \ + "should be one of the following: {}.". \ + format(scenario['time_period'], list(self.days_json.keys())) + logging.error(message) + return status, message, payload + self.scenario['time_period'] = scenario['time_period'] + warmup_period = 7*24*3600. + key = self.scenario['time_period'] + start_time = self.days_json[key]*24*3600.-7*24*3600. + end_time = start_time + 14*24*3600. + except: + status = 400 + message = "Invalid values for the scenario parameters: {}".format(traceback.format_exc()) + logging.error(message) + return status, message, payload + try: + if scenario['time_period']: + initialize_payload = self.initialize(start_time, warmup_period, end_time=end_time) + if initialize_payload[0] != 200: + status = 500 + message = initialize_payload[1] + logging.error(message) + return status, message, payload + payload['time_period'] = initialize_payload[2] + # It's needed to reset KPI Calculator when scenario is changed + self.cal.initialize() + except: + status = 500 + message = "Failed to set the test case scenario: {}".format(traceback.format_exc()) + payload = None + logging.error(message) + logging.info(message) - return result + return status, message, payload def get_scenario(self): - '''Returns the current case scenario.''' + '''Returns the current case scenario. - scenario = self.scenario + Parameters + ---------- + None - return scenario + Returns + ------- + status: int + Indicates whether a request for querying the scenario has been successfully completed + If 200, the scenario was successfully queried. + If 500, an internal error occurred. + message: str + Includes detailed debugging information + payload: dict + {'electricity_price': , + 'time_period': + } + + ''' + + payload = None + status = 200 + message = "Queried current test case for scenario successfully." + try: + payload = self.scenario + except: + status = 500 + message = "Failed to find a current test case scenario setting. Check it was set properly: {}".format(traceback.format_exc()) + logging.error(message) + logging.info(message) + + return status, message, payload def get_name(self): '''Returns the name of the test case fmu. @@ -579,17 +1018,26 @@ def get_name(self): Returns ------- - name : dict + status: int + Indicate whether a request for querying the name of the test case has been successfully completed. + If 200, the name was successfully queried. + If 500, an internal error occurred. + message: str + Includes detailed debugging information + payload : dict Name of test case as {'name': } ''' - name = {'name':self.name} + status = 200 + message = "Queried the name of the test case successfully." + payload = {'name': self.name} + logging.info(message) - return name + return status, message, payload - def get_elapsed_control_time_ratio(self): - '''Returns the elapsed control time ratio vector for the case. + def get_version(self): + '''Returns the version number of BOPTEST. Parameters ---------- @@ -597,17 +1045,134 @@ def get_elapsed_control_time_ratio(self): Returns ------- - elapsed_control_time_ratio : np array of floats - elapsed_control_time_ratio for each control step. + status: int + Indicate whether a request for querying the version number of BOPTEST has been completed. + If 200, the version was successfully queried. + If 500, an internal error occurred. + message: str + Includes detailed debugging information + payload: dict + Version of BOPTEST as {'version': } ''' - elapsed_control_time_ratio = self.elapsed_control_time_ratio + status = 200 + message = "Queried the version number successfully." + logging.info(message) + payload = {'version': self.version} - return elapsed_control_time_ratio + return status, message, payload - def get_version(self): - '''Returns the version number of BOPTEST. + def post_results_to_dashboard(self, api_key, tags, unit_test=False): + '''Posts test results to online dashboard at given server address. + + Parameters + ---------- + api_key : str + API key corresponding to user account for dashboard. + tags : list of str + List of tags to be included with result posting. + unit_test : bool + True if API being called for unit testing so as not to post to online dashboard. + + Returns + ------- + status: int + Indicates whether a posting to the dashboard has been successfully completed. + If 200, the posting was successful. + If 400, there was an input error. + If 500, an internal error occurred. + message: str + Includes detailed debugging information + payload: None + + ''' + + # Check formal scnenario test completed + if not self.scenario_end: + status = 500 + message = 'Formal test scenario, including time period, has not been completed. Initialize a test scenario using the PUT /scenario API and run it to completion before submitting results to dashboard.' + logging.error(message) + return status, message, None + # Check parameters + if not isinstance(api_key, str): + status = 400 + message = 'Invalid type for input api_key. It must be a string, but is a {0}.'.format(type(api_key)) + logging.error(message) + return status, message, None + + if not isinstance(tags, list): + status = 400 + message = 'Invalid type for input tags. It must be a list of strings, but is a {0}.'.format(type(tags)) + logging.error(message) + return status, message, None + + if len(tags)>10: + status = 400 + message = 'Invalid number of tags. The limit is 10, but there are {0}.'.format(len(tags)) + logging.error(message) + return status, message, None + + for tag in tags: + if not isinstance(tag, str): + status = 400 + message = 'Invalid type for one of the tag inputs. They must be strings, but one is a {0}.'.format(type(tag)) + logging.error(message) + return status, message, None + # Specify server address and payload + dash_server = os.environ['BOPTEST_DASHBOARD_SERVER'] + # Create payload + uid = str(uuid.uuid4()) + payload = { + "results": [ + { + "uid": uid, + "dateRun": str(datetime.now(tz=pytz.UTC)), + "boptestVersion": self.version, + "isShared": True, + "controlStep": str(self.get_step()[2]), + "account": { + "apiKey": api_key + }, + "tags": tags, + "kpis": self.get_kpis()[2], + "forecastParameters": self.get_forecast_parameters()[2], + "scenario": self.add_forecast_uncertainty(self.keys_to_camel_case(self.get_scenario()[2])), + "buildingType": { + "uid": self.get_name()[2]['name'] + } + } + ] + } + dash_url = "%s/api/results" % dash_server + # Post to dashboard + if not unit_test: + result = requests.post(dash_url, json=payload) + else: + message = '/submit API run in unit test mode' + logging.info(message) + status = 200 + payload_ret = {'dash_url':dash_url, 'payload':payload} + return status, message, payload_ret + + # Interpret response + status = result.status_code + if status == 200: + message = 'Results submitted successfully to dashboard at {0}.'.format(dash_server) + logging.info(message) + payload = {'identifier':uid} + else: + if 'Could not find any entity of type "accounts" matching' in result.json()['rejected'][0]['message']: + message = 'Error submitting results to dashboard at {0}. Check the dashboard user account API key is correct. Full dashboard response is: {1}.'.format(dash_server, result.json()) + else: + message = 'Error submitting results to dashboard at {0}. Full dashboard response is: {1}.'.format(dash_server, result.json()) + logging.error(message) + payload = None + + return status, message, payload + + def _get_elapsed_control_time_ratio(self): + '''Returns the elapsed control time ratio vector for the case. Parameters ---------- @@ -615,12 +1180,14 @@ def get_version(self): Returns ------- - version : dict - Version of BOPTEST as {'version': } + elapsed_control_time_ratio : np array of floats + elapsed_control_time_ratio for each control step. ''' - return {'version':self.version} + elapsed_control_time_ratio = self.elapsed_control_time_ratio + + return elapsed_control_time_ratio def _get_var_metadata(self, fmu, var_list, inputs=False): '''Build a dictionary of variables and their metadata. @@ -692,30 +1259,32 @@ def _check_value_min_max(self, var, value): ------ checked_value : float Value of variable truncated by min and max. - + message: str + Alert messages that input value is truncated to min/max ''' # Get minimum and maximum for variable mini = self.inputs_metadata[var]['Minimum'] maxi = self.inputs_metadata[var]['Maximum'] + message = None # Check the value and truncate if necessary if value > maxi: checked_value = maxi - print('WARNING: Value of {0} for {1} is above maximum of {2}. Using {2}.'.format(value, var, maxi)) + message = 'WARNING: value of {0} for {1} is above maximum of {2}. Using {2}. '.format(value, var, maxi) elif value < mini: checked_value = mini - print('WARNING: Value of {0} for {1} is below minimum of {2}. Using {2}.'.format(value, var, mini)) + message = 'WARNING: value of {0} for {1} is below minimum of {2}. Using {2}. '.format(value, var, mini) else: checked_value = value - return checked_value + return checked_value, message def _get_area(self): '''Get the building floor area in m^2. Returns ------- - area : float + area: float Building floor area in m^2 ''' @@ -729,7 +1298,7 @@ def _get_full_current_state(self): Returns ------- - z : dict + z: dict Combination of self.y and self.u dictionaries. ''' @@ -738,3 +1307,24 @@ def _get_full_current_state(self): z.update(self.u) return z + + def to_camel_case(self, snake_str): + components = snake_str.split('_') + # We capitalize the first letter of each component except the first one + # with the 'title' method and join them together. + return components[0] + ''.join(x.title() for x in components[1:]) + + def keys_to_camel_case(self, a_dict): + result = {} + for key, value in a_dict.items(): + result[self.to_camel_case(key)] = value + return result + + # weatherForecastUncertainty is required by the dashboard, + # however some testcases don't report it. + # This is a workaround + def add_forecast_uncertainty(self, scenario): + if not 'weatherForecastUncertainty' in scenario: + scenario['weatherForecastUncertainty'] = 'deterministic' + + return scenario diff --git a/testing/makefile b/testing/makefile index d1f7cd745..2ced62066 100755 --- a/testing/makefile +++ b/testing/makefile @@ -316,4 +316,4 @@ test_all: # Remove boptest base image make remove_boptest_image # Report test results - python report.py + python report.py \ No newline at end of file diff --git a/testing/references/bestest_air/submit.json b/testing/references/bestest_air/submit.json new file mode 100644 index 000000000..a17986bf9 --- /dev/null +++ b/testing/references/bestest_air/submit.json @@ -0,0 +1 @@ +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "bestest_air"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.0273067557575723, "emis_tot": 0.785588519629367, "ener_tot": 3.722732131848899, "idis_tot": 1220.1797359785448, "pdih_tot": null, "pele_tot": 0.010215810323849649, "pgas_tot": 0.12133181119488147, "tdis_tot": 5.687315396946064, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/bestest_hydronic/submit.json b/testing/references/bestest_hydronic/submit.json new file mode 100644 index 000000000..382f2169f --- /dev/null +++ b/testing/references/bestest_hydronic/submit.json @@ -0,0 +1 @@ +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "bestest_hydronic"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.465938501176076, "emis_tot": 1.628640744152675, "ener_tot": 9.000706489138203, "idis_tot": 0.0, "pdih_tot": null, "pele_tot": 0.00025517153990852034, "pgas_tot": 0.11798039028403248, "tdis_tot": 18.217583154564775, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/bestest_hydronic_heat_pump/submit.json b/testing/references/bestest_hydronic_heat_pump/submit.json new file mode 100644 index 000000000..d5097d5f9 --- /dev/null +++ b/testing/references/bestest_hydronic_heat_pump/submit.json @@ -0,0 +1 @@ +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "bestest_hydronic_heat_pump"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.8828705411678542, "emis_tot": 0.5808444758926209, "ener_tot": 3.4781106340875496, "idis_tot": 0.0, "pdih_tot": null, "pele_tot": 0.018913031427716383, "pgas_tot": null, "tdis_tot": 8.382467492017371, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/multizone_office_simple_air/submit.json b/testing/references/multizone_office_simple_air/submit.json new file mode 100644 index 000000000..8f99419d9 --- /dev/null +++ b/testing/references/multizone_office_simple_air/submit.json @@ -0,0 +1 @@ +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "multizone_office_simple_air"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.16537569781601175, "emis_tot": 0.6104861553474377, "ener_tot": 1.790281980491019, "idis_tot": 0.0, "pdih_tot": null, "pele_tot": 0.03334238212125819, "pgas_tot": null, "tdis_tot": 11.835569616547645, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/multizone_residential_hydronic/submit.json b/testing/references/multizone_residential_hydronic/submit.json new file mode 100644 index 000000000..697df510d --- /dev/null +++ b/testing/references/multizone_residential_hydronic/submit.json @@ -0,0 +1 @@ +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "multizone_residential_hydronic"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.7912501281574299, "emis_tot": 1.4253043873549793, "ener_tot": 8.140085161898376, "idis_tot": 9114.593818178417, "pdih_tot": null, "pele_tot": 0.0017390231869758264, "pgas_tot": 0.09592720495532536, "tdis_tot": 22.003355978645242, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/singlezone_commercial_hydronic/submit.json b/testing/references/singlezone_commercial_hydronic/submit.json new file mode 100644 index 000000000..03411c22e --- /dev/null +++ b/testing/references/singlezone_commercial_hydronic/submit.json @@ -0,0 +1 @@ +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "singlezone_commercial_hydronic"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.26318819272610183, "emis_tot": 0.3835824174014755, "ener_tot": 3.216803183654829, "idis_tot": 5.423240054209877, "pdih_tot": 0.08966901817018418, "pele_tot": 0.004907824412797784, "pgas_tot": null, "tdis_tot": 7.9949290180759505, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "peak_heat_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/testcase1/kpis_javascript.csv b/testing/references/testcase1/kpis_javascript.csv index d16eeab03..c0fc74621 100644 --- a/testing/references/testcase1/kpis_javascript.csv +++ b/testing/references/testcase1/kpis_javascript.csv @@ -1,10 +1,10 @@ keys,value -tdis_tot,10.632461888543997 -idis_tot,1016.9445000958008 -ener_tot,2.147326216620148 -cost_tot,0.1503128351634103 -emis_tot,0.4294652433240296 +tdis_tot,10.632461888544 +idis_tot,1016.9445000958 +ener_tot,2.14732621662015 +cost_tot,0.15031283516341 +emis_tot,0.42946524332403 pele_tot, -pgas_tot,0.0961576827281649 +pgas_tot,0.096157682728165 pdih_tot, -time_rat,0.0006078806712671 +time_rat,0.0005918023307566 diff --git a/testing/references/testcase1/submit.json b/testing/references/testcase1/submit.json new file mode 100644 index 000000000..f826d7abe --- /dev/null +++ b/testing/references/testcase1/submit.json @@ -0,0 +1 @@ +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "testcase1"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.8993115895520294, "emis_tot": 2.5694616844343696, "ener_tot": 12.847308422171846, "idis_tot": 7118.611500670603, "pdih_tot": null, "pele_tot": null, "pgas_tot": 0.11536491775500209, "tdis_tot": 503.7616200965347, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "test_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/testcase2/kpis_javascript.csv b/testing/references/testcase2/kpis_javascript.csv index 39d1edac6..a81857513 100644 --- a/testing/references/testcase2/kpis_javascript.csv +++ b/testing/references/testcase2/kpis_javascript.csv @@ -1,10 +1,10 @@ keys,value -tdis_tot,0.2857490911774727 -idis_tot,0.0 -ener_tot,2.387280414343804 -cost_tot,0.4774560828687608 -emis_tot,1.193640207171902 -pele_tot,0.0856042840563063 +tdis_tot,0.285749091177473 +idis_tot,0 +ener_tot,2.3872804143438 +cost_tot,0.477456082868761 +emis_tot,1.1936402071719 +pele_tot,0.085604284056306 pgas_tot, pdih_tot, -time_rat,0.0005853307426527 +time_rat,0.0004832534767963 diff --git a/testing/references/testcase2/submit.json b/testing/references/testcase2/submit.json new file mode 100644 index 000000000..e2468be4e --- /dev/null +++ b/testing/references/testcase2/submit.json @@ -0,0 +1 @@ +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "testcase2"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 1.5461632218795267, "emis_tot": 5.779914422546377, "ener_tot": 11.559828845092754, "idis_tot": 0.0, "pdih_tot": null, "pele_tot": 0.08727961692454471, "pgas_tot": null, "tdis_tot": 0.587950393326939, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "test_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/references/testcase3/submit.json b/testing/references/testcase3/submit.json new file mode 100644 index 000000000..69fad8e12 --- /dev/null +++ b/testing/references/testcase3/submit.json @@ -0,0 +1 @@ +{"dash_url": "https://api.boptest.net:8081//api/results", "payload": {"results": [{"account": {"apiKey": "valid_api_key"}, "boptestVersion": "0.x.x\n", "buildingType": {"uid": "testcase3"}, "controlStep": "86400.0", "dateRun": "2020-05-17 00:00:00", "forecastParameters": {"horizon": 3600.0, "interval": 300.0}, "isShared": true, "kpis": {"cost_tot": 0.9675513120754404, "emis_tot": 2.764432320215544, "ener_tot": 13.822161601077719, "idis_tot": 3606.22674562425, "pdih_tot": null, "pele_tot": null, "pgas_tot": 0.12017492256506475, "tdis_tot": 443.50197076537665, "time_rat": 0}, "scenario": {"electricityPrice": "dynamic", "timePeriod": "test_day", "weatherForecastUncertainty": "deterministic"}, "tags": ["baseline", "unit_test"], "uid": "1"}]}} diff --git a/testing/test_bestest_hydronic_heat_pump.py b/testing/test_bestest_hydronic_heat_pump.py index 91f8c42e1..8a163fc80 100644 --- a/testing/test_bestest_hydronic_heat_pump.py +++ b/testing/test_bestest_hydronic_heat_pump.py @@ -62,14 +62,14 @@ def test_event(self): # Initialize test case requests.put('{0}/initialize'.format(self.url), data={'start_time':start_time, 'warmup_period':0}) # Get default simulation step - step_def = requests.get('{0}/step'.format(self.url)).json() + step_def = requests.get('{0}/step'.format(self.url)).json()['payload'] # Simulation Loop for i in range(int(length/step_def)): # Advance simulation #switch pump on/off for each timestep pump = 0 if (i % 2) == 0 else 1 u = {'ovePum_activate':1, 'ovePum_u':pump} - requests.post('{0}/advance'.format(self.url), data=u).json() + requests.post('{0}/advance'.format(self.url), data=u).json()['payload'] # Check results points = self.get_all_points(self.url) df = self.results_to_df(points, start_time, start_time+length, self.url) diff --git a/testing/test_singlezone_commercial_hydronic.py b/testing/test_singlezone_commercial_hydronic.py index 31a65d855..dc5dacead 100644 --- a/testing/test_singlezone_commercial_hydronic.py +++ b/testing/test_singlezone_commercial_hydronic.py @@ -9,6 +9,7 @@ import os import requests import utilities +import pandas as pd class Run(unittest.TestCase, utilities.partialTestTimePeriod, utilities.partialTestSeason): '''Tests the example test case. @@ -59,14 +60,14 @@ def test_zero_flow(self): # Initialize test case requests.put('{0}/initialize'.format(self.url), data={'start_time':start_time, 'warmup_period':7*24*3600}) # Get default simulation step - step_def = requests.get('{0}/step'.format(self.url)).json() + step_def = requests.get('{0}/step'.format(self.url)).json()['payload'] # Simulation Loop for i in range(int(length/step_def)): # Advance simulation #switch pump on/off for each timestep pump = 0 if (i % 2) == 0 else 1 u = {'ovePum_activate':1, 'ovePum_u':pump} - requests.post('{0}/advance'.format(self.url), data=u).json() + requests.post('{0}/advance'.format(self.url), data=u).json()['payload'] # Check results points = self.get_all_points(self.url) df = self.results_to_df(points, start_time, start_time+length, self.url) diff --git a/testing/test_testcase1.py b/testing/test_testcase1.py index d45b8216f..95719dc96 100644 --- a/testing/test_testcase1.py +++ b/testing/test_testcase1.py @@ -15,6 +15,7 @@ from examples.python import testcase1 from examples.python import testcase1_scenario + class ExampleProportionalPython(unittest.TestCase, utilities.partialChecks): '''Tests the example test of proportional feedback controller in Python. @@ -163,7 +164,7 @@ def test_min(self): # Run test requests.put('{0}/initialize'.format(self.url), data={'start_time':0, 'warmup_period':0}) - y = requests.post('{0}/advance'.format(self.url), data={"oveAct_activate":1,"oveAct_u":-500000}).json() + y = requests.post('{0}/advance'.format(self.url), data={"oveAct_activate":1,"oveAct_u":-500000}).json()['payload'] # Check kpis value = float(y['PHea_y']) self.assertAlmostEqual(value, 10101.010101010103, places=3) @@ -175,7 +176,7 @@ def test_max(self): # Run test requests.put('{0}/initialize'.format(self.url), data={'start_time':0, 'warmup_period':0}) - y = requests.post('{0}/advance'.format(self.url), data={"oveAct_activate":1,"oveAct_u":500000}).json() + y = requests.post('{0}/advance'.format(self.url), data={"oveAct_activate":1,"oveAct_u":500000}).json()['payload'] # Check kpis value = float(y['PHea_y']) self.assertAlmostEqual(value, 10101.010101010103, places=3) @@ -198,20 +199,20 @@ def test_extra_step(self): ''' - scenario = {'time_period':'test_day'} + scenario = {'time_period': 'test_day'} requests.put('{0}/scenario'.format(self.url), data=scenario) # Try simulating past test period step = 7*24*3600 requests.put('{0}/step'.format(self.url), data={'step':step}) - for i in [0,1,2]: - y = requests.post('{0}/advance'.format(self.url), data={}).json() + for i in [0, 1, 2]: + y = requests.post('{0}/advance'.format(self.url), data={}).json()['payload'] # Check y[2] indicates no simulation (empty dict) self.assertDictEqual(y,dict()) # Check results points = self.get_all_points(self.url) df = self.results_to_df(points, -np.inf, np.inf, self.url) ref_filepath = os.path.join(utilities.get_root_path(), 'testing', 'references', self.name, 'results_time_period_end_extra_step.csv') - self.compare_ref_timeseries_df(df,ref_filepath) + self.compare_ref_timeseries_df(df, ref_filepath) def test_larger_step(self): '''Test that simulation stops if try to take larger step than scenario. @@ -223,7 +224,7 @@ def test_larger_step(self): # Try simulating past test period step = 5*7*24*3600 requests.put('{0}/step'.format(self.url), data={'step':step}) - requests.post('{0}/advance'.format(self.url), data={}).json() + requests.post('{0}/advance'.format(self.url), data={}) # Check results points = self.get_all_points(self.url) df = self.results_to_df(points, -np.inf, np.inf, self.url) @@ -235,11 +236,11 @@ def test_longer_initialize(self): ''' start_time = 14*86400 - requests.put('{0}/initialize'.format(self.url), data={'start_time':start_time, 'warmup_period':0}).json() + requests.put('{0}/initialize'.format(self.url), data={'start_time':start_time, 'warmup_period':0}) # Try simulating past a typical test period step = 5*7*24*3600 requests.put('{0}/step'.format(self.url), data={'step':step}) - y = requests.post('{0}/advance'.format(self.url), data={}).json() + y = requests.post('{0}/advance'.format(self.url), data={}).json()['payload'] # Check results self.assertEqual(y['time'], start_time+step) @@ -253,7 +254,7 @@ def test_return(self): scenario_time = {'time_period':'test_day'} scenario_elec = {'electricity_price':'dynamic'} # Both - res = requests.put('{0}/scenario'.format(self.url), data=scenario_both).json() + res = requests.put('{0}/scenario'.format(self.url), data=scenario_both).json()['payload'] # Check return is valid for electricity price self.assertTrue(res['electricity_price']) # Check return is valid for time period @@ -262,7 +263,7 @@ def test_return(self): ref_filepath = os.path.join(utilities.get_root_path(), 'testing', 'references', self.name, 'initial_values_set_scenario.csv') self.compare_ref_values_df(df, ref_filepath) # Time only - res = requests.put('{0}/scenario'.format(self.url), data=scenario_time).json() + res = (requests.put('{0}/scenario'.format(self.url), data=scenario_time).json()['payload']) # Check return is valid for electricity price self.assertTrue(res['electricity_price'] is None) # Check return is valid for time period @@ -271,7 +272,7 @@ def test_return(self): ref_filepath = os.path.join(utilities.get_root_path(), 'testing', 'references', self.name, 'initial_values_set_scenario.csv') self.compare_ref_values_df(df, ref_filepath) # Electricity price only - res = requests.put('{0}/scenario'.format(self.url), data=scenario_elec).json() + res = requests.put('{0}/scenario'.format(self.url), data=scenario_elec).json()['payload'] # Check return is valid for electricity price self.assertTrue(res['electricity_price']) # Check return is valid for time period @@ -296,12 +297,12 @@ def test_constant_step(self): # Run test requests.put('{0}/initialize'.format(self.url), data={'start_time':0, 'warmup_period':0}) - step = requests.get('{0}/step'.format(self.url)).json() - for i in range(5): - requests.post('{0}/advance'.format(self.url), data={}).json() + step = requests.get('{0}/step'.format(self.url)).json()['payload'] + for i in range(10): + requests.post('{0}/advance'.format(self.url), data={}) time.sleep(2) # Check kpis - kpi = requests.get('{0}/kpi'.format(self.url)).json() + kpi = requests.get('{0}/kpi'.format(self.url)).json()['payload'] self.assertAlmostEqual(kpi['time_rat'], 2.0/step, places=2) requests.put('{0}/step'.format(self.url), data={'step':step}) @@ -312,14 +313,14 @@ def test_variable_step(self): # Run test requests.put('{0}/initialize'.format(self.url), data={'start_time':0, 'warmup_period':0}) - step = requests.get('{0}/step'.format(self.url)).json() + step = requests.get('{0}/step'.format(self.url)).json()['payload'] for i in range(5): if i > 2: requests.put('{0}/step'.format(self.url), data={'step':2*step}) - requests.post('{0}/advance'.format(self.url), data={}).json() + requests.post('{0}/advance'.format(self.url), data={}) time.sleep(2) # Check kpis - kpi = requests.get('{0}/kpi'.format(self.url)).json() + kpi = requests.get('{0}/kpi'.format(self.url)).json()['payload'] self.assertAlmostEqual(kpi['time_rat'], (3*2.0/step+2*2.0/(2*step))/5, places=2) requests.put('{0}/step'.format(self.url), data={'step':step}) @@ -338,8 +339,9 @@ def setUp(self): self.name = 'testcase1' self.url = 'http://127.0.0.1:5000' - self.step_ref = 60.0 + self.step_ref = 60 self.test_time_period = 'test_day' + requests.put('{0}/step'.format(self.url), data={'step': self.step_ref}) if __name__ == '__main__': utilities.run_tests(os.path.basename(__file__)) diff --git a/testing/test_testcase2.py b/testing/test_testcase2.py index e34e69098..ace627c40 100644 --- a/testing/test_testcase2.py +++ b/testing/test_testcase2.py @@ -134,8 +134,8 @@ def test_min(self): ''' # Run test - requests.put('{0}/initialize'.format(self.url)) - y = requests.post('{0}/advance'.format(self.url), data={"oveTSetRooHea_activate":1,"oveTSetRooHea_u":273.15}).json() + requests.put('{0}/initialize'.format(self.url), data={'start_time':0, 'warmup_period':0}) + y = requests.post('{0}/advance'.format(self.url), data={"oveTSetRooHea_activate":1,"oveTSetRooHea_u":273.15}).json()['payload'] # Check kpis value = float(y['oveTSetRooHea_u']) self.assertAlmostEqual(value, 273.15+10, places=3) @@ -146,8 +146,8 @@ def test_max(self): ''' # Run test - requests.put('{0}/initialize'.format(self.url)) - y = requests.post('{0}/advance'.format(self.url), data={"oveTSetRooHea_activate":1,"oveTSetRooHea_u":310.15}).json() + requests.put('{0}/initialize'.format(self.url), data={'start_time':0, 'warmup_period':0}) + y = requests.post('{0}/advance'.format(self.url), data={"oveTSetRooHea_activate":1,"oveTSetRooHea_u":310.15}).json()['payload'] # Check kpis value = float(y['oveTSetRooHea_u']) self.assertAlmostEqual(value, 273.15+35, places=3) @@ -167,7 +167,7 @@ def setUp(self): self.name = 'testcase2' self.url = 'http://127.0.0.1:5000' - self.step_ref = 3600.0 + self.step_ref = 3600 self.test_time_period = 'test_day' if __name__ == '__main__': diff --git a/testing/test_testcase3.py b/testing/test_testcase3.py index e4977dcfb..7b6a842af 100644 --- a/testing/test_testcase3.py +++ b/testing/test_testcase3.py @@ -55,7 +55,7 @@ def setUp(self): self.name = 'testcase3' self.url = 'http://127.0.0.1:5000' - self.step_ref = 60.0 + self.step_ref = 60 self.test_time_period = 'test_day' if __name__ == '__main__': diff --git a/testing/utilities.py b/testing/utilities.py index 109bd0f16..46783c70a 100644 --- a/testing/utilities.py +++ b/testing/utilities.py @@ -13,6 +13,7 @@ import pandas as pd import re import matplotlib.pyplot as plt +from datetime import datetime def get_root_path(): @@ -353,9 +354,9 @@ def results_to_df(self, points, start_time, final_time, url='http://127.0.0.1:50 ---------- points: list of str List of points to retrieve from boptest api. - start_time: float + start_time: int Starting time of data to get in seconds. - final_time: float + final_time: int Ending time of data to get in seconds. url: str URL pointing to deployed boptest test case. @@ -370,7 +371,7 @@ def results_to_df(self, points, start_time, final_time, url='http://127.0.0.1:50 df = pd.DataFrame() for point in points: - res = requests.put('{0}/results'.format(url), data={'point_name':point,'start_time':start_time, 'final_time':final_time}).json() + res = requests.put('{0}/results'.format(url), data={'point_name':point,'start_time':start_time, 'final_time':final_time}).json()['payload'] df = pd.concat((df,pd.DataFrame(data=res[point], index=res['time'],columns=[point])), axis=1) df.index.name = 'time' @@ -392,12 +393,18 @@ def get_all_points(self, url='localhost:5000'): ''' - measurements = requests.get('{0}/measurements'.format(url)).json() - inputs = requests.get('{0}/inputs'.format(url)).json() + measurements = requests.get('{0}/measurements'.format(url)).json()['payload'] + inputs = requests.get('{0}/inputs'.format(url)).json()['payload'] points = list(measurements.keys()) + list(inputs.keys()) return points + def compare_error_code(self, response, message=None, code_ref=400): + status_code = response.status_code + if message is None: + message = response.message + self.assertEqual(status_code, code_ref, message) + class partialTestAPI(partialChecks): '''This partial class implements common API tests for test cases. @@ -424,7 +431,7 @@ def test_get_version(self): ''' # Get version from BOPTEST API - version = requests.get('{0}/version'.format(self.url)).json() + version = requests.get('{0}/version'.format(self.url)).json()['payload'] # Create a regex object as three decimal digits seperated by period r_num = re.compile('\d.\d.\d') r_x = re.compile('0.x.x') @@ -439,7 +446,7 @@ def test_get_name(self): ''' - name = requests.get('{0}/name'.format(self.url)).json() + name = requests.get('{0}/name'.format(self.url)).json()['payload'] self.assertEqual(name['name'], self.name) def test_get_inputs(self): @@ -447,7 +454,7 @@ def test_get_inputs(self): ''' - inputs = requests.get('{0}/inputs'.format(self.url)).json() + inputs = requests.get('{0}/inputs'.format(self.url)).json()['payload'] ref_filepath = os.path.join(get_root_path(), 'testing', 'references', self.name, 'get_inputs.json') self.compare_ref_json(inputs, ref_filepath) @@ -456,7 +463,7 @@ def test_get_measurements(self): ''' - measurements = requests.get('{0}/measurements'.format(self.url)).json() + measurements = requests.get('{0}/measurements'.format(self.url)).json()['payload'] ref_filepath = os.path.join(get_root_path(), 'testing', 'references', self.name, 'get_measurements.json') self.compare_ref_json(measurements, ref_filepath) @@ -465,7 +472,7 @@ def test_get_step(self): ''' - step = requests.get('{0}/step'.format(self.url)).json() + step = requests.get('{0}/step'.format(self.url)).json()['payload'] df = pd.DataFrame(data=[step], index=['step'], columns=['value']) df.index.name = 'keys' ref_filepath = os.path.join(get_root_path(), 'testing', 'references', self.name, 'get_step.csv') @@ -476,10 +483,10 @@ def test_set_step(self): ''' - step_current = requests.get('{0}/step'.format(self.url)).json() + step_current = requests.get('{0}/step'.format(self.url)).json()['payload'] step = 101 requests.put('{0}/step'.format(self.url), data={'step':step}) - step_set = requests.get('{0}/step'.format(self.url)).json() + step_set = requests.get('{0}/step'.format(self.url)).json()['payload'] self.assertEqual(step, step_set) requests.put('{0}/step'.format(self.url), data={'step':step_current}) @@ -491,10 +498,10 @@ def test_initialize(self): # Get measurements and inputs points = self.get_all_points(self.url) # Get current step - step = requests.get('{0}/step'.format(self.url)).json() + step = requests.get('{0}/step'.format(self.url)).json()['payload'] # Initialize start_time = 0.5*24*3600 - y = requests.put('{0}/initialize'.format(self.url), data={'start_time':start_time, 'warmup_period':0.5*24*3600}).json() + y = requests.put('{0}/initialize'.format(self.url), data={'start_time':start_time, 'warmup_period':0.5*24*3600}).json()['payload'] # Check that initialize returns the right initial values and results df = pd.DataFrame.from_dict(y, orient = 'index', columns=['value']) df.index.name = 'keys' @@ -507,7 +514,7 @@ def test_initialize(self): # Check results self.compare_ref_timeseries_df(df,ref_filepath) # Check kpis - res_kpi = requests.get('{0}/kpi'.format(self.url)).json() + res_kpi = requests.get('{0}/kpi'.format(self.url)).json()['payload'] df = pd.DataFrame.from_dict(res_kpi, orient='index', columns=['value']) df.index.name = 'keys' ref_filepath = os.path.join(get_root_path(), 'testing', 'references', self.name, 'kpis_initialize_initial.csv') @@ -515,7 +522,7 @@ def test_initialize(self): # Advance step_advance = 1*24*3600 requests.put('{0}/step'.format(self.url), data={'step':step_advance}) - y = requests.post('{0}/advance'.format(self.url),data = {}).json() + y = requests.post('{0}/advance'.format(self.url),data = {}).json()['payload'] # Check trajectories df = self.results_to_df(points, start_time, start_time+step_advance, self.url) # Set reference file path @@ -523,7 +530,7 @@ def test_initialize(self): # Check results self.compare_ref_timeseries_df(df,ref_filepath) # Check kpis - res_kpi = requests.get('{0}/kpi'.format(self.url)).json() + res_kpi = requests.get('{0}/kpi'.format(self.url)).json()['payload'] df = pd.DataFrame.from_dict(res_kpi, orient='index', columns=['value']) df.index.name = 'keys' ref_filepath = os.path.join(get_root_path(), 'testing', 'references', self.name, 'kpis_initialize_advance.csv') @@ -541,7 +548,7 @@ def test_advance_no_data(self): requests.put('{0}/initialize'.format(self.url), data={'start_time':0, 'warmup_period':0}) requests.put('{0}/step'.format(self.url), data={'step':self.step_ref}) - y = requests.post('{0}/advance'.format(self.url), data=dict()).json() + y = requests.post('{0}/advance'.format(self.url), data=dict()).json()['payload'] df = pd.DataFrame.from_dict(y, orient = 'index', columns=['value']) df.index.name = 'keys' ref_filepath = os.path.join(get_root_path(), 'testing', 'references', self.name, 'advance_no_data.csv') @@ -556,19 +563,23 @@ def test_advance_false_overwrite(self): ''' if self.name == 'testcase1': - u = {'oveAct_activate':0, 'oveAct_u':1500} + u = {'oveAct_activate': 0, 'oveAct_u': 1500} elif self.name == 'testcase2': - u = {'oveTSetRooHea_activate':0, 'oveTSetRooHea_u':273.15+22} + u = {'oveTSetRooHea_activate': 0, 'oveTSetRooHea_u': 273.15+22} elif self.name == 'testcase3': - u = {'oveActNor_activate':0, 'oveActNor_u':1500, - 'oveActSou_activate':0, 'oveActSou_u':1500} + u = {'oveActNor_activate': 0, 'oveActNor_u': 1500, + 'oveActSou_activate': 0, 'oveActSou_u': 1500} elif self.name == 'bestest_air': - u = {'fcu_oveTSup_activate':0, 'fcu_oveTSup_u':290} + u = {'fcu_oveTSup_activate': 0, 'fcu_oveTSup_u': 290} elif self.name == 'bestest_hydronic': - u = {'oveTSetSup_activate':0, 'oveTSetSup_u':273.15+60, - 'ovePum_activate':0, 'ovePum_u':1} + u = { + 'oveTSetSup_activate': 0, + 'oveTSetSup_u': 273.15+60, + 'ovePum_activate': 0, + 'ovePum_u': 1 + } elif self.name == 'bestest_hydronic_heat_pump': - u = {'oveTSet_activate':0, 'oveTSet_u':273.15+22} + u = {'oveTSet_activate': 0, 'oveTSet_u': 273.15+22} elif self.name == 'multizone_residential_hydronic': u = {'conHeaRo1_oveTSetHea_activate':0, 'conHeaRo1_oveTSetHea_u':273.15+22, 'oveEmiPum_activate':0, 'oveEmiPum_u':1} @@ -581,9 +592,9 @@ def test_advance_false_overwrite(self): raise Exception('Need to specify u for this test case') requests.put('{0}/initialize'.format(self.url), data={'start_time':0, 'warmup_period':0}) - requests.put('{0}/step'.format(self.url), data={'step':self.step_ref}) - y = requests.post('{0}/advance'.format(self.url), data=u).json() - df = pd.DataFrame.from_dict(y, orient = 'index', columns=['value']) + requests.put('{0}/step'.format(self.url), data={'step': self.step_ref}) + y = requests.post('{0}/advance'.format(self.url), data=u).json()['payload'] + df = pd.DataFrame.from_dict(y, orient='index', columns=['value']) df.index.name = 'keys' ref_filepath = os.path.join(get_root_path(), 'testing', 'references', self.name, 'advance_false_overwrite.csv') self.compare_ref_values_df(df, ref_filepath) @@ -598,7 +609,7 @@ def test_get_forecast_default(self): # Initialize requests.put('{0}/initialize'.format(self.url), data={'start_time':0, 'warmup_period':0}) # Test case forecast - forecast = requests.get('{0}/forecast'.format(self.url)).json() + forecast = requests.get('{0}/forecast'.format(self.url)).json()['payload'] df_forecaster = pd.DataFrame(forecast).set_index('time') # Set reference file path ref_filepath = os.path.join(get_root_path(), 'testing', 'references', self.name, 'get_forecast_default.csv') @@ -611,16 +622,16 @@ def test_put_and_get_parameters(self): ''' # Define forecast parameters - forecast_parameters_ref = {'horizon':3600, 'interval':300} + forecast_parameters_ref = {'horizon': 3600, 'interval':300} # Set forecast parameters ret = requests.put('{0}/forecast_parameters'.format(self.url), - data=forecast_parameters_ref) + data=forecast_parameters_ref).json()['payload'] # Get forecast parameters - forecast_parameters = requests.get('{0}/forecast_parameters'.format(self.url)).json() + forecast_parameters = requests.get('{0}/forecast_parameters'.format(self.url)).json()['payload'] # Check the forecast parameters self.assertDictEqual(forecast_parameters, forecast_parameters_ref) # Check the return on the put request - self.assertDictEqual(ret.json(), forecast_parameters_ref) + self.assertDictEqual(ret, forecast_parameters_ref) def test_get_forecast_with_parameters(self): '''Check that the forecaster is able to retrieve the data. @@ -630,14 +641,14 @@ def test_get_forecast_with_parameters(self): ''' # Define forecast parameters - forecast_parameters_ref = {'horizon':3600, 'interval':300} + forecast_parameters_ref = {'horizon': 3600, 'interval':300} # Initialize requests.put('{0}/initialize'.format(self.url), data={'start_time':0, 'warmup_period':0}) # Set forecast parameters requests.put('{0}/forecast_parameters'.format(self.url), - data=forecast_parameters_ref) + data=forecast_parameters_ref) # Test case forecast - forecast = requests.get('{0}/forecast'.format(self.url)).json() + forecast = requests.get('{0}/forecast'.format(self.url)).json()['payload'] df_forecaster = pd.DataFrame(forecast).set_index('time') # Set reference file path ref_filepath = os.path.join(get_root_path(), 'testing', 'references', self.name, 'get_forecast_with_parameters.csv') @@ -650,11 +661,11 @@ def test_set_get_scenario(self): ''' # Set scenario - scenario_current = requests.get('{0}/scenario'.format(self.url)).json() + scenario_current = requests.get('{0}/scenario'.format(self.url)).json()['payload'] scenario = {'electricity_price':'highly_dynamic', 'time_period':self.test_time_period} requests.put('{0}/scenario'.format(self.url), data=scenario) - scenario_set = requests.get('{0}/scenario'.format(self.url)).json() + scenario_set = requests.get('{0}/scenario'.format(self.url)).json()['payload'] self.assertEqual(scenario, scenario_set) # Check initialized correctly points = self.get_all_points(self.url) @@ -671,19 +682,29 @@ def test_set_get_scenario(self): # Return scenario to original requests.put('{0}/scenario'.format(self.url), data=scenario_current) - def test_partial_results_inner(self): '''Test getting results for start time after and final time before. ''' - - requests.put('{0}/initialize'.format(self.url), data={'start_time':0, 'warmup_period':0}) - requests.put('{0}/step'.format(self.url), data={'step':self.step_ref}) - measurements = requests.get('{0}/measurements'.format(self.url)).json() - requests.post('{0}/advance'.format(self.url), data=dict()).json() - res_inner = requests.put('{0}/results'.format(self.url), data={'point_name':list(measurements.keys())[0], \ - 'start_time':self.step_ref*0.25, \ - 'final_time':self.step_ref*0.75}).json() + measurement_list = {'testcase1': 'PHea_y', + 'testcase2': 'PFan_y', + 'testcase3': 'CO2RooAirSou_y', + 'bestest_hydronic':'reaQHea_y', + 'bestest_air':'zon_weaSta_reaWeaSolHouAng_y', + 'bestest_hydronic_heat_pump':'weaSta_reaWeaPAtm_y', + 'multizone_residential_hydronic':'weatherStation_reaWeaWinSpe_y', + 'singlezone_commercial_hydronic':'ahu_reaTRetAir_y', + 'multizone_office_simple_air':'hvac_reaAhu_PPumHea_y'} + requests.put('{0}/initialize'.format(self.url), data={'start_time': 0, 'warmup_period': 0}) + requests.put('{0}/step'.format(self.url), data={'step': self.step_ref}) + measurements = requests.get('{0}/measurements'.format(self.url)).json()['payload'] + y = requests.post('{0}/advance'.format(self.url), data=dict()).json()['payload'] + point = measurement_list[self.name] + if point not in measurements: + raise KeyError('Point {0} not in measurements list.'.format(point)) + res_inner = requests.put('{0}/results'.format(self.url), data={'point_name': point, + 'start_time': self.step_ref*0.25, + 'final_time': self.step_ref*0.75}).json()['payload'] df = pd.DataFrame.from_dict(res_inner).set_index('time') ref_filepath = os.path.join(get_root_path(), 'testing', 'references', self.name, 'partial_results_inner.csv') self.compare_ref_timeseries_df(df, ref_filepath) @@ -692,18 +713,294 @@ def test_partial_results_outer(self): '''Test getting results for start time before and final time after. ''' - - requests.put('{0}/initialize'.format(self.url), data={'start_time':0, 'warmup_period':0}) + measurement_list = {'testcase1': 'PHea_y', + 'testcase2': 'PFan_y', + 'testcase3': 'CO2RooAirSou_y', + 'bestest_hydronic':'reaQHea_y', + 'bestest_air':'zon_weaSta_reaWeaSolHouAng_y', + 'bestest_hydronic_heat_pump':'weaSta_reaWeaPAtm_y', + 'multizone_residential_hydronic':'weatherStation_reaWeaWinSpe_y', + 'singlezone_commercial_hydronic':'ahu_reaTRetAir_y', + 'multizone_office_simple_air':'hvac_reaAhu_PPumHea_y'} + requests.put('{0}/initialize'.format(self.url), data={'start_time': 0, 'warmup_period': 0}) requests.put('{0}/step'.format(self.url), data={'step':self.step_ref}) - measurements = requests.get('{0}/measurements'.format(self.url)).json() - requests.post('{0}/advance'.format(self.url), data=dict()).json() - res_outer = requests.put('{0}/results'.format(self.url), data={'point_name':list(measurements.keys())[0], \ - 'start_time':0-self.step_ref, \ - 'final_time':self.step_ref*2}).json() + measurements = requests.get('{0}/measurements'.format(self.url)).json()['payload'] + y = requests.post('{0}/advance'.format(self.url), data=dict()).json()['payload'] + point = measurement_list[self.name] + if point not in measurements: + raise KeyError('Point {0} not in measurements list.'.format(point)) + res_outer = requests.put('{0}/results'.format(self.url), data={'point_name': point, + 'start_time': 0-self.step_ref, + 'final_time': self.step_ref*2}).json()['payload'] df = pd.DataFrame.from_dict(res_outer).set_index('time') ref_filepath = os.path.join(get_root_path(), 'testing', 'references', self.name, 'partial_results_outer.csv') self.compare_ref_timeseries_df(df, ref_filepath) + def test_submit(self): + '''Test the submit API. + + ''' + + # Get current scenario and step + scenario_current = requests.get('{0}/scenario'.format(self.url)).json()['payload'] + step_current = requests.get('{0}/step'.format(self.url)).json()['payload'] + api_key = "valid_api_key" + # Set testing scenario + scenario = {"time_period":self.test_time_period, + "electricity_price":"dynamic"} + # Set test case scenario + y = requests.put("{0}/scenario".format(self.url), + data=scenario).json()["payload"]["time_period"] + # Set step so doesn't take too long + requests.put('{0}/step'.format(self.url), data={'step':86400}) + # Simulation Loop + while y: + # Compute control signal + u = {} + # Advance simulation with control signal + y = requests.post("{0}/advance".format(self.url), data=u).json()["payload"] + payload = requests.post("{0}/submit".format(self.url), data={"api_key": api_key, + "tag1":"baseline", + "tag2":"unit_test", + "unit_test":"True"}).json()['payload'] + payload['payload']['results'][0]['kpis']['time_rat'] = 0 + payload['payload']['results'][0]['uid'] = '1' + payload['payload']['results'][0]['dateRun'] = str(datetime(2020, 5, 17)) + ref_filepath = os.path.join(get_root_path(), 'testing', 'references', self.name, 'submit.json') + self.compare_ref_json(payload, ref_filepath) + # Return scenario and step to original + requests.put('{0}/scenario'.format(self.url), data=scenario_current) + requests.put('{0}/step'.format(self.url), data={'step':step_current}) + + def test_invalid_step(self): + '''Test set step with invalid non-numeric and negative values returns a 400 error. + + ''' + + # Try setting non-numeric step + step = "5*7*24*3600" + payload = requests.put('{0}/step'.format(self.url), data={'step': step}) + self.compare_error_code(payload, "Invalid step in set_step did not return 400 message.") + # Try setting negative step + step = -5*7*24*3600 + payload = requests.put('{0}/step'.format(self.url), data={'step': step}) + self.compare_error_code(payload, "Negative step int set_step did not return 400 message.") + + def test_invalid_forecast_parameters(self): + '''Check that the setting forecast parameter with invalid start or horizon returns 400 error. + + ''' + + # Try setting non-numeric horizon + forecast_parameters_ref = {'horizon': 'foo', 'interval': 300} + payload = requests.put('{0}/forecast_parameters'.format(self.url), + data=forecast_parameters_ref) + self.compare_error_code(payload, "Invalid horizon in forecast_parameters request did not return 400 message.") + # Try setting non-numeric interval + forecast_parameters_ref = {'horizon': 3600, 'interval': 'bar'} + payload = requests.put('{0}/forecast_parameters'.format(self.url), + data=forecast_parameters_ref) + self.compare_error_code(payload, "Invalid interval in forecast_parameters request did not return 400 message.") + # Try setting negative horizon + forecast_parameters_ref = {'horizon': -3600, 'interval': 300} + payload = requests.put('{0}/forecast_parameters'.format(self.url), + data=forecast_parameters_ref) + self.compare_error_code(payload, "Invalid interval in forecast_parameters request did not return 400 message.") + # Try setting negative interval + forecast_parameters_ref = {'horizon': 3600, 'interval': -300} + payload = requests.put('{0}/forecast_parameters'.format(self.url), + data=forecast_parameters_ref) + self.compare_error_code(payload, "Invalid interval in forecast_parameters request did not return 400 message.") + + def test_invalid_scenario(self): + '''Test setting scenario with invalid identifier returns 400 error. + + ''' + + # Set scenario + scenario_current = requests.get('{0}/scenario'.format(self.url)).json()['payload'] + # Try setting invalid electricity price + scenario = {'electricity_price': 'invalid_scenario', 'time_period': self.test_time_period} + payload = requests.put('{0}/scenario'.format(self.url), data=scenario) + self.compare_error_code(payload, + "Invalid value for electricity_price in set_scenario request did not return 400 message.") + # Try setting invalid time period + scenario = {'electricity_price': 'highly_dynamic', 'time_period': "invalid_time_period"} + payload = requests.put('{0}/scenario'.format(self.url), data=scenario) + self.compare_error_code(payload, + "Invalid value for time_period in set_scenario request did not return 400 message.") + # Return scenario to original + requests.put('{0}/scenario'.format(self.url), data=scenario_current) + + def test_invalid_initialize(self): + '''Test initialization of test simulation with invalid start_time and warmup_period returns 400 error. + + ''' + + # Try setting non-numeric start_time + start_time = "0.5 * 24 * 3600" + warmup_period = 0.5*24*3600 + y = requests.put('{0}/initialize'.format(self.url), + data={'start_time': start_time, 'warmup_period': warmup_period}) + self.compare_error_code(y, "Invalid start_time to initialize request did not return 400 message.") + # Try setting non-numeric warmup_period + start_time = 0.5*24*3600 + warmup_period = "0.5 * 24 * 3600" + y = requests.put('{0}/initialize'.format(self.url), + data={'start_time': start_time, 'warmup_period': warmup_period}) + self.compare_error_code(y, "Invalid warmup_period in initialize request did not return 400 message.") + # Try setting negative start_time + start_time = -0.5*24*3600 + warmup_period = 0.5*24*3600 + y = requests.put('{0}/initialize'.format(self.url), + data={'start_time': start_time, 'warmup_period': warmup_period}) + self.compare_error_code(y, "Negative start_time in initialize request did not return 400 message.") + # Try setting negative warmup_period + start_time = 0.5*24*3600 + warmup_period = -0.5*24*3600 + y = requests.put('{0}/initialize'.format(self.url), + data={'start_time': start_time, 'warmup_period': warmup_period}) + self.compare_error_code(y, "Negative warmup_period in initialize request did not return 400 message.") + + def test_invalid_advance_value(self): + '''Test advancing of simulation with invalid input data type (non-numerical) will return 400 error. + + This is a basic test of functionality. + + ''' + + if self.name == 'testcase1': + u = {'oveAct_activate': 0, 'oveAct_u': 1500} + elif self.name == 'testcase2': + u = {'oveTSetRooHea_activate': 0, 'oveTSetRooHea_u': 273.15 + 22} + elif self.name == 'testcase3': + u = {'oveActNor_activate': 0, 'oveActNor_u': 1500, + 'oveActSou_activate': 0, 'oveActSou_u': 1500} + elif self.name == 'bestest_air': + u = {'fcu_oveTSup_activate': 0, 'fcu_oveTSup_u': 290} + elif self.name == 'bestest_hydronic': + u = {'oveTSetSup_activate': 0, + 'oveTSetSup_u': 273.15 + 60, + 'ovePum_activate': 0} + elif self.name == 'bestest_hydronic_heat_pump': + u = {'oveTSet_activate': 0, 'oveTSet_u': 273.15 + 22} + elif self.name == 'multizone_residential_hydronic': + u = {'conHeaRo1_oveTSetHea_activate': 0, 'conHeaRo1_oveTSetHea_u': 273.15 + 22, + 'oveEmiPum_activate': 0, 'oveEmiPum_u': 1} + elif self.name == 'singlezone_commercial_hydronic': + u = {'oveTSupSet_activate': 0, 'oveTSupSet_u': 273.15 + 25, + 'oveTZonSet_activate': 0, 'oveTZonSet_u': 273.15 + 25} + elif self.name == 'multizone_office_simple_air': + u = {'hvac_oveAhu_TSupSet_activate': 0, 'hvac_oveAhu_TSupSet_u': 273.15 + 22} + else: + raise Exception('Need to specify u for this test case') + for key, value in u.items(): + if '_activate' in key: + for value in ['invalid', 1.2, '1.2']: + u[key] = value + y = requests.post('{0}/advance'.format(self.url), data=u) + self.compare_error_code(y, "Invalid advance request for _activate did not return 400 message.") + else: + u[key] = "invalid" + y = requests.post('{0}/advance'.format(self.url), data=u) + self.compare_error_code(y, "Invalid advance request for _u did not return 400 message.") + + def test_invalid_advance_name(self): + '''Test advancing of simulation with invalid input parameter name will return 400 error. + + This is a basic test of functionality. + + ''' + + u = {'invalid': 0} + y = requests.post('{0}/advance'.format(self.url), data=u) + self.compare_error_code(y, "Invalid advance request for _u did not return 400 message.") + + + def test_invalid_get_results(self): + '''Test getting results for start time before and final time after. + + ''' + measurement_list = {'testcase1': 'PHea_y', + 'testcase2': 'PFan_y', + 'testcase3': 'CO2RooAirSou_y', + 'bestest_hydronic':'reaQHea_y', + 'bestest_air':'zon_weaSta_reaWeaSolHouAng_y', + 'bestest_hydronic_heat_pump':'weaSta_reaWeaPAtm_y', + 'multizone_residential_hydronic':'weatherStation_reaWeaWinSpe_y', + 'singlezone_commercial_hydronic':'ahu_reaTRetAir_y', + 'multizone_office_simple_air':'hvac_reaAhu_PPumHea_y'} + requests.put('{0}/initialize'.format(self.url), data={'start_time': 0, 'warmup_period': 0}) + requests.put('{0}/step'.format(self.url), data={'step':self.step_ref}) + measurements = requests.get('{0}/measurements'.format(self.url)).json()['payload'] + requests.post('{0}/advance'.format(self.url), data=dict()).json()['payload'] + point = measurement_list[self.name] + if point not in measurements: + raise KeyError('Point {0} not in measurements list.'.format(point)) + # Try getting invalid start_time + res = requests.put('{0}/results'.format(self.url), data={'point_name': point, + 'start_time': "foo", + 'final_time': self.step_ref*2}) + self.compare_error_code(res, "Invalid start_time in get_results request did not return a 400 error.") + # Try getting invalid final_time + res = requests.put('{0}/results'.format(self.url), data={'point_name': point, + 'start_time': 0.0 - self.step_ref, + 'final_time': "foo"}) + self.compare_error_code(res, "Invalid final_time in get_results request did not return a 400 error.") + # Try getting invalid point_name + res = requests.put('{0}/results'.format(self.url), data={'point_name': "foo", + 'start_time': 0.0 - self.step_ref, + 'final_time': self.step_ref*2.0}) + self.compare_error_code(res, "Invalid point_name in get_results request did not return a 400 error.") + + def test_invalid_submit(self): + '''Test the submit API with invalid usage. + + ''' + + # Get current scenario and step + scenario_current = requests.get('{0}/scenario'.format(self.url)).json()['payload'] + step_current = requests.get('{0}/step'.format(self.url)).json()['payload'] + api_key = "valid_api_key" + # Set testing scenario + scenario = {"time_period":self.test_time_period, + "electricity_price":"dynamic"} + # Set test case scenario + y = requests.put("{0}/scenario".format(self.url), + data=scenario).json()["payload"]["time_period"] + # Set step so doesn't take too long + requests.put('{0}/step'.format(self.url), data={'step':86400}) + # Simulation Loop + while y: + # Compute control signal + u = {} + # Advance simulation with control signal but stop after one iteration + y = requests.post("{0}/advance".format(self.url), data=u).json()["payload"] + y = False + res = requests.post("{0}/submit".format(self.url), data={"api_key": api_key, + "tag1":"baseline", + "tag2":"unit_test", + "unit_test":"True"}) + self.compare_error_code(res, "Invalid time run in submit request did not return a 500 error.", code_ref=500) + # Continue simulation Loop + y = True + while y: + # Compute control signal + u = {} + # Advance simulation to end of time period scenario + y = requests.post("{0}/advance".format(self.url), data=u).json()["payload"] + # Test invalid tag number + res = requests.post("{0}/submit".format(self.url), data={"api_key": api_key, + "tag1":"1", "tag2":"2", "tag3":"3", + "tag4":"4", "tag5":"5", "tag6":"6", + "tag7":"7", "tag8":"2", "tag9":"3", + "tag10":"1", "tag11":"2", + "unit_test":"True"}) + self.compare_error_code(res, "Invalid tag number in submit request did not return a 400 error.") + # Return scenario and step to original + requests.put('{0}/scenario'.format(self.url), data=scenario_current) + requests.put('{0}/step'.format(self.url), data={'step':step_current}) + class partialTestTimePeriod(partialChecks): '''Partial class for testing the time periods for each test case @@ -729,7 +1026,7 @@ def run_time_period(self, time_period): y = 1 while y: # Advance simulation - y = requests.post('{0}/advance'.format(self.url), data={}).json() + y = requests.post('{0}/advance'.format(self.url), data={}).json()['payload'] # Check results df = self.results_to_df(self.points_check, -np.inf, np.inf, self.url) ref_filepath = os.path.join(get_root_path(), 'testing', 'references', self.name, 'results_{0}.csv'.format(time_period)) @@ -739,7 +1036,7 @@ def run_time_period(self, time_period): # Set scenario requests.put('{0}/scenario'.format(self.url), data={'electricity_price':price_scenario}) # Report kpis - res_kpi = requests.get('{0}/kpi'.format(self.url)).json() + res_kpi = requests.get('{0}/kpi'.format(self.url)).json()['payload'] # Check kpis df = pd.DataFrame.from_dict(res_kpi, orient='index', columns=['value']) df.index.name = 'keys' @@ -779,11 +1076,11 @@ def run_season(self, season): # Initialize test case requests.put('{0}/initialize'.format(self.url), data={'start_time':start_time, 'warmup_period':0}) # Get default simulation step - step_def = requests.get('{0}/step'.format(self.url)).json() + step_def = requests.get('{0}/step'.format(self.url)).json()['payload'] # Simulation Loop for i in range(int(length/step_def)): # Advance simulation - requests.post('{0}/advance'.format(self.url), data={}).json() + requests.post('{0}/advance'.format(self.url), data={}).json()['payload'] requests.put('{0}/scenario'.format(self.url), data={'electricity_price':'constant'}) # Check results points = self.get_all_points(self.url) @@ -795,7 +1092,7 @@ def run_season(self, season): # Set scenario requests.put('{0}/scenario'.format(self.url), data={'electricity_price':price_scenario}) # Report kpis - res_kpi = requests.get('{0}/kpi'.format(self.url)).json() + res_kpi = requests.get('{0}/kpi'.format(self.url)).json()['payload'] # Check kpis df = pd.DataFrame.from_dict(res_kpi, orient='index', columns=['value']) df.index.name = 'keys'