Skip to content

Commit

Permalink
updated calculations
Browse files Browse the repository at this point in the history
  • Loading branch information
@jrfep
jrfep committed Jun 7, 2024
1 parent 640b089 commit c9efdef
Show file tree
Hide file tree
Showing 220 changed files with 2,860 additions and 70 deletions.
2 changes: 1 addition & 1 deletion _freeze/how2/additional-map-details/execute-results/html.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"hash": "5d66f62e41dbc687e88bebfe7a3b33eb",
"result": {
"engine": "knitr",
"markdown": "---\ntitle: Map information in database\nexecute: \n eval: true\n---\n\n\nWe will query some information regarding map versions and metadata that is stored in our internal database. \n\n## Set up work directory\n\nWe use library `here` to define relative paths\n\n::: {.cell}\n\n```{.r .cell-code}\nhere::i_am(\"how2/additional-map-details.qmd\")\n```\n\n::: {.cell-output .cell-output-stderr}\n\n```\nhere() starts at /Users/z3529065/proyectos/typology-website/typology-map-info\n```\n\n\n:::\n:::\n\n\n## Database\n\n### Database connection\n\nWe store databse credentials in the `_environment.local` file in the project directory. See file `_environment.required` for the format. \n\n\n::: {.cell}\n\n```{.r .cell-code}\nlibrary(dplyr)\nlibrary(stringr)\nlibrary(tidyr)\nlibrary(lubridate)\nlibrary(readr)\nlibrary(\"RPostgreSQL\")\n\ndrv <- dbDriver(\"PostgreSQL\") ## remember to update .pgpass file\n\ncon <- dbConnect(drv, \n dbname = Sys.getenv(\"DBNAME\"),\n host = Sys.getenv(\"DBHOST\"),\n port = Sys.getenv(\"DBPORT\"),\n user = Sys.getenv(\"DBUSER\"))\n```\n:::\n\n \n### Queries\n\n#### Biome names\nThis information is actually missing from the profile information workbook:\n\n\n::: {.cell}\n\n```{.r .cell-code}\nqry <- \"select biome_code, name from biomes;\"\nbiome_info <- dbGetQuery(con,qry)\nwrite_csv(biome_info, file = here::here(\"gisdata\",\"profiles\",\"biome-names.csv\"))\n```\n:::\n\n::: {.cell}\n\n```{.r .cell-code}\nqry <- \"\nSELECT code, map_code, map_version, license \nFROM map_evaluation \nLEFT JOIN map_metadata \nUSING(map_code, map_version) \nWHERE status = 'valid'\nAND map_type = 'Indicative Map'\nORDER BY license, map_code;\n\"\nmaps_licenses <- dbGetQuery(con,qry)\n\nmaps_licenses |> \n mutate(biome = str_extract(code, \"[MFST0-9]+\")) |> \n group_by(biome, license) |> \n summarise(total = n_distinct(code), .groups = 'drop') |> \n pivot_wider(names_from=license,values_from = total) |>\n DT::datatable()\n```\n\n::: {.cell-output-display}\n\n```{=html}\n<div class=\"datatables html-widget html-fill-item\" id=\"htmlwidget-7d4cf6cd3c18ccf34aa5\" style=\"width:100%;height:auto;\"></div>\n<script type=\"application/json\" data-for=\"htmlwidget-7d4cf6cd3c18ccf34aa5\">{\"x\":{\"filter\":\"none\",\"vertical\":false,\"data\":[[\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\",\"8\",\"9\",\"10\",\"11\",\"12\",\"13\",\"14\",\"15\",\"16\",\"17\",\"18\",\"19\",\"20\",\"21\",\"22\",\"23\",\"24\",\"25\"],[\"F1\",\"F2\",\"F3\",\"FM1\",\"M1\",\"M2\",\"M3\",\"M4\",\"MFT1\",\"MT1\",\"MT2\",\"MT3\",\"S1\",\"S2\",\"SF1\",\"SF2\",\"SM1\",\"T1\",\"T2\",\"T3\",\"T4\",\"T5\",\"T6\",\"T7\",\"TF1\"],[1,1,4,null,5,null,6,null,1,2,1,null,null,1,null,null,null,1,1,2,3,null,1,3,4],[6,9,1,3,4,5,null,2,1,2,1,1,null,null,1,1,3,3,4,2,1,4,4,2,2],[null,null,null,null,null,null,1,null,null,null,null,null,1,null,1,1,null,null,null,null,1,1,null,null,null],[null,null,null,null,null,null,null,null,1,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],[null,null,null,null,null,null,null,null,null,null,null,null,1,null,null,null,null,null,1,null,null,null,null,null,null]],\"container\":\"<table class=\\\"display\\\">\\n <thead>\\n <tr>\\n <th> <\\/th>\\n <th>biome<\\/th>\\n <th>Commercial use<\\/th>\\n <th>Non-commercial use<\\/th>\\n <th>Undocumented<\\/th>\\n <th>See contact info<\\/th>\\n <th>NA<\\/th>\\n <\\/tr>\\n <\\/thead>\\n<\\/table>\",\"options\":{\"columnDefs\":[{\"className\":\"dt-right\",\"targets\":[2,3,4,5,6]},{\"orderable\":false,\"targets\":0},{\"name\":\" \",\"targets\":0},{\"name\":\"biome\",\"targets\":1},{\"name\":\"Commercial use\",\"targets\":2},{\"name\":\"Non-commercial use\",\"targets\":3},{\"name\":\"Undocumented\",\"targets\":4},{\"name\":\"See contact info\",\"targets\":5},{\"name\":\"NA\",\"targets\":6}],\"order\":[],\"autoWidth\":false,\"orderClasses\":false}},\"evals\":[],\"jsHooks\":[]}</script>\n```\n\n:::\n:::\n\n\n### Close connection\n\n::: {.cell}\n\n```{.r .cell-code}\ndbDisconnect(con)\n```\n\n::: {.cell-output .cell-output-stdout}\n\n```\n[1] TRUE\n```\n\n\n:::\n:::",
"markdown": "---\ntitle: Map information in database\nexecute: \n eval: true\n---\n\n\nWe will query some information regarding map versions and metadata that is stored in our internal database. \n\n## Set up work directory\n\nWe use library `here` to define relative paths\n\n::: {.cell}\n\n```{.r .cell-code}\nhere::i_am(\"how2/additional-map-details.qmd\")\n```\n\n::: {.cell-output .cell-output-stderr}\n\n```\nhere() starts at /Users/z3529065/proyectos/typology-website/typology-map-info\n```\n\n\n:::\n:::\n\n\n## Database\n\n### Database connection\n\nWe store databse credentials in the `_environment.local` file in the project directory. See file `_environment.required` for the format. \n\n\n::: {.cell}\n\n```{.r .cell-code}\nlibrary(dplyr)\nlibrary(stringr)\nlibrary(tidyr)\nlibrary(lubridate)\nlibrary(readr)\nlibrary(\"RPostgreSQL\")\n\ndrv <- dbDriver(\"PostgreSQL\") ## remember to update .pgpass file\n\ncon <- dbConnect(drv, \n dbname = Sys.getenv(\"DBNAME\"),\n host = Sys.getenv(\"DBHOST\"),\n port = Sys.getenv(\"DBPORT\"),\n user = Sys.getenv(\"DBUSER\"))\n```\n:::\n\n \n### Queries\n\n#### Biome names\nThis information is actually missing from the profile information workbook:\n\n\n::: {.cell}\n\n```{.r .cell-code}\nqry <- \"select biome_code, name from biomes;\"\nbiome_info <- dbGetQuery(con,qry)\nwrite_csv(biome_info, file = here::here(\"gisdata\",\"profiles\",\"biome-names.csv\"))\n```\n:::\n\n::: {.cell}\n\n```{.r .cell-code}\nqry <- \"\nSELECT code, map_code, map_version, license \nFROM map_evaluation \nLEFT JOIN map_metadata \nUSING(map_code, map_version) \nWHERE status = 'valid'\nAND map_type = 'Indicative Map'\nORDER BY license, map_code;\n\"\nmaps_licenses <- dbGetQuery(con,qry)\n\nmaps_licenses |> \n mutate(biome = str_extract(code, \"[MFST0-9]+\")) |> \n group_by(biome, license) |> \n summarise(total = n_distinct(code), .groups = 'drop') |> \n pivot_wider(names_from=license,values_from = total) |>\n DT::datatable()\n```\n\n::: {.cell-output-display}\n\n```{=html}\n<div class=\"datatables html-widget html-fill-item\" id=\"htmlwidget-003889b05e2bddd6f05c\" style=\"width:100%;height:auto;\"></div>\n<script type=\"application/json\" data-for=\"htmlwidget-003889b05e2bddd6f05c\">{\"x\":{\"filter\":\"none\",\"vertical\":false,\"data\":[[\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\",\"8\",\"9\",\"10\",\"11\",\"12\",\"13\",\"14\",\"15\",\"16\",\"17\",\"18\",\"19\",\"20\",\"21\",\"22\",\"23\",\"24\",\"25\"],[\"F1\",\"F2\",\"F3\",\"FM1\",\"M1\",\"M2\",\"M3\",\"M4\",\"MFT1\",\"MT1\",\"MT2\",\"MT3\",\"S1\",\"S2\",\"SF1\",\"SF2\",\"SM1\",\"T1\",\"T2\",\"T3\",\"T4\",\"T5\",\"T6\",\"T7\",\"TF1\"],[1,1,4,null,5,null,6,null,1,2,1,null,null,1,null,null,null,1,1,2,3,null,1,3,4],[6,9,1,3,4,5,null,2,1,2,1,1,null,null,1,1,3,3,4,2,1,4,4,2,2],[null,null,null,null,null,null,1,null,null,null,null,null,1,null,1,1,null,null,null,null,1,1,null,null,null],[null,null,null,null,null,null,null,null,1,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null],[null,null,null,null,null,null,null,null,null,null,null,null,1,null,null,null,null,null,1,null,null,null,null,null,null]],\"container\":\"<table class=\\\"display\\\">\\n <thead>\\n <tr>\\n <th> <\\/th>\\n <th>biome<\\/th>\\n <th>Commercial use<\\/th>\\n <th>Non-commercial use<\\/th>\\n <th>Undocumented<\\/th>\\n <th>See contact info<\\/th>\\n <th>NA<\\/th>\\n <\\/tr>\\n <\\/thead>\\n<\\/table>\",\"options\":{\"columnDefs\":[{\"className\":\"dt-right\",\"targets\":[2,3,4,5,6]},{\"orderable\":false,\"targets\":0},{\"name\":\" \",\"targets\":0},{\"name\":\"biome\",\"targets\":1},{\"name\":\"Commercial use\",\"targets\":2},{\"name\":\"Non-commercial use\",\"targets\":3},{\"name\":\"Undocumented\",\"targets\":4},{\"name\":\"See contact info\",\"targets\":5},{\"name\":\"NA\",\"targets\":6}],\"order\":[],\"autoWidth\":false,\"orderClasses\":false}},\"evals\":[],\"jsHooks\":[]}</script>\n```\n\n:::\n:::\n\n\n### Close connection\n\n::: {.cell}\n\n```{.r .cell-code}\ndbDisconnect(con)\n```\n\n::: {.cell-output .cell-output-stdout}\n\n```\n[1] TRUE\n```\n\n\n:::\n:::",
"supporting": [],
"filters": [
"rmarkdown/pagebreak.lua"
Expand Down
8 changes: 3 additions & 5 deletions _freeze/how2/populate-templates-R/execute-results/html.json
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
{
"hash": "81c8be8e56b516e84eb985100b2f7dbd",
"hash": "02e8c8f57b69d3c62319a2f38952eb47",
"result": {
"engine": "knitr",
"markdown": "---\ntitle: Populate templates with R\nexecute: \n eval: true\n---\n\n\n## Set up work directory\n\nWe use library `here` to define relative paths\n\n::: {.cell}\n\n```{.r .cell-code}\nhere::i_am(\"how2/populate-templates-R.qmd\")\n```\n\n::: {.cell-output .cell-output-stderr}\n\n```\nhere() starts at /Users/z3529065/proyectos/typology-website/typology-map-info\n```\n\n\n:::\n:::\n\n\n## Read profile information\n\nWe read the profile information from the workbook downloaded from OSF and complement this with information about the biomes.\n\n\n::: {.cell}\n\n```{.r .cell-code}\nlibrary(readxl)\nlibrary(dplyr)\nlibrary(readr)\n\nout_folder <- here::here(\"gisdata\", \"profiles\")\ndest_file <- here::here(out_folder, \"biome-names.csv\")\nbiome_names <- read_csv(dest_file, show_col_types = FALSE) |>\n rename(\"biome code\" = \"biome_code\" , \"biome name\" = \"name\") \n\ndest_file <- here::here(out_folder, \"IUCN-GET-profiles-exported.xlsx\")\nprofile_info <- read_excel(dest_file, sheet = 2) |> \n left_join(biome_names, by = \"biome code\")\n```\n:::\n\n\n## Output initial qmd file\n\nNow we can export this information into a template `qmd` file to show the results for all functional groups.\n\n\n::: {.cell}\n\n```{.r .cell-code}\ntemplate_text <- \"---\ntitle: \\\"%s\\\"\nsubtitle: \\\"%s\\\"\nparams:\n efgcode: %s\ncategories:\n - %s\nformat:\n html: default\nexecute:\n echo: false\n---\n\n{{< include _short-descriptions.qmd >}}\n{{< include _map.qmd >}}\n{{< include _profile-versions.qmd >}}\n\"\n\noutdata <- profile_info |> \n mutate(out = sprintf(template_text, \n name, `biome name`, code, `biome code`), \n outfile = here::here(\"efg-info\", sprintf(\"%s.qmd\", gsub(\"\\\\.\", \"_\", code)))) |>\n select(outfile, out) \n\nfor (j in seq(along=outdata$out)) {\n outfile <- slice(outdata,j) |> pull(outfile)\n outtext <- slice(outdata,j) |> pull(out)\n \n if (!file.exists(outfile)) {\n cat(outtext, file = outfile)\n }\n}\n```\n:::\n\n\nAnd similar for all valid maps:\n\n\n::: {.cell}\n\n```{.r .cell-code}\ntoday <- format(Sys.time(), \"%d %b %Y\")\ntemplate_text <- \"---\ntitle: \\\"Valid map for EFG %s\\\"\nsubtitle: \\\"%s\\\"\ndate: %s\nparams:\n efgcode: %s\ncategories:\n - %s\n - valid\nformat:\n html: default\nexecute:\n echo: false\n---\n\n{{< include _valid_map_info.qmd >}}\n{{< include _valid_maps_database.qmd >}}\n\"\n\noutdata <- profile_info |> \n mutate(out = sprintf(template_text, \n `short name`, `biome name`, today, code, `biome code`), \n outfile = here::here(\"map-info\", sprintf(\"%s_valid.qmd\", gsub(\"\\\\.\", \"_\", code)))) |>\n select(outfile, out) \n\nfor (j in seq(along=outdata$out)) {\n outfile <- slice(outdata,j) |> pull(outfile)\n outtext <- slice(outdata,j) |> pull(out)\n \n if (!file.exists(outfile)) {\n cat(outtext, file = outfile)\n }\n}\n```\n:::",
"supporting": [
"populate-templates-R_files"
],
"markdown": "---\ntitle: Populate templates with R\nexecute: \n eval: true\n---\n\n\n## Set up work directory\n\nWe use library `here` to define relative paths\n\n::: {.cell}\n\n```{.r .cell-code}\nhere::i_am(\"how2/populate-templates-R.qmd\")\n```\n\n::: {.cell-output .cell-output-stderr}\n\n```\nhere() starts at /Users/z3529065/proyectos/typology-website/typology-map-info\n```\n\n\n:::\n:::\n\n\n## Read profile information\n\nWe read the profile information from the workbook downloaded from OSF and complement this with information about the biomes.\n\n\n::: {.cell}\n\n```{.r .cell-code}\nlibrary(readxl)\nlibrary(dplyr)\nlibrary(readr)\n\nout_folder <- here::here(\"gisdata\", \"profiles\")\ndest_file <- here::here(out_folder, \"biome-names.csv\")\nbiome_names <- read_csv(dest_file, show_col_types = FALSE) |>\n rename(\"biome code\" = \"biome_code\" , \"biome name\" = \"name\") \n\ndest_file <- here::here(out_folder, \"IUCN-GET-profiles-exported.xlsx\")\nprofile_info <- read_excel(dest_file, sheet = 2) |> \n left_join(biome_names, by = \"biome code\")\n```\n:::\n\n\n## Output initial qmd file\n\nNow we can export this information into a template `qmd` file to show the results for all functional groups.\n\n\n::: {.cell}\n\n```{.r .cell-code}\ntemplate_text <- \"---\ntitle: \\\"%s\\\"\nsubtitle: \\\"%s\\\"\nparams:\n efgcode: %s\ncategories:\n - %s\nformat:\n html: default\nexecute:\n echo: false\n---\n\n{{< include _short-descriptions.qmd >}}\n{{< include _map.qmd >}}\n{{< include _profile-versions.qmd >}}\n\"\n\noutdata <- profile_info |> \n mutate(out = sprintf(template_text, \n name, `biome name`, code, `biome code`), \n outfile = here::here(\"efg-info\", sprintf(\"%s.qmd\", gsub(\"\\\\.\", \"_\", code)))) |>\n select(outfile, out) \n\nfor (j in seq(along=outdata$out)) {\n outfile <- slice(outdata,j) |> pull(outfile)\n outtext <- slice(outdata,j) |> pull(out)\n \n if (!file.exists(outfile)) {\n cat(outtext, file = outfile)\n }\n}\n```\n:::\n\n\nAnd similar for all valid maps:\n\n\n::: {.cell}\n\n```{.r .cell-code}\ntoday <- format(Sys.time(), \"%d %b %Y\")\ntemplate_text <- \"---\ntitle: \\\"EFG %s -- current maps\\\"\nsubtitle: \\\"%s\\\"\ndate: %s\nparams:\n efgcode: %s\ncategories:\n - %s\n - valid\nformat:\n html: default\nexecute:\n echo: false\n---\n\n{{< include _valid_map_info.qmd >}}\n{{< include _valid_maps_database.qmd >}}\n\"\n\noutdata <- profile_info |> \n mutate(out = sprintf(template_text, \n `short name`, `biome name`, today, code, `biome code`), \n outfile = here::here(\"map-info\", sprintf(\"%s_valid.qmd\", gsub(\"\\\\.\", \"_\", code)))) |>\n select(outfile, out) \n\nfor (j in seq(along=outdata$out)) {\n outfile <- slice(outdata,j) |> pull(outfile)\n outtext <- slice(outdata,j) |> pull(out)\n \n if (!file.exists(outfile)) {\n cat(outtext, file = outfile)\n }\n}\n```\n:::\n\n\nAnd the complete version history of the maps.\n\n\n::: {.cell}\n\n```{.r .cell-code}\ntemplate_text <- \"---\ntitle: \\\"EFG %s -- previous maps\\\"\nsubtitle: \\\"%s\\\"\ndate: %s\nparams:\n efgcode: %s\n efgname: \\\"%s\\\"\ncategories:\n - %s\n - alt versions\nformat:\n html: default\nexecute:\n echo: false\n---\n\n{{< include _alt_maps_database.qmd >}}\n\"\n\noutdata <- profile_info |> \n mutate(out = sprintf(template_text, \n `short name`, `biome name`, today, code, `short name`, `biome code`), \n outfile = here::here(\"map-info\", sprintf(\"%s_history.qmd\", gsub(\"\\\\.\", \"_\", code)))) |>\n select(outfile, out) \n\nfor (j in seq(along=outdata$out)) {\n outfile <- slice(outdata,j) |> pull(outfile)\n outtext <- slice(outdata,j) |> pull(out)\n \n if (!file.exists(outfile)) {\n cat(outtext, file = outfile)\n }\n}\n```\n:::",
"supporting": [],
"filters": [
"rmarkdown/pagebreak.lua"
],
Expand Down
Loading

0 comments on commit c9efdef

Please sign in to comment.