IMPRO-1779 get local day (metoppv#1375)

* Starts to add a TimezoneExtraction plugin

* Sets up output_cube

- Adds method and test to build the output_cube with appropriate meta-data

* Sets up output_cube

- Adds support for multiple utc_times

* Removes xy-coord function (unnecessary)

- And tidies up a bit

* Completes plugin and unit-tests.

- Removes support for multiple output UTC times (no use-case)
- Adds tests to ensure input cubes are compatible

* Adds check for spatial coords

* Improves doc-string

* isort

* Improves documentation

- Input datetime is now called local_time as this better describes what it is for
- time_units definition moved to __init__ as not dependent on any inputs

* Adds dtype checking

* generate_timezone_mask now outputs masks as int8, not int32

* generate_timezone_mask now outputs cubes with UTC_offset coords that have units

* generate_timezone_mask now outputs cubes with correctly-defined UTC_offset coords.

* Adds CLI and acceptance tests

* Adds unit-test for TypeError

* Adds handling for time-bounds on input_cube and refactors create_output_cube to be run last.

* Bug fix and update of checksums following inclusion of time-bounds.

* Removes the trailing Z from input local-time argument.

* Extends unit-test coverage

- Tests input as either cube or list
- Tests input with and without bounds on the time coord

* Changes coord representation of local time from "utc" to "time_in_local_timezone"

- Updates code and unit-tests
- Updates doc-strings
- Updates add_coordinate method to be more explicit in which time coords require an update of forecast_period coord
- Updates acceptance test data checksums

* Updates acceptance-test checksums following merge with least-significant-digit change.

* Simplifies creation of time_in_local_timezone coord to avoid using add_coordinate method for this simple requirement.

* First review

- Improves an error message
- Tweaks doc-strings
- Simplifies unit-tests slightly

* Second review

- Corrects calculation of local-time using the offset (reverses sign)

* Second review

- Recreates acceptance test inputs and KGO
- Because the UK input times were asymmetric, I needed to recreate them. Because I used pytest parameterize (and I got the data from the Alpha suite), I therefore had to recreate the global too.

* Second review

- Improves doc-strings and comments
- Refactors dtype enforcement from temporal.py and cube_combiner.py into check_datatypes.py

* Modifies code to handle an extra dimension (e.g. percentile) on the input cube.

- Modifies the unit tests so that the dimensions are not all len(3) as this can mask broadcasting errors.

* Second review

- Modifies plugin to copy any cell_methods on the input_cube onto the output_cube
- Improves doc-strings and comments

* isort eyesore a puddy tat

* Methods that aren't tested directly are now private methods.

* Second review: Moves call to spatial_coords_match into check_input_cube_dims.

* Updates checksums for cell-method change.

* Corrects test for spatial coords mismatch so that it raises an error if necessary.

improver/cli/map_to_timezones.py

@@ -0,0 +1,67 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+# -----------------------------------------------------------------------------
+# (C) British Crown Copyright 2017-2020 Met Office.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# * Redistributions of source code must retain the above copyright notice, this
+#   list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright notice,
+#   this list of conditions and the following disclaimer in the documentation
+#   and/or other materials provided with the distribution.
+#
+# * Neither the name of the copyright holder nor the names of its
+#   contributors may be used to endorse or promote products derived from
+#   this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
+"""Script to map multiple forecast times into a local time grid"""
+
+from improver import cli
+
+
+@cli.clizefy
+@cli.with_output
+def process(
+    timezone_cube: cli.inputcube, local_time: str, *cubes: cli.inputcube,
+):
+    """Calculates timezone-offset data for the specified UTC output times
+
+    Args:
+        timezone_cube (iris.cube.Cube):
+            Cube describing the UTC offset for the local time at each grid location.
+            Must have the same spatial coords as input_cube.
+            Use generate-timezone-mask-ancillary to create this.
+        local_time (str):
+            The "local" time of the output cube as %Y%m%dT%H%M. This will form a
+            scalar "time_in_local_timezone" coord on the output cube, while the "time"
+            coord will be auxillary to the spatial coords and will show the UTC time
+            that matches the local_time at each point.
+        cubes (list of iris.cube.Cube):
+            Source data to be remapped onto time-zones. Must contain an exact 1-to-1
+            mapping of times to time-zones. Multiple input files will be merged into one
+            cube.
+
+    Returns:
+        iris.cube.Cube:
+            Processed cube.
+    """
+    from datetime import datetime
+    from improver.utilities.temporal import TimezoneExtraction
+
+    local_datetime = datetime.strptime(local_time, "%Y%m%dT%H%M")
+    return TimezoneExtraction()(cubes, timezone_cube, local_datetime)

improver/cube_combiner.py

@@ -36,6 +36,7 @@
 from iris.exceptions import CoordinateNotFoundError
 
 from improver import BasePlugin
+from improver.metadata.check_datatypes import enforce_dtype
 from improver.metadata.probabilistic import (
     extract_diagnostic_name,
     find_threshold_coordinate,
@@ -278,13 +279,8 @@ def process(
         if self.operation == "mean":
             result.data = result.data / len(cube_list)
 
-        # Check resulting dtype and modify if necessary
-        if result.dtype == np.float64:
-            unique_cube_types = set([c.dtype for c in cube_list])
-            raise TypeError(
-                f"Operation {self.operation} on types {unique_cube_types} results in "
-                "float64 data which cannot be safely coerced to float32"
-            )
+        # Check resulting dtype
+        enforce_dtype(self.operation, cube_list, result)
 
         # where the operation is "multiply", retain all coordinate metadata
         # from the first cube in the list; otherwise expand coordinate bounds

improver/generate_ancillaries/generate_timezone_mask.py

@@ -42,6 +42,7 @@
 from pytz import timezone
 
 from improver import BasePlugin
+from improver.metadata.constants.time_types import TIME_COORDS
 from improver.metadata.utilities import (
     create_new_diagnostic_cube,
     generate_mandatory_attributes,
@@ -256,7 +257,7 @@ def _create_template_cube(self, cube):
         attributes["includes_daylight_savings"] = str(self.include_dst)
 
         return create_new_diagnostic_cube(
-            "timezone_mask", 1, cube, attributes, dtype=np.int32
+            "timezone_mask", 1, cube, attributes, dtype=np.int8
         )
 
     def _group_timezones(self, timezone_mask):
@@ -348,8 +349,12 @@ def process(self, cube):
         # Create a cube containing the timezone UTC offset information.
         timezone_mask = iris.cube.CubeList()
         for offset in range(min_offset, max_offset + 1):
-            zone = (grid_offsets != offset).astype(np.int32)
-            coord = iris.coords.DimCoord([offset], long_name="UTC_offset")
+            zone = (grid_offsets != offset).astype(np.int8)
+            coord = iris.coords.DimCoord(
+                np.array([offset], dtype=np.int32),
+                long_name="UTC_offset",
+                units="hours",
+            )
             tz_slice = template_cube.copy(data=zone)
             tz_slice.add_aux_coord(coord)
             timezone_mask.append(tz_slice)
@@ -358,4 +363,11 @@ def process(self, cube):
             timezone_mask = self._group_timezones(timezone_mask)
 
         timezone_mask = timezone_mask.merge_cube()
+        timezone_mask.coord("UTC_offset").convert_units(TIME_COORDS["UTC_offset"].units)
+        timezone_mask.coord("UTC_offset").points = timezone_mask.coord(
+            "UTC_offset"
+        ).points.astype(TIME_COORDS["UTC_offset"].dtype)
+        timezone_mask.coord("UTC_offset").bounds = timezone_mask.coord(
+            "UTC_offset"
+        ).bounds.astype(TIME_COORDS["UTC_offset"].dtype)
         return timezone_mask

improver/metadata/check_datatypes.py

@@ -104,6 +104,33 @@ def check_dtype(obj):
     return dtype_ok
 
 
+def enforce_dtype(operation, inputs, result):
+    """
+    Ensures that result has not been automatically promoted to float64.
+
+    Args:
+        operation (str):
+            The operation that was performed (for the error message)
+        inputs (list):
+            The Numpy arrays or cubes that the operation was performed on (for the
+            error message)
+        result (np.array or iris.cube.Cube):
+            The result of the operation
+
+    Raises:
+        TypeError:
+            If result.dtype does not match the meta-data standard.
+
+    """
+    if not check_dtype(result):
+        unique_cube_types = set([c.dtype for c in inputs])
+        raise TypeError(
+            f"Operation {operation} on types {unique_cube_types} results in "
+            "float64 data which cannot be safely coerced to float32 (Hint: "
+            "combining int8 and float32 works)"
+        )
+
+
 def get_required_units(obj):
     """
     Returns the mandatory units for the supplied obj. Only time coords have

improver/metadata/constants/time_types.py

@@ -44,6 +44,8 @@
 
 TIME_COORDS = {
     "time": _TIME_REFERENCE_SPEC,
+    "time_in_local_timezone": _TIME_REFERENCE_SPEC,
     "forecast_reference_time": _TIME_REFERENCE_SPEC,
     "forecast_period": _TIME_INTERVAL_SPEC,
+    "UTC_offset": _TIME_INTERVAL_SPEC,
 }

improver/synthetic_data/set_up_test_cubes.py

@@ -592,7 +592,7 @@ def add_coordinate(
 
         # recalculate forecast period if time or frt have been updated
         if (
-            "time" in coord_name
+            coord_name in ["time", "forecast_reference_time"]
             and coord_units is not None
             and Unit(coord_units).is_time_reference()
         ):