Added 40.timex resolution (#430)

tracyboehrer · Virtual-Josh · commit 78099740f332 · 2019-11-20T15:49:40.000-08:00
* Unfinished push until recognizers-text is updated.

* Added 40.timex-resolution
diff --git a/samples/40.timex-resolution/README.md b/samples/40.timex-resolution/README.md
@@ -0,0 +1,51 @@
+# Timex Resolution
+
+This sample shows how to use TIMEX expressions.
+
+## Concepts introduced in this sample
+
+### What is a TIMEX expression?
+
+A TIMEX expression is an alpha-numeric expression derived in outline from the standard date-time representation ISO 8601.
+The interesting thing about TIMEX expressions is that they can represent various degrees of ambiguity in the date parts. For example, May 29th, is not a
+full calendar date because we haven't said which May 29th - it could be this year, last year, any year in fact.
+TIMEX has other features such as the ability to represent ranges, date ranges, time ranges and even date-time ranges.
+
+### Where do TIMEX expressions come from?
+
+TIMEX expressions are produced as part of the output of running a DateTimeRecognizer against some natural language input. As the same
+Recognizers are run in LUIS the result returned in the JSON from a call to LUIS also contains the TIMEX expressions.
+
+### What can the library do?
+
+It turns out that TIMEX expressions are not that simple to work with in code. This library attempts to address that. One helpful way to
+think about a TIMEX expression is as a partially filled property bag. The properties might be such things as "day of week" or "year."
+Basically the more properties we have captured in the expression the less ambiguity we have.
+
+The library can do various things:
+
+- Parse TIMEX expressions to give you the properties contained there in.
+- Generate TIMEX expressions based on setting raw properties.
+- Generate natural language from the TIMEX expression. (This is logically the reverse of the Recognizer.)
+- Resolve TIMEX expressions to produce example date-times. (This produces the same result as the Recognizer (and therefore LUIS)).
+- Evaluate TIMEX expressions against constraints such that new more precise TIMEX expressions are produced.
+
+### Where is the source code?
+
+The TIMEX expression library is contained in the same GitHub repo as the recognizers. Refer to the further reading section below.
+
+## Running the sample
+- Clone the repository
+```bash
+git clone https://github.com/Microsoft/botbuilder-python.git
+```
+- Activate your desired virtual environment
+- Bring up a terminal, navigate to `botbuilder-python\samples\40.timex-resolution` folder
+- In the terminal, type `pip install -r requirements.txt`
+- In the terminal, type `python main.py`
+
+## Further reading
+
+- [TIMEX](https://en.wikipedia.org/wiki/TimeML#TIMEX3)
+- [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601)
+- [Recognizers Text](https://github.com/Microsoft/recognizers-text)
diff --git a/samples/40.timex-resolution/ambiguity.py b/samples/40.timex-resolution/ambiguity.py
@@ -0,0 +1,78 @@
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+
+from recognizers_date_time import recognize_datetime, Culture
+
+
+class Ambiguity:
+    """
+    TIMEX expressions are designed to represent ambiguous rather than definite dates. For
+    example: "Monday" could be any Monday ever. "May 5th" could be any one of the possible May
+    5th in the past or the future. TIMEX does not represent ambiguous times. So if the natural
+    language mentioned 4 o'clock it could be either 4AM or 4PM. For that the recognizer (and by
+    extension LUIS) would return two TIMEX expressions. A TIMEX expression can include a date and
+    time parts. So ambiguity of date can be combined with multiple results. Code that deals with
+    TIMEX expressions is frequently dealing with sets of TIMEX expressions.
+    """
+
+    @staticmethod
+    def date_ambiguity():
+        # Run the recognizer.
+        results = recognize_datetime(
+            "Either Saturday or Sunday would work.", Culture.English
+        )
+
+        # We should find two results in this example.
+        for result in results:
+            # The resolution includes two example values: going backwards and forwards from NOW in the calendar.
+            # Each result includes a TIMEX expression that captures the inherent date but not time ambiguity.
+            # We are interested in the distinct set of TIMEX expressions.
+            # There is also either a "value" property on each value or "start" and "end".
+            distinct_timex_expressions = {
+                value["timex"]
+                for value in result.resolution["values"]
+                if "timex" in value
+            }
+            print(f"{result.text} ({','.join(distinct_timex_expressions)})")
+
+    @staticmethod
+    def time_ambiguity():
+        # Run the recognizer.
+        results = recognize_datetime(
+            "We would like to arrive at 4 o'clock or 5 o'clock.", Culture.English
+        )
+
+        # We should find two results in this example.
+        for result in results:
+            # The resolution includes two example values: one for AM and one for PM.
+            # Each result includes a TIMEX expression that captures the inherent date but not time ambiguity.
+            #  We are interested in the distinct set of TIMEX expressions.
+            distinct_timex_expressions = {
+                value["timex"]
+                for value in result.resolution["values"]
+                if "timex" in value
+            }
+
+            # TIMEX expressions don't capture time ambiguity so there will be two distinct expressions for each result.
+            print(f"{result.text} ({','.join(distinct_timex_expressions)})")
+
+    @staticmethod
+    def date_time_ambiguity():
+        # Run the recognizer.
+        results = recognize_datetime(
+            "It will be ready Wednesday at 5 o'clock.", Culture.English
+        )
+
+        # We should find a single result in this example.
+        for result in results:
+            # The resolution includes four example values: backwards and forward in the calendar and then AM and PM.
+            # Each result includes a TIMEX expression that captures the inherent date but not time ambiguity.
+            # We are interested in the distinct set of TIMEX expressions.
+            distinct_timex_expressions = {
+                value["timex"]
+                for value in result.resolution["values"]
+                if "timex" in value
+            }
+
+            # TIMEX expressions don't capture time ambiguity so there will be two distinct expressions for each result.
+            print(f"{result.text} ({','.join(distinct_timex_expressions)})")
diff --git a/samples/40.timex-resolution/constraints.py b/samples/40.timex-resolution/constraints.py
@@ -0,0 +1,31 @@
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+
+import datetime
+
+from datatypes_timex_expression import TimexRangeResolver, TimexCreator
+
+
+class Constraints:
+    """
+    The TimexRangeResolved can be used in application logic to apply constraints to a set of TIMEX expressions.
+    The constraints themselves are TIMEX expressions. This is designed to appear a little like a database join,
+    of course its a little less generic than that because dates can be complicated things.
+    """
+
+    @staticmethod
+    def examples():
+        """
+        When you give the recognizer the text "Wednesday 4 o'clock" you get these distinct TIMEX values back.
+        But our bot logic knows that whatever the user says it should be evaluated against the constraints of
+        a week from today with respect to the date part and in the evening with respect to the time part.
+        """
+
+        resolutions = TimexRangeResolver.evaluate(
+            ["XXXX-WXX-3T04", "XXXX-WXX-3T16"],
+            [TimexCreator.week_from_today(), TimexCreator.EVENING],
+        )
+
+        today = datetime.datetime.now()
+        for resolution in resolutions:
+            print(resolution.to_natural_language(today))
diff --git a/samples/40.timex-resolution/language_generation.py b/samples/40.timex-resolution/language_generation.py
@@ -0,0 +1,33 @@
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+
+import datetime
+
+from datatypes_timex_expression import Timex
+
+
+class LanguageGeneration:
+    """
+    This language generation capabilities are the logical opposite of what the recognizer does.
+    As an experiment try feeding the result of language generation back into a recognizer.
+    You should get back the same TIMEX expression in the result.
+    """
+
+    @staticmethod
+    def examples():
+        LanguageGeneration.__describe(Timex("2019-05-29"))
+        LanguageGeneration.__describe(Timex("XXXX-WXX-6"))
+        LanguageGeneration.__describe(Timex("XXXX-WXX-6T16"))
+        LanguageGeneration.__describe(Timex("T12"))
+
+        LanguageGeneration.__describe(Timex.from_date(datetime.datetime.now()))
+        LanguageGeneration.__describe(
+            Timex.from_date(datetime.datetime.now() + datetime.timedelta(days=1))
+        )
+
+    @staticmethod
+    def __describe(timex: Timex):
+        # Note natural language is often relative, for example the sentence "Yesterday all my troubles seemed so far
+        # away." Having your bot say something like "next Wednesday" in a response can make it sound more natural.
+        reference_date = datetime.datetime.now()
+        print(f"{timex.timex_value()} : {timex.to_natural_language(reference_date)}")
diff --git a/samples/40.timex-resolution/main.py b/samples/40.timex-resolution/main.py
@@ -0,0 +1,23 @@
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+
+from ambiguity import Ambiguity
+from constraints import Constraints
+from language_generation import LanguageGeneration
+from parsing import Parsing
+from ranges import Ranges
+from resolution import Resolution
+
+if __name__ == "__main__":
+    # Creating TIMEX expressions from natural language using the Recognizer package.
+    Ambiguity.date_ambiguity()
+    Ambiguity.time_ambiguity()
+    Ambiguity.date_time_ambiguity()
+    Ranges.date_range()
+    Ranges.time_range()
+
+    # Manipulating TIMEX expressions in code using the TIMEX Datatype package.
+    Parsing.examples()
+    LanguageGeneration.examples()
+    Resolution.examples()
+    Constraints.examples()
diff --git a/samples/40.timex-resolution/parsing.py b/samples/40.timex-resolution/parsing.py
@@ -0,0 +1,45 @@
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+
+from datatypes_timex_expression import Timex, Constants
+
+
+class Parsing:
+    """
+    The Timex class takes a TIMEX expression as a string argument in its constructor.
+    This pulls all the component parts of the expression into properties on this object. You can
+    then manipulate the TIMEX expression via those properties.
+    The "types" property infers a datetimeV2 type from the underlying set of properties.
+    If you take a TIMEX with date components and add time components you add the
+    inferred type datetime (its still a date).
+    Logic can be written against the inferred type, perhaps to have the bot ask the user for
+    disambiguation.
+    """
+
+    @staticmethod
+    def __describe(timex_pattern: str):
+        timex = Timex(timex_pattern)
+
+        print(timex.timex_value(), end=" ")
+
+        if Constants.TIMEX_TYPES_DATE in timex.types:
+            if Constants.TIMEX_TYPES_DEFINITE in timex.types:
+                print("We have a definite calendar date.", end=" ")
+            else:
+                print("We have a date but there is some ambiguity.", end=" ")
+
+        if Constants.TIMEX_TYPES_TIME in timex.types:
+            print("We have a time.")
+        else:
+            print("")
+
+    @staticmethod
+    def examples():
+        """
+        Print information an various TimeX expressions.
+        :return: None
+        """
+        Parsing.__describe("2017-05-29")
+        Parsing.__describe("XXXX-WXX-6")
+        Parsing.__describe("XXXX-WXX-6T16")
+        Parsing.__describe("T12")
diff --git a/samples/40.timex-resolution/ranges.py b/samples/40.timex-resolution/ranges.py
@@ -0,0 +1,51 @@
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+
+from recognizers_date_time import recognize_datetime
+from recognizers_text import Culture
+
+
+class Ranges:
+    """
+    TIMEX expressions can represent date and time ranges. Here are a couple of examples.
+    """
+
+    @staticmethod
+    def date_range():
+        # Run the recognizer.
+        results = recognize_datetime(
+            "Some time in the next two weeks.", Culture.English
+        )
+
+        # We should find a single result in this example.
+        for result in results:
+            # The resolution includes a single value because there is no ambiguity.
+            # We are interested in the distinct set of TIMEX expressions.
+            distinct_timex_expressions = {
+                value["timex"]
+                for value in result.resolution["values"]
+                if "timex" in value
+            }
+
+            # The TIMEX expression can also capture the notion of range.
+            print(f"{result.text} ({','.join(distinct_timex_expressions)})")
+
+    @staticmethod
+    def time_range():
+        # Run the recognizer.
+        results = recognize_datetime(
+            "Some time between 6pm and 6:30pm.", Culture.English
+        )
+
+        # We should find a single result in this example.
+        for result in results:
+            # The resolution includes a single value because there is no ambiguity.
+            # We are interested in the distinct set of TIMEX expressions.
+            distinct_timex_expressions = {
+                value["timex"]
+                for value in result.resolution["values"]
+                if "timex" in value
+            }
+
+            # The TIMEX expression can also capture the notion of range.
+            print(f"{result.text} ({','.join(distinct_timex_expressions)})")
diff --git a/samples/40.timex-resolution/requirements.txt b/samples/40.timex-resolution/requirements.txt
@@ -0,0 +1,3 @@
+recognizers-text>=1.0.2a2
+datatypes-timex-expression>=1.0.2a2
+
diff --git a/samples/40.timex-resolution/resolution.py b/samples/40.timex-resolution/resolution.py
@@ -0,0 +1,26 @@
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+
+import datetime
+
+from datatypes_timex_expression import TimexResolver
+
+
+class Resolution:
+    """
+    Given the TIMEX expressions it is easy to create the computed example values that the recognizer gives.
+    """
+
+    @staticmethod
+    def examples():
+        # When you give the recognizer the text "Wednesday 4 o'clock" you get these distinct TIMEX values back.
+
+        today = datetime.datetime.now()
+        resolution = TimexResolver.resolve(["XXXX-WXX-3T04", "XXXX-WXX-3T16"], today)
+
+        print(f"Resolution Values: {len(resolution.values)}")
+
+        for value in resolution.values:
+            print(value.timex)
+            print(value.type)
+            print(value.value)

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,3 @@`
	`1`	`+recognizers-text>=1.0.2a2`
	`2`	`+datatypes-timex-expression>=1.0.2a2`
	`3`	`+`