Add date/time, list format, relative date/time data. Some executors added in NodeJS and ICU4X. #183

sven-oly · 2024-02-21T23:14:11Z

No description provided.

sven-oly · 2024-03-22T21:55:00Z

Please take a look. Note that CPP datetime instances all fail now, and CPP doesn't have list format.

sffc · 2024-03-26T18:13:15Z

executors/cpp/datetime_fmt.cpp

+
+  string era_str;
+  string year_str;
+  string month_str;


Question: I don't see where you read these variables. Intentional?

They will be used when I figure out the API to handle individual fields in CPP version of DateTime format.

sffc · 2024-03-26T18:14:32Z

executors/cpp/datetime_fmt.cpp

+  // Prefer milliseconds as input.
+  json_object *input_millis = json_object_object_get(json_in, "input_millis");
+  if (input_millis) {
+    testDateTime = json_object_get_double(input_millis);
+  } else {


This is a bit messy; you should either have the timestamp or the ISO string but having both just creates confusion. I prefer the ISO string because it is less ambiguous.

OK, I'm removing the milliseconds as an output field.

Reversal - parsing the date isn't yet working, so I'm keeping the milliseconds for now.

Let's chat about how to use absl here, which I think is the right thing to do.

sffc · 2024-03-26T18:16:21Z

executors/cpp/datetime_fmt.cpp

+      UnicodeString parse_skeleton = "YYYY-MM-DD HH:mm:ss";
+
+      DateFormat* dparser = DateFormat::createInstanceForSkeleton(parse_skeleton,
+                                                                  displayLocale,
+                                                                  status);


You shouldn't use an ICU4C DateFormat to parse an ISO-8601 string. You should be able to parse it directly or using a helper library such as absl.

Good idea! I'll add this.

Parsing date string isn't yet working.

Fixed this!

sffc · 2024-03-26T18:21:25Z

testgen/generators/list_fmt_gen.js

+const locales = ['und',
+                 'en-US', 'zh-TW', 'pt', 'vi', 'el', 'mt-MT', 'ru', 'en-GB',
+                 'bn', 'ar','mr', 'zu'];


Suggestion: include Spanish, which uses inflection to pick between o and u and between y and e, and make sure there is a test that exercises this code path.

Good idea! Added!

echeran

initial comments. will continue

echeran · 2024-03-29T17:51:14Z

executors/cpp/datetime_fmt.cpp

+    // Data specifies a date time skeleton. Make a formatter based on this.
+    string skeleton_string = json_object_get_string(date_skeleton_obj);
+
+    cout << "# Skelecton = " << skeleton_string << endl;


Suggested change

cout << "# Skelecton = " << skeleton_string << endl;

cout << "# Skeleton = " << skeleton_string << endl;

echeran · 2024-03-29T18:13:04Z

executors/cpp/datetime_fmt.cpp

+
+    cout << "# Skelecton = " << skeleton_string << endl;
+
+    UnicodeString u_skeleton = skeleton_string.c_str();


Apparently, this isn't the way to convert C String -> UnicodeString. You should use this UnicodeString constructor, and only if the text is ASCII. If the char * text is not ASCII, then you need to do a codepage conversion first before converting to UnicodeString.

(I asked Shane b/c I didn't see any char * arg operator=() for UnicodeString. He learned the above info from Markus, so you can double check with him to make sure that's correct.)

Thanks. Constructor is now in place.

sven-oly · 2024-03-30T00:21:04Z

OK, this is moving along. I've got a form of skeletons (not debugged) for use in ICU4C. However, it's not yet using Calendar or Timezone, so most cases still fail.

sven-oly · 2024-04-01T22:57:14Z

PTAL. Note that Date Time formatting still has substantial test failure due to calendar not handled correctly.

New test data generator uses Node's formatToParts to get test data that is compatible with ICU4J/C with NBSP.

sffc · 2024-04-01T23:07:33Z

schema/datetime_fmt/test_schema.json

+         "input": {
+           "description": "date time specification in XYZ format",
+           "type": "string"
+         },
+         "input_millis": {
+           "description": "numeric value for time in milliseconds since January 1, 1970, UTC (the epoch)",
+           "type": "string"
+         },
+         "input_string": {
+           "description": "String in ISO 8601 form, e.g. YYYY-MM-DD hh:mm:ss.sss",
+           "type": "string"
+         },


Are all three of these fields valid input?

Removed input and input_millis fields.

sffc · 2024-04-01T23:08:08Z

schema/list_fmt/test_schema.json

+     "description": "version of conformance data",
+     "type": "string"
+   },
+   "test_ environmment": {


Extra space character?

Good catch! Fixed!

sven-oly added 5 commits February 8, 2024 17:41

Starting DateTime format with CPP

e027bfa

Adding date time schema prototypes

b1239a4

Start test generator

40b4478

Parsing format.txt

fca9392

Defer updating date time test data

9e9743e

sven-oly force-pushed the startDateTime branch from c0430df to 9e9743e Compare March 8, 2024 18:34

sven-oly added 10 commits March 8, 2024 10:38

Move generator for DateTime to generators

572d120

Merge remote-tracking branch 'upstream/main' into startDateTime

dab45cf

Starting DateTime generate parsing

d86eaae

Starting DateTime generate parsing

27ab673

Merge remote-tracking branch 'upstream/main' into startDateTime

de0c346

Added NodeJS datetime_fmt

9d72878

Generators for datetime and list formatting (from NodeJS)

6d7c98b

Starting on CPP for date time and list formatting

b4f1016

Merge remote-tracking branch 'origin' into startDateTime

b46cb48

Configure node for date time and list formatting

720741f

sven-oly force-pushed the startDateTime branch from df91606 to 720741f Compare March 22, 2024 21:32

sven-oly changed the title ~~Start date time in CPP~~ Add date/time and list format test data via Node. Test these in NodeJS. Mar 22, 2024

Add list format generator

9434024

sven-oly assigned echeran and sffc Mar 22, 2024

sven-oly added 4 commits March 23, 2024 20:23

Adding first try with CPP list format

2891410

Update input_string to ISO formatted date/time

683a078

Add relative date/time format data and Node executor

d37d074

Update to CPP DT format, still not working

70c8121

sven-oly changed the title ~~Add date/time and list format test data via Node. Test these in NodeJS.~~ Add date/time, list format, relative date/time data. Some executors added in NodeJS and ICU4X. Mar 26, 2024

sffc reviewed Mar 26, 2024

View reviewed changes

echeran reviewed Mar 29, 2024

View reviewed changes

echeran mentioned this pull request Mar 29, 2024

Add schema validation for date/time, list format, relative date/time format. #213

Closed

DateTime improvements - some passing

f5ec893

sven-oly added 2 commits April 1, 2024 14:42

DT test generator to use format_to_parts. DT icu4c with tz and calendar

7ab23d6

More updates to date/time generation

0392036

Characterize exponent differences for Number Format verifier

263d9a7

sffc reviewed Apr 1, 2024

View reviewed changes

Update schema details

d8dec7c

sffc approved these changes Apr 1, 2024

View reviewed changes

sven-oly merged commit 75b477e into unicode-org:main Apr 1, 2024

sven-oly deleted the startDateTime branch April 1, 2024 23:59

	cout << "# Skelecton = " << skeleton_string << endl;
	cout << "# Skeleton = " << skeleton_string << endl;


		cout << "# Skelecton = " << skeleton_string << endl;

		UnicodeString u_skeleton = skeleton_string.c_str();

Uh oh!

Add date/time, list format, relative date/time data. Some executors added in NodeJS and ICU4X. #183

Add date/time, list format, relative date/time data. Some executors added in NodeJS and ICU4X. #183

Uh oh!

Conversation

sven-oly commented Feb 21, 2024

Uh oh!

sven-oly commented Mar 22, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

echeran left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sven-oly commented Mar 30, 2024

Uh oh!

sven-oly commented Apr 1, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!