Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add format to the in-memory Column #6538

Merged
merged 65 commits into from
May 9, 2023
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
0306efc
Date.format locale
GregoryTravis Apr 25, 2023
2238e81
wip
GregoryTravis Apr 25, 2023
c6df9fd
dt tu builtin
GregoryTravis Apr 26, 2023
5b0381f
Merge branch 'develop' into wip/gmt/6329-format
GregoryTravis Apr 26, 2023
f891786
fixed builtin problem, first test
GregoryTravis Apr 26, 2023
b6e3f69
Merge branch 'develop' into wip/gmt/6329-format
GregoryTravis Apr 26, 2023
b8bd9f5
_ syntax
GregoryTravis Apr 26, 2023
7f70d81
date locale
GregoryTravis Apr 26, 2023
526f748
bad format
GregoryTravis Apr 26, 2023
32cca98
illegal_argument_to_warning
GregoryTravis Apr 26, 2023
df6b63f
reuse that
GregoryTravis Apr 26, 2023
3fdc36c
+map_2_to_text_over_storage
GregoryTravis Apr 26, 2023
05bed43
locale for all
GregoryTravis Apr 27, 2023
6795e2a
local example
GregoryTravis Apr 27, 2023
dc31013
doc
GregoryTravis Apr 27, 2023
56a738c
builder param
GregoryTravis Apr 27, 2023
fd39742
do not use builtins
GregoryTravis Apr 27, 2023
2ad0f7f
format coulumn
GregoryTravis Apr 27, 2023
e91af1b
column mismatch test
GregoryTravis Apr 28, 2023
7448d87
multiple warnings
GregoryTravis Apr 28, 2023
9c200bf
fix
GregoryTravis Apr 28, 2023
0e57314
max errors but not working
GregoryTravis Apr 28, 2023
47cfd06
format column type error
GregoryTravis May 1, 2023
50e28e0
dead
GregoryTravis May 1, 2023
f58bc42
Date_Time
GregoryTravis May 1, 2023
6fbb25c
Time_Of_Day
GregoryTravis May 1, 2023
bd12a86
docs
GregoryTravis May 1, 2023
a5ef7ab
dead
GregoryTravis May 1, 2023
0f85dbd
Merge branch 'develop' into wip/gmt/6329-format
GregoryTravis May 1, 2023
06d70a8
bool
GregoryTravis May 1, 2023
1ebfe9d
rename cols
GregoryTravis May 1, 2023
0f72842
boolean bad format
GregoryTravis May 1, 2023
533e315
dead
GregoryTravis May 1, 2023
e339273
hard error on first format error
GregoryTravis May 2, 2023
399f609
dead
GregoryTravis May 2, 2023
57ec504
unused
GregoryTravis May 2, 2023
bdc2a18
Integer
GregoryTravis May 2, 2023
8668819
Integer and Float
GregoryTravis May 2, 2023
d262dcf
bad format
GregoryTravis May 2, 2023
ff14870
doc
GregoryTravis May 2, 2023
e2350e8
examples
GregoryTravis May 2, 2023
bb04b84
supported types
GregoryTravis May 2, 2023
4f0cc74
cleanup
GregoryTravis May 3, 2023
1648ad4
remove on_problems, Database stub
GregoryTravis May 3, 2023
0c33f94
empty/Nothing pattern
GregoryTravis May 3, 2023
d926a71
simpler
GregoryTravis May 3, 2023
05fb0a2
edge cases + errors
GregoryTravis May 3, 2023
2df0a87
cleanup
GregoryTravis May 3, 2023
1f1aefa
cleanup
GregoryTravis May 3, 2023
7f3991b
M`Integer` or `Float`erge branch 'develop' into wip/gmt/6329-format
GregoryTravis May 3, 2023
c66b564
changelog
GregoryTravis May 3, 2023
c1f6bc6
merge
GregoryTravis May 4, 2023
7d0bdd3
no overload
GregoryTravis May 4, 2023
96df7e4
succinct
GregoryTravis May 4, 2023
6addd10
todo
GregoryTravis May 4, 2023
9d0aa81
separate file
GregoryTravis May 4, 2023
b5281c0
Review.
GregoryTravis May 4, 2023
c98079f
Merge branch 'develop' into wip/gmt/6329-format
GregoryTravis May 5, 2023
da353bb
review
GregoryTravis May 5, 2023
4d7d162
merge
GregoryTravis May 6, 2023
298b3ed
make tests TZ portable
radeusgd May 8, 2023
ac6d443
fix a few tests
radeusgd May 8, 2023
f8a2e69
Merge branch 'develop' into wip/gmt/6329-format
GregoryTravis May 8, 2023
5b5226f
missing import
GregoryTravis May 8, 2023
602794c
Merge branch 'wip/gmt/6329-format' of github.com:enso-org/enso into w…
GregoryTravis May 8, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -408,7 +408,8 @@
`Text.write`.][6459]
- [Implemented `create_database_table` allowing saving queries as database
tables.][6467]
- [Implemented `Column.format` for in-memory `Columns`.][6538]
- [Implemented `Column.format` for in-memory `Column`s.][6538]
- [Moved `Redshift` connector into a separate `AWS` library.][6550]

[debug-shortcuts]:
https://github.com/enso-org/enso/blob/develop/app/gui/docs/product/shortcuts.md#debug
Expand Down Expand Up @@ -616,6 +617,7 @@
[6459]: https://github.com/enso-org/enso/pull/6459
[6467]: https://github.com/enso-org/enso/pull/6467
[6538]: https://github.com/enso-org/enso/pull/6538
[6550]: https://github.com/enso-org/enso/pull/6550

#### Enso Compiler

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -659,4 +659,4 @@ type Date_Time
format : Text -> Locale -> Text
format self pattern locale=Locale.default = case locale of
Nothing -> Time_Utils.date_time_format self pattern
_ -> Time_Utils.date_time_format self pattern locale.java_locale
_ -> Time_Utils.date_time_format_with_locale self pattern locale.java_locale
198 changes: 22 additions & 176 deletions distribution/lib/Standard/Table/0.0.0-dev/src/Data/Column.enso
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from Standard.Base import all
from Standard.Base import all
from project.Data.Column_Format import all

import Standard.Base.Data.Array_Proxy.Array_Proxy
import Standard.Base.Data.Locale.Locale
import Standard.Base.Errors.Common.Index_Out_Of_Bounds
import Standard.Base.Errors.Illegal_Argument.Illegal_Argument
import Standard.Base.Errors.Illegal_State.Illegal_State
Expand All @@ -16,7 +16,6 @@ import project.Data.Table.Table
import project.Internal.Java_Problems
import project.Internal.Naming_Helpers.Naming_Helpers
import project.Internal.Parse_Values_Helper
import project.Internal.Problem_Builder.Problem_Builder
import project.Internal.Widget_Helpers
import project.Data.Type.Value_Type_Helpers

Expand All @@ -25,8 +24,6 @@ from project.Data.Type.Value_Type import Value_Type, Auto
from project.Errors import No_Index_Set_Error, Floating_Point_Equality, Invalid_Value_Type, Inexact_Type_Coercion
from project.Internal.Java_Exports import make_string_builder

polyglot java import java.lang.IllegalArgumentException
polyglot java import java.time.temporal.UnsupportedTemporalTypeException
polyglot java import org.enso.table.data.column.operation.map.MapOperationProblemBuilder
polyglot java import org.enso.table.data.column.storage.Storage as Java_Storage
polyglot java import org.enso.table.data.table.Column as Java_Column
Expand Down Expand Up @@ -1190,7 +1187,7 @@ type Column
! Error Conditions

- If the format is incorrectly formed, or if some values in the column
did not match the expected datatype format, an `Invalid_Argument`
did not match the expected datatype format, an `Illegal_Argument`
error is thrown.

? Supported Types
Expand All @@ -1208,6 +1205,11 @@ type Column
as "3 Dec 2011". See https://docs.oracle.com/en/java/javase/18/docs/api/java.base/java/time/format/DateTimeFormatter.html
for a complete format specification.

Note that the format string can specify decimal point and digit
separators, but these characters are interpreted in the context of the
Locale used. The format string specifies their location, but the Locale
has the final decision about which characters are used.
GregoryTravis marked this conversation as resolved.
Show resolved Hide resolved

? `Value_Type.Integer`, `Value_Type.Float` format strings

Numeric format strings are specified by the Java DecimalFormat class.
Expand Down Expand Up @@ -1249,6 +1251,17 @@ type Column
# ==> ["t", "f"]
format : Text | Column -> Locale -> Column ! Illegal_Argument
format self format=Nothing locale=Locale.default =
create_formatter = case self.value_type of
Value_Type.Date -> make_value_formatter locale
Value_Type.Date_Time _ -> make_value_formatter locale
Value_Type.Time -> make_value_formatter locale
Value_Type.Boolean -> make_boolean_formatter
Value_Type.Integer _ -> make_value_formatter locale
Value_Type.Float _ -> make_value_formatter locale
bad_type ->
msg = "Cannot format a Column of type " + bad_type.to_text
Error.throw (Illegal_Argument.Error msg)
GregoryTravis marked this conversation as resolved.
Show resolved Hide resolved

new_column = case format of
"" ->
formatter = .to_text
Expand All @@ -1257,41 +1270,11 @@ type Column
formatter = .to_text
map_over_storage self formatter make_string_builder
_ : Text ->
formatter = case self.value_type of
Value_Type.Date ->
make_datelike_formatter format locale
Value_Type.Date_Time _ ->
make_datelike_formatter format locale
Value_Type.Time ->
make_datelike_formatter format locale
Value_Type.Boolean ->
make_boolean_formatter format
Value_Type.Integer _ ->
make_number_formatter format locale
Value_Type.Float _ ->
make_number_formatter format locale
bad_type ->
msg = "Cannot format a Column of type " + bad_type.to_text
Error.throw (Illegal_Argument.Error msg)
formatter = create_formatter
formatter.if_not_error <|
map_over_storage self formatter make_string_builder
map_over_storage self (formatter format=format) make_string_builder
format_column : Column -> Value_Type.expect_text format_column <|
formatter = case self.value_type of
Value_Type.Date ->
make_datelike_with_column_formatter locale
Value_Type.Date_Time _ ->
make_datelike_with_column_formatter locale
Value_Type.Time ->
make_datelike_with_column_formatter locale
Value_Type.Boolean ->
make_boolean_with_column_formatter
Value_Type.Integer _ ->
make_number_with_column_formatter locale
Value_Type.Float _ ->
make_number_with_column_formatter locale
bad_type ->
msg = "Cannot format a Column of type " + bad_type.to_text
Error.throw (Illegal_Argument.Error msg)
formatter = create_formatter
formatter.if_not_error <|
map_2_over_storage self format_column formatter make_string_builder
_ -> Error.throw <| Illegal_Argument.Error <| "Unsupported format type: " + format.to_text
Expand Down Expand Up @@ -1862,140 +1845,3 @@ wrap_text_argument_as_value_provider val =
col : Column ->
storage = col.java_column.getStorage
i-> storage.getItemBoxed i

## PRIVATE
Create a date/time formatter for the given format string. The `datelike`
parameter can be a `Date`, `Date_Time`, or `Time_Of_Day`.
make_datelike_formatter : Text -> Locale -> (Date -> Text)
make_datelike_formatter format locale =
datelike->
handle_illegal_argument_exception format <|
datelike.format format locale

## PRIVATE
Create a date/time formatter that takes the format string as the second
parameter. The `datelike` parameter can be a `Date`, `Date_Time`, or
`Time_Of_Day`.
make_datelike_with_column_formatter : Locale -> (Date -> Text -> Text)
make_datelike_with_column_formatter locale =
empty_format_uses_to_text <|
datelike-> format->
handle_illegal_argument_exception format <|
datelike.format format locale

## PRIVATE
Create a `Boolean` formatter for the given format string.
make_boolean_formatter : Text -> (Date -> Text) ! Illegal_Argument
make_boolean_formatter format =
data_formatter = Data_Formatter.Value.with_format Value_Type.Boolean format
data_formatter.if_not_error
bool->
handle_illegal_argument_exception format <|
data_formatter.format bool

## PRIVATE
Create a `Boolean` formatter that takes the format string as the second
parameter.
make_boolean_with_column_formatter : (Date -> Text -> Text)
make_boolean_with_column_formatter =
empty_format_uses_to_text <|
bool-> format->
data_formatter = Data_Formatter.Value.with_format Value_Type.Boolean format
data_formatter.if_not_error
data_formatter.format bool

## PRIVATE
Create a numeric formatter for the given format string. The `number`
parameter can be an `Integer` or `Float`.
make_number_formatter : Text -> Locale -> (Date -> Text)
make_number_formatter format locale =
number->
handle_illegal_argument_exception format <|
number.format format locale

## PRIVATE
Create a numeric formatter that takes the format string as the second
parameter. The `number` parameter can be an `Integer` or `Float`.
make_number_with_column_formatter : Locale -> (Date -> Text -> Text)
make_number_with_column_formatter locale =
empty_format_uses_to_text <|
number-> format->
handle_illegal_argument_exception format <|
number.format format locale

## PRIVATE
Wrap a text formatter so that, if passed `""` or `Nothing` for the format,
uses .to_text instead of the normal .format method.
empty_format_uses_to_text : (Any -> Any -> Any) -> (Any -> Any -> Any)
empty_format_uses_to_text function = x-> format->
case format of
_ : Nothing -> x.to_text
"" -> x.to_text
_ -> function x format

## PRIVATE
Rethrow a Java IllegalArgumentException as an Illegal_Argument.
handle_illegal_argument_exception : Text -> Any -> Any
handle_illegal_argument_exception format_string ~action =
handler cause =
msg = cause.payload.getMessage + ' in \"' + format_string + '\"'
Error.throw (Illegal_Argument.Error msg)
Panic.catch IllegalArgumentException handler=handler <|
Panic.catch UnsupportedTemporalTypeException handler=handler action

## PRIVATE
Iterate over a range, exiting early if the body produces an `Error`.
TODO: Move into Range
each_propagate : Range -> (Number -> Any) -> Nothing ! Error
each_propagate range function =
if range.step == 0 then Error.throw (Illegal_State.Error "A range with step = 0 is ill-formed.") else
end_condition = if range.step > 0 then (>=) else (<=)
go current =
if end_condition current range.end then Nothing else
result = function current
if result.is_error then result else
@Tail_Call go current+range.step
go range.start

## PRIVATE
Map a text-returning function over the column values, using Storage directly.
The output column has the same name as the input.
map_over_storage : Column -> (Any -> Text) -> (Integer -> Any) -> Boolean -> Column
map_over_storage input_column function builder skip_nothing=True =
input_storage = input_column.java_column.getStorage
num_input_rows = input_storage.size
output_storage_builder = builder num_input_rows
ok = each_propagate (0.up_to num_input_rows) i->
input_value = input_storage.getItemBoxed i
if skip_nothing && input_value.is_nothing then output_storage_builder.append Nothing else
output_value = function input_value
output_value.if_not_error
output_storage_builder.append output_value
ok.if_not_error <|
output_storage = output_storage_builder.seal
Column.from_storage input_column.name output_storage

## PRIVATE
Map a text-returning function over the values of two columns, using Storage
directly. The output column has the same name as the first input column.
`skip_nothing` applies to the first input to the function, not both inputs.
map_2_over_storage : Column -> Column -> (Any -> Any -> Text) -> (Integer -> Any) -> Boolean -> Column
map_2_over_storage input_column_0 input_column_1 function builder skip_nothing=True =
input_storage_0 = input_column_0.java_column.getStorage
input_storage_1 = input_column_1.java_column.getStorage
case input_storage_0.size != input_storage_1.size of
True ->
msg = "Column lengths differ: " + input_storage_0.size.to_text + " != " + input_storage_1.size.to_text
Error.throw (Illegal_Argument.Error msg)
False ->
num_input_rows = input_storage_0.size
output_storage_builder = builder num_input_rows
ok = each_propagate (0.up_to num_input_rows) i->
input_value_0 = input_storage_0.getItemBoxed i
input_value_1 = input_storage_1.getItemBoxed i
if skip_nothing && input_value_0.is_nothing then output_storage_builder.append Nothing else
output_value = function input_value_0 input_value_1
output_storage_builder.append output_value
ok.if_not_error <|
output_storage = output_storage_builder.seal
Column.from_storage input_column_0.name output_storage
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
from Standard.Base import all
import Standard.Base.Errors.Illegal_Argument.Illegal_Argument
import Standard.Base.Errors.Illegal_State.Illegal_State

import project.Data.Column.Column
import project.Data.Data_Formatter.Data_Formatter
import project.Data.Type.Storage

from project.Data.Type.Value_Type import Value_Type
GregoryTravis marked this conversation as resolved.
Show resolved Hide resolved
from project.Internal.Java_Exports import make_string_builder

polyglot java import java.lang.IllegalArgumentException
polyglot java import java.time.temporal.UnsupportedTemporalTypeException
polyglot java import org.enso.table.data.column.operation.map.MapOperationProblemBuilder
polyglot java import org.enso.table.data.column.storage.Storage as Java_Storage
polyglot java import org.enso.table.data.table.Column as Java_Column
polyglot java import org.enso.table.operations.OrderBuilder

## PRIVATE
Create a formatter for the given format string.
The `value` parameter has to have a `format` method that takes a format and
locale.
make_value_formatter : Locale -> (Any -> Text)
make_value_formatter locale = value-> format->
handle_illegal_argument_exception format <|
if format.is_nothing || format.is_empty then value.to_text else
value.format format locale
## PRIVATE
Create a `Boolean` formatter that takes the format string as the second
parameter.
make_boolean_formatter : (Boolean -> Text -> Text)
make_boolean_formatter = bool-> format->
if format.is_nothing || format.is_empty then bool.to_text else
data_formatter = Data_Formatter.Value.with_format Value_Type.Boolean format
data_formatter.format bool

## PRIVATE
Rethrow a Java IllegalArgumentException as an Illegal_Argument.
handle_illegal_argument_exception : Text -> Any -> Any
handle_illegal_argument_exception format_string ~action =
handler cause =
msg = cause.payload.getMessage + ' in \"' + format_string + '\"'
Error.throw (Illegal_Argument.Error msg)
Panic.catch IllegalArgumentException handler=handler <|
Panic.catch UnsupportedTemporalTypeException handler=handler action

## PRIVATE
Iterate over a range, exiting early if the body produces an `Error`.
each_propagate : Range -> (Number -> Any) -> Nothing ! Error
each_propagate range function =
if range.step == 0 then Error.throw (Illegal_State.Error "A range with step = 0 is ill-formed.") else
end_condition = if range.step > 0 then (>=) else (<=)
go current =
if end_condition current range.end then Nothing else
result = function current
result.if_not_error <|
@Tail_Call go current+range.step
go range.start

## PRIVATE
Map a text-returning function over the column values, using Storage directly.
The output column has the same name as the input.
map_over_storage : Column -> (Any -> Text) -> (Integer -> Any) -> Boolean -> Column
GregoryTravis marked this conversation as resolved.
Show resolved Hide resolved
map_over_storage input_column function builder skip_nothing=True =
input_storage = input_column.java_column.getStorage
num_input_rows = input_storage.size
output_storage_builder = builder num_input_rows
ok = each_propagate (0.up_to num_input_rows) i->
input_value = input_storage.getItemBoxed i
if skip_nothing && input_value.is_nothing then output_storage_builder.append Nothing else
output_value = function input_value
output_value.if_not_error
output_storage_builder.append output_value
ok.if_not_error <|
output_storage = output_storage_builder.seal
Column.from_storage input_column.name output_storage

## PRIVATE
Map a text-returning function over the values of two columns, using Storage
directly. The output column has the same name as the first input column.
`skip_nothing` applies to the first input to the function, not both inputs.
map_2_over_storage : Column -> Column -> (Any -> Any -> Text) -> (Integer -> Any) -> Boolean -> Column
map_2_over_storage input_column_0 input_column_1 function builder skip_nothing=True =
input_storage_0 = input_column_0.java_column.getStorage
input_storage_1 = input_column_1.java_column.getStorage
case input_storage_0.size != input_storage_1.size of
True ->
msg = "Column lengths differ: " + input_storage_0.size.to_text + " != " + input_storage_1.size.to_text
Error.throw (Illegal_Argument.Error msg)
False ->
num_input_rows = input_storage_0.size
output_storage_builder = builder num_input_rows
ok = each_propagate (0.up_to num_input_rows) i->
input_value_0 = input_storage_0.getItemBoxed i
input_value_1 = input_storage_1.getItemBoxed i
if skip_nothing && input_value_0.is_nothing then output_storage_builder.append Nothing else
output_value = function input_value_0 input_value_1
output_storage_builder.append output_value
ok.if_not_error <|
output_storage = output_storage_builder.seal
Column.from_storage input_column_0.name output_storage
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ public static String date_time_format(ZonedDateTime dateTime, Object format) {
return DateTimeFormatter.ofPattern(format.toString()).format(dateTime);
}

public static String date_time_format(ZonedDateTime dateTime, Object format, Locale locale) {
public static String date_time_format_with_locale(ZonedDateTime dateTime, Object format, Locale locale) {
GregoryTravis marked this conversation as resolved.
Show resolved Hide resolved
return DateTimeFormatter.ofPattern(format.toString()).withLocale(locale).format(dateTime);
}

Expand Down
Loading