Skip to content

To dataframe improvements #1081

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Mar 7, 2025
Merged

To dataframe improvements #1081

merged 12 commits into from
Mar 7, 2025

Conversation

AndreiKingsley
Copy link
Collaborator

Fixed behavior for primitive arrays (due to compiler limitations, it is impossible to make extensions for primitive arrays), added extensions for common value types.

@@ -65,6 +65,9 @@ object Names {
val LOCAL_DATE_CLASS_ID = kotlinx.datetime.LocalDate::class.classId()
val LOCAL_DATE_TIME_CLASS_ID = kotlinx.datetime.LocalDateTime::class.classId()
val INSTANT_CLASS_ID = kotlinx.datetime.Instant::class.classId()
val DATE_TIME_PERIOD_CLASS_ID = kotlinx.datetime.DateTimePeriod::class.classId()
Copy link
Collaborator

@koperagen koperagen Feb 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll later remove dependency on kotlinx.datetime, please call ClassId constructor with Fqname + String (check examples above) :)

), session
) ||
this.isSubtypeOf(
ConeClassLikeTypeImpl(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's refactor it a little, extract class ids in Names object and then call Names.TEMPORAL.constructClassLikeType instead of ConeClassLikeTypeImpl...

Copy link
Collaborator Author

@AndreiKingsley AndreiKingsley Feb 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, btw why here was isNullable=false?

Copy link
Collaborator

@koperagen koperagen Feb 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! For number type i have two subtypeOf, one with nullable type and other with non-nullable. For Temporal i missed one. But 1 check with nullable = true will be enough and it's more correct. Test case that works incorrectly now:

class Test(val temporal: Temporal?)

listOf(Test(LocalDateTime.parse("01/01/08 00:02:04", DateTimeFormatter.ofPattern("MM/dd/yy HH:mm:ss"))), Test(null)).toDataFrame(maxDepth = 2)   

Copy link
Collaborator

@koperagen koperagen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a test for compiler plugin part covering the new case

Copy link
Collaborator

@Jolanrensen Jolanrensen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice stability and consistency improvements!

@@ -38,20 +39,37 @@ import kotlin.reflect.jvm.javaField
import kotlin.reflect.typeOf

private val valueTypes = setOf(
Any::class,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uhm, Any doesn't cause issues with other classes? I guess this doesn't mean "a subtype of", rather the exact type? Maybe mention this in kdoc for future us

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

User can provide Iterable<Any?>.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And yeah, these are exact types.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so now if a user has something like listOf(cat, dog, house, table).toDataFrame() they will get a value column, right?

@Jolanrensen
Copy link
Collaborator

Fixed behavior for primitive arrays (due to compiler limitations, it is impossible to make extensions for primitive arrays), added extensions for common value types.

sad we cannot have this for arrays :/ maybe mark the original issue as "won't fix" or "impossible" or something

@AndreiKingsley
Copy link
Collaborator Author

sad we cannot have this for arrays

But as @koperagen said we don't need it since we have plugin !

*/
@PublishedApi
internal val KClass<*>.hasProperties: Boolean
get() = this.memberProperties.isNotEmpty()
Copy link
Collaborator

@Jolanrensen Jolanrensen Mar 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might cause issues with POJO's, java objects with getters+setters instead of memberProperties. We support those with properties(). Maybe you can reuse some of the logic there

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added check

# Conflicts:
#	plugins/kotlin-dataframe/src/org/jetbrains/kotlinx/dataframe/plugin/impl/api/toDataFrame.kt
#	plugins/kotlin-dataframe/src/org/jetbrains/kotlinx/dataframe/plugin/utils/Names.kt
@AndreiKingsley AndreiKingsley merged commit 7c090b4 into master Mar 7, 2025
6 checks passed
@AndreiKingsley AndreiKingsley deleted the to_dataframe_imrovements branch March 7, 2025 17:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Missing overloads for Iterable<ArrayLike>.toDataFrame()
3 participants