Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release v2 #240

Merged
merged 107 commits into from
Jul 17, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
107 commits
Select commit Hold shift + click to select a range
b61ff5f
fix: Assume kotlin.Pair as a normal data class instead of an union
Chuckame Jan 21, 2024
79a132e
Merge pull request #174 from Chuckame/fix/kotlin-pair
Chuckame Jan 27, 2024
6a839b2
feat!: No more reflection and customizable logical types
Chuckame Jan 21, 2024
17eaf60
Merge pull request #175 from Chuckame/reflectionless
Chuckame Jan 27, 2024
dc27771
Merge pull request #177 from avro-kotlin/main
Chuckame Jan 27, 2024
c8492de
Generalize encoding/decoding tests (#168)
thake Jan 28, 2024
7696fe1
Merge pull request #179 from Chuckame/test/generalize-tests
Chuckame Jan 28, 2024
a485a3c
chore(build): Add spotless with ktlint + editorconfig
Chuckame Jan 28, 2024
4d59d4d
chore(spotless): Apply spotless
Chuckame Jan 28, 2024
24c9f2b
Merge pull request #180 from Chuckame/chore/formatting
Chuckame Jan 28, 2024
563c20f
feat: Fully support value classes and remove @AvroInline
Chuckame Jan 28, 2024
91dea8e
feat: Handle contextual map keys #114
Chuckame Jan 28, 2024
f6b186e
remove useless InlineDecoder as it is natively handled now
Chuckame Jan 29, 2024
b6f93e3
@AvroDefault & @ScalePrecision no more available for classes, as it i…
Chuckame Feb 8, 2024
1574c53
added assertThat test utils
Chuckame Feb 9, 2024
ebf6cc1
fixed decoding for map nullable values
Chuckame Feb 9, 2024
98cc0ae
remove useless line
Chuckame Apr 10, 2024
fdb53fc
apply spotless
Chuckame Apr 10, 2024
3cf493a
Merge pull request #183 from Chuckame/feat/value-classes
Chuckame Apr 10, 2024
f85e658
feat: Separate naming strategies (#178), remove AvroName[space] and a…
Chuckame Jan 28, 2024
237a3a1
feat: Add naming cache
Chuckame Feb 3, 2024
f0ec3bb
Merge pull request #182 from Chuckame/feat/separated-naming-strategy
Chuckame Apr 11, 2024
8ce714f
feat: Merge ScalePrecision to AvroDecimalLogicalType
Chuckame Apr 11, 2024
9a126af
Merge pull request #191 from Chuckame/feat/merge-decimal-ann
Chuckame Apr 11, 2024
cb35a3b
chore: Upgrade github actions and use standard gradle actions
Chuckame Apr 11, 2024
8aa48f3
Merge pull request #192 from Chuckame/gh-actions-update
Chuckame Apr 11, 2024
ea31c5d
chore: Add kover
Chuckame Apr 21, 2024
9e7ab01
deps: Remove snappy library
Chuckame Apr 21, 2024
29cc5e1
feat: revamp the schema generation
Chuckame Apr 21, 2024
69a369a
remove namingStrategy cache as there is a schema cache
Chuckame Apr 15, 2024
eee8a81
Merge pull request #190 from Chuckame/descriptor-visitor
Chuckame Apr 22, 2024
e3fdf92
feat: Handle char type
Chuckame Apr 21, 2024
bddc501
feat: New Avro entrypoint
Chuckame Apr 22, 2024
b3bf051
chore: remove redondant null checks
Chuckame Apr 22, 2024
b27b66d
chore: rename RecordNamingStrategy to TypeNamingStrategy
Chuckame Apr 22, 2024
85dc5c5
Merge pull request #186 from Chuckame/avro-api
Chuckame Apr 22, 2024
a4223ac
feat: Support everything at root level
Chuckame May 3, 2024
a6b4817
Merge pull request #202 from Chuckame/anything-root
Chuckame May 3, 2024
a4af252
feat!: Set @AvroEnumDefault directly to the enum value instead of the…
Chuckame May 3, 2024
bb5b035
feat!: Merge AvroJsonProp to AvroProp #201
Chuckame May 3, 2024
091f36c
Merge pull request #203 from Chuckame/enum-default
Chuckame May 3, 2024
325bd3f
Merge pull request #204 from Chuckame/unify-avro-json-prop
Chuckame May 3, 2024
a6df754
build: Explicit API mode to prevent exposing internal stuff
Chuckame May 3, 2024
e66ed3e
Merge pull request #205 from Chuckame/explicit-api-mode
Chuckame May 3, 2024
a16ca0c
deps: Upgrade kotlinx-serialization and kotlin
Chuckame May 9, 2024
0ec51a5
perf: Improve encoding performances with inlined union resolving
Chuckame May 7, 2024
a0d4272
chore: Add benchmark module
Chuckame May 9, 2024
a84209c
Merge pull request #208 from Chuckame/union-perf-improvement
Chuckame May 9, 2024
f3b2243
Merge pull request #209 from Chuckame/deps
Chuckame May 9, 2024
31b7889
docs: Improve README.md
Chuckame May 9, 2024
a030681
chore!: Stop supporting Timestamp class
Chuckame May 9, 2024
7c07046
chore! Remove custom schema as it is not supported in the serialization
Chuckame May 9, 2024
f3964d0
chore!: Internalize AvroLogicalType annotation, waiting for the new k…
Chuckame May 9, 2024
0c0725c
Merge pull request #210 from Chuckame/docs
Chuckame May 9, 2024
c8437a5
add sealed class in benchmark
Chuckame May 10, 2024
e8993e4
test: Support record with different field positions
Chuckame May 21, 2024
2d919da
test: Added nullable test case for nullable logical types
Chuckame May 11, 2024
7afa6ad
perf: Stop using tagger encoder
Chuckame May 11, 2024
be56714
perf: Stop using tagger decoder
Chuckame May 11, 2024
0ccd5d5
Add ExperimentalSerializationApi marker
Chuckame May 21, 2024
34e6393
feat!: No more kotlin-reflect for logical types
Chuckame May 23, 2024
7c90993
upgrade github workflows
Chuckame May 21, 2024
9cef510
Add gradle build scan
Chuckame May 23, 2024
8a99c60
docs: Add generic encoding note
Chuckame May 23, 2024
41dfff3
Merge pull request #214 from Chuckame/no-reflection
Chuckame May 23, 2024
b7f3b6b
perf: Direct decoding & encoding
Chuckame May 28, 2024
31306c1
Merge pull request #215 from Chuckame/direct-encoding
Chuckame May 29, 2024
c36c443
feat: Allow generating a release on a non-main branch
Chuckame May 29, 2024
2040d25
Merge pull request #217 from Chuckame/fix-release
Chuckame May 29, 2024
d3bc54b
feat: Allow adding props to a given type using value classes
Chuckame May 30, 2024
afc5fff
Merge pull request #219 from Chuckame/fix/props-value-class-element
Chuckame May 30, 2024
8c7b221
deps: Use non-RC version of kotlinx-serialization
Chuckame Jun 20, 2024
4804a9a
Merge pull request #221 from Chuckame/deps/non-rc-kotlinx-serialization
Chuckame Jun 20, 2024
a88f5cc
deps: Upgrade plugins, trying to fix publication failing
Chuckame Jun 20, 2024
9df5c94
Merge pull request #222 from Chuckame/deps/upgrade-versions
Chuckame Jun 20, 2024
78737b4
fix: Removed AvroNamespaceOverride as it was not fully implemented
Chuckame Jun 25, 2024
b6e52fc
feat: Implement BinaryFormat
Chuckame Jun 25, 2024
87c58ac
Add migration guide
Chuckame Jun 25, 2024
25f3f38
Merge pull request #224 from Chuckame/docs
Chuckame Jun 25, 2024
e5f5137
benchmark: Added apache avro lib generic<->binary benchmark
Chuckame Jun 25, 2024
7793a06
little refactor
Chuckame Jun 25, 2024
948643e
unifying union checks
Chuckame Jun 25, 2024
23814d2
Merge pull request #225 from Chuckame/improve-benchmark
Chuckame Jun 25, 2024
933fabb
fix docs
Chuckame Jun 26, 2024
78cf46f
Create dependabot.yml
Chuckame Jul 1, 2024
71a8e3c
Merge pull request #227 from avro-kotlin/Dependabot
Chuckame Jul 1, 2024
83544bd
Merge pull request #226 from Chuckame/docs
Chuckame Jul 1, 2024
228fcd1
handle nullable bytearrays and add null values in benchmark
Chuckame Jul 5, 2024
976dee7
Merge pull request #228 from Chuckame/docs
Chuckame Jul 5, 2024
5df37fb
feat: Add duration logical type
Chuckame Jul 10, 2024
ad17bb3
Merge pull request #233 from Chuckame/feat/duration-logical-type
Chuckame Jul 10, 2024
83934fc
fix: Only handle ByteArrays as bytes, and collection of Byte as array…
Chuckame Jul 10, 2024
489b45a
Merge pull request #234 from Chuckame/fix/remove-bytes
Chuckame Jul 10, 2024
01ccbfc
fix: No more automatic padding for fixed type
Chuckame Jul 10, 2024
34c32bb
Merge pull request #235 from Chuckame/feat/no-padding
Chuckame Jul 10, 2024
76d0134
fix: Update docs
Chuckame Jul 10, 2024
143d6a9
Merge pull request #237 from Chuckame/docs
Chuckame Jul 11, 2024
837ea2f
feat: Add @AvroStringable
Chuckame Jul 10, 2024
b0aec20
Merge pull request #236 from Chuckame/feat/stringable
Chuckame Jul 11, 2024
eb37957
feat: Remove AvroDecimal defaults
Chuckame Jul 11, 2024
eda4375
Merge pull request #238 from Chuckame/feat/decimal-default
Chuckame Jul 11, 2024
999a90c
feat: Add implicitEmptyCollections configuration
Chuckame Jul 14, 2024
967f5ed
Merge pull request #239 from Chuckame/implicit-empty-collections
Chuckame Jul 15, 2024
6a8dd19
Merge branch 'main' into main-v2
Chuckame Jul 16, 2024
84e88f4
fix documentation
Chuckame Jul 16, 2024
b64158f
benchmark: add simple use case
Chuckame Jul 16, 2024
f617d83
spotless + apiDump
Chuckame Jul 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix: Removed AvroNamespaceOverride as it was not fully implemented
  • Loading branch information
Chuckame committed Jun 25, 2024
commit 78737b4441649fef0cd03f7544378953a06604a0
24 changes: 0 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -737,30 +737,6 @@ package my.package
data class MyData(val myField: String)
```

## Changing the namespace of all nested named type(s)

Sometimes, using classes from other packages or libraries, you may want to change the namespace of a nested named type. This is done annotating the field
with `@AvroNamespaceOverride`.

```kotlin
import kotlinx.serialization.Serializable
import com.github.avrokotlin.avro4k.AvroNamespaceOverride

@Serializable
data class MyData(
@AvroNamespaceOverride("new.namespace") val myField: NestedRecord
)

// ...
package external.package.name

@Serializable
data class NestedRecord(val field: String)
```

> [!NOTE]
> This impacts the schema generation, the serialization and the deserialization.

## Change type name (FIXED only)

> [!WARNING]
Expand Down
9 changes: 0 additions & 9 deletions api/avro4k-core.api
Original file line number Diff line number Diff line change
Expand Up @@ -164,15 +164,6 @@ public final class com/github/avrokotlin/avro4k/AvroKt {
public static final fun schema (Lcom/github/avrokotlin/avro4k/Avro;Lkotlinx/serialization/KSerializer;)Lorg/apache/avro/Schema;
}

public abstract interface annotation class com/github/avrokotlin/avro4k/AvroNamespaceOverride : java/lang/annotation/Annotation {
public abstract fun value ()Ljava/lang/String;
}

public synthetic class com/github/avrokotlin/avro4k/AvroNamespaceOverride$Impl : com/github/avrokotlin/avro4k/AvroNamespaceOverride {
public fun <init> (Ljava/lang/String;)V
public final synthetic fun value ()Ljava/lang/String;
}

public final class com/github/avrokotlin/avro4k/AvroObjectContainerFile {
public fun <init> ()V
public fun <init> (Lcom/github/avrokotlin/avro4k/Avro;)V
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,6 @@ import kotlinx.serialization.descriptors.SerialDescriptor
import org.apache.avro.LogicalType
import org.intellij.lang.annotations.Language

/**
* When annotated on a property, deeply overrides the namespace for all the nested named types (records, enums and fixed).
*
* Works with standard classes and inline classes.
*/
@SerialInfo
@ExperimentalSerializationApi
@Target(AnnotationTarget.PROPERTY)
public annotation class AvroNamespaceOverride(
val value: String,
)

/**
* Adds a property to the Avro schema or field. Its value could be any valid JSON or just a string.
*
Expand Down Expand Up @@ -83,6 +71,11 @@ public annotation class AvroFixed(val size: Int)
/**
* Sets the default avro value for a record's field.
*
* - Records and maps have to be represented as a json object
* - Arrays have to be represented as a json array
* - Nulls have to be represented as a json `null`. To set the string `"null"`, don't forget to quote the string, example: `""""null""""` or `"\"null\""`.
* - Any non json content will be treated as a string
*
* Ignored in inline classes.
*/
@SerialInfo
Expand Down Expand Up @@ -112,7 +105,7 @@ public annotation class AvroEnumDefault
* }
* ```
*
* For more complex needs, please file an issue [here](https://github.com/avro-kotlin/avro4k/issues).
* For more complex needs, please file an feature request [here](https://github.com/avro-kotlin/avro4k/issues).
*/
@ExperimentalSerializationApi
public fun SerialDescriptor.asAvroLogicalType(): SerialDescriptor {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ import com.github.avrokotlin.avro4k.AvroDefault
import com.github.avrokotlin.avro4k.internal.isStartingAsJson
import com.github.avrokotlin.avro4k.internal.jsonNode
import com.github.avrokotlin.avro4k.internal.nonNullSerialName
import com.github.avrokotlin.avro4k.internal.overrideNamespace
import kotlinx.serialization.SerializationException
import kotlinx.serialization.builtins.serializer
import kotlinx.serialization.descriptors.SerialDescriptor
Expand Down Expand Up @@ -99,25 +98,20 @@ internal class ClassVisitor(
elementDescriptor: SerialDescriptor,
elementSchema: Schema,
): Schema.Field {
var finalSchema: Schema = annotations.namespaceOverride?.value?.let { elementSchema.overrideNamespace(it) } ?: elementSchema
val fieldDefault = getFieldDefault(annotations.default, elementSchema, elementDescriptor)

val fieldDefault = getFieldDefault(annotations.default, finalSchema, elementDescriptor)

if (fieldDefault != null) {
reorderUnionIfNeeded(fieldDefault, finalSchema)?.let {
finalSchema = it
val finalSchema =
if (fieldDefault != null) {
reorderUnionIfNeeded(fieldDefault, elementSchema)
} else {
elementSchema
}
}

val field =
Schema.Field(
// name =
fieldName,
// schema =
finalSchema,
// doc =
annotations.doc?.value,
// defaultValue =
fieldDefault
)
annotations.aliases.flatMap { it.value.asSequence() }.forEach { field.addAlias(it) }
Expand All @@ -130,23 +124,23 @@ internal class ClassVisitor(
*/
private fun reorderUnionIfNeeded(
fieldDefault: Any,
finalSchema: Schema,
): Schema? {
if (finalSchema.isUnion && finalSchema.isNullable) {
schema: Schema,
): Schema {
if (schema.isUnion && schema.isNullable) {
var nullNotFirst = false
if (fieldDefault is Collection<*>) {
nullNotFirst = fieldDefault.any { it != JsonProperties.NULL_VALUE }
} else if (fieldDefault != JsonProperties.NULL_VALUE) {
nullNotFirst = true
}
if (nullNotFirst) {
val nullIndex = finalSchema.types.indexOfFirst { it.type == Schema.Type.NULL }
val nonNullTypes = finalSchema.types.toMutableList()
val nullIndex = schema.types.indexOfFirst { it.type == Schema.Type.NULL }
val nonNullTypes = schema.types.toMutableList()
val nullType = nonNullTypes.removeAt(nullIndex)
return Schema.createUnion(nonNullTypes + nullType)
}
}
return null
return schema
}

private fun getFieldDefault(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ package com.github.avrokotlin.avro4k.internal.schema

import com.github.avrokotlin.avro4k.internal.copy
import com.github.avrokotlin.avro4k.internal.jsonNode
import com.github.avrokotlin.avro4k.internal.overrideNamespace
import kotlinx.serialization.descriptors.SerialDescriptor
import org.apache.avro.Schema

Expand All @@ -24,14 +23,13 @@ internal class InlineClassVisitor(
return ValueVisitor(context.copy(inlinedAnnotations = inlinedAnnotations)) { generatedSchema ->
val annotations = InlineClassFieldAnnotations(inlineClassDescriptor)

var schema = generatedSchema
if (annotations.namespaceOverride != null) {
schema = schema.overrideNamespace(annotations.namespaceOverride.value)
}
val props = annotations.props.toList()
if (props.isNotEmpty()) {
schema = schema.copy(additionalProps = props.associate { it.key to it.jsonNode })
}
val schema =
if (props.isNotEmpty()) {
generatedSchema.copy(additionalProps = props.associate { it.key to it.jsonNode })
} else {
generatedSchema
}

onSchemaBuilt(schema)
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ import com.github.avrokotlin.avro4k.AvroAlias
import com.github.avrokotlin.avro4k.AvroDefault
import com.github.avrokotlin.avro4k.AvroDoc
import com.github.avrokotlin.avro4k.AvroFixed
import com.github.avrokotlin.avro4k.AvroNamespaceOverride
import com.github.avrokotlin.avro4k.AvroProp
import com.github.avrokotlin.avro4k.internal.AnnotatedLocation
import com.github.avrokotlin.avro4k.internal.findAnnotation
Expand All @@ -29,11 +28,9 @@ internal fun VisitorContext.resetNesting() = copy(inlinedAnnotations = null)
* Contains all the annotations for a field of a class (kind == CLASS && isInline == true).
*/
internal data class InlineClassFieldAnnotations(
val namespaceOverride: AvroNamespaceOverride?,
val props: Sequence<AvroProp>,
) {
constructor(inlineClassDescriptor: SerialDescriptor) : this(
inlineClassDescriptor.findElementAnnotation<AvroNamespaceOverride>(0),
sequence {
yieldAll(inlineClassDescriptor.findAnnotations<AvroProp>())
yieldAll(inlineClassDescriptor.findElementAnnotations<AvroProp>(0))
Expand All @@ -53,14 +50,12 @@ internal data class FieldAnnotations(
val aliases: Sequence<AvroAlias>,
val doc: AvroDoc?,
val default: AvroDefault?,
val namespaceOverride: AvroNamespaceOverride?,
) {
constructor(descriptor: SerialDescriptor, elementIndex: Int) : this(
descriptor.findElementAnnotations<AvroProp>(elementIndex).asSequence(),
descriptor.findElementAnnotations<AvroAlias>(elementIndex).asSequence(),
descriptor.findElementAnnotation<AvroDoc>(elementIndex),
descriptor.findElementAnnotation<AvroDefault>(elementIndex),
descriptor.findElementAnnotation<AvroNamespaceOverride>(elementIndex)
descriptor.findElementAnnotation<AvroDefault>(elementIndex)
) {
require(descriptor.kind == StructureKind.CLASS) {
"${FieldAnnotations::class.qualifiedName} is only for classes, but trying at element index $elementIndex with non class descriptor $descriptor"
Expand Down
Loading