A way to copy an instance while replacing some fields

[bluetech]

When programming with an immutable/frozen data structure (which I personally prefer to do whenever it's reasonable), perhaps the most common operation is to replace some subset of the fields with new values, returning a new instance. It would be nice if dataclasses had similar functionality, for frozen classes at least.

Some prior art:

---

Python 3 dict:

a = {'key1': 'value1', 'key2': 'value2'}
b = {**a, 'key1': 'VALUE1'}

---

namedtuple _replace:

NT = namedtuple('NT', ('field1', 'field2'))
a = NT(field1='value1', field2='value2')
b = a._replace(field1='VALUE1')

---

attrs evolve:

@attr.s
class C:
     field1 = attr.ib()
     field2 = attr.ib()
a = C(field1='value1', field2='value2')
b = attr.evolve(a, field1='VALUE1')

---

Kotlin data class copy:

data class C(val field1: String, val field2: String)
val a = C(field1 = "value1", field2 = "value2")
val b = a.copy(field1 = "VALUE1")

---

Clojure record assoc:

(defrecord R [field1 field2])
(def a (R. "value1" "value2"))
(def b (assoc a :field1 "VALUE1"))

---

There are of course other example, but I think this shows the variety of names.

[ilevkivskyi]

Big +1 from me. Somehow I forgot about this functionality when adding asdict and astuple. I think this should be a module level helper replace(), rather than a method (for consistency with fields(), asdict(), and astuple()).

[ericvsmith]

Agreed. The API should be replace(inst, **kwargs), to match namedtuple.

[ericvsmith]

attrs has 2 methods for doing this:

• The deprecated assoc: http://www.attrs.org/en/stable/api.html#attr.assoc, which copies all of the fields of the original object, and replaces fields with the supplied ones
• evolve: http://www.attrs.org/en/stable/api.html#attr.evolve, which creates a new object by calling __init__ with object's fields plus replaced fields as supplied. This won't work with a dataclass with init=False at the class level.

There's discussion at python-attrs/attrs#116 and python-attrs/attrs#124. I think the difference matters more to attrs because of their validators, but it still makes a difference here. With the evolve()-style behavior, which calls __init__, then:

      @dataclass
      class C:
          x: int
          y: int = field(init=False, default=10)

      c = C(1)
      c.y = 20

      c1 = replace(c, x=5)

would give c1.y = 10, since it would be set that way in __init__. I'm thinking that we want the first, assoc-like behavior, where c1.y would be 20.

I'll admit, though, that I haven't completely thought it through. In the attrs discussion there's talk of a speed difference between the two approaches. I have a PR for the evolve-like behavior, that I'll post shortly.

[gvanrossum]

That can't be right, either approach should result in y being 20. The difference would matter when __init__ does something funny. There's a similar thing with unpickle and copy, which don't call __init__. On Sep 17, 2017 4:44 PM, "ericvsmith" <notifications@github.com> wrote: attrs has 2 methods for doing this: - The deprecated assoc: http://www.attrs.org/en/ stable/api.html#attr.assoc, which copies all of the fields of the original object, and replaces fields with the supplied ones - evolve: http://www.attrs.org/en/stable/api.html#attr.evolve, which creates a new object by calling __init__ with object's fields plus replaced fields as supplied. This won't work with a dataclass with init=False at the class level. There's discussion at python-attrs/attrs#116 <python-attrs/attrs#116> and python-attrs/attrs#124 <python-attrs/attrs#124>. I think the difference matters more to attrs because of their validators, but it still makes a difference here. With the evolve()-style behavior, which calls __init__, then: @DataClass class C: x: int y: int = field(init=False, default=10) c = C(1) c.y = 20 c1 = replace(c, x=5) would give c1.y = 10, since it would be set that way in __init__. I'm thinking that we want the first, assoc-like behavior, where c1.y would be 20. I'll admit, though, that I haven't completely thought it through. In the attrs discussion there's talk of a speed difference between the two approaches. I have a PR for the evolve-like behavior, that I'll post shortly. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACwrMkB3R2IxIbEC81ab0FuNCKKT1bsNks5sja7egaJpZM4PSScQ> .

[ericvsmith]

Well for the evolve()-like code, it gives y=10 because that's the default value set in __init__. Here's a more complete example:

      @dataclass
      class C:
          x: int
          y: int = field(init=False, default=10)
          z: int

      c = C(1, 4)  # those are x and z values: no value for y, since it has init=False
      c.y = 20
      # now, c == C(x=1, y=20, z=4)

      c1 = replace(c, x=5)
      # Creates a new instance by calling C(x=5, z=c.z)
      # So c1 == C(x=5, y=10, z=4)

The y=10 for c1 comes from it being set in __init__(), since that's the default value.

I'm not saying that's the preferred behavior, I'm saying that's how attrs works.

In my earlier message I said I prefer the evolve()-like behavior, but that was a typo (that I'm not going to edit, since it might change the responses).

I think the preferred approach to c1 = replace(c, x=5) would give c1 as C(x=5, y=20, z=4). That is, the equivalent of:

c1 = <new C object, not created by calling __init__>
c1.x = 5  # the new value given in the replace() call
c1.y = c.y  # for fields not given in replace(), use the value from the source object
c1.z = c.z

Even if it's slower, this is the path of least surprise.

[gvanrossum]

Oh, wow, the evolve() behavior for something with init=False is indeed counter-intuitive. I'm sure @hynek can explain this, perhaps the use case for init=False is not what I think.

I agree with your preference for replace() semantics except I'm not so sure about not calling __init__() -- what if __init__() is overridden to have some registration side effect? Copied objects would not get registered.

[ericvsmith]

Just to show the attr.evolve() behavior is as described:

>>> @attr.s
... class C:
...    x = attr.ib()
...    y = attr.ib(init=False, default=10)
...    z = attr.ib()
...
>>> c = C(1, 4)
>>> c.y = 20
>>> c
C(x=1, y=20, z=4)
>>>
>>> c1 = attr.evolve(c, x=5)
>>> c1
C(x=5, y=10, z=4)

For dataclasses.replace(), I guess I could call __init__() (with only the init=True params), then set all of the init=False fields after the object is created.

In both passes, the value to be used would come from the source object, or the supplied params if given.

[hynek]

I'm on a vacation/phone only ATM. It does look like a weird edge case that nobody hasn't run into tho. Maybe @Tinche can make more sense of it short term.

[Tinche]

Eric got it right.

Initially assoc was copy-based (as in copy.copy), but it was replaced with evolve for several reasons:

• copy.copy is really slow
• copying the instance and replacing the fields doesn't trigger the validators, converters or __attrs_post_init__
• conceptually we wanted evolve to basically be sugar for creating a new instance, but in a more convenient way (especially if the class in question has many fields, and you only want to change a few)

This means evolve doesn't work as well if the class in question has a non-standard __init__. If you've written a totally custom __init__, then it won't work at all. If you have fields with init=False on them, it doesn't touch them.

I actually haven't considered just assigning fields that are init=False but have defaults after the new instance is created. At first blush it sounds like a good enhancement.

[ericvsmith]

I've created PR #67 for this. I'll mull it over for a while, and if no comments, merge it.