Skip to content

Fix propagation of ExitCase in Resource#{both,combineK} #3307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
djspiewak merged 9 commits into from
Dec 24, 2022
Merged

Fix propagation of ExitCase in Resource#{both,combineK} #3307

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8d70133
Correctly handle exit cases in `Resource#both`
armanbilge Dec 6, 2022
1a37a02
Revert changes to `Resource#both`
armanbilge Dec 6, 2022
d82fba1
Handle `ExitCase` in `Resource#fold`, fix `both`, `combineK`
armanbilge Dec 6, 2022
605dccf
Split into smaller tests
armanbilge Dec 6, 2022
d2d27d6
Add `ExitCase` tests for `Resource#combineK`
armanbilge Dec 6, 2022
9e8d092
Fix awk wording
armanbilge Dec 6, 2022
484a984
Document `ExitCase` semantics for `Resource#both`
armanbilge Dec 24, 2022
d1f0397
Add more test cases for r `Resource#combineK`
armanbilge Dec 24, 2022
be51ea6
Fix scaladoc linking
armanbilge Dec 24, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 42 additions & 19 deletions kernel/shared/src/main/scala/cats/effect/kernel/Resource.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ sealed abstract class Resource[F[_], +A] extends Serializable {

private[effect] def fold[B](
onOutput: A => F[B],
onRelease: F[Unit] => F[Unit]
onRelease: (ExitCase => F[Unit], ExitCase) => F[Unit]
)(implicit F: MonadCancel[F, Throwable]): F[B] = {
sealed trait Stack[AA]
case object Nil extends Stack[A]
Expand All @@ -178,7 +178,7 @@ sealed abstract class Resource[F[_], +A] extends Serializable {
}
} {
case ((_, release), outcome) =>
onRelease(release(ExitCase.fromOutcome(outcome)))
onRelease(release, ExitCase.fromOutcome(outcome))
}
case Bind(source, fs) =>
loop(source, Frame(fs, stack))
Expand All @@ -204,7 +204,7 @@ sealed abstract class Resource[F[_], +A] extends Serializable {
* the result of applying [F] to
*/
def use[B](f: A => F[B])(implicit F: MonadCancel[F, Throwable]): F[B] =
fold(f, identity)
fold(f, _.apply(_))

/**
* For a resource that allocates an action (type `F[B]`), allocate that action, run it and
Expand Down Expand Up @@ -251,6 +251,10 @@ sealed abstract class Resource[F[_], +A] extends Serializable {
* _each_ of the two resources, nested finalizers are run in the usual reverse order of
* acquisition.
*
* The same [[Resource.ExitCase]] is propagated to every finalizer. If both resources acquired
* successfully, the [[Resource.ExitCase]] is determined by the outcome of [[use]]. Otherwise,
* it is determined by which resource failed or canceled first during acquisition.
*
* Note that `Resource` also comes with a `cats.Parallel` instance that offers more convenient
* access to the same functionality as `both`, for example via `parMapN`:
*
Expand Down Expand Up @@ -281,19 +285,31 @@ sealed abstract class Resource[F[_], +A] extends Serializable {
def both[B](
that: Resource[F, B]
)(implicit F: Concurrent[F]): Resource[F, (A, B)] = {
type Update = (F[Unit] => F[Unit]) => F[Unit]
type Finalizer = Resource.ExitCase => F[Unit]
type Update = (Finalizer => Finalizer) => F[Unit]

def allocate[C](r: Resource[F, C], storeFinalizer: Update): F[C] =
r.fold(_.pure[F], release => storeFinalizer(F.guarantee(_, release)))

val bothFinalizers = F.ref(F.unit -> F.unit)

Resource.make(bothFinalizers)(_.get.flatMap(_.parTupled).void).evalMap { store =>
val thisStore: Update = f => store.update(_.bimap(f, identity))
val thatStore: Update = f => store.update(_.bimap(identity, f))
r.fold(
_.pure[F],
(release, _) => storeFinalizer(fin => ec => F.unit >> fin(ec).guarantee(release(ec)))
)

val noop: Finalizer = _ => F.unit
val bothFinalizers = F.ref((noop, noop))

Resource
.makeCase(bothFinalizers) { (finalizers, ec) =>
finalizers.get.flatMap {
case (thisFin, thatFin) =>
F.void(F.both(thisFin(ec), thatFin(ec)))
}
}
.evalMap { store =>
val thisStore: Update = f => store.update(_.bimap(f, identity))
val thatStore: Update = f => store.update(_.bimap(identity, f))

(allocate(this, thisStore), allocate(that, thatStore)).parTupled
}
F.both(allocate(this, thisStore), allocate(that, thatStore))
}
}

/**
Expand Down Expand Up @@ -661,12 +677,19 @@ sealed abstract class Resource[F[_], +A] extends Serializable {
implicit F: MonadCancel[F, Throwable],
K: SemigroupK[F],
G: Ref.Make[F]): Resource[F, B] =
Resource.make(Ref[F].of(F.unit))(_.get.flatten).evalMap { finalizers =>
def allocate(r: Resource[F, B]): F[B] =
r.fold(_.pure[F], (release: F[Unit]) => finalizers.update(_.guarantee(release)))

K.combineK(allocate(this), allocate(that))
}
Resource
.makeCase(Ref[F].of((_: Resource.ExitCase) => F.unit))((fin, ec) =>
fin.get.flatMap(_(ec)))
.evalMap { finalizers =>
def allocate(r: Resource[F, B]): F[B] =
r.fold(
_.pure[F],
(release, _) =>
finalizers.update(fin => ec => F.unit >> fin(ec).guarantee(release(ec)))
)

K.combineK(allocate(this), allocate(that))
}

}

Expand Down
142 changes: 142 additions & 0 deletions tests/shared/src/test/scala/cats/effect/ResourceSpec.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -598,6 +598,84 @@ class ResourceSpec extends BaseSpec with ScalaCheck with Discipline {
leftReleased must beTrue
rightReleased must beTrue
}

"propagate the exit case" in {
import Resource.ExitCase

"use succesfully, test left" >> ticked { implicit ticker =>
var got: ExitCase = null
val r = Resource.onFinalizeCase(ec => IO { got = ec })
r.both(Resource.unit).use(_ => IO.unit) must completeAs(())
got mustEqual ExitCase.Succeeded
}

"use successfully, test right" >> ticked { implicit ticker =>
var got: ExitCase = null
val r = Resource.onFinalizeCase(ec => IO { got = ec })
Resource.unit.both(r).use(_ => IO.unit) must completeAs(())
got mustEqual ExitCase.Succeeded
}

"use errored, test left" >> ticked { implicit ticker =>
var got: ExitCase = null
val ex = new Exception
val r = Resource.onFinalizeCase(ec => IO { got = ec })
r.both(Resource.unit).use(_ => IO.raiseError(ex)) must failAs(ex)
got mustEqual ExitCase.Errored(ex)
}

"use errored, test right" >> ticked { implicit ticker =>
var got: ExitCase = null
val ex = new Exception
val r = Resource.onFinalizeCase(ec => IO { got = ec })
Resource.unit.both(r).use(_ => IO.raiseError(ex)) must failAs(ex)
got mustEqual ExitCase.Errored(ex)
}

"right errored, test left" >> ticked { implicit ticker =>

@filipwiech filipwiech Dec 10, 2022 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be a good idea to describe exit case propagation semantics in the scaladoc for the Resource#both? 🙂

@armanbilge armanbilge Dec 24, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, even in two weeks I've forgotten what it was 😂

var got: ExitCase = null
val ex = new Exception
val r = Resource.onFinalizeCase(ec => IO { got = ec })
r.both(Resource.eval(IO.sleep(1.second) *> IO.raiseError(ex))).use_ must failAs(ex)
got mustEqual ExitCase.Errored(ex)
}

"left errored, test right" >> ticked { implicit ticker =>
var got: ExitCase = null
val ex = new Exception
val r = Resource.onFinalizeCase(ec => IO { got = ec })
Resource.eval(IO.sleep(1.second) *> IO.raiseError(ex)).both(r).use_ must failAs(ex)
got mustEqual ExitCase.Errored(ex)
}

"use canceled, test left" >> ticked { implicit ticker =>
var got: ExitCase = null
val r = Resource.onFinalizeCase(ec => IO { got = ec })
r.both(Resource.unit).use(_ => IO.canceled) must selfCancel
got mustEqual ExitCase.Canceled
}

"use canceled, test right" >> ticked { implicit ticker =>
var got: ExitCase = null
val r = Resource.onFinalizeCase(ec => IO { got = ec })
Resource.unit.both(r).use(_ => IO.canceled) must selfCancel
got mustEqual ExitCase.Canceled
}

"right canceled, test left" >> ticked { implicit ticker =>

@filipwiech filipwiech Dec 10, 2022

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would happen if right was canceled and left errored out (and vice versa)? Would they both get ExitCase.Errored? Maybe it could be a good idea to add tests for those cases. 🙂

@armanbilge armanbilge Dec 24, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would happen if right was canceled and left errored out (and vice versa)?

Depends which happens first. If right cancels first, it's covered by this test. If left errors first it's covered by "left errored, test right"

var got: ExitCase = null
val r = Resource.onFinalizeCase(ec => IO { got = ec })
r.both(Resource.eval(IO.sleep(1.second) *> IO.canceled)).use_ must selfCancel
got mustEqual ExitCase.Canceled
}

"left canceled, test right" >> ticked { implicit ticker =>
var got: ExitCase = null
val r = Resource.onFinalizeCase(ec => IO { got = ec })
Resource.eval(IO.sleep(1.second) *> IO.canceled).both(r).use_ must selfCancel
got mustEqual ExitCase.Canceled
}
}
}

"releases both resources on combineK" in ticked { implicit ticker =>
Expand Down Expand Up @@ -642,6 +720,70 @@ class ResourceSpec extends BaseSpec with ScalaCheck with Discipline {
lhs eqv rhs
}
}

"propagate the exit case" in {
import Resource.ExitCase

"use succesfully, test left" >> ticked { implicit ticker =>
var got: ExitCase = null
val r = Resource.onFinalizeCase(ec => IO { got = ec })
r.combineK(Resource.unit).use(_ => IO.unit) must completeAs(())
got mustEqual ExitCase.Succeeded
}

"use errored, test left" >> ticked { implicit ticker =>
var got: ExitCase = null
val ex = new Exception
val r = Resource.onFinalizeCase(ec => IO { got = ec })
r.combineK(Resource.unit).use(_ => IO.raiseError(ex)) must failAs(ex)
got mustEqual ExitCase.Errored(ex)
}

"left errored, test left" >> ticked { implicit ticker =>

@filipwiech filipwiech Dec 10, 2022

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to add tests for "left canceled, test left/right" too? 🙂

@armanbilge armanbilge Dec 24, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If left cancels, no resources are acquired, so no finalizers are run.

var got: ExitCase = null
val ex = new Exception
val r = Resource.onFinalizeCase(ec => IO { got = ec }) *>
Resource.eval(IO.raiseError(ex))
r.combineK(Resource.unit).use_ must completeAs(())
got mustEqual ExitCase.Succeeded
}
Comment on lines +742 to +749

@armanbilge armanbilge Dec 6, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This semantic is kind of counter-intuitive. Even though the left Resource errors, ultimately the combineKed Resource is acquired and used successfully. So the left finalizer is passed Succeeded.

This is a by-product of the fact that the left Resource is not released eagerly even if it errors. Maybe this semantic is undesirable?

@filipwiech filipwiech Dec 10, 2022 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After mulling this over I think it makes sense and is expected, although I could be wrong (I went back and forth on this in my mind a couple times before settling on the conclusion). It would be consistent with what Resource.both is doing, the reasoning being as follows:

Resource.both requires both Resources to be acquired successfully - if any of them errors out (or gets canceled), then all of them get an ExitCase.Errored (or ExitCase.Canceled respectively), even the one that succeded, because the operation as a whole has failed.

Conversely, Resource.combineK needs any of the combined Resources to be acquired successfully - if just one of them succedes, then all of them (well, the ones that were actually evaluated) get an ExitCase.Succeeded, even the ones that failed, because the operation as a whole has succeded.

The fact that ExitCase from the use operation propagates to all the input Resources, regardless of their own, seems to be aligned as well.

I hope it makes sense. In any case, whichever semantic will be introduced, it could be a good idea to document it in the scaladoc, especially if it seems to be counter-intuitive. 🙂

However, I'm not sure if any sort of consistency between Reource.both and combineK is desired. There's also the case of Resource.race which will release loser eagerly in #3226.

@filipwiech filipwiech Dec 10, 2022 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that ExitCase from the use operation propagates to all the input Resources, regardless of their own, seems to be aligned as well.

One more thought regarding this: one could say that in the case of combineK only one Resource is used in the end (similarly to the winner of Resource.race and unlike Resource.both which utilizes all input Resources), so there's no need to propagate ExitCase from use to the one that wasn't. So I guess that could be the reason to maybe release eagerly in combineK. Sorry for the rambling, perhaps I'm not as convinced as I was before. 😄

@djspiewak djspiewak Dec 24, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this is somewhat similar to the following:

case object TestException extends RuntimeException

Resource.raiseError[IO, Unit, Throwable](TestException)
  .onFinalizeCase(IO.println(_))
  .voidError

Do we print out Succeeded here? If so, then this combineK semantic is reasonable. If not, then we're being inconsistent.

@armanbilge armanbilge Dec 24, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following program doesn't print.

//> using lib "org.typelevel::cats-effect::3.4.2"

import cats.effect._
import cats.syntax.all._

object App extends App {
  case object TestException extends RuntimeException

  def run = Resource
    .raiseError[IO, Unit, Throwable](TestException)
    .onFinalizeCase(IO.println(_))
    .voidError
    .use_

}

@djspiewak djspiewak Dec 24, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It prints for me? Succeeded. Here's an even more interesting example:

(Resource.unit.onFinalizeCase(IO.println(_)) *> Resource.raiseError[IO, Unit, Throwable](TestException)).voidError.use_

Prints Succeeded. Meanwhile:

(Resource.unit.onFinalizeCase(IO.println(_)) *> Resource.raiseError[IO, Unit, Throwable](TestException)).use_

Prints Errored(...).

I think combineK is fine as-is. It's rather weird, but we're at least consistent with the backpropagation of final state.


"left errored, test right" >> ticked { implicit ticker =>

@filipwiech filipwiech Dec 10, 2022

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case one of the Resources errors out during acqusition and the other one self-cancels, will the ExitCase of the right Resource propagate/overwrite the result on the left? Could be worth it to add tests for those scenarios. 🙂

@armanbilge armanbilge Dec 24, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a resource errors out or cancels during acquisition, then it has no exit case because the finalizer never runs.

var got: ExitCase = null
val ex = new Exception
val r = Resource.onFinalizeCase(ec => IO { got = ec })
Resource.eval(IO.raiseError(ex)).combineK(r).use_ must completeAs(())
got mustEqual ExitCase.Succeeded
}

"left errored, use errored, test right" >> ticked { implicit ticker =>
var got: ExitCase = null
val ex = new Exception
val r = Resource.onFinalizeCase(ec => IO { got = ec })
Resource
.eval(IO.raiseError(new Exception))
.combineK(r)
.use(_ => IO.raiseError(ex)) must failAs(ex)
got mustEqual ExitCase.Errored(ex)
}

"use canceled, test left" >> ticked { implicit ticker =>

@filipwiech filipwiech Dec 10, 2022 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to add cases for "use errored/canceled, test right" (for when left Resource fails to acquire, but right one succedes)? 🙂

@armanbilge armanbilge Dec 24, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it would!

var got: ExitCase = null
val r = Resource.onFinalizeCase(ec => IO { got = ec })
r.combineK(Resource.unit).use(_ => IO.canceled) must selfCancel
got mustEqual ExitCase.Canceled
}

"left errored, use canceled, test right" >> ticked { implicit ticker =>
var got: ExitCase = null
val r = Resource.onFinalizeCase(ec => IO { got = ec })
Resource
.eval(IO.raiseError(new Exception))
.combineK(r)
.use(_ => IO.canceled) must selfCancel
got mustEqual ExitCase.Canceled
}
}
}

"surround" should {
Expand Down