Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PingSource broken when created before the KSVC #2994

Open
matzew opened this issue Apr 15, 2020 · 11 comments
Open

PingSource broken when created before the KSVC #2994

matzew opened this issue Apr 15, 2020 · 11 comments
Labels
area/sources kind/bug Categorizes issue or PR as related to a bug. triage/accepted Issues which should be fixed (post-triage)
Milestone

Comments

@matzew
Copy link
Member

matzew commented Apr 15, 2020

Describe the bug

create a ping source before a ksvc, and notice that it will never go to READY state.

pingsource.sources.knative.dev/test-ping-source    False   NotFound          4m

Expected behavior

once the ksvc is there, it should go to ready state.

To Reproduce

  1. Install eventing first
  2. Creates a pingsource
  3. Install Serving
  4. Creates a ksvc
  5. Creates a new PingSource
  6. Creates a new ksvc
  7. Notice that none of the PingSources becomes Ready

Knative release version

  • 0.13
  • 0.14

Additional context

here is the yamls, I used:

  • ping.yaml:
apiVersion: sources.knative.dev/v1alpha2
kind: PingSource
metadata:
  name: test-ping-source
spec:
  schedule: "*/2 * * * *"
  jsonData: '{"message": "Hello world!"}'
  sink:
    ref:
      apiVersion: serving.knative.dev/v1
      kind: Service
      name: event-display
  • ksvc.yaml:
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: event-display
spec:
  template:
    spec:
      containers:
        - image: quay.io/openshift-knative/knative-eventing-sources-event-display:v0.13.2
  • ping2.yaml:
apiVersion: sources.knative.dev/v1alpha2
kind: PingSource
metadata:
  name: test-ping-source2
spec:
  schedule: "*/2 * * * *"
  jsonData: '{"message": "Hello world!"}'
  sink:
    ref:
      apiVersion: serving.knative.dev/v1
      kind: Service
      name: event-display2
  • ksvc2.yaml:
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: event-display2
spec:
  template:
    spec:
      containers:
        - image: quay.io/openshift-knative/knative-eventing-sources-event-display:v0.13.2
@matzew matzew added the kind/bug Categorizes issue or PR as related to a bug. label Apr 15, 2020
@matzew
Copy link
Member Author

matzew commented Apr 15, 2020

In the eventing controller log I see something like:

{
  "level": "info",
  "ts": "2020-04-15T12:46:25.361Z",
  "logger": "controller.event-broadcaster",
  "caller": "record/event.go:274",
  "msg": "Event(v1.ObjectReference{Kind:\"PingSource\", Namespace:\"default\", Name:\"test-ping-source2\", UID:\"9c85dc7b-fc75-438d-8f0a-3c5973afd4fd\", APIVersion:\"sources.knative.dev/v1alpha1\", ResourceVersion:\"1931\", FieldPath:\"\"}): type: 'Warning' reason: 'SinkNotFound' Sink not found: {\"ref\":{\"kind\":\"Service\",\"namespace\":\"default\",\"name\":\"event-display2\",\"apiVersion\":\"serving.knative.dev/v1\"}}",
  "commit": "02bc516"
}

@matzew
Copy link
Member Author

matzew commented Apr 15, 2020

the ksvc objects are READY:

 k get ksvc
NAME             URL                                         LATESTCREATED          LATESTREADY            READY   REASON
event-display    http://event-display.default.example.com    event-display-rntw2    event-display-rntw2    True    
event-display2   http://event-display2.default.example.com   event-display2-rwscr   event-display2-rwscr   True    

but the not so the sources:

k get sources
NAME                                               READY   REASON     SINK   AGE
pingsource.sources.knative.dev/test-ping-source    False   NotFound          14m
pingsource.sources.knative.dev/test-ping-source2   False   NotFound          12m

@matzew
Copy link
Member Author

matzew commented Apr 15, 2020

when I kill the eventing controller pod,

the sources are correctly reconciled and eventually will be true. but only when I kill the pod...

@n3wscott
Copy link
Contributor

The first failure is expected. The watch on the ksvc kind fails.

Why it does not recover with the new source is interesting...

@lionelvillard
Copy link
Member

@matzew this scenario works with mtping because the receive adapter is created after the sink existence check. However there is no sink tracker, strange...

@feitnomore
Copy link

feitnomore commented Jul 28, 2020

I'm getting a similar behavior on 0.16:

source.yaml:

apiVersion: sources.knative.dev/v1alpha2
kind: PingSource
metadata:
  name: source
spec:
  schedule: "* * * * *"
  jsonData: '{"message": "Hello world from ping!"}'
  sink:
    ref:
      apiVersion: serving.knative.dev/v1
      kind: Service
      name: service

service.yaml:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: service
spec:
  template:
    spec:
      containers:
        - image: docker.io/meteatamel/event-display:v1

error:
{"level":"info","ts":"2020-07-28T17:40:31.192Z","logger":"controller.event-broadcaster","caller":"record/event.go:281","msg":"Event(v1.ObjectReference{Kind:\"PingSource\", Namespace:\"default\", Name:\"source\", UID:\"0739a3e2-2f22-4b5e-b905-10a6292eb92a\", APIVersion:\"sources.knative.dev/v1alpha2\", ResourceVersion:\"5439586\", FieldPath:\"\"}): type: 'Warning' reason: 'SinkNotFound' Sink not found: {\"ref\":{\"kind\":\"Service\",\"namespace\":\"default\",\"name\":\"service\",\"apiVersion\":\"serving.knative.dev/v1\"}}","commit":"acdd118"}

kubectl get sources:

NAME                                    READY   REASON     SINK   AGE
pingsource.sources.knative.dev/source   False   NotFound          2m5s

feitnomore added a commit to feitnomore/knative-tutorial that referenced this issue Aug 2, 2020
There is an existing issue with the eventing controller that
sometimes prevents the PingSource from reconcile. This will
eventually cause our source to fail with SinkNotFound exception.
In this case, we need to have the Sink/Service in place before
we create the Source.

More details on #2994
Reference: knative/eventing#2994
@grantr grantr added this to the Backlog milestone Aug 24, 2020
@github-actions
Copy link

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

@github-actions github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 25, 2020
@himanshumps
Copy link

Is there a solution to this problem ? I am using camel k and have the same issue. These are the steps I am doing.

echo "Creating the event watcher service account"
envsubst < createEventsSA.yaml | oc apply -f -
echo "Creating the event watcher role"
envsubst < eventWatcherRole.yaml | oc apply -f -
echo "Creating the event watcher role binding"
envsubst < eventWatcherRoleBinding.yaml | oc apply -f -
echo "Creating the event watcher channel"
envsubst < eventWatcherChannel.yaml | oc apply -f -
echo "Creating the event watcher apiserversource"
envsubst < eventWatcherapiserversource.yaml | oc apply -f -

and then I start the service but facing the same issue.

kamel run --dev --property 'quarkus.log.category."org.apache.camel".level'=DEBUG --dependency=camel-jackson --dependency=camel-rest NamespaceEventWatcher.java -n testproject

All Yaml files attached with the java code

issueWithApiServerSources.zip

@himanshumps
Copy link

@feitnomore @grantr - Can you pls assist here

@lionelvillard
Copy link
Member

/remove-lifecycle stale
/reopen
/triage accepted

@knative-prow-robot knative-prow-robot added the triage/accepted Issues which should be fixed (post-triage) label Jan 25, 2022
@knative-prow-robot
Copy link
Contributor

@lionelvillard: Reopened this issue.

In response to this:

/remove-lifecycle stale
/reopen
/triage accepted

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@knative-prow-robot knative-prow-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/sources kind/bug Categorizes issue or PR as related to a bug. triage/accepted Issues which should be fixed (post-triage)
Projects
None yet
Development

No branches or pull requests

7 participants