construct base object: Failed to filter components; the following components don't exist: [ 'kubeflow-core' ]

[amq]

When following the guide from readme, I'm stuck at the last command

ks apply default -c kubeflow-core

... which fails with

construct base object: Failed to filter components; the following components don't exist: [ 'kubeflow-core' ]

Possible questions

Were all the previous commands successful? - Yes

Did I try starting from scratch? - Yes

Does kubeflow-core.jsonnet exist in components/? - Yes

How do I run the commands? - Tried Docker Quickstart Terminal and MINGW64

Does kubectl get pods work? - Yes

Versions

ks version
ksonnet version: 0.9.1
jsonnet version: v0.9.5
client-go version: 1.8

kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.2", GitCommit:"5fa2db2bd46ac79e5e00a4e6ed24191080aa463b", GitTreeState:"clean", BuildDate:"2018-01-18T10:09:24Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.2", GitCommit:"5fa2db2bd46ac79e5e00a4e6ed24191080aa463b", GitTreeState:"clean", BuildDate:"2018-01-18T09:42:01Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}

Tried minikube 0.25.0 and Docker for Windows 18.03

[jlewi]

It looks like there was a problem creating your ksonnet app. Are you sure you created a component kubeflow-core?

Can you run

ls -la -R

in your ksonnet app dir and share the output

Do you see a file

components/kubeflow-core.jsonnet

[pdmack]

This is an issue using ksonnet with the MinGW terminal. I suspect it is not parsing the path separator for Win32.

[amq]

@pdmack I just removed my-kubeflow/ and re-run the setup in Bash (Ubuntu with Windows WSL), and got the same result. Then using Procmon I've double-checked that components/kubeflow-core.jsonnet is actually being read by ks, it is.

Interesting thing: when I remove components/kubeflow-core.jsonnet, the ks apply default -c kubeflow-core command succeedes, but, as expected, nothing gets deployed.

@jlewi here is the output of ls -la -R

.:
total 4
drwxrwxrwx 0 root root  512 Mar 22 21:22 .
drwxrwxrwx 0 root root  512 Mar 22 21:22 ..
-rwxrwxrwx 1 root root 1056 Mar 22 21:23 app.yaml
drwxrwxrwx 0 root root  512 Mar 22 21:23 components
drwxrwxrwx 0 root root  512 Mar 22 21:22 environments
drwxrwxrwx 0 root root  512 Mar 22 21:22 .ksonnet
drwxrwxrwx 0 root root  512 Mar 22 21:22 lib
drwxrwxrwx 0 root root  512 Mar 22 21:23 vendor

./components:
total 4
drwxrwxrwx 0 root root 512 Mar 22 21:23 .
drwxrwxrwx 0 root root 512 Mar 22 21:22 ..
-rwxrwxrwx 1 root root 487 Mar 22 21:23 kubeflow-core.jsonnet
-rwxrwxrwx 1 root root 842 Mar 22 21:23 params.libsonnet

./environments:
total 0
drwxrwxrwx 0 root root 512 Mar 22 21:22 .
drwxrwxrwx 0 root root 512 Mar 22 21:22 ..
-rwxrwxrwx 1 root root 115 Mar 22 21:22 base.libsonnet
drwxrwxrwx 0 root root 512 Mar 22 21:22 default

./environments/default:
total 0
drwxrwxrwx 0 root root 512 Mar 22 21:22 .
drwxrwxrwx 0 root root 512 Mar 22 21:22 ..
-rwxrwxrwx 1 root root 289 Mar 22 21:22 main.jsonnet
-rwxrwxrwx 1 root root 256 Mar 22 21:22 params.libsonnet

./.ksonnet:
total 0
drwxrwxrwx 0 root root 512 Mar 22 21:22 .
drwxrwxrwx 0 root root 512 Mar 22 21:22 ..
drwxrwxrwx 0 root root 512 Mar 22 21:23 registries

./.ksonnet/registries:
total 0
drwxrwxrwx 0 root root 512 Mar 22 21:23 .
drwxrwxrwx 0 root root 512 Mar 22 21:22 ..
drwxrwxrwx 0 root root 512 Mar 22 21:22 incubator
drwxrwxrwx 0 root root 512 Mar 22 21:23 kubeflow

./.ksonnet/registries/incubator:
total 4
drwxrwxrwx 0 root root 512 Mar 22 21:22 .
drwxrwxrwx 0 root root 512 Mar 22 21:23 ..
-rwxrwxrwx 1 root root 654 Mar 22 21:22 40285d8a14f1ac5787e405e1023cf0c07f6aa28c.yaml

./.ksonnet/registries/kubeflow:
total 0
drwxrwxrwx 0 root root 512 Mar 22 21:23 .
drwxrwxrwx 0 root root 512 Mar 22 21:23 ..
-rwxrwxrwx 1 root root 327 Mar 22 21:23 febb21d1eca9e6800e6ca8c27ed98c88473e346e.yaml

./lib:
total 0
drwxrwxrwx 0 root root 512 Mar 22 21:22 .
drwxrwxrwx 0 root root 512 Mar 22 21:22 ..
drwxrwxrwx 0 root root 512 Mar 22 21:22 v1.7.0

./lib/v1.7.0:
total 4132
drwxrwxrwx 0 root root     512 Mar 22 21:22 .
drwxrwxrwx 0 root root     512 Mar 22 21:22 ..
-rwxrwxrwx 1 root root 2411234 Mar 22 21:22 k8s.libsonnet
-rwxrwxrwx 1 root root    2077 Mar 22 21:22 k.libsonnet
-rwxrwxrwx 1 root root 1813365 Mar 22 21:22 swagger.json

./vendor:
total 0
drwxrwxrwx 0 root root 512 Mar 22 21:23 .
drwxrwxrwx 0 root root 512 Mar 22 21:22 ..
drwxrwxrwx 0 root root 512 Mar 22 21:23 kubeflow

./vendor/kubeflow:
total 0
drwxrwxrwx 0 root root 512 Mar 22 21:23 .
drwxrwxrwx 0 root root 512 Mar 22 21:23 ..
drwxrwxrwx 0 root root 512 Mar 22 21:23 core
drwxrwxrwx 0 root root 512 Mar 22 21:23 tf-job
drwxrwxrwx 0 root root 512 Mar 22 21:23 tf-serving

./vendor/kubeflow/core:
total 92
drwxrwxrwx 0 root root   512 Mar 22 21:23 .
drwxrwxrwx 0 root root   512 Mar 22 21:23 ..
-rwxrwxrwx 1 root root   500 Mar 22 21:23 all.libsonnet
-rwxrwxrwx 1 root root  6539 Mar 22 21:23 ambassador.libsonnet
-rwxrwxrwx 1 root root  4511 Mar 22 21:23 cert-manager.libsonnet
-rwxrwxrwx 1 root root 23964 Mar 22 21:23 iap.libsonnet
-rwxrwxrwx 1 root root  6964 Mar 22 21:23 jupyterhub.libsonnet
-rwxrwxrwx 1 root root  4058 Mar 22 21:23 jupyterhub_spawner.py
-rwxrwxrwx 1 root root  8664 Mar 22 21:23 nfs.libsonnet
-rwxrwxrwx 1 root root   816 Mar 22 21:23 parts.yaml
drwxrwxrwx 0 root root   512 Mar 22 21:23 prototypes
-rwxrwxrwx 1 root root  2055 Mar 22 21:23 README.md
-rwxrwxrwx 1 root root  2599 Mar 22 21:23 spartakus.libsonnet
drwxrwxrwx 0 root root   512 Mar 22 21:23 tests
-rwxrwxrwx 1 root root 11386 Mar 22 21:23 tf-job.libsonnet
-rwxrwxrwx 1 root root   934 Mar 22 21:23 util.libsonnet

./vendor/kubeflow/core/prototypes:
total 12
drwxrwxrwx 0 root root  512 Mar 22 21:23 .
drwxrwxrwx 0 root root  512 Mar 22 21:23 ..
-rwxrwxrwx 1 root root 1867 Mar 22 21:23 all.jsonnet
-rwxrwxrwx 1 root root 1324 Mar 22 21:23 cert-manager.jsonnet
-rwxrwxrwx 1 root root 2044 Mar 22 21:23 iap-ingress.jsonnet

./vendor/kubeflow/core/tests:
total 44
drwxrwxrwx 0 root root  512 Mar 22 21:23 .
drwxrwxrwx 0 root root  512 Mar 22 21:23 ..
-rwxrwxrwx 1 root root 5833 Mar 22 21:23 ambassador_test.jsonnet
-rwxrwxrwx 1 root root 3982 Mar 22 21:23 iap_test.jsonnet
-rwxrwxrwx 1 root root 8989 Mar 22 21:23 jupyterhub_test.jsonnet
-rwxrwxrwx 1 root root 1819 Mar 22 21:23 nfs_test.jsonnet
-rwxrwxrwx 1 root root 2286 Mar 22 21:23 spartakus_test.jsonnet
-rwxrwxrwx 1 root root 5458 Mar 22 21:23 tf-job_test.jsonnet
-rwxrwxrwx 1 root root 1016 Mar 22 21:23 util_test.jsonnet

./vendor/kubeflow/tf-job:
total 12
drwxrwxrwx 0 root root  512 Mar 22 21:23 .
drwxrwxrwx 0 root root  512 Mar 22 21:23 ..
-rwxrwxrwx 1 root root  820 Mar 22 21:23 parts.yaml
drwxrwxrwx 0 root root  512 Mar 22 21:23 prototypes
-rwxrwxrwx 1 root root 2514 Mar 22 21:23 README.md
-rwxrwxrwx 1 root root 1274 Mar 22 21:23 tf-job.libsonnet

./vendor/kubeflow/tf-job/prototypes:
total 12
drwxrwxrwx 0 root root  512 Mar 22 21:23 .
drwxrwxrwx 0 root root  512 Mar 22 21:23 ..
-rwxrwxrwx 1 root root 4443 Mar 22 21:23 tf-cnn-benchmarks.jsonnet
-rwxrwxrwx 1 root root 2348 Mar 22 21:23 tf-job.jsonnet

./vendor/kubeflow/tf-serving:
total 20
drwxrwxrwx 0 root root  512 Mar 22 21:23 .
drwxrwxrwx 0 root root  512 Mar 22 21:23 ..
-rwxrwxrwx 1 root root  854 Mar 22 21:23 parts.yaml
drwxrwxrwx 0 root root  512 Mar 22 21:23 prototypes
-rwxrwxrwx 1 root root 2228 Mar 22 21:23 README.md
-rwxrwxrwx 1 root root 9923 Mar 22 21:23 tf-serving.libsonnet

./vendor/kubeflow/tf-serving/prototypes:
total 4
drwxrwxrwx 0 root root 512 Mar 22 21:23 .
drwxrwxrwx 0 root root 512 Mar 22 21:23 ..
-rwxrwxrwx 1 root root 794 Mar 22 21:23 tf-serving-all-features.jsonnet

[pdmack]

C:\my-kubeflow>ks -vvvv apply default
DEBUG ←[0mAdding jsonnet search path C:\my-kubeflow/vendor
DEBUG ←[0mAdding jsonnet search path C:\my-kubeflow\lib/v1.7.0
DEBUG ←[0mAdding jsonnet search path C:\my-kubeflow/environments
←[31mERROR ←[0mgenerate objects for namespace C:\my-kubeflow\components\: unable to read C:\my-kubeflow/environments/default/main.jsonnet: <extvar:__ksonnet/components>:2:4-43 Unknown escape sequence in string literal: \m

  "←[31mC:\my-kubeflow\components\kubeflow-core←[0m": import "C:\my-kubeflow\components\kubeflow-core.jsonnet",

Search paths are being constructed but the object generator is struggling with the "\" part of the path. I think there are some breadcrumbs in the ks code about this being a potential pitfall.

[amq]

bash:

amq@ubuntu:/mnt/c/Users/amq/tensorflow/my-kubeflow
$ ks.exe -vvvv apply default -c kubeflow-core
construct base object: Failed to filter components; the following components don't exist: [ 'kubeflow-core' ]

$ ks.exe -vvvv apply default
Adding jsonnet search path C:\Users\amq\tensorflow\my-kubeflow/vendor
Adding jsonnet search path C:\Users\amq\tensorflow\my-kubeflow\lib/v1.7.0
Adding jsonnet search path C:\Users\amq\tensorflow\my-kubeflow/environments
generate objects for namespace C:\Users\amq\tensorflow\my-kubeflow\components\: unable to read C:\Users\amq\tensorflow\my-kubeflow/environments/default/main.jsonnet: <extvar:__ksonnet/components>:2:4-72 Unknown escape sequence in string literal: \U

  "C:\Users\amq\tensorflow\my-kubeflow\components\kubeflow-core": import "C:\Users\amq\tensorflow\my-kubeflow\components\kubeflow-core.jsonnet",

mingw:

amq@windows ~/tensorflow/my-kubeflow
$ ks.exe -vvvv apply default -c kubeflow-core
construct base object: Failed to filter components; the following components don't exist: [ 'kubeflow-core' ]

$ ks.exe -vvvv apply default
Adding jsonnet search path C:\Users\amq\tensorflow\my-kubeflow/vendor
Adding jsonnet search path C:\Users\amq\tensorflow\my-kubeflow\lib/v1.7.0
Adding jsonnet search path C:\Users\amq\tensorflow\my-kubeflow/environments
generate objects for namespace C:\Users\amq\tensorflow\my-kubeflow\components\: unable to read C:\Users\amq\tensorflow\my-kubeflow/environments/default/main.jsonnet: <extvar:__ksonnet/components>:2:4-72 Unknown escape sequence in string literal: \U

  "C:\Users\amq\tensorflow\my-kubeflow\components\kubeflow-core": import "C:\Users\amq\tensorflow\my-kubeflow\components\kubeflow-core.jsonnet",

[pdmack]

Linux (RHEL) bash:

# ks -vvvv apply default 
DEBUG Adding jsonnet search path /root/my-kubeflow/vendor
DEBUG Adding jsonnet search path /root/my-kubeflow/lib/v1.7.0
DEBUG Adding jsonnet search path /root/my-kubeflow/environments
....

[jlewi]

@amq Does ksonnet work for you at all? For example can you follow the ksonnet tutorial and create the examples mentioned there?

[pdmack]

Raised ksonnet/ksonnet/issues/382 for this. The basic guestbook tutorial also fails on Windows (i.e., not directly a Kubeflow issue).

@jlewi We may want to consider a (temporary?) Windows disclaimer in the README since it appears at the moment that there are some rough edges with ksonnet.

[jlewi]

Thanks. Filed #501