Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Errors in topic create don't propagate to client #135

Open
candlerb opened this issue Jun 22, 2018 · 0 comments
Open

Errors in topic create don't propagate to client #135

candlerb opened this issue Jun 22, 2018 · 0 comments

Comments

@candlerb
Copy link
Contributor

If an error occurs in topic create, the client thinks it was successful. A simple way to trigger this is to create a topic with empty name:

# cmd/jocko/jocko topic create --topic ""
created topic:
# echo $?
0

But at the server side you get:

2018-06-22T05:56:26.119Z	ERROR	fsm/commands.go:97	EnsurePartition failed	{"id": 0, "broker addr": "0.0.0.0:9092", "serf addr": "0.0.0.0:9094", "raft addr": "127.0.0.1:9093", "id": 0, "raft addr": "127.0.0.1:9093", "error": "failed inserting partition: object missing primary index"}
github.com/travisjeffery/jocko/log.(*logger).Error
	/root/go/src/github.com/travisjeffery/jocko/log/logger.go:38
github.com/travisjeffery/jocko/jocko/fsm.(*FSM).applyRegisterPartition
	/root/go/src/github.com/travisjeffery/jocko/jocko/fsm/commands.go:97
github.com/travisjeffery/jocko/jocko/fsm.New.func1
	/root/go/src/github.com/travisjeffery/jocko/jocko/fsm/fsm.go:81
github.com/travisjeffery/jocko/jocko/fsm.(*FSM).Apply
	/root/go/src/github.com/travisjeffery/jocko/jocko/fsm/fsm.go:98
github.com/travisjeffery/jocko/vendor/github.com/hashicorp/raft.(*Raft).runFSM.func1
	/root/go/src/github.com/travisjeffery/jocko/vendor/github.com/hashicorp/raft/fsm.go:57
github.com/travisjeffery/jocko/vendor/github.com/hashicorp/raft.(*Raft).runFSM
	/root/go/src/github.com/travisjeffery/jocko/vendor/github.com/hashicorp/raft/fsm.go:120
github.com/travisjeffery/jocko/vendor/github.com/hashicorp/raft.(*Raft).(github.com/travisjeffery/jocko/vendor/github.com/hashicorp/raft.runFSM)-fm
	/root/go/src/github.com/travisjeffery/jocko/vendor/github.com/hashicorp/raft/api.go:506
github.com/travisjeffery/jocko/vendor/github.com/hashicorp/raft.(*raftState).goFunc.func1
	/root/go/src/github.com/travisjeffery/jocko/vendor/github.com/hashicorp/raft/state.go:146

Furthermore, this error is persistent: each time you kill and restart the broker, the same error appears again (presumably it is re-running the commit log?)

I don't know the correct approach to this problem. Is raft Apply required to succeed always? In which case, maybe the only solution is additional pre-flight checks before sending the change to raft. If so, what are the rules for validating the topic?

But I think there is a deeper issue with error handling. For example: topic create has a pre-flight check to reject a topic which already exists. But this leads to a race condition: what happens if two clients try to create the topic at the same time, both pre-flight checks pass, and then both commit?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant