kafkagosaur

Kafkagosaur is a Kafka client for Deno built using WebAssembly. The project binds to the kafka-go library.

See this article for an overview of its inner workings.

Supported features

• Writer
• Reader
• SASL
• TLS
• Deno streams

Examples

For comprehensive examples on how to use kafkagosaur, head over to the provided examples.

KafkaWriter

To write a message, first create KafkaWriter instance using the createWriter function on the KafkaGoSaur instance:

import KafkaGoSaur from "https://deno.land/x/kafkagosaur/mod.ts";

const kafkaGoSaur = new KafkaGoSaur();
const writer = await kafkaGoSaur.createWriter({
  broker: "localhost:9092",
  topic: "test-0",
});

const enc = new TextEncoder();
const msgs = [{ value: enc.encode("value") }];

await writer.writeMessages(msgs);

KafkaReader

To read a message, first create KafkaReader instance using the createReader function on the KafkaGoSaur instance:

import KafkaGoSaur from "https://deno.land/x/kafkagosaur/mod.ts";

const kafkaGoSaur = new KafkaGoSaur();
const reader = await kafkaGoSaur.createReader({
  brokers: ["localhost:9092"],
  topic: "test-0",
});

const readMsg = await reader.readMessage();

Provided examples

To run the provided examples, ensure you have docker up and running. Then start the kafka broker using

make docker-up

To run the writer example

deno run --allow-read --allow-net --unstable examples/writer.ts

To run the reader example

deno run --allow-read --allow-net --unstable examples/reader.ts

Documentation

The API documentation is hosted here.

Development

To build the WebAssembly module, first run

make build-wasm

To run the tests, ensure first you have docker up and running. Then start the kafka broker using

make docker-up

Then run

make test

Performance benchmarks

The Deno benchmarks are located in bench and can be run via

deno run --allow-read --allow-net --allow-env --unstable bench/reader.ts

deno run --allow-read --allow-net --allow-env --unstable bench/writer.ts

Results

	kafka-go1	kafkagosaur (DialBackend.Node)	kafkagosaur (DialBackend.Deno)
fetchMessage	5236 ± 743	2760 ± 376	2678 ± 361
writeMessages2	5333 ± 148	4455 ± 159	N/A3

Environment

• 2,6 GHz 6-Core Intel Core i7
• Confluent Cloud Basic cluster; 6 partitions

Contributing

Kafkgagosaur is in early stage of development. Nevertheless your contributions are highly valued and welcomed! Feel free to ask for new features, report bugs, or submit your code.

Footnotes

1. Using a single goroutine. ↩

2. Batching 10.000 messages. ↩

3. DialBackend.Deno not yet functional for writing. ↩