A lot of ingest architectures jump straight to Kafka. Callra does not, at least not in v1.
The rule that matters most
The ingest API should only acknowledge a request after it has been durably written locally.
In Callra, that means each request becomes:
- one
*.ndjsonbatch file on local disk - one SQLite queue record that tracks its state
Only after that does the API return success.
Why this is a good first version
It protects the write path from temporary analytics database outages while keeping the system inspectable. If a batch is stuck, you can look at the queue row, the batch file, and the flusher logs directly.
That is a better v1 trade than introducing broker operations before the product has earned them.
What Kafka would buy later
Kafka can still make sense when Callra needs:
- multiple downstream consumers
- much higher sustained throughput
- longer shared retention outside node-local storage
- broader fan-out and replay semantics
The point is not that Kafka is wrong. The point is that durable local batches solve the first production problem with much less machinery.