Topics are not automatically created when needed

[objectuser]

They are with Kaffe.Consumer. It seems like something we should add to the group consumer.

[sheshankkodam]

@objectuser Do we have a solution for this?

[objectuser]

@sheshankkodam The approach we've used is to just create the topics prior to usage. This seems to always be what we want in any case.

Would this work in your case?

[sheshankkodam]

@objectuser Yes, you're right. The topics are available in production and staging environments and it works fine there.

However, in the test environment, we are using docker and docker-compose to create an environment. In this case, we wanted to create topics dynamically by setting allow_topic_creation=true.

As a workaround, I'm running these 2 commands to create the topics prior to running the application

docker exec kafka kafka-topics --create --topic "topic1"
docker-compose up application_name

This works but it would be nice to run the application in 1 command like docker-compose up

[objectuser]

Yes, I do see what you mean.

What we do is have an init script that creates the topics when the Docker container starts. So we still only do docker-compose up but we need to maintain the scripts.

Since we have a different path, this would be pretty low on our priority list, but if you'd like to do a PR, that would be 🥇 worthy.