Odtnhj

We have two separate kafka clusters in two datacenters and have configured Mirrormaker to replicate a set of topics. Each datacenter is running 3 nodes with kafka and mirrormaker. This setup is running correctly.

To enable more security, we need SASL_PLAINTEXT, or better SASL_SSL.

I configured SASL_PLAINTEXT in one of our datacenters on all three nodes:

Added in /etc/sysconfig/kafka:

Djava.security.auth.login.config=/etc/kafka/kafka_server.jaas"

Created /etc/kafka/kafka_server.jaas:

KafkaServer 
 org.apache.kafka.common.security.plain.PlainLoginModule required
 username="admin"
 password="admin-secret"

;
KafkaClient 
 org.apache.kafka.common.security.plain.PlainLoginModule required
 username="admin"
 password="admin-secret"
 ;
;

Changed listeners in /etc/kafka/server.properties:

listeners=PLAINTEXT://0.0.0.0:443,SASL_PLAINTEXT://0.0.0.0:29093,SASL_SSL://0.0.0.0:29094
advertised.listeners=PLAINTEXT://node1.app.gen.local:443,SASL_PLAINTEXT://node1.app.gen.local:29093,SASL_SSL://node1.app.gen.local:29094

Added to /etc/kafka/server.properties:

authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
allow.everyone.if.no.acl.found=true
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN
super.users=User:admin
zookeeper.set.acl=false

After these changes a successful test has been performed on the first kafka node to test the SASL port like this:

export KAFKA_OPTS="-Djava.security.auth.login.config=/etc/kafka/kafka_server.jaas"
/u01/app/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:29093 --topic topictest1 --from-beginning --consumer.config=/etc/kafka/consumer.properties

/etc/kafka/consumer.properties:

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
bootstrap.servers=localhost:29093

/etc/kafka/producer.properties:

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
bootstrap.servers=localhost:29093

So the consumer on the newly created SASL port can read the existing topics in the cluster on which no ACLs have been set yet. This was also the expected behaviour.

The only problem now is that mirrormaker is broken between the new secured cluster and the second non secured cluster. Tried to fix it by making the following changes to the file that holds the consumer properties for mirrormaker: /etc/kafka/mirrormaker-consumer.properties

bootstrap.servers=node1.app.gen.local:29093,node2.app.gen.local:29093,node3.app.gen.local:29093
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required
username="admin"
password="admin-secret";


Mirrormaker starts, lots of entries in the DEBUG log, but the messages are not copied from the secured cluster to the (still) unsecured cluster.

What could I have missed?

Thanks, I solved this by your suggestion to test consumer and producer separately. It appeared that I had an ACL defined with topic '*' in there that prevented Mirrormaker's consumer to read the topics from its PLAINTEXT port.

What I have learned: if you put an ACL on a topic, it can no longer be read from the PLAINTEXT port :-)