This guide was written originally for dFlow~IQ. Authoritative sources were used to verify technical facts and are credited in the references section. Source wording is not reproduced.
Treat security as three independent layers
TLS normally provides encryption and server identity. SASL credentials or a client certificate establish client identity. Kafka ACLs or platform RBAC decide which topics, groups and operations that principal may use.
A connection can therefore authenticate successfully and still fail to list a topic. It can also be encrypted without authenticating the client.
| Layer | Question |
|---|---|
| Encryption | Can an observer read traffic in transit? |
| Authentication | Who is the client or server? |
| Authorization | What may that identity do? |
Configure Kafka and Registry separately
Kafka brokers use the Kafka protocol and return advertised broker addresses after bootstrap. Schema Registry is a separate HTTP service with its own URL, trust and credentials.
Successful Kafka access does not prove Registry access. Broker SASL credentials are not automatically Registry Basic credentials, and each service may use a different certificate authority.
Validate trust and identity with TLS
The client validates certificate dates, chain signatures, trust anchors and hostname identity. Every broker hostname returned in metadata must match its certificate, not only the initial bootstrap address.
One-way TLS authenticates the server. Mutual TLS additionally requires a client certificate and matching private key. A public .crt file alone cannot provide client identity.
A certificate or hostname bypass permits endpoint impersonation. Import the correct CA and use certificate names that match the real host.
Choose SASL/PLAIN or SCRAM deliberately
SASL/PLAIN sends a reusable username and password through the SASL exchange, so it must be protected by TLS. SCRAM uses salted challenge-response proofs and reduces password exposure, but it should still run over TLS to authenticate the server and resist capture-based attacks.
The broker must enable the exact mechanism and provision the user for it. Authentication alone grants no topic access.
| Protocol | Use |
|---|---|
| PLAINTEXT | Unencrypted, unauthenticated local development only |
| SSL | TLS, optionally client-certificate authentication |
| SASL_PLAINTEXT | Authenticated but unencrypted; avoid on untrusted networks |
| SASL_SSL | Normal production choice for password authentication |
Know what certificate files contain
- JKS is a structured Java store that may hold trusted certificates or private-key entries.
- PEM is textual encoding identified by BEGIN/END labels; inspect whether it contains certificates or a private key.
- .crt and .cer are naming conventions and may contain PEM or binary DER certificates.
- A truststore contains accepted issuers; a keystore used for mTLS contains the client private key and certificate chain.
- Protect private keys and never place credentials or JAAS strings in source control or screenshots.
Protect Schema Registry independently
Use an HTTPS Registry URL and provide the Registry CA when it is not already trusted. HTTP Basic credentials are only Base64 encoded, not encrypted, and therefore require HTTPS.
Registry authorization can separately govern listing subjects, retrieving schemas, registering versions and testing compatibility. Grant only the operations dFlow~IQ users require.
Configure a secure dFlow~IQ workspace
- Obtain bootstrap addresses, security protocol, SASL mechanism, dedicated credentials, CA chain and required Kafka permissions.
- Create the workspace and configure Kafka transport and authentication first.
- Open Settings → Connections & TLS and enable Kafka TLS where required.
- Select a readable JKS, PEM, CRT or CER trust file, or use the JVM defaults when the CA is already trusted.
- Test Kafka and verify allowed topic operations.
- Configure the Registry HTTPS URL and separate Basic credentials if required.
- Test Registry, then verify schema-backed produce and consume on an approved topic.
Troubleshoot by layer
| Symptom | Likely layer |
|---|---|
| Connection refused or timeout | Network, listener, VPN or firewall |
| Bootstrap works; broker calls fail | Unreachable advertised.listeners addresses |
| PKIX path building failed | Missing or wrong CA chain |
| Hostname verification failure | Certificate SAN does not match contacted host |
| SASL authentication failed | Wrong mechanism, username or credential |
| TopicAuthorizationException | Kafka ACL/RBAC, not authentication |
| Registry 401 / 403 | Registry authentication / authorization |
| Unknown schema ID | Wrong Registry environment or wire format |
References
References support factual claims in this original guide. They are not required reading.
- Apache Kafka: Security Overview Used to verify Kafka encryption, authentication and authorization concepts.
- Apache Kafka: SASL Authentication Used to verify SASL protocols, PLAIN and SCRAM configuration.
- Apache Kafka: SSL Used to verify TLS trust, hostname validation and client certificates.
- RFC 7617: HTTP Basic Authentication Used to verify Basic credential encoding and the need for protected transport.