Kafka Developer

Kafka DeveloperWe are looking for a hands-on Kafka Developer to design, build, and maintain real-time data pipelines and event-driven applications. You will work on the application layer—writing producers, consumers, stream processing topologies, and connectors—not on cluster administration. You’ll collaborate with product, architecture, and platform teams to deliver scalable, fault-tolerant streaming solutions.Key ResponsibilitiesDesign and develop high-throughput Kafka producers and consumers for reliable event publishing and processing.Build and optimize stream processing applications using Kafka Streams, KSQL, or similar frameworks.Implement data integration pipelines with Kafka Connect (source/sink connectors) to sync systems like databases, object storage, and third-party APIs.Define and enforce data models: select partitioning keys, design topics, and manage schemas (Avro/Protobuf/JSON Schema) via Schema Registry.Ensure exactly-once semantics, idempotent writes, and proper error/retry handling in message flows.Write unit, integration, and performance tests using TopologyTestDriver, TestContainers, or embedded Kafka clusters.Monitor application-level metrics (lag, throughput, error rates) and tune producer/consumer configurations (batching, compression, fetch sizes).Collaborate on event-driven architecture, domain event design, and streaming data contracts.Participate in code reviews, CI/CD pipeline integration, and documentation of data flows.Required Skills & ExperienceProgramming: Strong proficiency in Java or Scala (Python/Kotlin a plus).Kafka Client APIs: Deep experience with Producer, Consumer, and AdminClient APIs.Stream Processing: Proven track record with Kafka Streams DSL (or KSQL) – stateful/stateless operations, windowing, joins, interactive queries.Kafka Connect: Hands-on experience building or configuring connectors (JDBC, Debezium, S3, Elasticsearch, etc.).Data Formats & Schema Management: Avro, Protobuf, or JSON Schema; practical use of Confluent Schema Registry (or similar).Testing: Familiarity with TopologyTestDriver, EmbeddedKafka, or TestContainers for integration tests.General Engineering: Git, CI/CD tools, microservices patterns, understanding of distributed systems principles.Preferred / Nice-to-HaveExperience with Confluent Cloud, Redpanda, or managed Kafka services.Knowledge of Kafka security (SASL, TLS, ACLs) from a developer’s perspective.Exposure to observability tools (Prometheus, Grafana, DataDog) for stream-level monitoring.Contributions to internal Kafka libraries or stream-processing frameworks.Domain-Driven Design (DDD) and event storming facilitation.