Senior Data Engineer (Kafka & Snowflake)
We are seeking a Senior Data Engineer (Kafka & Snowflake) to join a brand-new regulatory reporting team being built from scratch for a global payments technology company. In this role, you will work alongside a small group of engineers and the client’s team to build reliable streaming pipelines on Kafka and reporting models in Snowflake, ensure the data behind regulatory reports is accurate, auditable and reconciled, and help define how the team works and how its architecture takes shape from day one.
Essential functions
Streaming Pipelines: Design and build Kafka-based data flows with Kafka Streams and/or Kafka Connect, choosing the right delivery guarantees, from at-least-once to exactly-once with idempotent producers and transactions.
Resilient Event Processing: Implement error handling with retries, dead-letter queues and reprocessing, preserve message ordering, and manage schema evolution through Schema Registry with Avro or Protobuf.
Reporting Data in Snowflake: Model facts, dimensions and SCD for regulatory reporting, and build ingestion and processing with Streams & Tasks, Snowpipe and Time Travel.
Data Accuracy & Reconciliation: Implement reconciliation, validation and deduplication so that every reported figure can be trusted and traced.
Performance & Monitoring: Monitor and tune consumer lag and throughput, and optimise Snowflake queries through execution plans, clustering keys, pruning and warehouse sizing.
Shaping a New Team: Help define processes and architecture from scratch in a newly formed team.
Qualifications
Commercial experience as a data or software engineer building production streaming and data pipelines.
Kafka Expertise: Strong knowledge of Kafka architecture (topics, partitions, consumer groups, offsets, replication), delivery guarantees, Kafka Streams and/or Kafka Connect, and Schema Registry with Avro or Protobuf.
Advanced SQL in Snowflake: Complex queries with window functions, CTEs, aggregations and large-volume joins, plus hands-on query optimisation.
Snowflake Platform: Practical experience with Streams & Tasks, Snowpipe and Time Travel.
Reporting Data Modelling & Quality: Experience modelling data for reporting (facts, dimensions, SCD) and implementing reconciliation, validation and deduplication.
Ownership & Communication: Independent and proactive, comfortable in a newly formed team, with good communication in English.
Would be a plus
Apache Flink: Experience with the DataStream API or Flink SQL, stateful processing, checkpointing, event time, watermarks and windowing.
Java Ecosystem: Java 11/17+, Spring Boot and Spring Kafka, with testing in JUnit, Mockito and Testcontainers.
Platform & Delivery: Kubernetes (deployments, Helm, scaling), CI/CD, Git and cloud experience.
Regulatory & Financial Reporting: Experience in banking or payments reporting and a solid understanding of data auditability, lineage and accuracy requirements.
We offer
- Opportunity to work on bleeding-edge projects
- Work with a highly motivated and dedicated team
- Competitive salary
- Flexible schedule
- Benefits package - medical insurance, sports
- Corporate social events
- Professional development opportunities
- Well-equipped office
About us
Grid Dynamics (NASDAQ: GDYN) is a leading provider of technology consulting, platform and product engineering, AI, and advanced analytics services. Fusing technical vision with business acumen, we solve the most pressing technical challenges and enable positive business outcomes for enterprise companies undergoing business transformation. A key differentiator for Grid Dynamics is our 8 years of experience and leadership in enterprise AI, supported by profound expertise and ongoing investment in data, analytics, cloud & DevOps, application modernization and customer experience. Founded in 2006, Grid Dynamics is headquartered in Silicon Valley with offices across the Americas, Europe, and India.Apply to the position
Information on personal data processing
You cannot apply for a position without accepting “INFORMATION ON PERSONAL DATA PROCESSING”
Applications for this job are no longer accepted. Please explore other open opportunities on our platform.
Thank you!
You applied for the position Senior Data Engineer (Kafka & Snowflake) successfully. We will get back to you soon. Have a great day!
Something went wrong...
There are possible difficulties with connection or other issues. Please try to use another browser (it's recommended to use the latest version of Google Chrome browser). If the problem still persists, please send your application to cv@griddynamics.com
Something went wrong...
Please double-check the information filled in the form, and make sure to provide valid data.
Don’t see the right opportunity?
Contact us anyway and let’s talk! To apply, send your resume and cover letter to jobs@griddynamics.com
Grid Dynamics is an equal opportunity employer. We are committed to creating an inclusive environment for all employees during their employment and for all candidates during the application process.
All qualified applicants will receive consideration for employment without regard to, and will not be discriminated against based on, age, race, gender, color, religion, national origin, sexual orientation, gender identity, veteran status, disability or any other protected category. All employment is decided on the basis of qualifications, merit, and business need.