Senior BigData Developer (Python, PySpark, Kafka, GCP)
Our client is the world’s most iconic denim brand, who believe that technology is a key to business success.
The project is about re-platforming from AWS to GCP. Project goal is:
-to build an initial version of infrastructure using GCP that will host specific services and applications to be migrated from AWS during the next project phases;
-complete the initial research, identify the first applications to be migrated on later program phases, and build a migration plan for selected applications or services.
Project scope includes establishing of services for ingestion of data into GCS storage layer, data transformation using Spark, implementation of Cloud Composer (managed by Airflow), Google Data Catalog.
- 2+ years of solid experience working with Python
- Good knowledge PySpark
- Extensive knowledge of Kafka and Spark Streaming
- Working experience with GCP
- Expertise in standard software engineering methodology, e.g. unit testing, code reviews, design documentation
- English level: Upper-Intermediate and higher
Nice to have:
- Working experience with Apache Hadoop, Apache Airflow
- Good knowledge of Dataproc, GCS, BigQuery
- Experience with DataBricks
- Opportunity to work on bleeding-edge projects
- Work with a highly motivated and dedicated team
- Competitive salary
- Flexible schedule
- Medical insurance
- Benefits program
- Corporate social events
Grid Dynamics is the engineering services company known for transformative, mission-critical cloud solutions for retail, finance and technology sectors. We architected some of the busiest e-commerce services on the Internet and have never had an outage during the peak season. Founded in 2006 and headquartered in San Ramon, California with offices throughout the US and Eastern Europe, we focus on big data analytics, scalable omnichannel services, DevOps, and cloud enablement.
Grid Dynamics is an equal opportunity employer. We are committed to creating an inclusive environment for all employees during their employment and for all candidates during the application process. All qualified applicants will receive consideration for employment without regard to, and will not be discriminated against based on, age, race, gender, color, religion, national origin, sexual orientation, gender identity, veteran status, disability or any other protected category. All employment is decided on the basis of qualifications, merit, and business need.