Senior Big Data Engineer- Remote
We are looking for a Senior/Staff Software/Platform Engineer that can design and code critical components for a platform that will interconnect data pipelines in a team of 5-10 engineers. Our customer is one of the world's largest tCPG companies based in Beaverton, Oregon with operations all over the world. The ideal candidate is a proactive, driven, and technology-proficient software engineer with strong data engineering background, experience event messaging system design, and strong coding experience. The candidate will need to quickly design, build and extend services from scratch.
- Develop and extend a recently started data platform to support big data pipelines in the consumer data space
- Drive/remain responsible for development of end-to-end for specific components
- Contribute to project discussions, collaborate directly with architect team and present results to key stakeholders
- Design, build and continuously enhance the project codebase
- Act as an onsite-timezone force multiplier for a distributed team of engineers and managers
- Write detailed design documentation, present decisions and motivate these
- Work inside a team of industry experts on the cutting edge Big Data technologies to develop solutions for deployment at massive scale
- Design data infrastructure with privacy and security being cross-cutting concerns
- Set coding and deployment best practices
- +6 years experience designing and coding platform solutions for Big Data pipelines
- +3 years of experience working with event-messaging systems - Kafka is a big plus
- +2 years coded and deploying services running on Kubernetes
- Python and Spark knowledge is required
- Experience working with AWS
- Experience with enterprise data warehouse
- Strong understanding of the challenges in building end-to-end big data pipelines for a large variety of use-cases at scale
- Strong communication skills
What will be a big plus:
- Experience with Scala
- Experience with EMR
- Experience with Snowflake
- Understanding Microservices and how to architect/design scalable solutions on Kubernetes
- Understanding challenges of working with many disjunct big data technologies
- Worked with big data pipelines at terabyte/petabyte scale
- Worked with HDFS
- Understanding how to run Spark on Kubernetes
- Experience working with Big Data scheduling technologies and their APIs - Airflow
- Experience with JVM build systems (Gradle, Maven)
- Opportunity to work on bleeding-edge projects
- Work with a highly motivated and dedicated team
- Competitive salary
- Flexible schedule
- Medical insurance
- Benefits program
- Corporate social events
Placement and Staffing Agencies need not apply. We do not work with C2C at this time.
At this moment, we are not able to process H1B transfers.
Grid Dynamics is the engineering services company known for transformative, mission-critical cloud solutions for retail, finance and technology sectors. We architected some of the busiest e-commerce services on the Internet and have never had an outage during the peak season. Founded in 2006 and headquartered in San Ramon, California with offices throughout the US and Eastern Europe, we focus on big data analytics, scalable omnichannel services, DevOps, and cloud enablement.
Grid Dynamics is an equal opportunity employer. We are committed to creating an inclusive environment for all employees during their employment and for all candidates during the application process. All qualified applicants will receive consideration for employment without regard to, and will not be discriminated against based on, age, race, gender, color, religion, national origin, sexual orientation, gender identity, veteran status, disability or any other protected category. All employment is decided on the basis of qualifications, merit, and business need.