Engineer

Job Title Sr Data Engineer

Relevant Experience (in Yrs) 8-10 yrs

Must Have Technical/Functional Skills Apache Spark, Scala/ Python and PySpark

Experience Required Min. 5-7 years hands on experience working in data modeling

Roles & Responsibilities

• Work on migrating applications from an on-premises location to the cloud service providers.
• • Develop products and services on the latest technologies through contributions in development, enhancements, testing and implementation.
• • Develop, modify, extend code for building cloud infrastructure, and automate using CI/CD pipeline.
• • Partners with business and peers in the pursuit of solutions that achieve business goals through an agile software development methodology.
• • Perform problem analysis, data analysis, reporting, and communication.
• • Work with peers across the system to define and implement best practices and standards.
• • Assess applications and help determine the appropriate application infrastructure patterns.
• • Use best practices and knowledge of internal or external drivers to improve products or services.

What we are looking for:

• • Hands-on experience in building/implementing cloud platforms/applications on AWS platform.
• • Experience in developing data pipeline solutions to ingest and exploit new and existing data sources.
• • Expertise in leveraging SQL, programming language like Python and ETL tools like Databricks
• • Perform code reviews to ensure fit to requirements, optimal execution patterns and adherence to established standards.
• • Expertise in AWS Compute (EC2, EMR), AWS Storage (S3, EBS), AWS Databases (RDS, DynamoDB), AWS Data Integration (Glue).
• • Advanced understanding of Container Orchestration services including Docker and Kubernetes, and a variety of AWS tools and services.
• • Good understanding of AWS Identify and Access management, AWS Networking and AWS Monitoring tools.
• • Proficiency in CI/CD and deployment automation using GITLAB pipeline.
• • Proficiency in Cloud infrastructure provisioning tools e.g., Terraform.
• • Proficiency in one or more programming languages e.g., Python, Scala.