Senior Data Engineer

What you'll do...

Position: Senior Data Engineer

Job Location: 860 W. California Avenue, Sunnyvale, CA 94086

Duties: Design, develop, and maintain scalable batch/streaming data pipelines within Google Cloud Platform (GCP), utilizing technologies such as Dataproc, Scala, Spark, Kafka, Google Cloud Storage (GCS), BigQuery, and SQL.

Continuously optimize data processing and storage systems for maximum performance, scalability, and cost effectiveness. Schedule production jobs using schedulers such as Automic and Airflow. Maintain Production workflows, fix data issues/workflow failures in a timely manner, and provide Root Cause analysis (RCA). Work on code management and deployment via CI/CD using GitHub and Jenkins.

Design and implement data models using Erwin. Collaborate with cross-functional teams to understand data requirements and ensure high-quality, accessible data. Create scalable APIs in Java coupled with Druid database to expose curated data to different stakeholders with sub-second SLA adherence.

Enhance monitoring, logging, and alerting mechanisms to track API usage and proactively identify and address issues. Create architecture diagram and comprehensive documentation using Draw IO, Lucid chart, and Confluence to facilitate seamless data consumption and analysis. Follow Scrum methodology and Agile approach to managing tasks and responsibilities.

Minimum education and experience required: Master's degree or equivalent in Computer Science, Information Technology, Engineering (any), or related field and 1 year of experience in data engineering, database engineering, business intelligence, or business analytics or related field; OR Bachelor's degree or equivalent in Computer Science, Information Technology, Engineering (any), or related field and 3 years of experience in data engineering, database engineering, business intelligence, or business analytics or related field.

Skills required: Experience designing and developing scalable batch and streaming pipelines using big data frameworks including Spark, Dataproc, and Kafka to process and analyze a large volume of data.

Experience developing and maintaining data lake/warehouses using Google Cloud Storage (GCS), Google BigQuery, and Hive for structured data storage. Experience identifying and resolving issues, performing root cause analysis, and implementing corrective actions. Experience fine tuning spark job and SQL queries to ensure optimal query performance and resource utilization with minimum cost.

Experience constructing data models, data quality checks, and validation rules. Experience designing and implementing REST API web services using Java, coupled with different databases. Experience using Git for source code version control and CI/CD (Jenkins) to automate testing, build, and deployment processes.

Experience coding in functional or object-oriented programming language including Java, Python, or Scala. Employer will accept any amount of experience with the required skills.

Salary Range: $117,000/year to $234,000/year. Additional compensation includes annual or quarterly performance incentives.

  Additional compensation for certain positions may also include: Regional Pay Zone (RPZ) (based on location) and Stock equity incentives.

Benefits: At Walmart, we offer competitive pay as well as performance-based incentive awards and other great benefits for a happier mind, body, and wallet.

Health benefits include medical, vision and dental coverage. Financial benefits include 401(k), stock purchase and company-paid life insurance. Paid time off benefits include PTO (including sick leave), parental leave, family care leave, bereavement, jury duty and voting. Other benefits include short-term and long-term disability, education assistance with 100% company paid college degrees, company discounts, military service pay, adoption expense reimbursement, and more.

Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to a specific plan or program terms. For information about benefits and eligibility, see One.Walmart.com.

#LI-DNP #LI-DNI

Wal-Mart is an Equal Opportunity Employer.