Senior Data Engineer

Microsoft Corporation Reston , VA 20190

Posted 2 months ago

The Azure Customer Advisory Team (AzureCAT) is an elite team of engineers and architects within the Azure engineering organization that works with interesting and strategic customer deployments on Azure.

Required qualifications for this role:

  • Must have TS SCI clearance.

  • Must have deep experience designing, developing, tuning and troubleshooting complex data-intensive applications, using Hadoop, Spark, relational databases, and other related technologies.

  • Knowledge of Cloud (Azure or AWS or Google Cloud) platform infrastructure to run and optimize distributed data applications.

  • Ability and flexibility to learn and adapt to a spectrum of data technologies running on multiple platforms.

  • 5+ years of experience in building or shipping products or services or building and architecting software solutions.

  • Excellent communication and presentation skills.

Preferred skills for this role include:

  • Bachelors or higher degree in technology, Computer Science or Engineering or related discipline

  • Ability to design and build distributed data systems

  • Working knowledge of big data components like HDFS, Spark, Hive, MapReduce, Kafka, Elastic Search, NoSQL technologies, and various file formats such as Avro, Parquet, etc.

  • Comfortable writing code in either Java, Scala, Python or .Net

  • Knowledge of data warehousing and data pipeline concepts, such as ETL, data integration, streaming

  • Experienced in defining and recommending best practices and principles of data architecture.

  • Experience working with multiple top tier customer scenarios and workloads to meet customer needs for performance, scalability, latency, reliability.

  • Practical experience in planning, sizing & migrating large enterprise workloads from on-premises to cloud platform.

  • Knowledge of other open source database technologies is an added advantage.

Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings:

  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form.

Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.

#AZCAT

In this role, you will have the opportunity to work closely with strategic customers to help build data-intensive applications, and influence the overall direction of the platform to unblock these strategic customers. You will be required to synthesize and present customer challenges to engineering leadership, present solutions and guidance to architects, developers and senior customer executives. This team is geographically dispersed, this specific position is based in Reston, VA, USA. About 25% travel expected.


icon no score

See how you match
to the job

Find your dream job anywhere
with the LiveCareer app.
Mobile App Icon
Download the
LiveCareer app and find
your dream job anywhere
App Store Icon Google Play Icon
lc_ad

Boost your job search productivity with our
free Chrome Extension!

lc_apply_tool GET EXTENSION

Similar Jobs

Want to see jobs matched to your resume? Upload One Now! Remove
Senior Data Engineer

Leidos

Posted 1 week ago

VIEW JOBS 4/13/2019 12:00:00 AM 2019-07-12T00:00 Job Description: Leidos is seeking a proven Senior Data Engineer to join an internal team responsible for developing, deploying and maintaining a customer facing solution framework for Bids & Proposals in support of Coding Challenges and Technical Demonstrations. This team will work directly with corporate Technical Core Competency leads to inject discriminating capability into new business acquisitions. This team will also perform Independent Research and Development (IRAD) in support of the organization's Agile SecDevOps program. The Senior Data Engineer will be a critical member of this dynamic team, working with corporate SMEs, business partners, and customers to drive new business through innovative approaches to critical customer problems. Responsibilities: * Develops automated tools supporting CI/CD and automated testing. Installs, configures, and administers infrastructure (CloudFormation, OpenShift) supporting Docker container deployment in AWS, Azure * Designs and develops software tools to perform extract, transform, and load of new data assets into the data model on the cloud. Develops tests verifying correct transformation and load and adherence to the security model. Develops data load solutions meeting performance requirements. * Designs and develops workflow components to integrate analytics into automated processing infrastructure. Integrates analytics with data stores and with event-driven / messaging architecture underlying automated workflows. Develops component-level tests for verification. * Bring a passion to stay on top of tech trends, experiment with and learn new technologies, participate in internal & external technology communities Minimum Qualifications: * Experience with Scrum/Agile development methodologies * Significant CI/CD experience and strong AWS system administration skills * Extensive experience using CloudFormation or Terraform * Experience with cloud platform services: Azure or AWS * Experience with configuration management of Cloud based resources * AWS Professional Certifications Additional Desired Qualifications: * Build Servers and Tools: Jenkins, Bamboo * Dependency Management tools: Maven, Gradle * Configuration Management tools: Ansible, Puppet, or Chef * Versioning and Git based source control EDUCATION & EXPERIENCE: Requires BS degree and 12 - 15 years of prior relevant experience or Masters with 10 - 13 years of prior relevant experience. May possess a Doctorate in technical domain. Leidos is a Fortune 500® information technology, engineering, and science solutions and services leader working to solve the world's toughest challenges in the defense, intelligence, homeland security, civil, and health markets. The company's 31,000 employees support vital missions for government and commercial customers. Headquartered in Reston, Virginia, Leidos reported annual revenues of approximately $10.17 billion for the fiscal year ended December 29, 2017. (NYSE: LDOS) All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or veteran status. Leidos Reston VA

Senior Data Engineer

Microsoft Corporation