Sr. Principal Engineer, Data

Overview

PURPOSE STATEMENT:

We are looking for an experienced Senior Principal Data Engineer to architect, build, and maintain the data infrastructure and pipelines that power our analytics and machine learning platforms. In this role, you will collaborate with stakeholders across business units to understand data needs and then design optimal technical solutions leveraging tools like Snowflake, Airflow, etc on Cloud platforms (Azure preferred). This can be 100% remote.

Responsibilities

ESSENTIAL FUNCTIONS:

• Design, implement, and optimize cloud data architecture including databases, schemas, tables, views, clusters in cloud data platform tech stack

• Develop end-to-end data pipelines that from source systems ingestion to data warehouse or data lakes and to data mart, eventually service BI and AI/ML for visualization and analytical needs

• Extract, load and transform curated data assets from various enterprise systems and SaaS solutions into enterprise data platform with access and governance in place for analytical workloads

• Monitor data pipelines, infrastructure health metrics. Automate for fault tolerant and self-healing, optimize for maximal throughput and performance

• Automation and integration of data quality components in the data pipelines.

• Provide day to day operational support, optimization and innovation for enterprise data platform

• Set, document and maintain standards for data assets inventory, meta data management and data lineage

• Research and evaluate new data management tools and approaches for potential integration

• Coach and mentor junior data engineers in data modeling, pipeline development, troubleshooting issues

STANDARD EXPECTATIONS:

• Complies with organizational policies, procedures, performance improvement initiatives and maintains organizational and industry policies regarding confidentiality

• Communicate clearly and effectively to person(s) receiving services and their family members, guests, and other members of the health care team

• Develops constructive and cooperative working relationships with others and maintains them over time

• Encourages and builds mutual trust, respect, and cooperation among team members

• Maintains regular and predictable attendance

Qualifications

EDUCATION/EXPERIENCE/SKILL REQUIREMENTS:

• BS/MS degree in Computer Science, Engineering or equivalent field

• Deep hands-on development and administration experience with Snowflake as a must-have

• Deep hands-on experience with Apache Airflow (Astronomer) and Python as a must-have

• 7+ years hands-on engineering and administration experience in data engineering on cloud platforms (Azure preferred) with RDBMS such as SQL Server, PostgreSQL, MySQL.

• Extensive working experience with Azure ADF, ADLS and Synapse.

• Expert knowledge of SQL and experience with Spark, Python for data transformation/processing

• Experience building and optimizing data pipelines at scale with orchestration tools like AirFlow and observability tools like ELK

• Excellent communication skills collaborating cross-functionally with stakeholders

• Self-directed and passionate about keeping up with latest innovations in the data space

• API development experience is a plus

SUPERVISORY REQUIREMENTS:

None

While this job description is intended to be an accurate reflection of the requirements of the job, management reserves the right to add or remove duties from particular jobs when circumstances (e.g. emergencies, changes in workload, rush jobs or technological developments) dictate.

AHCORP

#remote

#LI-APD1