Senior Data Engineer

Job Type:

Regular

Plains All American is an industry-leading transportation and midstream provider specializing in transportation, storage, processing and marketing solutions for crude oil and natural gas liquids (NGLs). We own an extensive network of pipeline transportation, terminalling, storage and gathering assets in key crude oil and NGL producing basins and transportation corridors and at major market hubs in the United States and Canada. On average, Plains handles more than 6 million barrels per day of crude oil and NGL in its Transportation segment. The company is headquartered in Houston, Texas.

We are looking for highly-motivated individuals who enjoy working as part of a team in a dynamic and entrepreneurial work environment.

Position Summary:

Plains is initiating new initiatives within the Enterprise Data Engineering team where an experienced Azure Data Engineer. We are seeking someone who can demonstrate their qualifications and experience in all the items listed below.

Reporting to the Data Engineering Manager, the Sr Data Engineer is a software engineer responsible for leading the technical direction of a delivery team. This individual will ensure alignment of technical deliverables with both business requirements and enterprise architecture principles, functioning as a domain single point of contact for technical coordination across Strategy, Architecture, IS Operations, Infrastructure, and third-party providers (AMS/Outsource Service providers). Providing strong technical direction involves establishing a technical vision and standards, resolving technical disagreements, and managing the technical quality of team deliverables.

Effective technical engineer ensures the team uses appropriate software development practices, driving continual improvements to tooling or technical debt and ensuring that systems evolve to meet changing business needs. The Snr. Data Engineer understands broader business and team goals and establishes appropriate technical vision and practices, working with the delivery team and counterparts accordingly.

This role emphasizes an entrepreneurial spirit, creative and critical thinking.

The selected candidate will be eligible for company benefits including Medical/Dental/Vision, vacation time off, and flex days. Additionally, this position comes with the option of a hybrid work schedule, combining working in our Houston office with working remotely.

Core Focus Areas:

Development - A Sr Data Engineer Work closely with data architects and stakeholders to design and develop scalable and efficient data solutions on the Azure platform. This includes data ingestion, transformation, storage, and visualization. They should be able to help their team with any technical challenges they have, even when they are not necessarily the resident expert.

Job Responsibilities:

• Responsible for defining and driving the data engineering strategy for the Plains, working closely with senior leadership to align data initiatives with business goals.

• Establish software development standards and processes for code reviews. Oversee code standards and technical execution for all development staff including AMS team members.

• Lead designing workshops to fulfill functional and non-functional requirements effectively and efficiently.

• Data Integration and ETL: Develop and maintain advanced Data Engineering processes and patterns using Spark, Python, Scala, Databricks, Azure Data Factory and other relevant tools to integrate data from various sources.

• Azure Service Implementation: Implement and configure Azure services such as Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Azure SQL Database, and Azure Cosmos DB to build and manage data pipelines and data processing workflows.

• Data Modeling and Optimization: Create best practices for data modeling and indexing to ensure optimal performance and scalability.

• Data Security and Compliance: Oversee the implementation of data security measures and ensure compliance with data privacy regulations and industry standards. This includes implementing encryption, access controls, and monitoring mechanisms to protect sensitive data.

• Performance Tuning and Monitoring: Monitor and optimize the performance of data solutions by identifying and addressing bottlenecks, fine-tuning queries, and ensuring efficient resource utilization.

• Provide architectural and design direction to development and AMS teams in alignment with principles defined by Architecture team.

• Responsible for high level and detailed technical design and explores solution alternatives in collaboration with technical standards defined by architecture group.

• Translate business requirements to design specifications.

• Ensures quality integration between various systems.

• Design and implements standards, best practices, and accountable for technical delivery of data solutions.

• Promote collaboration and engagement.

• Own the technical design and have domain decision rights in alignment with Enterprise Architecture.

• Acts as technical escalations point of contact in respective Delivery Team for technical coordination across IS Operations, Infrastructure, and integrations teams.

Position Requirements:

• Bachelor's degree in Math, Computer Science, Engineering, Business Administration, or relevant practical experience preferred.

• 10+ years of experience in data engineering or software engineering, and a proven track record of leading successful data initiatives in large organizations.

• Proven technology experience in a complex enterprise environment

• Strong communication skills, both written and spoken

• Proficiency in Azure Synapse Analytics, Azure Data Factory, Azure Databricks, Azure SQL Database

• Experience in data modeling, data warehousing, and data lake design and implementation.

• Experience with programming languages such as SQL, Python, or Scala for data manipulation and scripting.

• Knowledge of data integration techniques, ETL processes, and data pipeline orchestration.

• Knowledge of software engineering best practices such as code reviews, testing frameworks, maintainability and readability would be an asset

• Knowledge of AI and ML is a plus

• Experience with agile delivery frameworks

• Cleared criminal history (background) and satisfactory reference checks

• Compliance with the Company's drug and alcohol policy including pre-employment D&A testing.

#Plains

At Plains, our employees are our most valuable asset. Hard work is rewarded with competitive compensation and a top-tier benefits program designed to keep our employees safe, healthy and happy. We work hard to deliver the best results to our stakeholders and we also respect our employees need for personal and family time, which is reflected in our benefits program.

Salary details estimated by job boards such as Indeed, Glassdoor, and LinkedIn do not represent Plains' compensation structure. We thank all candidates for their interest; however, only those selected for an interview will be contacted.

By submitting your resume, you consent to the collection, use and necessary disclosure of the personal information provided during the application and selection process. Learn more.