Senior Data Engineer

Copart Dallas , TX 75201

Posted 4 months ago

The Sr. Data Engineer will be part of the Data Services Team. The Data Services team works very closely with all aspects of data, both internal and external. We are looking for a Data Engineer with the Software Engineering skill to build data pipelines for efficient and reliable data movement across systems, and also to build the next generation of data tools to enable us to take full advantage of this data. In this role, your work will broadly influence the company's data consumers, executives and analysts.

Key Responsibilities:

  • Work and lead the team responsible design and develop Real-time dataflows and other data pipeline solutions

  • Lead analysis, architecture, design, and development of data warehouse and business intelligence solutions and using next generation data platforms.

  • Mentor other team members

  • Bachelor's degree or higher in computer science, Engineering or similar

  • 5+ years of experience programming enterprise solutions in Java/Scala

  • 3+ years of experience in Apache Kafka, Apache Spark, Apache HBase or any other big data technologies

  • Experience building and shipping highly scalable clickstream data pipelines and analytics systems on distributed data systems (Hadoop/AWS/Azure/GCP)

  • Experience in streaming technologies like AWS Kinesis or Kafka.

  • Experience building batch, real-time and streaming analytics pipelines with data from event data streams, NoSQL and APIs.

  • Experience with batch & stream processing technologies such Spark, KStreams, Samza etc.

  • Worked on two or more of the following databases: Oracle, MySQL, DB2 LUW, Cassandra or MongoDB

  • Experience with enterprise search technology (ElasticSearch, Solr, Lucene, etc.)

Desirable:

  • Experience in Machine Learning -- Weka, MLlib, TensorFlow, etc

  • Experience in Business Intelligence tools and technologies -- Tableau, Superset, Metabase

icon no score

See how you match
to the job

Find your dream job anywhere
with the LiveCareer app.
Mobile App Icon
Download the
LiveCareer app and find
your dream job anywhere
App Store Icon Google Play Icon
lc_ad

Boost your job search productivity with our
free Chrome Extension!

lc_apply_tool GET EXTENSION

Similar Jobs

Want to see jobs matched to your resume? Upload One Now! Remove
Senior Data Engineer

Cervello

Posted 2 weeks ago

VIEW JOBS 9/26/2019 12:00:00 AM 2019-12-25T00:00 Summary: You have experience with client projects and in handling vast amounts of data – working on database design and development, data integration and ingestion, designing ETL architectures using a variety of ETL tools and techniques. You are someone with a drive to implement the best possible solutions for clients and work closely with a highly skilled Data Science team. Lead on projects from a data engineering perspective, working with our clients to model their data landscape, obtain data extracts and define secure data exchange approaches * Plan and execute secure, good practice data integration strategies and approaches * Acquire, ingest, and process data from multiple sources and systems into Big Data platforms * Create and manage data environments in the Cloud * Collaborate with our data scientists to map data fields to hypotheses and curate, wrangle, and prepare data for use in their advanced analytical models * Have a strong understanding of Information Security principles to ensure compliant handling and management of client data * This is a fantastic opportunity to be involved in end-to-end data management for cutting edge Advanced Analytics and Data Science Qualifications: * Commercial experience leading on client-facing projects, including working in close-knit teams * 5+ years of experience and interest in Big Data technologies (Hadoop / Spark / NoSQL DBs) * 5+ years of experience working on projects within the cloud ideally AWS or Azure * 5+ years of experience working with streaming architectures and patterns like Kafka, Kinesis, Flink, or Confluent * Experience with open source tools like Apache Airflow and Griffin * Experience with DevOps and DataOps patterns and tools like Jenkins, Kubernetes, Docker, and Terraform * Data Warehousing experience with cloud products like Snowflake, Azure DW, or Redshift * Experience building operational ETL data pipelines across a number of sources, and constructing relational and dimensional data models * Experience with one or more ETL/ELT tools like Talend, Matillion, FiveTran, or Alooma * Experience building automated data quality and testing into data pipelines * Experience with AI, NLP, Machine Learning, etc. is a plus * Strong development background with experience in at least two scripting, object oriented or functional programming language, etc. SQL, Python, Java, Scala, C#, R * Experience working on lively projects and a consulting setting, often working on different and multiple projects at the same time * Excellent interpersonal skills when interacting with clients in a clear, timely, and professional manner. * A deep personal motivation to always produce outstanding work for your clients and colleagues * Excel in team collaboration and working with others from diverse skill-sets and backgrounds Cervello is a dynamic technology company that is focused on business analytics and planning. We take an innovative approach to making complex solutions simple so our clients can focus on running their businesses. Our services and applications enable our clients to gain the benefits of a world-class analytics and planning capability without the headaches. Cervello Dallas TX

Senior Data Engineer

Copart