Senior Data Engineer

Infotech Talent Chantilly , VA 20151

Posted 1 week ago

Our client is working on developing mission-critical systems that help keep people safe. You will be joining a team that helps our heroes make a true impact. The work you will get to do is important and make a huge impact. The team you would join deploys advanced technologies such as Artificial Intelligence, Blockchain, AR/VR, Cloud Native, and Quantum Physics to solve their client's missions in cyber, RF, undersea, interstellar space, and everything in between.

As a Senior Data Engineer, youll lead model and simulation activities and participate in requirements analysis and management, functional analysis, performance analysis, system design, trade studies, systems integration, and test (verification). As our Senior Data Engineer, you will get to support the Insider Threat mission. Our Data Engineers work with various security system data owners to automate data integration and collection strategies. Work closely with the data science team to ensure data cleanliness and accuracy.

Are you a creative thinker who can multi-task and have a solid understanding of data engineering concepts, principles, and theories?

Are you ready to step up to the challenge and prove youre ready to lead in our world?

What You Will Be Doing:

  • You will support the data science team by designing, developing, and implementing scalable ETL process for disparate datasets into a Hadoop infrastructure
  • You will design, develop, implement and maintain data ingestion process from various disparate datasets using StreamSets (experience with StreamSets not mandatory)
  • You will develop processes to identify data drift and malformed records
  • You will develop technical documentation and standard operating procedures
  • We will count on you to mentor new and junior data engineers and lead technical tasks for small teams or projects

What You Should Have:

  • Working knowledge of entity resolution systems
  • Working experience with ETL processing
  • Working experience with data workflow products like StreamSets or NiFi
  • Working experience with Python RESTful API services, JDBC
  • Experience with messages systems like Kafka
  • Experience with NoSQL and/or graph databases like MongoDB or ArangoDB AND any of the following databases: SQL, MongoDB, Oracle, Postgres
  • Experience with Hadoop and Hive/Impala
  • Experience with Cloudera Data Science Workbench is a plus
  • Understanding of pySpark Leadership experience

Minimum Qualifications:

  • TS/SCI with Polygraph (within a reasonable time of hire)
  • Bachelors Degree in Engineering, a related specialized area or field.
  • 8+ years of relevant experience in test engineering.
  • Experience with Continuous Integration Tools (i.e. Jenkins, QuickBuild, etc.)
  • Experience with Configuration Management version control systems (i.e. SVN, GIT, etc.)
  • Experience with Agile software issues tracking tools (i.e. JIRA, Trac, Redmine, etc.)
  • Experience with Automated Testing

KEYWORDS: Automated Testing, JIRA, Trac, Redmine, Agile, Configuration Management, version control systems, SVN, GIT, Continuous Integration, Jenkins, QuickBuild, Test Engineering, Entity Resolution Systems, Kafka, NoSQL, graph databases, MongoDB, ArangoDB, SQL, MongoDB, Oracle, Postgres, ETL processing, data workflow products, StreamSets, NiFi, Python RESTful API services, JDBC, Hadoop, Hive, Impala, Cloudera Data Science Workbench, pySpark

icon no score

See how you match
to the job

Find your dream job anywhere
with the LiveCareer app.
Mobile App Icon
Download the
LiveCareer app and find
your dream job anywhere
App Store Icon Google Play Icon
lc_ad

Boost your job search productivity with our
free Chrome Extension!

lc_apply_tool GET EXTENSION

Similar Jobs

Want to see jobs matched to your resume? Upload One Now! Remove
Senior Data Engineer

General Dynamics

Posted 2 weeks ago

VIEW JOBS 9/30/2020 12:00:00 AM 2020-12-29T00:00 Responsibilities for this Position Senior Data Engineer Job ID: 2020-45922 USA-VA-Chantilly Required Clearance: TS/SCI with Polygraph, obtainable within reasonable time based on requirements Posted Date: 9/28/2020 Category: Engineering-Other Employment Type: Full Time Hiring Company: General Dynamics Mission Systems Basic Qualifications Bachelor's degree in systems engineering, a related specialized area or field is required (or equivalent experience) plus a minimum of 8 years of relevant experience; or Master's degree plus a minimum of 6 years of relevant experience. Agile experience preferred. CLEARANCE REQUIREMENTS: A TS/SCI security clearance with the ability to obtain a Polygraph is required at time of hire. Candidates must be able to obtain the Polygraph within a reasonable amount of time from date of hire. Applicants selected will be subject to a U.S. Government security investigation and must meet eligibility requirements for access to classified information. Due to the nature of work performed within our facilities, U.S. citizenship is required. Responsibilities for this Position Developing mission-critical systems that help keep people safe is what we do. At General Dynamics Mission Systems, you'll be part of the team that helps heroes make a true impact. The work we do is important. The challenges we face are career-defining. The opportunity we can offer is one-of-a-kind. We apply advanced technologies such as Artificial Intelligence, Blockchain, AR/VR, Cloud Native and Quantum Physics to solve our customers' missions in cyber, RF, undersea, interstellar space and everything in between. As a Senior Data Engineer, you'll lead model and simulation activities as you participate in requirements analysis and management, functional analysis, performance analysis, system design, trade studies, systems integration and test (verification). It's your chance to step up to the challenge and prove you're ready to lead the world. REPRESENTATIVE DUTIES AND TASKS: We are seeking a Senior Data Engineer to support the Insider Threat mission. Data Engineers work with various security system data owners to automate data integration and collection strategies. Work closely with the data science team to ensure data cleanliness and accuracy. * Support data science team by designing, developing and implementing scalable ETL process for disparate datasets into a Hadoop infrastructure * Design, develop, implement and maintain data ingestion process from various disparate datasets using StreamSets (experience with StreamSets not mandatory) * Develop processes to identify data drift and malformed records * Develop technical documentation and standard operating procedures * Mentor new and junior data engineers * Leads technical tasks for small teams or projects KNOWLEDGE SKILLS AND ABILITIES: * Working knowledge of entity resolution systems * Experience with messages systems like Kafka * Experience with NoSQL and/or graph databases like MongoDB or ArangoDB * Any of the following databases: SQL, MongoDB, Oracle, Postgres * Working experience with ETL processing * Working experience with data workflow products like StreamSets or NiFi * Working experience with Python RESTful API services, JDBC * Experience with Hadoop and Hive/Impala * Experience with Cloudera Data Science Workbench is a plus * Understanding of pySpark Leadership experience * Creative thinker * Ability to multi-task * Excellent use and understanding of data engineering concepts, principles, and theories #CJ3 #CB Company Overview General Dynamics Mission Systems (GDMS) engineers a diverse portfolio of high technology solutions, products and services that enable customers to successfully execute missions across all domains of operation. With a global team of 13,000+ top professionals, we partner with the best in industry to expand the bounds of innovation in the defense and scientific arenas. Given the nature of our work and who we are, we value trust, honesty, alignment and transparency. We offer highly competitive benefits and pride ourselves in being a great place to work with a shared sense of purpose. You will also enjoy a flexible work environment where contributions are recognized and rewarded. If who we are and what we do resonates with you, we invite you to join our high performance team! General Dynamics is an Equal Opportunity/Affirmative Action Employer that is committed to hiring a diverse and talented workforce. EOE/Disability/Veteran General Dynamics Chantilly VA

Senior Data Engineer

Infotech Talent