Sorry, this job is no longer accepting applications. See below for more jobs that match what you’re looking for!

Super Hot: Data Engineer: Spark Developer

Expired Job

Diverse Lynx Llc Sunnyvale , CA 94087

Posted 2 months ago

Role: Data Engineer: Spark Developer
Location: Bradenton FL/BentonVille AR/ Sunnyvale, CA- Multiple position
Experience: 8-10 Year
Duration: 10-12 Months

Skill Needs
Expertise working with large scale distributed systems (Hadoop, Spark).
Strong understanding of the big data cluster, and its architecture
Experience building and optimizing big data ETL pipelines.
Advanced programming skills with Python, Java, Scala
Good knowledge of spark internals and performance tuning of spark jobs.
Strong SQL skills and is comfortable operating with relational data models and structure.
Capable of accessing data via a variety of API/RESTful services.
Experience with messaging systems like Kafka.
Experience with No SQL databases. Neo4j, mongo, etc.
Expertise with Continuous Integration/Continuous Delivery workflows and supporting applications.
Exposure to cloud environments and architectures. (preferably Azure)
Ability to work collaboratively with other teams.
Experience with containerization using tools such as Docker.
Strong knowledge of Linux and Bash. Can interact with the OS at the command line and create shell scripts to automate workflows.
Advanced understanding of software development and collaboration, including experience with tools such as Git.
Excellent written and verbal communication skills, comfortable presenting in front of non-technical audiences.

Essential Responsibilities include but not limited to:
Design and develop ETL workflows to migrate data from varied data sources including SQL Server, Netezza, Kafka etc. in batch and real-time.
Develop checks and balances to ensure integrity of the ingested data.
Design and Develop Spark jobs as per requirements for data processing needs.
Work with Analysts and Data Scientists to assist them in building scalable data products.
Designs systems, alerts, and dashboards to monitor data products in production

Thanks & Regards

Rama Shanker Verma
Phone: 732-452-1006x206
Direct :

yahoo IM- rshanker09, Gtalk: amanverma.9

Skills:
Spark and Hadoop
Contract


See if you are a match!

See how well your resume matches up to this job - upload your resume now.

Find your dream job anywhere
with the LiveCareer app.
Download the
LiveCareer app and find
your dream job anywhere
lc_ad

Boost your job search productivity with our
free Chrome Extension!

lc_apply_tool GET EXTENSION

Similar Jobs

Want to see jobs matched to your resume? Upload One Now! Remove
Sr Software Engineer ( Data Pipeline Java Hive Spark Storm )

Redolent, Inc

Posted 2 days ago

VIEW JOBS 11/14/2018 12:00:00 AM 2019-02-12T00:00 <style type="text/css"><!--td {border: 1px solid #ccc;}br {mso-data-placement:same-cell;}--> </style> <strong>We have following urgent role with our  DIRECT client</strong><br /> <br /> <strong>Title:  </strong><b>Sr Software Engineer (  Data pipeline - Java, Hive, Spark, Storm )</b> <br /> LOCATION: Sunnyvale, CA<br /> DURATION: 6+ months <br /> Compensation: Competitive ( DOE )<br /> <br /> <b>About the specific projects:</b><br /> The candidate could be working in the Data Pipeline team that builds, manages, and supports: <ul> <li style="padding: 0; margin: 0;">Data pipelines written in Hive/Java/Bash, Spark and related stack</li> <li style="padding: 0; margin: 0;">Data pipelines written in scalding/Scala</li> <li style="padding: 0; margin: 0;">Realtime stream processors using Storm/Kafka</li> <li style="padding: 0; margin: 0;">The candidate should be willing to try new relevant tech stack. </li> </ul> <strong>Job Description:</strong> <ul> <li style="padding: 0; margin: 0;">Strong Java programming experience</li> <li style="padding: 0; margin: 0;">Strong development skills around <b>Hadoop, Hive, Pig Latin, Spark, Scalding, Map Reduce, Storm, Kafka</b></li> <li style="padding: 0; margin: 0;"> Knowledge of <b>Spark</b> and <b>NoSQL databases</b></li> <li style="padding: 0; margin: 0;"> Strong skills in object oriented design,<b> Core Java, Scala, shell scripting/Python</b></li> <li style="padding: 0; margin: 0;"> Excellent problem solving and analytical skills</li> <li style="padding: 0; margin: 0;"> Proven background in Distributed Computing, <b>Data Warehousing, ETL development, and large scale data processing</b></li> <li style="padding: 0; margin: 0;"> Strong development skills in<b> Python and SQL</b></li> </ul> <b>Top 3 mandatory skills:</b> <ul> <li style="padding: 0; margin: 0;">Apache Hive and Spark</li> <li style="padding: 0; margin: 0;">Java and Bash as programming/scripting language</li> <li style="padding: 0; margin: 0;">Storm or any real-time stream processing technology</li> </ul> <br /> <br />   <div style="padding: 0; margin: 0;"> </div> <style type="text/css"><!--td {border: 1px solid #ccc;}br {mso-data-placement:same-cell;}--> </style> Redolent, Inc Sunnyvale CA

Super Hot: Data Engineer: Spark Developer

Expired Job

Diverse Lynx Llc