Data Engineer

Globex Digital Solutions Philadelphia , PA 19102

Posted 4 weeks ago

Hands on experience in developing big data pipelines end to end

· 5-8 years of Java and Python experience

· Experience in software development of largescale distributed systems including proven track record of delivering backend systems that participate in a complex ecosystem

· Experience working with imperfect data sets that, at times, will require improvements to process, definition and collection

· Experience with real-time data pipelines and components including Kafka, Spark Streaming

· Proficient in Unix/Linux environments

· AWS experience developing data streaming pipelines

· Deep Spark understanding

· Must Have Skills : Spark, SQL,Kinesis/Kafka, python for scripting on AWS and Java for APIs

icon no score

See how you match
to the job

Find your dream job anywhere
with the LiveCareer app.
Mobile App Icon
Download the
LiveCareer app and find
your dream job anywhere
App Store Icon Google Play Icon

Boost your job search productivity with our
free Chrome Extension!

lc_apply_tool GET EXTENSION

Similar Jobs

Want to see jobs matched to your resume? Upload One Now! Remove
Data Engineer

Harp Talent Solutions, LLC

Posted 1 week ago

VIEW JOBS 11/23/2020 12:00:00 AM 2021-02-21T00:00 <p><em>The Data Engineer delivers solution to client&rsquo;s technical problems to maximize client value, using technical skills and knowledge to deliver operational impact.</em>&nbsp;The ideal candidate will have a strong background in technical engineering design and systems configuration, and data modeling, calculations development, and pipeline management. The Delivery Engineer is expected to tailor their solution to each individual client&rsquo;s problem or needs, creating the technical design and configuration to address defined business requirements. The role is critical to managing relationships with technical client stakeholders. The Delivery Engineer collaborates with the Delivery team to align business expectations with technical delivery to produce world-class solutions for our clients. A highly effective Delivery Engineer will leverage the full capabilities of the solution that meets our client&rsquo;s business goals and expectations.</p><p>&nbsp;<strong>Responsibilities</strong></p><ul><li>You will own the project technical workstream, ensuring that the final technical solution addresses and satisfies all business requirements.</li><li>You will translate business needs into technical design and requirements.</li><li>You will collaborate with client technical stakeholders to establish appropriate access to the client&rsquo;s environment and to identify the client&rsquo;s data requirements.</li><li>You will support an initial investigation into the quality and content of client data sources in support of the technical design and development of business requirements</li><li>You will establish the instance and technical environments for development, testing, and deployment</li><li>You will create the technical design documentation to reflect the technical requirements needed for successful delivery.</li><li>You will configure the core functionality to meet client needs.</li><li>You will create the data model and populate all needed data structures to ensure access to the right data to support data point development.</li><li>You will design, build, and test the calculation of data points and client data transformations to support use case requirements, including documentation and code execution</li><li>You will support testing and ongoing platform optimization efforts through effective data engineering and system configurations</li><li>You will maintain metadata, including data dictionaries, for custom data points and data transformations</li></ul><p><strong>Requirements</strong></p><ul><li>2+ years&rsquo; experience as a software developer/analyst or software tester.</li><li>Proficiency in Relational Databases, such as MySQL, MS SQL Server, or Oracle, and comfortable working with relational data structures.</li><li>Design and identify relationships between multiple data entities (e.g. CRM, data warehouse, proprietary product database)</li><li>Experience with technical coding standards in SW product development.</li><li>Client facing skills required.</li><li>Excellent oral and written communication skills</li><li>A self-starter - someone who is motivated by curiosity and is comfortable working in a dynamic environment with minimal direction.</li></ul><ul><li>Experience with Jira, Git/ Bitbucket, Gradle, Sourcetree, Pentaho Kettle, Rundeck, and/or other ETL tools or solutions is a plus. </li><li>Experience with Waterfall and/or Agile software methodologies.</li><li>Veeva CRM knowledge or experience a plus</li></ul><p><strong>No Visa Sponsorships available at this time</strong></p> Harp Talent Solutions, LLC Philadelphia PA

Data Engineer

Globex Digital Solutions