Sorry, this job is no longer accepting applications. See below for more jobs that match what you’re looking for!

Senior Data Engineer

Expired Job

Gravity Infosolutions, Inc McLean , VA 22102

Posted 3 months ago

seeking a Big Data Engineer who has Nifi (primary focus) and Kafka skillsets to work on its big data acquisition team. Additionally, Python and PySpark skills and AWS experience is desired.

Responsibilities include:

Responsible for delivery in the areas of: big data engineering mainly data acquisition technology implementations (Nifi & Kafka)

Develop scalable and reliable data solutions to move data across systems from multiple sources in real time as well as batch modes (Nifi/Kafka/Spark)

Construct data staging layers and fast real-time systems

Utilize expertise in models that leverage the newest data sources, technologies, and tools, such as machine learning, Python, Hadoop, Spark, AWS, as well as other cutting-edge tools and applications for Big Data

Investigate the impact of new technologies, applications, and data sources on the future secondary mortgage business

Demonstrated ability to quickly learn new tools and paradigms to deploy cutting edge solutions

Develop both deployment architecture and scripts for automated system deployment in AWS

Create large scale deployments using newly researched methodologies

Work in Agile environment

Experience mentoring junior engineers

Qualifications

Bachelors degree in Mathematics, Statistics, Computer Science, related field, or equivalent work experience

Experience with Nifi & Kafka

Experience with Hadoop including Hive and Spark

Experience in Python & PySpark
Experience working with AWS
5-8 years of experience in Software development, ideally 8+ years

Skills:
Bigdata, Spark, Nifi, Kafka, Python, PySpark, AWS, Hive
Permanent
Full-time


See if you are a match!

See how well your resume matches up to this job - upload your resume now.

Find your dream job anywhere
with the LiveCareer app.
Download the
LiveCareer app and find
your dream job anywhere
lc_ad

Boost your job search productivity with our
free Chrome Extension!

lc_apply_tool GET EXTENSION

Similar Jobs

Want to see jobs matched to your resume? Upload One Now! Remove
Government & Public Sector Senior Technology Consulting Data Engineer Secret Clearance Tysons VA

Ernst & Young LLP

Posted 5 days ago

VIEW JOBS 11/14/2018 12:00:00 AM 2019-02-12T00:00 Ernst & Young LLP's (EY) US Government and Public Sector (GPS) practice is a robust and emerging practice with tremendous potential for growth - and we are prepared to tap into that potential. Our US GPS practice combines an understanding of the public sector's diverse needs, focused on delivering improved public services, with private sector leading practices. We respond to our client's issues by bringing together highly skilled professionals across Advisory, Assurance, Tax and Transaction Advisory services. This integrated approach facilitates multidimensional support and flexibility to meet our clients changing needs. We are seeing tremendous growth within the GPS team and there is a great need for rising leaders. Unprecedented challenges require us to be connected, responsive and insightful. Are you ready to join our team and help build a better working world? The opportunity A Data Engineer combines skills of database administration with those of a developer and has a strong foundation in Extract, Transform, and Load (ETL), data governance, and automation. A Data Engineer will work with myriad data management systems to feed the data needs of the business through efficient SQL querying, effective ETL development, and potentially other scripting languages, e.g. – Python, to cleanse and move data. Each data need and problem will have a unique solution which requires a creative mind to solve using a collection of development and data manipulation skills. In this role you will design and implement how data will be best retrieved, stored, and integrated into the existing data solution. You will also work with data consumers to meet specific requirements of a given data-related task. . Knowledge and Skills Requirements * SQL Databases: SQL Server, Oracle, Redshift, SQL Data Warehouse * NoSQL Databases: Cloudera, Hortonworks, HDInsight, MongoDB * Expertise in one of many industry ETL tools: Microsoft SSIS, Informatica, Alteryx, etc. * Experience integrating with APIs (SOAP and REST) for data extraction * Using Python, C#, other scripting languages to extract and manipulate data * Experience managing data extraction scripts and ETL packages via GIT or other code repository tools * Experience with streaming data sources, transformations, and real-time data architectures * General experience working on Business Intelligence and Analytics focused projects and initiatives * Ability to interface with data consumers to understand data requirements and translate to technical requirements * Experience working with data lakes in addition to traditional data warehousing * Working knowledge of OLAP cube building, maintenance, and tuning * Strong general technical skills and troubleshooting techniques * Experience on Agile projects a plus * Experience on projects utilizing continuous delivery and deployment methodologies a plus Your key responsibilities * Design and build of ETL programs, interfaces, and data reconciliation processes * Management of information assurance and data governance requirements * End-user support for data reconciliation and interface validation * Assistance in developing reports and dashboards for data consumers * Foster relationships with client personnel at appropriate levels. Consistently deliver quality client services. Drive high-quality work products within expected timeframes and on budget. Monitor progress, manage risk and ensure key stakeholders are kept informed about progress and expected outcomes. Manage expectations of client service delivery. Stay abreast of current business and industry trends relevant to the client's business. * Demonstrate in-depth technical capabilities and professional knowledge. Demonstrate ability to assimilate new knowledge. Possess good business acumen. Remain current on new developments in advisory services capabilities and industry knowledge. * Effectively manage and motivate client engagement teams with diverse skills and backgrounds. Provide constructive on-the-job feedback/coaching to team members. Foster an innovative and inclusive team-oriented work environment. Play an active role in counseling and mentoring junior consultants within the organization. * Cultivate and manage business development opportunities. Understand EY and its service lines and actively assess/present ways to serve clients. Develop and maintain long-term client relationships and networks. Develop relationships with team members across all EY practices to serve client needs. To qualify for the role you must have * A Bachelor's or advanced degree in Computer Science, Information Management Systems, Information Technology, Engineering or related field with approximately 4 years of related work experience * Strong analytical and problem-solving skills as well as excellent oral and written communication skills * A willingness to travel to meet client needs and a valid driver's license. * Active Secret Clearance or higher * Proximity to the Washington D.C. Metro area Skills and attributes for success * Demonstrated abilities and success with identifying and addressing client needs * Actively participating in client discussions and meetings * Demonstrated abilities and success as a team member * Created a positive environment by monitoring workloads of the team while meeting client expectations and respecting the work-life quality of team members * Provided candid, meaningful feedback in a timely manner, and keeping leadership informed of progress and issues * Excellent leadership, communication (written and oral) and interpersonal skills * PMP, TOGAF, DODAF and/or ITIL certification What we look for * The ideal candidate will consistently achieve project objectives on time, within budget and with the highest quality results * Candidate must possess excellent verbal and written communication skills, ability to interact with both business and technical stakeholders * Candidate must be a team player as well as be able to work independently and manage development resources * Candidate will recognize the potential risks and is able to provide solutions What working at EY offers We offer a competitive compensation package where you'll be rewarded based on your performance and recognized for the value you bring to our business. In addition, our Total Rewards package includes medical and dental coverage, both pension and 401(k) plans, a minimum of 15 days of vacation plus ten observed holidays and three paid personal days, and a range of programs and benefits designed to support your physical, financial and social well-being. Plus, we offer: * The opportunity to manage a portfolio of talent within a designated service line * A collaborative environment where everyone works together to create a better working world * Excellent training and development prospects, both through established programs and on-the-job training * An excellent team of senior colleagues, dedicated to managing and varying your workload About EY As a global leader in assurance, tax, transaction and advisory services, we hire and develop the most passionate people in their field to help build a better working world. This starts with a culture that believes in giving you the training, opportunities and creative freedom to make things better. So that whenever you join, however long you stay, the exceptional EY experience lasts a lifetime. Join us in building a better working world. Apply now. Ernst & Young LLP, an equal employment opportunity employer (Females/Minorities/Protected Veterans/Disabled), values the diversity of our workforce and the knowledge of our people All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity or national origin . Ernst & Young LLP Mclean VA

Senior Data Engineer

Expired Job

Gravity Infosolutions, Inc