Cardinal Technology Solutions New York City , NY 10017
Send your resume for immediate consideration with availability, contact and salary details to
Title: Data Engineer
Location: NYC, 10017
We are seeking an outstanding and forward-thinking Data Engineer to help manage our ever-growing information needs and answer increasingly complex business questions. This person will work on collecting, storing, processing, and analyzing huge sets of data.
The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. This person will also be responsible for integrating these data sets with the existing architecture used across the company.
Because we work on the cutting edge of a lot of technologies, we need someone who is a creative problem solver, resourceful in getting things done, and productive working independently or collaboratively.
As a member of the growing data engineering and business analytics team, this person will be a key contributor to the successful deployment of strategic future state capabilities critical for our business.
Gather, profile, analyze and process raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.)
Design and develop code to load and transform the source data from various formats (like relational, unstructured, semi-structured) into a form suitable for analysis
Identify reporting requirements and translate those into supporting data structures including dimensional (star-schema) marts, aggregate and normalized tables
Conduct ETL, SQL and DB performance tuning, troubleshooting and capacity estimation to meet SLA's and ensure highest data quality standards
Support business decisions with ad hoc analysis as needed
Bachelor's degree in related area (Computer Science, Information Systems, Engineering, Statistics) or an equivalent combination of education and experience
5+ years of design, development, testing and support for successfully designing and implementing big data-centric applications and programs
Experience working with streaming data ingestion services like Amazon Kinesis, Firehose, Kafka, Flume, etc
Proficient in at least one of the commonly used data integration tool like Microsoft's SSIS, Informatica, SnapLogic, PL/SQL, or any other open source tool
Powerful hands-on SQL based data extraction skills to manipulate data from common relational database systems (Oracle, Postgres, SQL Server), distributed systems (Amazon Redshift, Teradata), NoSQL databases (Mongo DB)
Good understanding of Lambda Architecture
Experience working on big data technologies like Hadoop v2.0, Spark, Amazon EMR, MapReduce, Cloudera or Hortonworks
Proficient in using one of the big data tools such as Pig, Hive, Spark SQL, Impala, Presto, Sqoop, Oozie, etc..
Experience with Business Intelligence and reporting tools (preferably Tableau, R, SSRS)
Understanding and experience in 3NF & dimensional (STAR schema) approaches for data warehouse environment
Understanding of various compression techniques and storage formats
Familiar with the workings of network, servers, AWS infrastructure & cloud services
Familiar in data mining, machine learning, natural language processing or Statistical analysis
Demonstrated experience translating business and technical requirements into data models
Soft Skills & Capabilities:
Ability to work independently on assignments with little oversight
High aptitude for logical thinking and problem solving techniques
Be proactive and passionate about delivering good quality product with efficient time management practices
Ability to sense risks and research issues before hand so as to analyze underlying problems before recommending possible solutions
Strong interpersonal skills (including written and verbal communication) to work effectively in a team environment and also part of a matrix organization
Work with a culturally diverse team from varying backgrounds
Must be flexible for changing business requirements and assignments with an ability to work on multiple projects simultaneously
Proven ability to mentor and develop others and help them grow their skills
Has a positive and creative attitude to unknown challenges
Passionate with an eagerness to learn & try out new open source and emerging technologies based on distributing and/or cloud based computing
Cardinal Technology Solution
Work: x 329
Data engineer, big data, etl, AWS,PYTHON