Data Engineering Lead

Tata Consultancy Services Malvern , PA 19355

Posted 2 months ago

Technology exp required:

Relational database, SQL, transaction log, replication, applying deltas

Amazon Web Services (AWS) - load data into S3 (ssh, sftp, AWS CLI)

Deep expertise with Python scripting preferably including Anaconda on AWS (processing CSV / pipe-delimited data files)

Experience in building reliable multi-step distributed data pipelines

Provide technical leadership for data engineering enabling us to advance architecture

The Architect would help guide the technical architecture for projects such as data replication from OnPrem to AWS, Data management, DBaaS, data modeling for the Data Lake on Cloud and data security.

Shaping an architecture to meet the needs of cloud data architecture to service identified use cases will be the primary technical objective of the role

Advanced knowledge of NoSQL and Graph databases, Hadoop, and ETL technologies will be needed to design overall data platform design

Experience designing systems to ingest, process, and get Big Data consumption ready for Data Science is required

Jupyter notebook on SageMaker for simple Python analytics

Duties and Responsibilities
1.Provides the architectural leadership in shaping strategic technology programs, which focus on both Business of IT (e.g., Unified Communication, SOI) and LOB-specific strategic technology programs.
2.Continuously pursues advanced level technical acumen:

  • Attends conferences and engages in associated activities (e.g., conducting presentations, leading workshops, etc.).

  • Consumes and contributes content from and to Open Source communities.
    3.Provides architecture thought leadership and expertise including cost optimization
    4.Defines reference and implementation architectures.
    5.Produces technology roadmaps in support of IT's vision and strategy.
    6.Develops proof-of-concept prototypes and initial implementation models.
    7.Monitors implementation activity to ensure architecture and design principles are upheld.
    8.Ensures implementation solutions support architecture objectives (availability, scalability, performance, security, etc.), as appropriate.
    9.Rolls up sleeves and does deep dives on Financial Services data sets to understand intricacies and come up with optimal data models

  1. Is adept with handling multiple data types from multiple data sources and strategize on the optimal way to analyze/store such data sets


  • Minimum 10 years relevant work experience.

  • Highly experienced with Big Data, Glue, EMR, and Data Quality tools/procedures for Cloud native data stores

  • Strong database and cloud expertise, along with experience with data replication technology required

  • Strong knowledge of micro-services and DDAAS required

  • Expert level AWS and Cloud native data experience is required

  • Minimum five years big data and advanced analytics experience required

  • Minimum 2 years of cloud-native database design, migration, and implementation experience required

  • Experience with Enterprise Information Management (EIM) is required

  • Experience with CDC tools (Attunity preferred) to move data from OnPrem sources

See if you are a match!

See how well your resume matches up to this job - upload your resume now.

Find your dream job anywhere
with the LiveCareer app.
Download the
LiveCareer app and find
your dream job anywhere

Boost your job search productivity with our
free Chrome Extension!

lc_apply_tool GET EXTENSION

Similar Jobs

Want to see jobs matched to your resume? Upload One Now! Remove
Data Scientist I


Posted 1 week ago

VIEW JOBS 11/6/2018 12:00:00 AM 2019-02-04T00:00 To build models using advanced analytic techniques, driving targeting, treatment and segmentation insights. Duties and Responsibilities: 1. Develops queries and performs extensive programming to access, transform and prepare data for statistical modeling. Identifies and diagnoses data inconsistencies and errors, forages to fill data gaps and blazes new data trails. Creates derived characteristics. 2. Designs and develops models to target marketing and service offers. Supports production implementation of models, and validates models. Creates model documentation. 3. Performs deep dive diagnostic, predictive and prescriptive analytics for customer behavioral segmentation, look-alike targeting, growth opportunity estimation and next best action identification. 4. Prepares and delivers insight presentations and action recommendations. Communicates complex material in layman's terms. Educates leaders and crew on analytical findings and implications using storytelling, data visualization and graphical artisanship. 5. Designs analytic plans, translating business problems/opportunities into analytic approaches. Engages with project sponsors and stakeholders to understand each business question. Probes for hidden questions and goals. Brings structure to each request and translates requirements into an analytic approach. 6. Analyzes data to describe client behavior and preferences, understand channel usage, diagnose client experience problems, size marketing opportunity, target clients for treatment and measure the impact of marketing and servicing initiatives. Identifies and utilizes appropriate analytic methods and specialized tools to generate insights, answer the business questions and fulfill project objectives. Seeks out innovative approaches and techniques. 7. Guides test design, research design and model validation. Provides statistical consultation services. Serves as the analytics expert on cross functional teams for large strategic initiatives. 8. Develops and applies domain knowledge of Vanguard's businesses and data to shape the analytic road map. 9. Contributes to the growth of the analytic community. 10. Participates in special projects and performs other duties as required. Qualifications: * Graduate degree in Analytics, Applied Mathematics, Economics, Statistics or related analytical field of study or equivalent combination of training and experience. Advanced degree preferred. * A minimum of five years general experience and five years relevant professional experience in analytical roles. * Experience developing creative analytical solutions to marketing and client service problems and opportunities. * Able to take disparate data and understand it, process it, extract value from it, visualize it and communicate it. * Experience consulting to understand business problems/opportunities, translating needs into analytic projects and applying results. * Data wrangling - Programming skills to access, transform and prepare large scale data for statistical modeling. * Experience utilizing statistical and machine learning methods: data imputation, propensity models, regression (logistic, linear), clustering (k-nearest neighbors, hierarchical), decision trees (CHAID, CART), random forest, uplift modeling, optimization, simulation, time series, hazard function and survival analysis, Bayesian methods, principal component analysis, support vector machines, association rules, ensemble methods, forecasting, model validation, design of experiments, text mining. * Skills in the data science tool stack: SAS (Base, Macro, Stat), SQL, KnowledgeSTUDIO, Python, R. * Domain experience in marketing analytics: customer acquisition, digital marketing, response modeling, identifying customer needs, predicting customer behavior, next best action, diagnosing customer struggle, attrition/churn, customer lifetime value, sales territory analytics. * Strong communicator of complex concepts and analysis, able to handle a variety of audience knowledge levels. Data visualization and storytelling skills. Vanguard is not offering visa sponsorship for this position. Vanguard Malvern PA

Data Engineering Lead

Tata Consultancy Services