Senior Big Data Engineer

Allstate Denver , CO 80208

Posted 2 months ago

Description :

Where good people build rewarding careers.

Think that working in the insurance field can't be exciting, rewarding and challenging? Think again. You'll help us reinvent protection and retirement to improve customers' lives. We'll help you make an impact with our training and mentoring offerings. Here, you'll have the opportunity to expand and apply your skills in ways you never thought possible. And you'll have fun doing it. Join a company of individuals with hopes, plans and passions, all using and developing our talents for good, at work and in life.

Job Description

Allstate Insurance Company has an exceptional career opportunity for a Senior Big Data Engineer!

This Senior Consultant is an experienced individual contributor who will leverage data and analytics to help automate and optimize Claims Analytics Data processes enabling our Claims employees to focus on serving our customers and delivering the most advanced claims experience on the planet. They will be responsible for helping to bring together complex data into clean and useful data structures making our valuable data more approachable.

Key Responsibilities:

  • Involved in the design, prototyping and delivery of software solutions within the big data eco-system

  • Work independently on big data projects and/or serving as analytics SME to provide new or enhanced data to the business

  • Improving data governance and quality increasing the reliability of our data

  • Influencing the creation of a single, trusted source for key Claims business data that can be shared across the Enterprise

  • Responsible for designing and building new Big Data systems for turning data into actionable insights

  • Train and mentor junior team members on Big Data/Hadoop tools and technologies

  • Identifies opportunities for improvement and presents recommendations to management

  • Develop solutions and iterates quickly to continuously improve

  • Seeks out and evaluates emerging big data technologies and open-source packages

  • Participate in strategic planning discussions with technical and non-technical partners

  • Uses, learns, teaches, and supports a wide variety of Big Data and Analytics tools to achieve results (i.e., Python, Hadoop, HIVE, Scala, Impala and others).

  • Uses, learns, teaches, and supports a wide variety of programming languages on Big Data and Analytics work (i.e. Java, Python, SQL, R)

Job Qualifications

  • Undergraduate degree in Computer Science, Mathematics, Engineering (or related field) or equivalent experience preferred

  • 3-5 years of experience preferred in a data integration, ETL and/or business intelligence/analytics related function

  • Ability to work with broad parameters in complex situations

  • Experience in developing, managing, and manipulating large, complex datasets

  • Expert high-level coding skills such as SQL and Python and/or other scripting languages(UNIX) required. Scala is a plus.

  • Some understanding and exposure to - streaming toolsets such as Kafka, FLINK, spark streaming a plus.

  • Experience with source control solutions (ex git, GitHub, Jenkins, Artifactory) required

  • At least 1-2 years of experience with big data and the Hadoop ecosystem (HDFS, SPARK, SQOOP, Hive, Impala, Parquet) required

  • Experience with Agile development methodologies and tools to iterate quickly on product changes, developing user stories and working through backlog (Continuous Integration and JIRA a plus)

  • Experience in any scheduling tools such as IBM Tivoli Workload Scheduler, Control M, AutoSys, etc required

  • Must be a self-starter and have excellent oral and communication skills

The candidate(s) offered this position will be required to submit to a background investigation, which includes a drug screen.

Good Work. Good Life. Good Hands.

As a Fortune 100 company and industry leader, we provide a competitive salary but that's just the beginning. Our Total Rewards package also offers benefits like tuition assistance, medical and dental insurance, as well as a robust pension and 401(k). Plus, you'll have access to a wide variety of programs to help you balance your work and personal life -- including a generous paid time off policy.

Learn more about life at Allstate. Connect with us on Twitter, Facebook, Instagram and LinkedIn or watch a video.

Allstate generally does not sponsor individuals for employment-based visas for this position.

Effective July 1, 2014, under Indiana House Enrolled Act (HEA) 1242, it is against public policy of the State of Indiana and a discriminatory practice for an employer to discriminate against a prospective employee on the basis of status as a veteran by refusing to employ an applicant on the basis that they are a veteran of the armed forces of the United States, a member of the Indiana National Guard or a member of a reserve component.

For jobs in San Francisco, please click "here" for information regarding the San Francisco Fair Chance Ordinance.

For jobs in Los Angeles, please click "here" for information regarding the Los Angeles Fair Chance Initiative for Hiring Ordinance.

It is the policy of Allstate to employ the best qualified individuals available for all jobs without regard to race, color, religion, sex, age, national origin, sexual orientation, gender identity/gender expression, disability, and citizenship status as a veteran with a disability or veteran of the Vietnam Era.

See if you are a match!

See how well your resume matches up to this job - upload your resume now.

Find your dream job anywhere
with the LiveCareer app.
Download the
LiveCareer app and find
your dream job anywhere

Boost your job search productivity with our
free Chrome Extension!

lc_apply_tool GET EXTENSION

Similar Jobs

Want to see jobs matched to your resume? Upload One Now! Remove
Big Data Engineer

Captech Richmond

Posted 2 days ago

VIEW JOBS 11/12/2018 12:00:00 AM 2019-02-10T00:00 Company Description CapTech is a national IT management consulting firm that bridges the gap between business and technology. We partner with some of the world's most successful companies to design, develop, and manage technical and digital solutions that delight customers, drive insights, and meet strategic objectives. At CapTech, you'll experience a flat organizational structure based upon a culture of mutual respect. Here, your career path isn't set in stone, and you have unlimited potential for growth and the resources to help you achieve your career goals. If you are creative, technically insightful, driven–and have a passion for the work that we do, then you might be a future CapTecher. For over five years, we have been in the Top 10 of Consulting Magazine's Best Firms to Work For, including #2 for IT Firms and #5 for Work/Life Balance. We are also recognized by's Top 50 Consulting Firms and have been on the Inc. 5000 list of fastest growing companies for over 10 years. Job Description The Big Data Engineer role falls into the Data & Analytics practice area at CapTech, through which our consultants provide a broad spectrum of services to help our clients define and implement a strategy to deliver lasting and mission-critical analytic capabilities. Our Big Data Engineers are data pipeline builders and data wranglers, implementing modern data architectures to deliver data to data analysts and scientists at a large scale. By partnering with clients to fully understand both their business philosophy and IT strategy, CapTech consultants maintain the vision that data pipelines should be built to help the organization make better decisions by providing the right data at the right time. Specific responsibilities for the Big Data Engineer position include: * Design, develop, and implement advanced data pipelines that bring together data from disparate sources, making it available to data scientists, analysts, and other users using a variety of programming languages (Java, Scala, Python, etc). * Facilitate cross-functional data-integration efforts upstream and downstream * Detect data quality issues, identify their root causes, implement fixes, and design data audits to capture issues * Extract data from multiple sources, and integrate them into a target database, application, or file using efficient programming processes at scale * Build pipelines on-premises, in the cloud, or in a hybrid architecture as needed for each client * Optimize and improve existing data pipelines by discovering and improving bottlenecks within the system as well as bringing in new data sources into data lakes and other datastores. * Migrate from traditional ETL to cloud-based or on-premises Big Data solutions using a variety of toolsets * Implement and deploy solutions in a CI/CD pipeline * Present programming documentation and design to team members and convey complex information in a clear and concise manner. * Communicate with all levels of stakeholders as appropriate, including executives, data modelers, application developers, business users, and customers * Participate in requirements gathering sessions with business and technical staff to distill technical requirements from business requests. * Partner with clients to fully understand business philosophy and IT Strategy; recommend process improvements to increase efficiency and reliability. * Collaborate with Quality Assurance resources to debug code and ensure the timely delivery of products. * Some of our technologies might include: Hadoop Ecosystem, HBase, Hive, Spark, Kafka, HDInsight, EMR, Elasticsearch, and Kinesis as well as various datastores such as Cassandra, MongoDB, Redis, Snowflake, Redshift, HDFS, S3, Aurora, Azure Storage, and Azure Data Warehouse Qualifications Specific qualifications for the Big Data Engineer position include: * Bachelor's degree in Computer Science, Software Engineering, MIS or equivalent combination of education and experience * Development experience building data pipelines at scale * Strong analytical skills working with structured, semi-structured, and unstructured datasets * Minimum of 3 years experience designing, developing, and testing software aligned with defined requirements * Experience with Apache Hive, Impala, and other technologies for querying distributed file systems * Development experience with at least two different programming languages (Scala, Python, Java, C, etc.) * Development experience with Unix tools and shell scripts * Experiencing with modeling of relational as well as NoSQL datastores * Experience with MapReduce and Apache Spark * Familiarity with streaming, pub-sub, and message queueing systems * Software engineering best-practices, including but not limited to version control (Git, TFS, Subversion, etc.), CI/CD (Jenkins, Maven, Gradle, Ant, Artifcatory, etc.), and unit testing * Experience with Cloud-based technologies preferred * Exposure to ETL, visualization, and analytic toolsets preferred * Experience with Cloudera, Hortonworks, MapR, or other Hadoop distribution preferred * AWS Big Data Specialty Cert, Cloudera CCA/CCP certification, or equivalent certification preferred Additional Information We offer challenging and impactful jobs with professional career paths. All CapTechers can keep their hands on technology no matter what position they hold. Our employees find their work exciting and rewarding in a culture filled with opportunities to have fun along the way. At CapTech we offer a competitive and comprehensive benefits package including, but not limited to: * Competitive salary with performance based bonus opportunities * Single and Family Health Insurance plans, including Dental coverage * Short-Term and Long-Term disability * Matching 401(k) * Competitive Paid Time Off * Training and Certification opportunities eligible for expense reimbursement * Team building and social activities * Mentor program to help you develop your career At this time, CapTech cannot transfer nor sponsor a work visa for this position. Applicants must be authorized to work directly for any employer in the United States without visa sponsorship. Candidates must be eligible to work in the U.S. for any employer directly (we are not open to contract or "corp to corp" agreements). 

 CapTech is an equal opportunity employer. 
 CapTech is a Drug-Free work place.
 Candidates must have the ability to work at CapTech's client locations. 
 All positions include the possibility of travel. CapTech has not contracted/does not contract with any outside vendors in its recruitment process. If you are interested in this position, please apply to CapTech directly. Captech Richmond Denver CO

Senior Big Data Engineer