Principal Data Lake Software Engineer - Cortex Platform

Palo Alto Networks Inc. Santa Clara , CA 95051

Posted 3 weeks ago


With Application Framework, we're building the foundations of the next inflection point in the security industry by combining our deep expertise in security with the state of the art in large-scale infrastructure for data processing and machine learning (Think exabytes of data).

We are seeking a Senior Principal Software Engineer to design, develop and deliver next-generation technologies within our Application Framework team that bridges cloud-based data processing and security disciplines to solve real-world problems. They build large-scale distributed software systems in Java, Python and other languages using open source technologies like Hadoop, Kafka, Redis, Spark, Flink etc, and other proprietary technologies.

We need passionate engineers who love to code and build great products. Engineers who bring new ideas in all facets of software development. We are looking for leaders who take ownership of their areas of focus and who are driven to solve problems at every level. Collaboration and teamwork are at the heart of our culture and we need engineers who can communicate at a high level and work well with others towards achieving a common goal.

This is a unique opportunity to help build the products and practices that will support our growth over the next decade. If you see the potential of how incredible people products can transform a business, this is the role for you. If you have an entrepreneurial spirit, know how to deliver, are deeply technical, highly innovative and long for the opportunity to build pioneering solutions to challenging problems, we want to talk to you.


  • Design, develop and implement different architectural models for our scalable data processing, as well as scalable data storage

  • Adapt to conventional Big Data frameworks and tools with the use-cases required by the project

  • Work with different development and quality assurances groups to achieve the best quality

  • Suggest and implement improvements to the development process

  • Work with research teams and data scientists, finding bottlenecks and resolving them

  • Build tools for proper data ingestion from multiple heterogeneous sources

  • Interview, mentor and coach new team members

  • Lead software development projects


  • Expert-level of experience in design and implementation in an environment with hundreds of terabytes of data

  • Works on significant and unique issues where analysis of situations or data requires an evaluation of intangibles

  • Exercises independent judgment in methods, techniques and evaluation criteria for obtaining results

  • Expert-level of experience with large data processing tools such as Hadoop, HBase, Elastic Search, Flink, Kafka, etc.

  • Expert-level of experience with Java development

  • Fast learner and eager to absorb new emerging technologies

  • Passion for doing research on large data sets containing ill-formatted data

  • Can-do attitude on problem-solving, quality and ability to execute

  • Enjoys working with many different teams with strong collaboration and communications skills

  • High energy and the ability to work in a fast-paced environment


  • BS/MS in Computer Science/Engineering, or equivalent experience
icon no score

See how you match
to the job

Find your dream job anywhere
with the LiveCareer app.
Mobile App Icon
Download the
LiveCareer app and find
your dream job anywhere
App Store Icon Google Play Icon

Boost your job search productivity with our
free Chrome Extension!

lc_apply_tool GET EXTENSION

Similar Jobs

Want to see jobs matched to your resume? Upload One Now! Remove
Principal Software Engineer Service Framework

Hewlett Packard Enterprise

Posted 2 days ago

VIEW JOBS 12/5/2019 12:00:00 AM 2020-03-04T00:00 At Hewlett Packard Enterprise, we bring together the brightest minds to create breakthrough technology solutions that advance the way people live and work. What sets us apart? Our people and our relentless dedication to helping our customers make their mark on the world. We are a team of doers, dreamers and visionaries; inspired by our purpose and driven by our strategy. We live by our three values: partner, innovate and act. Our legacy inspires us as we forge ahead, always pushing to discover what's next. Every day is a new opportunity to advance and grow ourselves, our company and the industry. Some people call it an obsession, we call it a way of life. What you need to know about the job Job ID: 1058298 Date Posted: 12/3/2019 Primary Location: Santa Clara, California Job Category: Engineering Schedule: Full time Shift: No shift premium (United States of America) The job family spans across multiple engineering disciplines. Work consists of designing, developing, analyzing, troubleshooting and debugging systems, software and solutions for research and/or research development of product, services, and solutions for company's portfolio. Requires a broad knowledge and application of engineering disciplines, methodologies and tools. Responsibilities: * Develops service framework and orchestration layer for complex distributed data platform which consist of software define distributed file system, No-SQL database and streaming engine. * Identifies and evaluates new technologies, innovations, and outsourced development partner relationships for alignment with technology roadmap and business value; creates plans for integration and update into architecture. * Reviews and evaluates designs and project activities for compliance with technology and development guidelines and standards; integrates suggestions, updates, and innovations into overall architecture. * Leverages recognized domain expertise, business acumen, and experience to influence decisions of executive business leadership, outsourced development partners, and industry standards groups. * Provides guidance and mentoring to less- experienced staff members to set an example of engineering design innovation and excellence. Education and Experience Required: * Bachelors or Master's degree in Computer Science, or equivalent. * Typically 10+ years' experience. Knowledge and Skills: * Prior experience working with complex distributed system orchestration, installation and configuration * Prior experience in working with Linux/Unix systems, good understanding of Kernel modules and dependencies * Prior experience building complex service frameworks using Java, Python and similar tools * Experience working with Container world, specially Docker, Kubernetes and Mesos * Using, evaluating, and developing appropriate engineering design tools and software packages. * Excellent analytical and problem solving skills. * Conducting reviews of material specifications and hardware and electrical component design * Evaluating and selecting forms of empirical analysis, modeling and testing methodologies to validate product designs and specifications * History of innovation with examples of developing new technologies and practices for designs * Excellent written and verbal communication skills; mastery in English and local language. Ability to effectively communicate product architectures, design proposals and negotiate options at business unit and executive levels. 1058298 HPE is an equal opportunity employer/Female/Minority/Individual with Disabilities/Protected Veteran Status Hewlett Packard Enterprise Santa Clara CA

Principal Data Lake Software Engineer - Cortex Platform

Palo Alto Networks Inc.