Sr. Scala Engineer, Database Engineering

Experfy Inc Miami , FL 33124

Posted 3 weeks ago

As a Sr. Software Engineer for our Data Platform Engineering team you will join skilled Scala engineers and core database developers responsible for developing hosted cloud analytics infrastructure (Apache Spark-based), distributed SQL processing frameworks, proprietary data science platforms, and core database optimization. This team is responsible for building the automated, intelligent, and highly performant query planner and execution engines, RPC calls between data warehouse clusters, shared secondary cold storage, etc. This includes building new SQL features and customer-facing functionality, developing novel query optimization techniques for industry-leading performance, and building a database system that's highly parallel, efficient and fault-tolerant. This is a vital role reporting to exec leadership and senior engineering leadership.

Requirements

  • Writing Scala code with tools like Apache Spark + Apache Arrow to build a hosted, multi-cluster data warehouse for Web3
  • Developing database optimizers, query planners, query and data routing mechanisms, cluster-to-cluster communication, and workload management techniques
  • Scaling up from proof of concept to cluster scale (and eventually hundreds of clusters with hundreds of terabytes each), in terms of both infrastructure/architecture and problem structure
  • Codifying best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases to facilitate meta data capturing and management
  • Managing a team of software engineers writing new code to build a bigger, better, faster, more optimized HTAP database (using Apache Spark, Apache Arrow and a wealth of other open source data tools)
  • Interacting with exec team and senior engineering leadership to define, prioritize, and ensure smooth deployments with other operational components
  • Highly engaged with industry trends within analytics domain from a data acquisition processing, engineering, management perspective
  • Understand data and analytics use cases across Web3 / blockchains
Skills & Qualifications
  • Bachelors degree in computer science or related technical field. Masters or PhD a plus.
  • 6+ years experience engineering software and data platforms / enterprise-scale data warehouses, preferably with knowledge of open source Apache stack (especially Apache Spark, Apache Arrow, and others)
  • 3+ years experience with Scala and Apache Spark
  • A track record of recruiting and leading technical teams in a demanding talent market
  • Rock solid engineering fundamentals; query planning, optimizing and distributed data warehouse systems experience is preferred but not required
  • Nice to have: Knowledge of blockchain indexing, web3 compute paradigms, Proofs and consensus mechanisms is a strong plus but not required
  • Experience with rapid development cycles in a web-based environment
  • Strong scripting and test automation knowledge
  • Nice to have: Passionate about Web3, blockchain, decentralization, and a base understanding of how data/analytics plays into this
icon no score

See how you match
to the job

Find your dream job anywhere
with the LiveCareer app.
Mobile App Icon
Download the
LiveCareer app and find
your dream job anywhere
App Store Icon Google Play Icon
lc_ad

Boost your job search productivity with our
free Chrome Extension!

lc_apply_tool GET EXTENSION

Similar Jobs

Want to see jobs matched to your resume? Upload One Now! Remove
Sr Software Database Engineer

Xylem Inc.

Posted 1 week ago

VIEW JOBS 11/17/2022 12:00:00 AM 2023-02-15T00:00 <p>Xylem |ˈzīləm|</p><ol><li><p>The tissue in plants that brings water upward from the roots;</p></li><li><p>a leading global water technology company.</p></li><br></ol><p>Xylem, a leading global water technology company dedicated to solving the world's most challenging water issues, is the leading global provider of efficient, innovative and sustainable water technologies improving the way water is used, managed, conserved and re-used. Our international team is unified in a common purpose: creating advanced technology and other trusted solutions to solve the world's water challenges. We are committed to creating an organization of inclusion and diversity, where everyone feels involved, respected, valued and connected, and where everyone is free to bring their authentic selves and ideas.</p><p>If you are excited and passionate about helping us #letssolvewater, we want to hear from you!</p><p>The Role: Sensus, a Xylem brand, seeks to hire a Sr Software Database Engineer, as a member of the Software Engineering Team. You will be responsible for developing, testing, improving and maintaining new and existing databases to help support of Advanced Meter Infrastructure (AMI) headend system. The successful candidate will have demonstrated the ability to succeed in a fast paced, fluid environment, while ensuring that project initiatives are met. If you are excited and passionate to work on state-of-the art technology trends we want to hear from you!</p><p>Essential Duties/Principal Responsibilities:</p><ul><li><p>Collaborate with other development teams to ensure products work flawlessly within Sensus solutions and with 3rd party offerings</p></li><li><p>Takes great personal pride in building robust modular software</p></li><li><p>Has strong sense of ownership and drive</p></li><li><p>Enjoys working in a fast-paced agile environment</p></li><li><p>Has excellent written and verbal communication skills</p></li><li><p>Has strong customer focus</p></li><li><p>Participation in Xylem Watermark volunteer activities</p></li></ul><p>Minimum Qualifications: Education, Experience, Skills, Abilities, License/Certification:</p><ul><li><p>BS/MS in Computer Science or equivalent</p></li><li><p>Minimum 5 years of experience in database design, coding, and documentation</p></li><li><p>SQL or Oracle database development experience</p></li><li><p>Ability to communicate effectively in writing, orally with both local and remote sites</p></li><li><p>Ability to work collaboratively within a team environment of engineers to meet aggressive goals and high-quality standards</p></li><li><p>Demonstrated experience working with cross functional teams</p></li></ul><p>Preferred Qualifications:</p><ul><li><p>Object-oriented design and coding skills (Java, C#, and/or C++), preferably on a UNIX or Linux platform developing Systems software</p></li><li><p>Solid understanding of data structures and algorithms</p></li><li><p>Online distributed systems, real-time systems, platform infrastructure design and implementation</p></li><li><p>Middleware software solutions design, development and integration</p></li><li><p>Multi-threaded/concurrent application development in large scale high availability systems</p></li><li><p>Code reviews and Unit testing</p></li><li><p>Continuous integration and deployment</p></li><li><p>Mission critical services or services operating in a 24x7 environment</p></li><li><p>Agile development processes</p></li></ul><p>We Offer You More Than Just &quot;A Job&quot;</p><ul><li>Professional Development</li></ul><ul><li>To advance the capabilities of our people, we offer a wide variety of experiences to support our employees' professional growth and continuous learning.</li></ul><ul><li>Total Rewards</li></ul><ul><li>We offer comprehensive programs for compensation, benefits, recognition, learning and development, work-life integration and corporate citizenship.</li></ul><ul><li>Watermark</li></ul><ul><li>Watermark is our corporate social responsibility program working to provide education and access to safe water to ensure healthy lives, gender equality, and resilient communities. Employees have the opportunity to learn and volunteer on various water-related projects.</li></ul><ul><li>Employee Networks</li></ul><ul><li>Our Employee Networks provide a professional, supportive network for employees from diverse backgrounds, including Women's, LGBT+ and Allies, Veteran's, People of Color and Allies, Hispanic Origin &amp; Latin Affinity, Emerging Leaders, Working Parents Networks and Non-Native English Speakers and Allies.</li></ul><p>Physical Demands/Work Environment:</p><p>(The physical demands and work environment described here are representative of those an employee encounters and must be met to successfully perform the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.)</p><ul><li><p>Regularly required to sit or stand, reach, bend and move about the facility</p></li><li><p>Office: Standard office equipment; work usually performed in an office setting free from any disagreeable elements.</p></li><li><p>Standard weekly job hours: 40 hours</p></li></ul><p>Xylem is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, national or ethnic origin, sex, sexual orientation, gender identity or expression, age, disability, protected veteran status or other characteristics protected by law.</p><br> Xylem Inc. Miami FL

Sr. Scala Engineer, Database Engineering

Experfy Inc