Senior Data Engineer

Change.Org San Francisco , CA 94118

Posted 4 months ago

At Change we are a unique blend of engineers, activists, marketers, designers and scientists with a common goal: to give the voiceless a voice and a chance to be heard in today's digital jungle; to allow everyone to connect with likeminded community and help people change the world for the better. People, the members, the activists, and our employees, are at the center of our mission and everything we do. Actually, employees at Change are a lot like the service itself: bright, brave, and innovative. Collaboration is the foundation of our workforce, and we're looking for smart individuals who are self-motivated and passionate to join us. Be a part of the team that creates a brighter future for everyone. Discover your future at Change!

Change has a great opportunity for a senior data engineer with several years of diverse experience who will help deliver Change to millions of people around the world. We are looking for someone who will own, lead, and execute projects. You should have outstanding analytical and programming skills with a deep understanding and proven track record of building robust, scalable, data processing pipelines. As a member of our highly motivated team, you should be dedicated to excellence and have a strong sense of personal responsibility. We hold ourselves to high standards and take pride in our work. We are looking for someone who is not afraid to get their hands dirty in data and be an integral part of the machine learning team's expansion.

What you'll do:

  • Work with data scientists and other data engineers to productize analytics and data models, developing and maintaining new ETL flows for new applications driven by model-based statistics and signals.

  • Push the envelope with scalable data processing and model deployment solutions leveraging technologies such as Spark, Kafka, Kinesis, Airflow, DynamoDB, TensorFlow, TensorBoard etc.

  • Build and support internal A/B testing and model evaluation platforms.

  • Tool our systems for observability, including logging, metrics monitoring, and dashboarding.

  • Have fun and make friends.


  • 5 years development experience of which 2-3 years are focused on data or analytics engineering working with big data technologies (Hadoop: i.e. MapReduce, HDFS/Hive, Spark)

  • Team player with excellent communication and interpersonal skills

  • Significant experience developing high quality software in Python, Scala or Java

  • Proficient with data preprocessing, data transformation, and integration of data from multiple data sources (ETL processes)

  • Experience with one of the following distributed relational databases: PostgreSQL, MySQL

  • Experience developing for Linux-based deployment platforms, developing scalable, multithreaded server side software for deployment

  • Experience designing and configuring hosted and cloud-based data and machine learning infrastructure

  • Experience unit testing with frameworks and a dedication to thorough testing to create high quality software, i.e. JUnit

  • Experience with productionizing feature engineering for machine/deep learning algorithms and exposure to machine learning algorithms and/or statistical modeling methods.

  • Experience with API design/development (i.e. RPC, REST, JSON, XML, SOAP)

  • Experience with NoSQL databases and key-value stores, such as Cassandra, Redis

Bonus skills:

  • Experience with recommender, or search/ranking systems

  • Experience with Kafka and Yarn or Mesos

  • Experience with AWS services (Athena, Glue, Redshift, Kinesis) or Google cloud services (BigQuery, BigTable)

  • BA/BS or above in Computer Science or a related field

This is a full-time opportunity. The position is located in San Francisco, CA, U.S.A. is committed to being a diverse and inclusive workplace. We encourage applicants of different backgrounds, cultures, genders, experiences, abilities and perspectives to apply.

All qualified applicants will receive consideration for employment without regard to race, color, national origin, religion, sexual orientation, gender, gender identity, age, physical disability, or length of time spent unemployed.

upload resume icon
See if you are a match!

See how well your resume matches up to this job - upload your resume now.

Find your dream job anywhere
with the LiveCareer app.
Mobile App Icon
Download the
LiveCareer app and find
your dream job anywhere
App Store Icon Google Play Icon

Boost your job search productivity with our
free Chrome Extension!

lc_apply_tool GET EXTENSION

Similar Jobs

Want to see jobs matched to your resume? Upload One Now! Remove
Senior Data Engineer


Posted 5 days ago

VIEW JOBS 1/13/2019 12:00:00 AM 2019-04-13T00:00 Company Description Software is changing the world, and we're at the center of it all. With more than 100,000 global customers (including 85 of the Fortune 100) and a highly disruptive business model, we're advancing the art of team collaboration with products like JIRA, Confluence, Trello and Stride–and we're just getting started. Driven by honest values, an amazing culture, and consistent revenue growth, we're out to unleash the potential of every team. From Amsterdam and Austin to Sydney and San Francisco, we're looking for people who are powered by passion and eager to do the best work of their lives in a highly autonomous yet collaborative, no B.S. environment. Job Description Atlassian is looking for a Senior Data Engineer to join our Data Engineering team and build world-class data solutions and applications that powers crucial business decisions throughout the organization. Your primary focus will be to build a state of the art HR datamart from the ground up. In this role, you'll enable our people team across the world to make data driven decisions. You'll enable analysts to do deeper analysis by providing clean and reliable data models. You'll enable data scientists to build models and there by change the way we operate. So, you'll have a big impact on our recruiting and talent management initiatives. On a typical day you may be consulted on the information architecture of our website or business applications, and help design the event collection infrastructure. Then you will move on to building the data models and ETL processes to provide this data for business use. As the data domain expert, you will be partnering with our technology teams, business teams, analytical teams, and data scientists across various initiatives. You will work with these partner teams to continually evolve solutions as business processes and the requirements change. You'll own a problem end-to-end, so data engineering skills will come in handy not just to collect, extract, and clean the data, but also to understand the systems that generated it, and automate your analyses and reporting. On an on-going basis, you'll be responsible for improving the data by adding new sources, coding business rules, and producing new metrics that support the business. Requirements will be vague. Iterations will be rapid. You will need to be nimble and take smart risks. MORE ABOUT YOU As a data engineer, you will have the opportunity to apply your strong technical experience on building analytics data models that supports a broad range of analytical requirements across the company. You've got industry experience working with large datasets. You have experience in handling and processing sensitive data. You enjoy working in a fast paced environment and you are able to take vague requirements and transform them into solid solutions. You are motivated by solving challenging problems, where creativity is as crucial as your ability to write code. On your first day, we'll expect you to have: * 5-10 years professional experience as a data engineer or in a similar role * Strong programming skills (some combination of Python, Java, and Scala preferred) * Experience building secure data marts for business areas like finance and HR in Spark or Hive. * Experience extracting data from HR systems (Workday, Lever etc.) * Experience building data pipelines using Spark and Hive * Experience writing SQL * Experience working with realtime streaming applications * A willingness to accept failure, learn and try again * An open mind to try solutions that may seem crazy at first It's great, but not required, if you have: * Experience working on Amazon Web Services (in particular using EMR, Kinesis, RDS and the like) * Experience in developing micro-services * You've used version control systems like Git and Subversion * You strive for continuous integration best practices * Bachelors in Computer Science or equivalent experience * Above all else, as a senior data engineer you will be leading the development process, driving architectural decisions, and incorporating business and technology strategy. You will be earning the trust of other developers in the team and then coaching and influencing them into the right behaviors to build the ultimate analytical data model and pipelines. MORE ABOUT THE TEAM The data engineering team is responsible for running multiple analytical data models and data pipelines all across Atlassian, including finance, growth, product analysis, customer support, sales, marketing, etc - so there are endless opportunities for growth. We maintain Atlassian's data lake and build creative and reliable analytics data model that provides a unified way of analyzing our customers, our products, our operations and the interactions among them. You'll be joining a team that is smart and very direct. We ask hard questions and challenge each other to constantly improve our work. We are self-driven but team oriented. We're all about enabling growth by delivering the right data and insights in the right way to partners across the company. More about our benefits Whether you work in an office or a distributed team, Atlassian is highly collaborative and yes, fun! To support you at work (and play) we offer some fantastic perks: ample time off to relax and recharge, flexible working options, five paid volunteer days a year for your favourite cause, an annual allowance to support your learning & growth, unique ShipIt days, a company paid trip after five years and lots more. More about Atlassian Software is changing the world, and we're at the center of it all. With a customer list that reads like a who's who in tech, and a highly disruptive business model, we're advancing the art of team collaboration with products like Jira Software, Confluence, Bitbucket, and Trello. Driven by honest values, an amazing culture, and consistent revenue growth, we're out to unleash the potential of every team. From Amsterdam and Austin to Sydney and San Francisco, we're looking for people who are powered by passion and eager to do the best work of their lives in a highly autonomous yet collaborative, no B.S. environment. Additional Information We believe that the unique contributions of all Atlassians is the driver of our success. To make sure that our products and culture continue to incorporate everyone's perspectives and experience we never discriminate on the basis of race, religion, national origin, gender identity or expression, sexual orientation, age, or marital, veteran, or disability status. All your information will be kept confidential according to EEO guidelines. Atlassian, Inc., will consider for employment qualified applicants with criminal histories in a manner consistent with the requirements of SFPC Art.49. Atlassian San Francisco CA

Senior Data Engineer