Software Development Engineer I - Data Platform

Twitch San Francisco , CA 94118

Posted 3 weeks ago

About Us

Launched in 2011, Twitch is a global community that comes together each day to create multiplayer entertainment: unique, live, unpredictable experiences created by the interactions of millions. We bring the joy of co-op to everything, from casual gaming to world-class esports to anime marathons, music, and art streams. Twitch also hosts TwitchCon, where we bring everyone together to celebrate, learn, and grow their personal interests and passions. We're always live at Twitch. Stay up to date on all things Twitch on LinkedIn, Twitter and on our Blog.

About the Role

Twitch is building the future of interactive entertainment. The Data Infrastructure team develops and operates the data platform that powers data systems. Our data pipe receives over 20 billion events a day, and we forward live data to tens of systems and powering decisions at all levels of Twitch. We provide tools to store, transform, move, and understand data.

As a software engineer on Data Infrastructure, you will provide the next generation of Twitch data technology. Using Go and Python, you will build tools which are reliable and scale. Reporting to our Data Engineering Manager, you'll review and shape the work of your peers, and participate in our team.

You Will:

  • Develop new capabilities in our data warehouses and pipelines.

  • Improve the reliability and scalability of our existing tools.

  • Collaborate on our vision as we scale to our next petabyte.

You Have:

  • 2+ years of software development experience.
  • 2+ years experience in Go or Python.
  • 1+ years working with distributed, highly-available systems.

Bonus Points

  • You've made petabytes of data usable.

  • You've worked with Amazon Web Services.

  • You contribute to open source.


  • Medical, Dental, Vision & Disability Insurance, 401(k)

  • Maternity & Parental Leave

  • Flexible PTO, Commuter Benefits

  • Amazon Employee Discount, Monthly Contribution & Discounts for Wellness Related Activities & Programs (e.g., gym memberships, off-site massages)

  • Breakfast, Lunch & Dinner Served Daily, Free Snacks & Beverages

We are an equal opportunity employer and value diversity at Twitch. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

icon no score

See how you match
to the job

Find your dream job anywhere
with the LiveCareer app.
Mobile App Icon
Download the
LiveCareer app and find
your dream job anywhere
App Store Icon Google Play Icon

Boost your job search productivity with our
free Chrome Extension!

lc_apply_tool GET EXTENSION

Similar Jobs

Want to see jobs matched to your resume? Upload One Now! Remove
Software Engineer Data Platform


Posted 2 days ago

VIEW JOBS 3/5/2021 12:00:00 AM 2021-06-03T00:00 <p>The data platform software engineer in SmartNews plays a key role in accelerating the products/business developments. Great efforts are paid on building a highly efficient and flexible data service for analytical and operational purposes.</p><p>To serve the internal users from analytics and product-dev teams, the goal and mission of data engineers is to create high-level, easy-to-use data services for simplifying the accessing, integration and consolidation of various data sets, and also building the platforms for executing tasks processing massive data in terms of TB per day.</p><p>Technology drives the growth of SmartNews, and thus we eagerly adopting cutting-edge technologies from the industry and academia especially, the open-source community.</p><p><br></p><h3>Responsibilities<br> </h3><p>This is a hybrid of system engineering and machine learning role:</p><ul> <li>Design, develop, setup and maintain new services, libraries, tools, frameworks for data processing or management, and investigate new algorithms to increase efficiency for Data Processing, such as ETL, Data Pipelines, OLAP DBMS, real-time messages and streams processing, data-sync between systems, etc.</li> <li>Develop in-house</li> <li>Develop toolings for system performance evaluation, monitoring and tuning of the data processing procedures or platforms, get insights of efficiency and stabilizability and make continuous improvement, such as optimizing distributed query engines, computing resource management and isolation, multi-tier storage systems, etc.</li> <li>Own and maintain the key data processing portfolios such as building and taking care of the environment, trouble-shooting and being responsible to the on-call of incidents. Work closely with data architecting/modeling roles to understand ways to implement the data service, and interact with Site Reliability Engineering (SRE) team to deploy the environments and drive production excellence.</li> <li>Device system, tooling and approaches for data privacy and security. Establish access control, create processes to handle sensitive data.</li> <li>Diagnose and resolve complex technical challenges for data accessing or processing. Using elegant and systematic rather than ad-hoc methods to help other teams tuning the performance and improving stability.</li> </ul><p><strong>Requirements</strong></p><p>Minimum Requirement</p><ul> <li>BS/MS degree in computer science or equivalent science/engineering degree with 5 or more years of experience</li> <li>Strong Programming skills and experiences with deep understanding of data structures and algorithms are required for building efficient and stable solutions</li> <li>Rich experiences with one or more programming languages such as Java, Scala, C++ or Python; familiar with agile development and manage testing skills</li> <li>Need certain knowledge on shell scripts and operating systems, especially on Linux</li> <li>Good understanding of modern bigdata technologies and ecosystems</li> <li>Familiar with Hadoop, Spark, Hive, Presto, Redis and Flink, be able to develop data processing programs with them in batch or streaming manner</li> <li>Familiar with modern data stores either RDBMS or NoSQL stores (such as MySQL, Cassandra or Druid, etc); have experiences on developing application or function-extensions on such data stores</li> <li>Be able to implement and tune complicated heavy-lifting data flows (ETLs or pipelines), familiar with certain toolings</li> <li>Capability of system design with good modularity and extensibility</li> <li>Familiar with system/module design methods and toolings such as UML</li> <li>Be able to draft the user-understandable blueprint and precise, detailed designs</li> <li>Experience of building highly scalable distributed systems</li> <li>Able to design and implement distributed services with scalability and performance in mind</li> <li>Able to debug and troubleshooting performance and reliability problems</li> </ul><p>Preferred Requirement</p><ul> <li>Experience with cloud based architecture (e.g. Amazon Web Services)</li> <li>Strong interest in high performance, high availability, hive volume data processing</li> <li>Good experience in data management, integration, security, and auditing</li> </ul><p><strong>Benefits</strong></p><ul> <li>Equity included</li> <li>100% medical, dental and vision insurance coverage (60% coverage for dependents)</li> <li>Monthly housing, commute, mobile phone and gym allowances</li> <li>401k matching program</li> <li>Free lunch, snacks, drinks, etc.</li> <li>Pet friendly office</li> </ul> Smartnews San Francisco CA

Software Development Engineer I - Data Platform