Sr. Big Data Engineer (Python)

Tiger Analytics Chicago , IL 60290

Posted 3 weeks ago

Tiger Analytics is a fast-growing advanced analytics consulting firm. Our consultants bring deep expertise in Data Science, Machine Learning and AI. We are the trusted analytics partner for several Fortune 100 companies, enabling them to generate business value from data. Our business value and leadership has been recognized by various market research firms, including Forrester and Gartner. We are looking for top-notch talent as we continue to build the best analytics global consulting team in the world.

You will be delivering data engineering solutions for multiple complex analytics or data science projects. You will create and stitch together structured and unstructured data from multiple sources and create data pipelines.

Job Duties

  • Ensure data quality, create new data pipelines, optimization and management of existing data pipelines,
  • Ingestion and curation of data sources for analytical purposes,
  • Transformations of structured and unstructured data into formats suitable for machine learning and advanced analytics
  • Work on deployment and making sure products are production-ready and function smoothly.

Requirements

  • 5+ years of experience in data engineering and data analytics
  • Strong data pipelining and ETL skills
  • Strong Python engineering skills and SQL skills
  • Ability to Clean, easy to maintain, modify and support code
  • Proficient in Python best practices, logging and strong debugging skills
  • Understanding of Python environments, package management
  • Strong basic SQL knowledge, able to read and understand SQL handed off
  • Able to do standard SQL joins, filtering, etc
  • Experience in distributed computing frameworks such as Spark
  • Able to work independently and ask for guidance when needed
  • Strong communication skills and delivery focused
  • Familiar with git source control
  • Understanding the pre-processing analytical methods is a plus
  • Bachelors degree in Computer Science or closely related field

Benefits

Significant career development opportunities exist as the company grows. The position offers a unique opportunity to be part of a small, challenging, and entrepreneurial environment, with a high degree of individual responsibility.

icon no score

See how you match
to the job

Find your dream job anywhere
with the LiveCareer app.
Mobile App Icon
Download the
LiveCareer app and find
your dream job anywhere
App Store Icon Google Play Icon
lc_ad

Boost your job search productivity with our
free Chrome Extension!

lc_apply_tool GET EXTENSION

Similar Jobs

Want to see jobs matched to your resume? Upload One Now! Remove
Senior Big Data Engineer

Integral

Posted 1 week ago

VIEW JOBS 2/26/2021 12:00:00 AM 2021-05-27T00:00 Integral Ad Science (IAS) is a global technology and data company that builds verification, optimization, and analytics solutions for the advertising industry and we're looking for a Senior Big Data Engineer to join our team. If you are excited by technology that has the power to handle hundreds of thousands of transactions per second; collect tens of billions of events each day; and evaluate thousands of data-points in real-time all while responding in just a few milliseconds, then IAS is the place for you! As a Senior Big Data Engineer you will design, implement, and maintain big data pipelines responsible for aggregating tens of billions of daily transactions. You will be part of a growing team to develop and mature IAS' capability in video ad verification, analytics and anti ad fraud software products. The ideal candidate is naturally curious, dedicated, detailed-oriented with a strong desire to work with awesome people in a highly collaborative environment. You should be able to not take yourself too seriously as well. What you'll get to do: * Architect, design, code and maintain components for aggregating tens of billions of daily transactions * Work on Big Data technologies such as Hadoop, MapReduce, Kafka, and/or Spark in columnar databases on AWS * Contribute to the entire software lifecycle including hands-on development, code reviews, testing, deployment, and documentation for streaming & batch ETL and RESTful APIs * Provide leadership, work collaboratively, and be a mentor in an awesome team You should apply if you have most of this: * Bachelors or Masters in Computer Engineering, Computer Science, Electrical Engineering or related field * 5+ years of recent hands-on in object oriented language (Java, Scala, Python) * 5+ years of experience designing and building data pipelines and data-intensive applications * Experience using Big Data frameworks (e.g., Hadoop, Spark) , databases (e.g., RedShift, Snowflake) for complex data assembly and transformation * Strong knowledge of collections, multi-threading, JVM memory model, etc. * In-depth understanding of algorithms, performance, scalability, and reliability in a Big Data setting * Solid understanding of OLTP and OLAP systems, database fundamentals * Solid knowledge of SQL * Experience in full software development, Agile, and CI/CD * Experience building production level systems in a cloud environment (AWS, Azure or GCP) What puts you over the top: * Orchestrating data pipelines using tools such as Airflow * Familiarity with messaging frameworks like Kafka or RabbitMQ * Experience with Spark streaming or Flink About Integral Ad Science Integral Ad Science (IAS) is the global market leader in digital ad verification, offering technologies that drive high-quality advertising media. IAS equips advertisers and publishers with both the insight and technology to protect their advertising investments from fraud and unsafe environments as well as to capture consumer attention, and drive business outcomes. Founded in 2009, IAS is headquartered in New York with global operations in 18 offices across 13 countries. IAS is part of the Vista Equity Partners portfolio of software companies. For more on how IAS is powering great impressions for top publishers and advertisers around the world, visit integralads.com. Equal Opportunity Employer: IAS is an equal opportunity employer, committed to our diversity and inclusiveness. We will consider all qualified applicants without regard to race, color, nationality, gender, gender identity or expression, sexual orientation, religion, disability or age. We strongly encourage women, people of color, members of the LGBTQIA community, people with disabilities and veterans to apply. California Applicant Pre-Collection Notice: We collect personal information (PI) from you in connection with your application for employment or engagement with IAS, including the following categories of PI: identifiers, personal records, commercial information, professional or employment or engagement information, non-public education records, and inferences drawn from your PI. We collect your PI for our purposes, including performing services and operations related to your potential employment or engagement. For additional details or if you have questions, contact us at compliance@integralads.com. To learn more about us, please visit http://integralads.com/ and https://muse.cm/2t8eGlN Attention agency/3rd party recruiters: IAS does not accept any unsolicited resumes or candidate profiles. If you are interested in becoming an IAS recruiting partner, please send an email introducing your company to recruitingagencies@integralads.com. We will get back to you if there's interest in a partnership. Integral Chicago IL

Sr. Big Data Engineer (Python)

Tiger Analytics