Big Data Engineer

Microagility Inc Ontario , CA 91761

Posted 2 months ago

Apply

This Job is not relevant Tell us why

Responsibilities

Create and maintain optimal data pipeline architecture.
Assemble large, complex data sets that meet functional / non-functional business requirements.
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
Keep our data separated and secure across national boundaries through multiple data centers and AWS regions.
Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
Work with data and analytics experts to strive for greater functionality in our data systems.

Qualifications

Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
Strong Java/Scala with spark developer experience in developing batch and real-time streaming applications.
We are looking for a candidate with 5+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:*
- Experience with big data tools: Hive, Spark, Kafka, Jenkins, Java 8, Scala
- Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
- Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
- Experience with AWS cloud services: EC2, EMR, RDS, Redshift
- Experience with stream-processing systems: Storm, Spark-Streaming, etc.
- Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
- Experience with Jenkins and ansible, understand CI and CD principles.
*

Show Full Description

See how you match
to the job

Upload my resume

Download the
LiveCareer app and find
your dream job anywhere

Similar Jobs

View All

Want to see jobs matched to your resume?
Upload One Now!

Systems Administrator Phmi Data Processing

Mission Regional Medical Center

Posted 3 days ago

VIEW JOBS

Principal Software Engineer Job

Acuity Lighting Group Inc

Posted 3 days ago

VIEW JOBS We Light the Way! Acuity Brands (NYSE: AYI) is transforming the lighting industry. We take pride in our 12,000 employees, and they take pride in working with the best in the industry. Join a team committed to lighting the way to a more brilliant, connected, and productive world. Individually we are talented, but together we are unstoppable. The Role As a Principal Software Engineer, you'll work on soon to be open-source (https://github.com/gneisstech/bedrock) internal platform powering Applications. Our ideal engineer is able to think holistically about applications, backend and infrastructure architecture, scalability, monitoring, and reliability. Your responsibilities * Design, build and ship core features to the platform * Contributing member of a high performing, agile team focused on next generation data & analytic technologies. * Create and enhance analytic platforms & tools that enable state of the art, next generation Big Data capabilities to analytic users and applications. * Understand business and engineering requirements and influence the Platform Engineering roadmap * Lead the design and definition of a Platform technical architecture and plan to achieve it * Architect key systems to handle the scale of our increasing customer load * Mentor and grow the most senior engineers within our teams We're looking for people with depth of knowledge in the following areas * 7+ years of professional software development experience * Security best practices - Proper storing of secrets and access controls * Queueing and big data systems - Kafka, Spark * Storage systems - Relational, Key-Value and OLAP * Infrastructure as code - Go, Shell, Linux, Docker, Kubernetes * Experience with large fleets of IOT controllers * Frontend - General awareness We invite you to apply today to join us as We Light the Way to a Brilliant, Productive, and Connected World! We value diversity and are an equal opportunity employer. All qualified applicants will be considered for employment without regards to race, color, age, gender, sexual orientation, gender identity and expression, ethnicity or national origin, disability, pregnancy, religion, covered veteran status, protected genetic information, or any other characteristic protected by law. Please click here and here for more information. Accommodation for Applicants with Disabilities: As an equal opportunity employer, Acuity Brands is committed to providing reasonable accommodations in its application process for qualified individuals with disabilities and disabled veterans. If you have difficulty using our online system due to a disability and need an accommodation, you may contact us at (770) 922-9000. Please clearly indicate what type of accommodation you are requesting and for what requisition. Any unsolicited resumes sent to Acuity Brands from a third party, such as an Agency recruiter, including unsolicited resumes sent to an Acuity Brands mailing address, fax machine or email address, directly to Acuity Brands employees, or to Acuity Brands resume database will be considered Acuity Brands property. Acuity Brands will NOT pay a fee for any placement resulting from the receipt of an unsolicited resume. Acuity Brands will consider any candidate for whom an Agency has submitted an unsolicited resume to have been referred by the Agency free of any charges or fees. This includes any Agency that is an approved/engaged vendor, but does not have the appropriate approvals to be engaged on a search. Req ID: 2082

Umapped Front End Engineer Toronto

Flight Centre Travel Group Limited

Posted 5 days ago

VIEW JOBS Front End Engineer - Toronto Are you looking for a challenge to be part of scaling a growing global technology platform and business? Umapped, a startup within Flight Centre Travel Group, is seeking a skilled and motivated Front End Engineer to join our Product and Engineering team, to help us build highly interactive interfaces and innovative new functionality for the Umapped itinerary management platform. The successful candidate is passionate about building user-friendly, engaging customer interfaces and highly scalable distributed systems for our customers. The Front End Engineer will work on cutting edge technology such as React, NodeJS, AWS, have strong technical skills, excellent organizational skills, great communication skills, and the motivation to achieve results in a fast-paced Agile environment. Umapped is the leading collaborative B2B2C trip management technology platform in the global travel industry. Our products help travel companies and professionals engage with their customers throughout the entire travel lifecycle. The company's core products include: * Consumer-facing web and mobile applications * Trip Publisher, an online itinerary and proposal builder for travel professionals * TripLinQ, a content and itinerary data interface accessed via API service Key Responsibilities: * Responsible for the design, development and testing of new features of the Umapped Technology platform. * Contribute to the ongoing support and monitoring of the technology infrastructure. * Ability to navigate through a large complex codebase. * Build an intuitive experience and feel for our mobile and web applications. * Converting UI/UX design into live code * Ensure that appropriate software engineering practices are employed to deliver high-quality results. * Writing unit/e2e tests to ensure code quality and increase test coverage * Work closely with the team to define the business requirements for new projects. * Collaborate across team boundaries with other teams * Communicate issues and technical roadblocks effectively Our Core Technology: * Languages: Java 8 and higher, Kotlin, Golang * Scripting Languages: JavaScript, Groovy, Python * JS-based web frameworks - Material-UI w/ React JS * SQL, NoSQL and graph databases and caching (PostgreSQL, DynamoDb, ElasticSearch, Redis, Neo4J, etc) * Linux * Git Core Expertise: * Great communication skills, both verbal and written * Ability to rapidly prototype and adjust in response to customer feedback * Cross-browser/Cross-Device web development * Exposure to Front-end libraries/framework - React, Angular, Vue * Ability to articulate the benefits and challenges of depending on front-end libraries, frameworks, and tool-sets. * Testing Exposure (functional, automated, load, platform reliability) * Drive and passion towards building great and sustainable code * University/College Degree in Computer Science/Computer Engineering or equivalent Nice to Have: * Experience in e-commerce platforms * Experience in the travel industry * Exposure to native mobile development (IOS and Android) * Exposure to React Native framework * Experience with AWS technologies is preferred (Cloudfront, RDS, infrastructure) If this sounds like the opportunity you have been waiting for, then APPLY NOW. Our promise is to provide an environment with equality of respect, dignity and opportunity for all our employees. We value an inclusive and supportive workplace which truly reflects the diversification of our society. Flight Centre Travel Group is committed to providing a barrier-free pathway throughout our recruitment selection process. If you require accommodation because of a disability at any stage of our hiring process, please let us know in advance by contacting our Recruitment Team at fccareers@flightcentre.ca

Apply