Big Data Developer

Infrastaff San Francisco , CA 94102

Posted 2 months ago

Apply

This Job is not relevant Tell us why

As developer within the Big Data team you will contribute to high quality technology solutions that address business needs by developing data applications for the customer business lines. You will contribute to the development and ongoing maintenance of a number of strategic data initiatives and data and analytic applications. The ability to communicate effectively is required as you will work closely with other groups, including development and testing efforts of your assigned application components to ensure the successful delivery of the project:

Experience with middle tier/backend systems development in Java/Linux
Minimum 3_5 years working experience with Hadoop in an enterprise setting
Experience with Java enterprise development, Python and Scalar
Hands on expertise with Sql & NoSql data platforms
Hands on expertise with Big data technologies (HBase, Hive, Sqoop)
Experience with Pub/Sub messaging (JMS, Kafka, etc.), stream processing (Storm, Spark Streaming, etc.)
Understanding and application of security best practices as they relate to big data technologies
Experience with horizontally scalable and highly available system design and implementation, with focus on performance and resiliency
Experience profiling, debugging, and performance tuning complex distributed systems
Experience with UNIX shell scripts and commands
Experience with data modeling
Ability to clearly document solution designs
Agile/Scrum methodology experience
Experience with ETL/ELT tools
Experience with BI solutions (Tableau, Microstrategy, D3 etc) Complexity Works on complex issues where analysis of situations and data requires an indepth evaluation of variable factors. Exercises judgment in selecting methods, techniques and evaluation criteria for obtaining results. Supervision Acts independently to determine methods and procedures on new assignments, and may provide work direction to others. Works under minimal supervision.
*

Minimum Education/Training/Certification :Bachelor's degree in an information technology area of study. Master's degree preferred specializing in Computer Science, Information Management, Data Science or equivalent combination of education and experience.

Responsibilities:

Hands on development role focused on creating big data and analytics solutions
Coding of mission critical components
Analyze business and functional requirements and contribute to overall solution
Participate in design reviews, provide input to the design recommendations
Participate in project planning sessions with project managers, business analysts and team members
Hands on expertise with Graph databases

Show Full Description

See how you match
to the job

Upload my resume

Download the
LiveCareer app and find
your dream job anywhere

Similar Jobs

View All

Want to see jobs matched to your resume?
Upload One Now!

Senior Data And Analytics Big Data Banking

Ernst & Young LLP

Posted 2 days ago

VIEW JOBS Senior - Financial Services - Data and Analytics - Data Architecture and Integration EY is the only professional services organization with a separate business unit (FSO) that is dedicated to the financial services marketplace. Our FSO teams have been at the forefront of every event that has reshaped and redefined the financial services industry. If you have a passion for rallying together to address the most complex challenges in the financial services industry, come join our dynamic FSO team! Data has yet to be utilized to its fullest potential. Financial institutions are looking to build smarter and more efficient ways to operate their business, through new opportunities uncovered by their data. Our clients look to us as trusted advisors to provide insights on how to ingest, interpret and manage their data. We focus on business intelligence, advanced analytics, and information management. The opportunity We provide constant room for growth through client engagements and personal development. Our practitioners develop and lead innovative data techniques and methods, supporting both business and technology leaders. This role provides the unique opportunity to build a professional network, tackle complex data issues, and drive cutting-edge approaches across industry-leading clients and domains. The experiences you gain here will be more valuable than anywhere else. Your key responsibilities You will be part of diverse teams of professionals across different geographies to deliver a wide range of data and analytics services. You will be addressing complex issues and driving growth across financial services. Skills and attributes for success * Fostering an innovative and inclusive team-oriented work environment * Leading and coaching diverse teams of professionals with different backgrounds * Demonstrating in-depth technical capabilities and professional knowledge * Establishing strong relationships with the clients * Working in an entrepreneurial environment to pave your own career path To qualify for the role you must have * A bachelor's or master's degree * Around two or more years of related work experience in banking, capital markets, insurance or asset management * Experience assisting clients with strategic Data Architecture/Integration or Big Data initiatives, preferably focused in financial services * Strong experience in data architecture and integration technologies (ETL, data ingestion and distribution patterns, SQL, large data processing on distributed platforms, data warehousing) * Hands-on experience with at least one major Hadoop platform (Cloudera, Hortonworks, MapR) * Hands-on experience in developing or architecting Big Data applications using Hadoop and related technologies such as Spark, MapReduce, YARN, HDFS, Hive, Impala, Oozie, NoSQL, HBase, Elasticsearch, Cassandra, Flink, Flume, Splunk, etc. * Hands-on experience working with at least one programming language and related technologies such as Java, Scala, Python, .NET and APIs * Hands-on experience with streaming platforms and applications such as Spark Streaming, Kafka, Storm, etc. * Hands-on experience with cloud-related technologies (such as AWS, Azure, Redshift, EMR, etc.) * Strong written and verbal communication skills * Strong analytical and problem-solving skills * Ability to assimilate new knowledge * Ability to understand business challenges and translate them into value add analytics solutions * Excellent leadership and teaming skills * A willingness and ability to travel to meet client needs; travel is estimated at 80%-90% * Valid driver's license in the US * Valid passport Ideally, you'll also have * A degree in computer science, engineering, accounting, finance or a related field; MBA or MS degree preferred * Consulting experience What we look for We're interested in self-motivated and passionate leaders with a strong vision and desire to stay on the cutting edge of data technology and science. We are looking for individuals who aspire to build collaborative team environments and possess effective management skills while demonstrating in-depth technical capabilities and industry knowledge. If you have a genuine drive for helping consumers achieve the full potential of their data while working toward your own development, this role is for you! What working at EY offers We offer a competitive compensation package where you'll be rewarded based on your performance and recognized for the value you bring to our business. In addition, our Total Rewards package includes medical and dental coverage, pension and 401(k) plans, a minimum of three weeks of vacation plus ten observed holidays and three paid personal days, and a range of programs and benefits designed to support your physical, financial and social well-being. Plus, we offer: * Support and coaching from some of the most engaging colleagues in the industry * Opportunities to develop new skills and progress your career * The freedom and flexibility to handle your role in a way that's right for you About EY As a global leader in assurance, tax, transaction and advisory services, we hire and develop the most passionate people in their field to help build a better working world. This starts with a culture that believes in giving you the training, opportunities and creative freedom to make things better. So that whenever you join, however long you stay, the exceptional EY experience lasts a lifetime. If you can confidently demonstrate that you meet the criteria above, please contact us as soon as possible. Make your mark. Apply today. EY provides equal employment opportunities to applicants and employees without regard to race, color, religion, age, sex, sexual orientation, gender identity/expression, national origin, protected veteran status, disability status, or any other legally protected basis, in accordance with applicable law.

Big Data Engineer

Afterpay Touch

Posted 2 days ago

VIEW JOBS Our Company Afterpay is transforming the way we pay by allowing customers to receive products immediately and pay for their purchases over four installments, always interest-free. The service is completely free for customers who pay on time - helping people spend responsibly without incurring interest, fees or extended debt! Afterpay is offered by more than 55,000 of the world's best retailers and has more than 10 million active customers globally. In the U.S. more than 5 million customers use Afterpay and is offered by more than 15,000 of the top fashion and beauty retailers across the country - including Anthropologie, Forever 21, DSW, GOAT, Finish Line, Levi's, Mac Cosmetics, Ray-Ban and many others. We are currently available in Australia, New Zealand, the United States and the United Kingdom where the brand is called Clearpay. Our U.S. headquarters are located at the heart of the retail and business district in San Francisco, California. Afterpay is on a mission to power an economy in which everyone wins. The Role Afterpay has unique opportunities for Big Data Engineers to join our Data Science Engineering Team and make an immediate impact on our product by getting involved early on at our Shanghai Tech Center. The engineers will influence many aspects of our business through architecting our data processing platform. This is an exciting opportunity to make a direct, tangible impact on our product and work on projects that are pivotal to our team's success and our company's growth. Responsibilities * Design and build self-serving data platform to handle new products and business requirements that securely scale over millions of users and their transactions * Develop, integrate and optimize end to end data pipeline * Partner with data scientists, data analysts and domain engineering teams to identify and execute on new opportunities. Qualifications * BA/MS/PhD in computer science or a related field * Experienced in building and maintaining applications that utilize a modern programming language, mainly Java and Python * Experience with Flink, Kafka, Spark, Cassandra and so on * Experience with big data cross platform compatible file formats like Apache Avro & Apache Parquet * Experience with AWS like S3, EMR, Redshift and so on * Experience on architecturing large scale distributed system * Enthusiastic about solving business problems with technology and can take ownership of an end-to-end solution * Passionate about continuously learning new technologies, frameworks, and services * Possess an execution mindset and the ability to deliver with cross-functional teams that are globally distributed * Serious about testing and have experience with automated testing frameworks * Exceptional written and verbal technical communication skills What We Offer * Competitive compensation packages * Medical, Dental, Vision insurance (100% covered!) * Fun virtual company events & collaborative team * Supportive, laid-back environment * Award winning culture How to Apply If you are brave, if you are committed to doing the right thing, if you always keep it real, and your background matches the description above then please apply today! Afterpay is continuing to hire for all open roles with all interviewing and on-boarding done virtually due to COVID-19. All new team members, in addition to current staff, will temporarily work from home until it is safe to return to our offices.

Big Data Production Support Team Member

Wells Fargo

Posted 6 days ago

VIEW JOBS Job Description Important Note: During the application process, ensure your contact information (email and phone number) is up to date and upload your current resume when submitting your application for consideration. To participate in some selection activities you will need to respond to an invitation. The invitation can be sent by both email and text message. In order to receive text message invitations, your profile must include a mobile phone number designated as "Personal Cell" or "Cellular" in the contact information of your application. At Wells Fargo, we want to satisfy our customers' financial needs and help them succeed financially. We're looking for talented people who will put our customers at the center of everything we do. Join our diverse and inclusive team where you'll feel valued and inspired to contribute your unique skills and experience. Help us build a better Wells Fargo. It all begins with outstanding talent. It all begins with you. Wells Fargo Technology is a team of more than 40,000 information technology and security professionals who help keep Wells Fargo at the forefront of America's diversified financial services companies. Employees execute an engineering-led IT strategy to deliver stable, secure, scalable and innovative services that provide Wells Fargo global customers 'round-the-clock' banking access through in-store, online, ATM, and other channels. Wells Fargo Technology plays a critical role in the company's customer and employee experience, business and risk management transformation, and growth agenda. Wells Fargo Technology's Data Technology Platform Services Team is looking for a Systems Operations Engineer to support the operations of multiple Enterprise scale Data Warehouse applications within the Production Support Shared Service organization. This position has the primary responsibility of providing production support, analyzing, troubleshooting, and resolving production job failures, data research, recommending and implementing code fixes as well as coordinating with the ETL Development, Database, and Business Analyst teams within each application team. * Primary production support for analytical data ETL jobs and Big Data Batch and Near Real-time processing jobs within the Enterprise Data Lake environment * Troubleshoot and analyze complex production problems related to data, network file delivery, and server and application issues independently and provide solutions to recovery * Drive and participate in postmortems to avoiding repeated incidents * Solve problems related to the Core services systems and the buildout of automation to prevent problem recurrence; with the goal of automating response to all non-exceptional service conditions * Collaborate with the business analysts and ETL development team members to resolve complex data design issues/provide optimal solutions that meet business requirements and benefits system performance * Collaborate with channel teams which includes offshore team members, work flexible hours as required to achieve a global follow-the-sun support model * Identify and implement operational best practices and process improvements within the following functional areas: Audit, Risk, Change, Event detection, Incident Management, Problem management, Vulnerability Management, Technical Refresh, Tools and Application Support * Encourage and participate in knowledge sharing as necessary * Develop and foster a positive relationship with team members, team leads and business partners * Develop and updates documentation, departmental technical procedures and user guides * Be willing to work non-standard business hours on an on-call basis in a 24x7x365 environment * Implement best practices for production environments * Encourage and participate in knowledge sharing as necessary Required Qualifications * 5+ years of systems support analysis experience demonstrated through work or military experience * 3+ years of experience working in managed services as demonstrated through work, military, or education * 3+ years of application production support experience * 3+ years of UNIX experience * 3+ years of job scheduling experience * 2+ years of Big Data experience Desired Qualifications * Experience supporting an enterprise-level environment * Ability to interact with all levels of an organization; including management * Strong team or technical leadership experience * Strong verbal, written, and interpersonal communication skills * Good verbal, written, and interpersonal communication skills * A BS/BA degree or higher in science or technology * Knowledge and understanding of incident management: gathering impacts and analyzing data * Strong organizational, multi-tasking, and prioritizing skills * Advanced problem solving and technical troubleshooting capabilities * Knowledge and understanding of Autosys * Knowledge and understanding of relational databases * 1+ year of SQL experience Other Desired Qualifications * 2+ years of production support or administrative experience with Hadoop tools such as MapR, Hortonworks, Yarn, Spark, and Hive * Experience with unstructured datasets, such as log files, email, and text Job Expectations * Flexibility to address incidents as needed 24 hours a day * Flexibility to frequently be on call beyond normal working hours * Ability to work on call as assigned Street Address NC-Charlotte: 301 S College St - Charlotte, NC AZ-Chandler: 2700 S Price Rd - Chandler, AZ AZ-PHX-Northwest Phoenix: 2222 W Rose Garden Ln - Phoenix, AZ NC-Charlotte: 401 S Tryon St - Charlotte, NC NC-Charlotte: 1525 W Wt Harris Blvd - Charlotte, NC MN-Minneapolis: 550 South 4th St - Minneapolis, MN CA-SF-South Of Market Area: 45 Fremont Street - San Francisco, CA MO-Saint Louis: 1 N Jefferson Ave - Saint Louis, MO TX-Irving: 2975 Regent Blvd - Irving, TX IA-West Des Moines: 7001 Westown Pkwy - West Des Moines, IA Disclaimer All offers for employment with Wells Fargo are contingent upon the candidate having successfully completed a criminal background check. Wells Fargo will consider qualified candidates with criminal histories in a manner consistent with the requirements of applicable local, state and Federal law, including Section 19 of the Federal Deposit Insurance Act. Relevant military experience is considered for veterans and transitioning service men and women. Wells Fargo is an Affirmative Action and Equal Opportunity Employer, Minority/Female/Disabled/Veteran/Gender Identity/Sexual Orientation.

Apply