Solution Architect- Bigdata Engineering
Boston , MA 02114
Posted 3 months ago
This Job is not relevant Tell us why
Role: Solution Architect- Bigdata Engineering
Experience Level: 10 to 16 Years
Work location: Boston, MA
Employment Mode: Full Time/ Permanent
- Work with and contribute to the Apache Hadoop and related open source communities
- Design and build a hybrid cloud solution combining on-premise and in-cloud clusters on Google Cloud Platform (GCP)
- Diagnose and troubleshoot complex distributed systems problems and develop solutions with a significant impact at our massive scale.
- Build tools to ingest and process more than 1 trillion data units per day.
- Design and develop next-gen storage and compute platforms used by dozens of engineering teams
- Communicate with a wide set of teams, including Hardware, Network, Linux kernel, JVM, SiteOps teams, and cloud vendors.
- Build advanced tooling for testing, monitoring, administration, and operations of multiple clusters across data centres, on-premise and in-cloud clusters
- Hands-on experience in distributed systems and ingestion/migration.
- Strong software development skills in at least one of: Java, C/C++, or Scala.
- Expertise on DBs, DWH, micro services and platform Engineering technologies
- BS, MS or PhD degree in Computer Science or Engineering, or equivalent experience.
- Experience building and supporting large-scale systems in a production environment.
Good to have:
- Running large infrastructure services on cloud- GCP/ AWS
- Experience with OSS technologies such as HDFS, YARN, Tez, Flume, HBase, Spark, Zookeeper.
- Solid knowledge of networking and Linux systems management.
- Experience with operating system internals, file systems, disk/storage technologies and storage protocols.