- Location: Remote / Detroit
- Visa Sponsorship: OK
- Compensation: Competitive Base Salary, Bonus, Equity
About the Client:
Exciting AI-based Enterprise Analytics platform with fresh capital infusion to support major growth.
About the Role:
Seeking experienced Big Date Engineer to oversee a wide variety of projects from planning through implementation. As part of the core development team, you will work closely with clients, delivery, data science, development operations engineers, and software developers on a cutting-edge technology platform and bring your expertise to continuously evolve our framework as we grow.
Requirements / Responsibilities:
- The role will be involved in designing, building, and deploying massively scalable, automated data ingestion and processing pipelines to store, analyze and derive insight from structured/unstructured data arriving from our clients and internal systems in multiple cloud environments.
- The role is expected to drive technology discussions and analyze the current landscape for gaps in addressing data, platform, and business need
- Review existing architecture and capabilities from a technical perspective and challenge the status quo to provide thought leadership/best practices to enhance our instance strategy, technical governance, core data, integrations and the overall technical health of our data and machine learning pipelines frameworks
- Interface with the customers and across the org in structuring and implementing solutions while demonstrating the ability to become a trusted advisor
- Develop CI/CD processes for the Engineering and Data Science groups
- Develop QA/QC suites to process, cleanse, and verify the integrity of incoming data, including data anomaly detection
- Undertake project specific proof of concepts activities to validate technical feasibility
- Discuss with onshore/offshore teams on tasks to be done, and mentor junior members of the team
- Perform design & code reviews for team members and develop test plans
- Assist DevOps in configuring, operating, and maintaining our infrastructure environment
Skills and Qualifications:
- Experience and desire to work in a fast-paced, collaborative, global delivery environment.
- 3+ years experience as a big data engineer at a data-driven, enterprise-grade technology working with large amounts of data in a continuously scaling environment
- Expertise in cloud-based platforms such as Amazon Web Services (AWS), Microsoft Azure, or Google Cloud (GC)
- Deep understanding of big data technologies and big data security models
- Prior software development experience with data-driven product companies using Spark, SQL, Python, Scala or advanced analytical tools is a plus
- Experience with the data collection process and evaluation methods for Machine Learning
- Proficient in designing efficient and robust ETL/ELT workflows, schedulers, and event-based triggers
- Database experience, including knowledge of storage internals of SQL and NoSQL services
- Experience with development methodologies like CI/CD, Agile, Git, Jenkins etc
- Performance Tuning & experience with building scalable Data platforms
- Education Qualification: Master’s Degree Computer Science, Engineering, Information Technology (IT) or related field
To Apply: Contact [email protected]