Job title: DataOps Engineer
Company: The Skills Mine
Job description: Job SummaryRequirements
\
- An MS or BS degree or the equivalent practical experience
- 3+ years’ experience working as a DataOps Engineer or in a similar role, with a strong focus on data engineering and automation
- 3+ years of experience using Python to develop enterprise-grade software using appropriate design patterns and principles
- Strong experience with cloud platforms such as AWS, Azure, or GCP, and expertise in utilizing cloud-based data services (e.g., AWS Glue, Azure Data Factory, Google Cloud Dataflow)
- Solid understanding of modern data architecture, including data modelling, ETL/ELT processes, and data warehousing concepts
- Experience with version control systems (e.g., Git), containerization technologies (e.g., Docker/Kubernetes), infrastructure as code (e.g., terraform), and CI/CD pipelines
- Familiarity with data governance, data security, and data privacy practices
Responsibilities
- Designed, developed, and maintained scalable data pipelines and workflows to support data acquisition, processing, and transformation activities in an agile and iterative manner
- Collaborated with data scientists, software engineers, and other stakeholders to understand data requirements and implemented efficient data solutions to foster team self-service and improve collaboration and communication between departments
- Implemented and optimized data integration processes, ensuring data consistency, accuracy, and reliability across different data sources and systems
- Automated data operations tasks, including data ingestion, data validation, and data quality monitoring, using appropriate tools and technologies
- Identified and resolved data-related issues, troubleshoot performance bottlenecks, and ensured optimal data processing and storage
The Skills MineCompany
Expected salary: R40000 per month
Location: Johannesburg, Gauteng
Job date: Sat, 10 Feb 2024 00:34:58 GMT
Apply for the job now!