Location: Atlanta, GA (3 days onsite, 2 days remote)
The Lead Data Engineer is responsible for managing and organizing enterprise data and overseeing the technical output of a team of data engineers. They will architect and designdata pipelines and dictate how information should be managed, consolidated, and stored for optimal use by the organization. Lead Data Engineers will support other Data Engineers, Data Architects, and Data Analysts on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. They will drive standardization and reduce effort through automation.
Essential Duties
Lead a team of engineers through architecture, design, demand delivery, code reviews, release management, implementation, presentations, and meetings.
Mentor fellow data engineers and contribute to ongoing process improvements for the team.
Evaluate business needsand objectives and align architecture/designs with business requirements
Build the data pipelines required for the optimal extraction, transformation, integration, and loading of raw data from a wide variety of data sources
Assemble large, complex data sets and model our data in a way that meets functional / non-functional business requirements
Create data tools for analytics team members that assist them in generating innovative industry insights that provide our business a competitive advantage
Implement data tagging mechanisms and metadata management so data is accurately classified and visible to the organization
Build processes to help identify and improve data quality, consistency, and effectiveness
Ensure our data is managed in a way that conforms to all information privacy and protection policies
Use agile software development processes to iteratively make improvements to our data management systems
Identify opportunities for automation
Be an advocate for best practices and continued learning
Requirements
Bachelor's/Tech School degree in Computer Science, MIS, or Engineering or relevant technical field and/or commensurate years of real-world experience in software engineering.
7+ years of relevant experience in data management
5+ years in data engineering with detailed knowledge of data warehouse technical architectures, infrastructure components, ETL/ ELT
3+ years of experience with Cloud based DW such as Redshift, Snowflake, etc.
Technical expertise with data models, data mining, and segmentation techniques
Experience in data acquisition, transformation, and storage design using design principles, patterns, and best practices
Experience with performance analysis and optimization
Experience with relational SQL and NoSQL databases, including Postgres and Cassandra
Experience with data pipeline and workflow management tools
Experience with AWS cloud services: EC2, EMR, RDS, Redshift, Modern data platforms, Snowflake, dbt, Fivetran, and Airflow.
Experience with stream-processing systems: Kafka, Storm, Spark-Streaming, etc. Data engineering certification is a plus.
Requirements
Bachelor's/Tech School degree in Computer Science, MIS, or Engineering or relevant technical field and/or commensurate years of real-world experience in software engineering.
7+ years of relevant experience in data management
5+ years in data engineering with detailed knowledge of data warehouse technical architectures, infrastructure components, ETL/ ELT
3+ years of experience with Cloud based DW such as Redshift, Snowflake, etc.
Technical expertise with data models, data mining, and segmentation techniques
Experience in data acquisition, transformation, and storage design using design principles, patterns, and best practices
Experience with performance analysis and optimization
Experience with relational SQL and NoSQL databases, including Postgres and Cassandra
Experience with data pipeline and workflow management tools
Experience with AWS cloud services: EC2, EMR, RDS, Redshift, Modern data platforms, Snowflake, dbt, Fivetran, and Airflow.
Experience with stream-processing systems: Kafka, Storm, Spark-Streaming, etc. Data engineering certification is a plus.
Intelliswift is consumed with the love for the new. Once a leading staffing company, Intelliswift now possesses the expertise to build data-rich modern platforms, and to create sophisticated systems for data management and analytics for thinking and connected enterprises. We are a global leader in delivering Digital Product Engineering, Data Management & Analytics, Cloud, Digital Enterprise and MSP/VMS staffing solutions. Led by a team of highly passionate and techno-centric innovators, we consciously embed the spirit of loving and embracing everything new in what we do. We ardently believe that companies that Love the New are at an advantage of being ahead of the curve in this age of digital.
Intelliswift’s unique digital transformation accelerators empower businesses with modern IT solutions to stay relevant for their end-customers. We constantly evolve our offerings and solutions to create a robust technology ecosystem– one that lays the foundation for a sustainable future.
