azaman

Highly experienced, multi-skilled programmer with a progressive career covering multiple generations of technology, culminating in good knowledge of Python, Snowflake and AWS. Meticulous in analysis, design and development of solutions and applications in an agile environment, with experience across the full software development life cycle (SDLC). Confident in balancing the requirements of stakeholders and reporting on build progress and defect remediation status. Task-driven and capable of self-governance in project management and equally strong operating as part of a technical team. I have been working in the area of data integration/data warehousing since 2000 , with particular focus on analysis, design, development and support of DW/ETL applications and completed over sixteen data integration projects in UK and across Europe, with sizes in terabytes, for clients such as London Transport, Vodafone UK, BT, Sky, Vodafone Netherlands, O2 , Sainsbury, Deutsche Bank, Barclaycard, e. on Energy Trading and orange, T-Mobile , ECHA ,Standard Life, Ingenico e-payments , Lidl and UBS.I have also been involved in a number of high profile data migration projects for clients such as Polish Ministry of Finance in Poland, Sky in Scotland and Vodafone in Netherlands.

Identifying requirements for data pipelines
Investigating and identifying available data points for each pipeline dataset
Agreeing record format (csv or json), data delivery time and data landing location (AWS S3) with source data provider
Performing Data Vault Modelling (conceptual and Physical)
Producing ETL Specification (source to target mapping) with data validation rules
Producing Outline design specification (including animated data pipeline data flow) for each dataset
Helping Product Owner with work breakdowns
Building, Testing and Deploying data pipeline using python, snowflake tasks, stored procedures, streams, AWS s3 and DBT
Designing automated data integration report for target tables in target database
Version controlling all artefacts in Github repository
Overseeing development (review Git-Hub PR, discuss technical option)
Dealing with any blocking issue reported during stand-up
Helping fellow engineers overcoming any technical challenges
Automating all backend tasks with shell scripts

Studied various aws services (EC2, S3, Lambda, Boto3, Dynamo DB, RDB)
Designed, built and deployed CICD pipeline using GitHub Actions
Designed and built Django-based testimonials website and deployed on AWS using Route53
Learnt advanced features of python (multithreading, multiprocessing, asynchronous programming and socket programming)
Built up expertise in snowflake and DBT
Become project base AWS solution architect

Identifying requirements for data pipelines
Investigating and identifying available data points for each pipeline dataset in kafka stream
Designing schema for target database
Designing flat file structures for HDFS
Producing ETL specification (transformation and validation rules) for oracle database target
Producing ETL specification (transformation and validation rules) for HDFS target
Creating test data for both database and HDFS targets
Building and testing data pipeline for target database
Building and testing pipeline for target HDFS
Version controlling all artefacts in Github
Manually deploy artefacts
Automating all backend tasks with shell scripts

Producing outline design for Data Lake from requirements
Producing ETL specification for json file target
Producing ETL specification for flat file target
Producing ETL specification for excel files target
Producing ETL specification for multiple oracle database targets
Building and testing pipeline for json target
Building and testing pipeline for flat file target
Building and testing pipeline for excel file target
Building and testing pipeline for database targets
Version controlling all artefacts into Github
Manually deploy all artefacts
Automating all backend tasks with shell scripts

Establishing required datasets for pipelines from requirements
Identifying data landing zone for data sources
Designing target schema for oracle database
Designing flat file structures for target HDFS
Producing ETL specification for target database
Producing ETL specification for HDFS
Developing and testing pipeline for oracle database
Developing and testing pipeline for HDFS
Version controlling all artefacts
Deploying all artefacts manually
Automating all backend tasks with shell scripts

Establishing required dataset from requirements
Identifying sources for dataset in kafka stream
Designing target database schema
Producing ETL specification
Creating test data
Building and testing data pipeline
Version controlling all artefacts
Manual deploying all artefacts
Automating all backend tasks with shell scripts
Establishing reporting data structure from requirements
Identifying sources for dataset in reporting database
Designing required json schema for the report
Designing required flat file record structure for the report
Building and testing json and csv reports
Designing and building module for emailing reports to clients
Manually deploying all artefacts

Evaluated the new Denodo data virtualisation platform for capability in integrating data for dashboard reporting
Focused on deliverables across data interfacing, relationship profiling, database design, ETL specifications, test strategy and PL/SQL procedures
Facilitated support in resolving ETL processing problems such as logic fixes, PL/SQL performance improvement and OWB module creation
Analysed new and existing star schemas against preliminary requirements to enable creation of a business object universe, operating within a regulatory environment

Formed part of a project management team to consult on a BI program for client EE as a senior developer and solution designer, from initial analysis to testing of data integration applications
Engaged with client Data Authority on data integration and warehousing projects
Liaised with Delivery Manager on high-level project planning and estimating
Built a sound understanding of upcoming projects using software requirements specifications