نبذة عني
I’m a Senior Analyst Data Engineer, Data Engineer Associate certified with Databricks, and Databricks Certified Associate Developer for Apache Spark 3.0 with Python 3.0. I am always looking for the best solution that off…
I’m a Senior Analyst Data Engineer, Data Engineer Associate certified with Databricks, and Databricks Certified Associate Developer for Apache Spark 3.0 with Python 3.0. I am always looking for the best solution that offers my clients quality in data and code. Currently, experiencing Azure cloud, I have appreciated the potentials of Databricks Lakehouse platform, in which I am managing software projects applying the most advanced ETL solutions with multi-hop architecture, detailed management of streaming and batch big data flows, and advanced and detailed data governance features. Experience of CD/CI, Gerrit, Jenkins, Terraform, Github, and Scrum. Previously, I experienced for several years the implementation of data marts in oracle DB using OWB, PL/SQL. I have worked since the beginning as Java Software Developer and Business Intelligent Developer. I have a long experience of Scrum and Agile frameworks. I got the Scrum Master Accredited Certification, 6/11/2020 authorized certification id 68758836121764 by International Scrum-Institute
الخبرة
Senior Analyst Data Engineer
Involved in a project to create a self service environment in Databricks for users to access fact view profiled and segregated by user and dimension tables inside Unity Catalog.
Managed grants to different catalogs, schemas, tables, views, and UDFs SQL used for RLS and CLS on fact views.
Implemented the Medallion architecture.
Ingested data from sources at bronze layer using Autoloader and Structured Streaming for initial loading.
Created tables in silver layer with different strategies such as SCD2, append, and overwrite.
Built initial loading and daily streaming flows in bronze and silver layers.
Parametrized code so tables could be added by modifying a JSON file without creating a new GIT version.
Created jobs called by REST API from Maestro.
Prepared tables and views to be used in PowerBI directly by users.
Used partitions on some tables.
Optimized Z-order.
Set the right size of clusters and SQL datawarehouse for ETL flows and user needs.
Deepened experience in performance and optimization techniques for cluster and SQL warehouse configuration, ETL flow, queries, data model design, and structures.
Applied Role Level Security rules for complex data segregation requirements.
Participated in specialized architectural meetings specifying the Medallion architecture used in the project.
Trained about 80 colleagues in a Databricks course inside the organization.
Senior Data Analyst engineer
I’m a Senior Analyst Data Engineer, Data Engineer Associate certified with Databricks, Databricks Certified Associate Developer for Apache Spark 3.0 with Python 3.0, and trainer databricks in my ITdigital department.
Actually I work in a specialized and unique team for advanced POC and projects applying all the best solution databricks at our platform. I am also working directly with Databricks consultants.
Cloud Data Engineer
Involved in different PoCs as a Databricks Data Engineer specialist to re-engineer the whole ETL process.
Applied Lakehouse features such as Delta Tables, Auto Loader, Delta Live Pipelines, and Databricks SQL.
Managed streaming and batch data flows in the same coding artifact.
Worked in Azure Cloud to get data from legacy sources and external sources such as Amazon S3, Marketo via API, SalesForce via MongoDB, Application Insight, and Google Ads via API.
Transformed and loaded data for Digital Marketing Team uses.
Used Talend to move data from sources to data lake storage.
Used Data Factory to orchestrate transformation and loading of data and refresh of the Analysis Services model.
Used Synapse to manage external tables, views, and procedures.
Worked on Analysis Services models using Visual Studio connected to GitHub.
Worked in a large team cooperating with people having different skills.
Simplified processes to reach better results.
Improved availability, stability, and data quality of an application built with Databricks, Data Factory, and Synapse.
Solved frequent failures on job runs and corrected the recovery data process.
Experienced performance and optimization techniques for cluster and SQL warehouse configuration, ETL flow, queries, data model design, and structures.
Applied Role Level Security rules for complex data segregation requirements.
Application Manager
Responsible for Qlik Training after learning it quickly.
Developed Qlik and BusinessObjects solutions to offer detailed monitoring of manufacturing WIP_CycleTime.
Developed Qlik and BusinessObjects solutions to offer detailed monitoring of performance process for IT production environment.
Developed Qlik and BusinessObjects solutions to offer detailed monitoring of license process for BO environment.
Responsible for a web application in Java, Oracle, and PL/SQL.
Consolidated old code and delivered many new releases with excellent time management.
Delivered releases just in time.
Kept errors for each deployment in Live close to zero.
Took quick actions on problem solving.
Applied pattern design to improve the code.
Worked with a small team with strong flexibility.
Certified Corporate trainer ST.
Responsible for Empowerment People project in IT Catania with a large team.
Trainer manager for BOXI and Web Intelligence.
Improved communication intra team and between team and manager.
Helped people focus on their resources and understand themselves and the working context.
Worked to improve the recognition of talents.
Founder and Direct
Owned a company dedicated to art graphic and web application.
Software Engineer
Responsible for a web application in Java, Oracle, PL/SQL, and Unix.
Took over an old application and delivered new releases always just in time.
Trained two people in Java, Oracle, PL/SQL, and Unix assigned to the project.
Attended exams as Software Configuration Manager according to CMM level 3.
Worked as Developer Business Object.
Freelence
Delivered a complex web application in Java and Access to manage daily activities for a support company for the health of the sick and the needs of the disabled and elderly.
Experimented with different roles and functions necessary to create, develop, and deliver a product as business analyst, product owner, architect, application manager, and developer.
Software Engineer
Worked as Java Developer as external counselor for Elsevier company for the migration to Java and Oracle.