Santos Saenz Ferrero

Designing the overall new architecture of the platform in terms of better data engineering, infrastructure design and CI/CD.
Looking after long-term effectiveness and optimization of the platform.
Pushing new features and best practises.
Close communication with clients and stakeholders.
Pressing collaboration with teams with a similar software stack, Platform team, Security team and teams within the division.
Supervising, providing oversight and advice to other teammates.
Bringing data from different clients to be consumed in the platform by developing data engineering and data modeling tasks with Azure Data Factory and Databricks (PySpark, Delta Lake, Mosaic).
Developing IaC with Terraform for the platform.
Collaboration in infrastructure at an enterprise level through communication and synchronization with the cloud team and security team.
Development of modules made available in Terraform Private Registry and policies with Sentinel.
Managing and developing CI/CD with GitHub, different GitHub Settings, GitHub Actions and GitHub Webhooks.

Division Digital Solutions

•Design of the overall new architecture of our platform, in terms of better data engineering, infrastructure & CI/CD. Looking after long-term effectiveness & optimization of our platform. Pushing new features and best practises

•Communication with clients & stakeholders

•Bringing data to be consumed in our platform by developing data engineering & data modelling tasks with Azure Data Factory & Databricks (PySpark, Delta Lake, Mosaic)

•Developing IaC with Terraform for our platform, & collaboration in Infrastructure at an enterprise level through communication & synchronization with the cloud team & security team

•Managing & developing CI/CD with GitHub, different GitHub Settings, GitHub Actions & GitHub Webhooks

•Governance at scale with Pureview, Azure Governance & Unity Catalog

•Assessing organization wide best practises and compliance through Azure Policies, Terraform Private Registry and Landing Zones

Helping in the design of the overall architecture of the platform in terms of data engineering and infrastructure.
Looking after long-term effectiveness and optimization of the systems.
Close communication with clients and stakeholders.
Self responsible in the selection of the most appropriate tools and strategies for developing the client demands.
Supervising, providing oversight and advice to other teammates.
Bringing the customer data to the Databricks Lakehouse Platform through medallion architecture ETL and data modeling using Data Factory and Databricks.
Using Scala Spark API and Delta Live Tables through Python in both batch and streaming processing paradigms.
Consuming data from different data systems like Relational Databases (SQL Database, SQL Server, MySQL), CRM Salesforce, API-REST, SFTP, Non-Relational Databases (InfluxDB).
Managing data governance through Unity Catalog, implementing data object access control, definition of service principals.
Creating and managing work clusters, all purpose clusters and DBSQL warehouses.
Handling Azure Data Lake Storage and Key Vault.
Managing CI/CD through Databricks Repos.
Using Git, Maven, PyPI, Jenkins and Artifactory.
Performed maintenance and development of new features for a Maven project built on Scala that through Azure SDK and Databricks SDK performs API calls to these platforms.
Used for non technical users to develop ETL in the lakehouse through the use of JSON files.
Developed and scheduled pipelines that handled and enriched millions of movements and transactions daily for consumption by clients from BBVA in Spain and Mexico.
Project responsibilities evolved to developing most complex, critical and time constraint modules of the project.
Supervising, providing oversight and advice to the rest of the teammates.
Close communication with client and other data producers and data consumer teams.
Collaboration with production and infrastructure team.
ETL developed using Spark with Scala.
Developed ETL using Java applications, SQL applications, Python scripts and Shell scripts through the Batch Processing Architecture.
Processing structured and semi-structured high volume data (Parquet, JSON, CSV).
Optimized performance of Spark and other ETL jobs by checking execution DAG and logs with tools like Atenea.
Used different data stores like S3 buckets, Oracle DB instances, Elasticsearch instances and HDFS.
Managed and checked job orchestration and execution through Control-M and scheduled job execution with a Scheduler as a Service.
Used Jenkins, Artifactory, Git and Maven.
Used containerization for local development with Docker.
Used CLI for managing different services.
Analysed quality of code with SonarQube.
Developed tests with JUnit and Mockito.
Used corporate tools for consultation and management of data governance, Data Quality and Traceability.

Developed REST API using Python, SQL and AWS Lambda.
Ingested data from customers, service stations and the stock market through API calls.
Transformed data and provided customer recommendations for refueling and predictions related to fuel prices using gradient boosted decision trees.
Developed different machine learning models.
Performed hyperparameter tuning using Bayesian optimization.
Developed ETL/ELT pipelines and calculated KPIs using Pandas and Python.
Used Git and PyPI.
Interacted with AWS through REST API.
Performed exploratory data analysis on different datasets, mostly related with fuel and time series data using Python, Plotly, Matplotlib, Seaborn, NumPy and Pandas.
Performed univariant and multivariant analysis.
Performed analysis of tendency, stationality, stationarity and noise.
Performed preprocessing of data and analysed best machine learning approaches and algorithms for forecasting.
Developed REST API using Python, SQL and AWS Lambda that reads financial structured data from customers, transforms it, summarizes the result and sends it by email automatically.
Worked with NumPy, SQL and Pandas for cleaning data from customers, transforming it and obtaining different insights about Return of Inversion, Sales and Investments.
Developed HTML documents with CSS that visually provide the insights for the customers.