Carlos.
I build data platforms
A passionate big data engineer. I tend to make use of modern technologies to implement, test, deploy, automate and maintain data driven apps.
Hi, my name is
About Me
I am a Software Engineer with a huge passion for data and analytics, and the Cloud computing ecosystem. I truly believe in the DevOps culture and its benefits. I cannot imagine a project without CI/CD, automation or monitoring. I also have a strong interest in the IoT world and its application in multiple fields.
I am currently working as a Big Data Engineer at Accenture, where I have designed, implemented, deployed, monitored and maintained multiple streaming applications for processing data in real time in the AWS cloud.
I was honored with the Final Degree Award for the highest GPA of the Software Engineer Degree.
Here are a few technologies I have been working with a lot recently:- Flink
- Kafka
- Redis
- ArangoDB
- EMR
- EKS
- Hadoop Yarn
- Kubernetes
- Ansible
- Docker
- Grafana
- Kibana
- Java
- Python
Experience
- Developed and end-to-end solution for monitoring Flink and EMR; which includes the collection, processing and loading of Flink/EMR metrics and logs; and the creation of dashboards in Grafana and Kibana for their visualization and analysis.
- Automated the configuration of the EMR clusters through Ansible, following a GitOps philosophy.
- Implemented a Continuous Delivery pipeline for the deployment of Flink jobs in EMR, based on the declarative paradigm.
- Developed a proof of concept of the new Flink SQL version, to homologate it and promote its usage within the project.
- Improved and evolved a Java framework based on Flink, including abstractions over the new Flink SQL API version, which simplified the development of streaming applications focused on the joins between streams of events.
- Reduced in a 60% the costs of the AWS infrastructure, due to the analysis of the consumptions of the EMR and ElastiCache services, and the optimization of the used instances (number and type of instances, as well as the use of spot instances).
- Kept a close contact with the development teams of the client, focused on the decision making and continuous improvement.
- Made guards focused on the maintenance of the production applications, solving numerous incidents.
- Was selected by Accenture to take part of the TechStar program (in which less than 1% of the analysts and consultants of Accenture Technology can participate), and in which I was able to reinforce my technical, soft and management skills.
- Interviewed 3 people to choose a new member for our project development team (technical interviews).
- Lead a team of 3 developers to carry out and deploy multiple projects developed in Flink.
- Participated in a RFP (Request For Proposals) for one of the largest insurance companies in the market, in which I designed a serverless microservices architecture on AWS, with high and low level diagrams, which was a complete success, given the fact that the project was awarded to Accenture, and we received the direct greetings from the CDO of the insurance company.
- Designed and implemented a Java framework based on Apache Flink which eased the development of Big Data applications for processing events in real time, that abstracts the following components: security configurations, read/write events from/to Kafka, N-N joins among events, generation of custom metrics, timestamps management, …
- Implemented and deployed in AWS multiple streaming applications employing the previous framework, using services such as EMR, S3, ElastiCache, EC2 and Certificate Manager, and technologies such as Kafka, Redis, CouchBase y Hadoop YARN.
- Implemented multiple scripts in Python and Bash for automating processes.
- Contributed to the adoption of a testing culture focused on specification-based techniques (Equivalent Class Partitioning) and structure-based techniques (MCDC - Modified condition/decision coverage).
- Took part of a multidisciplinary development team, guided by Agile methodologies, and focused in the continuous integration (version control with Git and GitLab, code reviews through Merge Requests, Jenkins, SonarQube and Artifactory).
- Developed a library for the identification of topics present in a collection of documents, the classification of documents in those topics, and the generation of extractive text summaries, using Natural Language Processing techniques (NLP), by means of supervised and unsupervised learning, in the Python programming language.
- Developed a REST API in Flask and a frontend in Angular which eased the interaction with a collection of documents.
- Installed diverse software packages and repaired multiple hardware equipments. Managed the inventory. Solved problems of various kinds, both from students and teachers (where stood out the problems related to the Active Directory).
Education
2015–2019
Software Engineer Degree
Universidad de Oviedo
GPA:
9.312 out of 10 · 3.66 out of 4
- 28 Honor Distinctions
- Final Degree Award for the highest GPA
- Grace Hooper Award for academic excellence
- Final Degree Project: Web application for the classification of text documents and the generation of extractive summaries
Achievements
Grace Hooper Award
I received this award for my academic excellence in the Software Engineer Degree.