Rafael Pereira

BigData Admin

Linkedin

For a more resumed version of my experience, visit my Linkedin page.

Visit

GIST

For a full overview of my code snippets, follow my GIST.

View

Disclaimer


Obligatory “Opinions are my own and do not necessarily reflect the views and opinions of my employer” :)

About me


I live and work in Portugal and I officially started my professional journey in 2017.

I created this website in order to share. Everything I know today was built on the backs of others, and I want to contribute to the Internet.

This page specifically is intended to be a sort of replacement of my CV. I think it’s a more updated format, and this website alongside my LinkedIn and Github/Gist should give interested parties enough context and insight into who I am and what I do.

Right now, my job description fits a BigData Administration, but I’m trying to expand more into the role of an Engineer. Recently with the Go To Cloud movements I’m trying to pick up knowledge surrounding Cloud Computing and Solutions.

I imagine myself picking up BigData On-Prem/Cloud Full Stack knowledge on my journey.

Current Position


Placeholder image

BigData Administrator

@Xpand IT, Mar 2019 - Present


I’m working currently, since mid-2019, as a BigData Admin/Engineer for Xpand IT, a technology consultancy company, in a department that is specialized in Big Data.

I work mostly with Cloudera Hadoop Technology and Microsoft Azure.

As a part of my job as a consultant I deploy, manage, maintain and upgrade Hadoop centered environments. Specifically I work with:

  • Cloudera Manager;
  • Hadoop (Cloudera):
    • HDFS;
    • Yarn;
    • MapReduce;
  • Kafka (Confluent/Cloudera);
  • Kudu (Cloudera);
  • Hbase (Cloudera);
  • Spark (Cloudera);
  • Impala (Cloudera);
  • Hive (Cloudera);
  • Docker;
  • OpenShift;
  • CentOS;
  • RedHat Enterprise Linux;
  • MySQL;
  • PostreSQL;
  • StreamSets DataCollector/Control Hub;
  • Ansible;

Among others, but these are the more recurrent technologies that I work with and have at least a modicum of knowledge about.

I am also called in for solution design, and assist developers develop and optimize their applications.

Certifications


Placeholder image

Cloudera Administrator

Credential ID: 100-022-970, Oct 2019


Placeholder image

Microsoft Certified: Azure Fundamentals

See Credential, May 2020


Placeholder image

Microsoft Certified: Azure Data Engineer Associate

See Credential, May 2020


Previous Positions

Technological


Placeholder image

BigData Developer

@Aubay, Sep 2018 - Mar 2019


At Aubay, I worked in a project that was set up around Microsoft Azure, developing and working with ETL pipelines for Business Analytics.

Some of my work consisted of:

  • ETL Pipelines with DataFactory, HDInsights, DataBricks.
  • Pipeline Monitoring and Support.
  • Migration of Data Pipelines to Azure.
  • Web App development.
  • Infrastructure Management.

From the development point of view, I can point out:

  • A project where I worked with a Node library, Puppeteer, that provides a High-Level API to interact with Chrome and Chromium, to develop an automated data extraction process;
  • Superficial .Net Core Development, focused on Azure Web Apps and Azure Functions.

I can also say that this being my first professional experience with BigData, I picked up valuable concepts and notions from the business side and the technological side, that became fundamental in my journey, especially when it comes to the Cloud.


Placeholder image

Internship - Android Developer

@Bridge Robotics, Jun 2017 - Nov 2017


Bridge Robotics is a small Startup that focuses on developing Robot Fleet Management Solutions.

During my stay, I picked up the challenge of contributing to the development of a native Android Mobile Application coming from an Electrical and Computer Engineering background.

It was an interesting shift, coming from programming in C, to enter the world of Java, and, as I later realized, and important one.

It was also my first professional contact with Python, Git, Docker and professional development in general.

By this time I was mostly interested in Robotics, BigData/System Administration wasn’t in my radar yet, and I picked up a thing or two about ROS and the industry as a whole.

Non-Technological


Placeholder image

Museum Guide

@MAAT - Electricity Museum , Sep 2018 - Mar 2019


MAAT is a Museum of Art, Architecture and Technology, and I was selected to be a part of an internship program, that only students could apply to, to become museum guides.

I worked in the part of the Museum dedicated to Technology, and old Power Station (Central Tejo) My work consisted of:

  • Giving guided tours to schools, families and all sorts of interested groups, to the Power Station and the rest of the museum;
  • Giving Workshops to kids, focused on Electricity;
  • Securing and giving assistance to the art exhibition rooms.

For 6 months I went to college and worked at this museum part-time, I learned a lot about the historical aspect of Electricity, in Portugal and the World, and developed important people skills.

There was a lot of preparation involved in the tours, but it was very rewarding to give them to all kinds of groups, from kids to senior citizens, and have their appreciation in the end.

The engineering nerd in me was always fascinated with the whole magnitude of what was accomplished at that now museum, and I discovered some taste and appreciation for contemporary art.


Placeholder image

Sales Promoter

@Hospedeiras de Portugal, Jun 2017 - Nov 2017


I challenged myself to start working while studying, and one of the jobs that I was able to do, because of the flexible schedule, was as a Sales Promoter.

I rotated between 3 positions:

  • Working with a specific product and promoting sales on-site.
  • Working with a set of products and providing assistance on site, while promoting sales.
  • And plain sales with the promotion aspect.

On the first position I worked mostly with Alcoholic Beverages in supermarkets, on the second my main go-to job was to assist people in the Garden/Pool area of a store with pool maintenance, and on the last one it was mostly doing sales on small carts in the middle of Shopping Centers.

It was an interesting gig, I met a lot of interesting people, and it made me look into sales positions from a completely different perspective. It can be simple, but it’s not an easy job.

Non-Professional Work


Placeholder image

President/Project Manager

@GEST, 2013 - 2016


During my college days, I worked with a Student Group called GEST, which was, in a sense, the nerd group.

I started out as a member, became Project Coordinator, and ended as President of the club. We organized several gaming competitions/gatherings, those mostly consisted in board games, trading card games, computer/console games and role playing games.

From my roles I can point out:

  • We gathered sponsorships for our small events, mostly local tech related companies around our campus;
  • We created 2 campus wide pop events, with all kinds of activities, being the first one a free entry one, TécniCon, where we counted around 600 people during 2 days, and the second one, SpringIT Con, where we sold 1600 tickets during 2 days;
  • We ran a social project, in which we took board games to play with kids in the hospital;
  • I learned a lot with dealing with volunteers, different organizations, and several different types of bureaucracy.


Placeholder image

Organizing Committee President

@AIESEC - Leadership Tournament, 2014 - 2015


AIESEC is the largest student organization in the world. It aims to have a positive impact on society by providing meaningful leadership experiences.

I started out my journey through this organization as a member of the Corporate Team, whose goal was to gather sponsored Internships in Portugal for people from abroad.

The opportunity to take point of the Leadership Tournament, an event that tried to promote leadership amongst students by putting them against them real life case studies from real companies, presented itself and my job consisted of:

  • Managing a Team;
  • Raising interest near students and organizations;
  • Partnership Management;
  • Event Planning;
  • Ensuring Event Quality;

The most important thing that I took note from AIESEC was what Leadership is, and what being a Leader means to me.