Social media scraping and Big Data

There’s not much to this work despite being almost two years on it. When I joined the company I was part of a project that generated reports on trending social media data by using services like Scrapify.io (not specifically, since it didn’t exist at the time, but a similar one) and scraping websites to get useful data to companies in an attempt to check the impact of advertising campaigns and also alarms to detect complains from unsatisfied customers to reach them and offer a solution in order to prevent it from going viral. That was an interesting job. The backend was in PHP, the main database was in MongoDB and the frontend was a tool made in vanilla-ish javascript that allowed analyzers to present charts and information and generate a periodic PDF for the clients to check. There were some major companies using this service.

Web factory

After a year, the company shifted priorities and reorganized departments. In this new stage my main work was creating institutional websites for customers and advertisement campaigns. It was a stream of short and quick websites mainly by a stack consisting in Wordpress for the admin, then the data was exported in a way that was picked up by a frontend made in Codeigniter, jQuery and javascript in general. The main idea was to take advantage of a quick admin setup using Wordpress and a popular Advanced Custom Fields Plugin that allowed to use Wordpress as a quick headless admin, without forcing ourselves to do the frontend the wordpress way, which at the time it was pretty painful.

Offboarding

The idea of my work of being mostly a part of a production chain for websites was not very appealing to me, especially because it meant to become more like an automata than an investigator or problem solver, which is much more interesting to me.

Luckily, an offering came and I was able to switch to the next company, when my professional experience had much more room to improve.