Home

Diogo Castro

Software Engineer

Experience

Back-End Software Engineer · Zyte (Formerly Scrapinghub)
Remote, Ireland | Jun 2022 – Now
Web Scraping focused company · Creator of the open-source Scrapy project
  • Building and maintaining distributed systems and APIs handling data at scale, with a strong focus on reliability and observability.
  • Developed web crawlers using many different Python packages.
  • Implemented the already acquired knowledge regarding HTML, CSS and Javascript to help on working developing better crawlers.
  • Built CI/CD pipelines using Github Actions, responsible for deploying the application to the production environment and checking for any issue related to test or code formatting
  • Responsible for reviewing and suggesting changes for less experienced developers by analyzing their Pull Requests.
  • Developed new features and worked on internal projects using Django.
Back-End Software Engineer · Americanas S.A
Remote, Rio de Janeiro | Feb 2021 – Jun 2022
One of the biggest brazilian e-commerce companies
  • Developed crawlers using Scrapy, Beautiful Soup, and Playwright to handle session management.
  • Created and maintained scripts to handle collected data with the Pandas framework under Apache Airflow.
  • Used Docker to build, test, and deploy the crawlers quickly.
  • Worked on AWS environment to manage S3 buckets and orchestrate containers responsible to run the crawlers using ECS.
  • Worked on optimization for queries under BigQuery data warehouse.
  • Maintained the Back-End for a RESTful API CRUD written in Django Rest Framework responsible for letting other co-workers check what was scraped for the latest crawls.
Full-Stack Software Engineer · LyMTech
Remote, Niterói | Nov 2019 – May 2021
Company specialized in creating SaaS solutions for various industries
  • Rewrote a legacy Ruby on Rails SaaS in Django.
  • Refactored and developed new extensions for Odoo, an open-source ERP fully written in Python.
  • Worked on creating chat-bots using Botpress, an Open-Source Javascript framework based on Node.js and React.
Full-Stack Software Engineer · Sinn Tecnologia
Rio de Janeiro, Rio de Janeiro | Apr 2016 – Sep 2019
Software House focused on selling solutions to brazilian government bodies
  • Development and maintenance of applications for a Brazilian public company, using the most varied languages and technologies to achieve results.
  • Implemented dashboards and Front-End components using React and Vue to add custom functionalities to the intranet employees’ system.
  • Developed RESTful APIs using Spring Framework.
  • Created and developed scripts to containerize applications using Docker.

Education

B.Sc. in Computer Science Universidade Federal Fluminense (UFF) Niterói, Rio de Janeiro, Brazil | Mar. 2018 - Dec. 2021

Skills

  • Languages: Python · Javascript · Java · Bash · C · SQL · NoSQL · Scrapy · AsyncIO
  • Web Development: React · Node.js · Express · Django · Django Rest Framework · FastAPI · Flask · HTML/CSS
  • Web Scraping: Scrapy · AsyncIO · BeautifulSoup · Playwright · Puppeteer · Selenium
  • Databases: PostgreSQL · MySQL · Elasticsearch · MongoDB · Redis · DynamoDB · SQLite
  • Infrastructure: Linux · Docker · Containers · CI/CD · Git · NGINX · AWS · GCP