Valdir Stumm Junior
I'm Valdir, a Brazilian Software Engineer passionate about software development, Python, best practices and writing.
I'm a self-motivated engineer, always looking for opportunities to improve the projects I'm working on. I believe in collaboration and that's something that I'm always promoting and trying to improve in the teams I take part.
My background includes web development with Python and Django, large scale web crawling with Scrapy and years of teaching and mentoring experience.
Lead Software Engineer
Leading a full stack team responsible for developing services for a web based insurance platform.
Main technologies: Vue.js, Django, Django REST Framework and several AWS services.
2018 - Now
Senior Crawling Engineer
Development of large scale web crawlers, crawling around 1.5k websites daily. The activities included crawler development, CI setup, management of servers, technical seminars, mentoring.
Experience working in a team of 15 developers spread all around the world and also as a solo developer responsible for handling a project for an important client.
Main technologies: Python, Lua, Scrapy, Pandas, Docker, ScrapyRT, CircleCI, S3 and Scrapy Cloud.
2016 - 2018
2015 - 2016
I bootstrapped the technical training program for Scrapinghub, turning it into a profitable product.
2015 - 2017
Computer Science Teacher
Besides teaching programming with Python and Java, I did several other activities such as development of web applications using Python & Django used by thousands of students, setup and administration of the Moodle Learning Management System and coordination of a graduation course.
2010 - 2015
Worked as a developer for a large scale ERP, used by some of the biggest cities and universities in Brazil.
Main stack: Delphi, Python, Plone.
2006 - 2008
Support, maintenance and automation in assorted servers and services.
Main technologies: GNU/Linux, iptables, Apache, Sendmail, Bash.
2004 - 2005
MSc in Computer Science
Federal University of Santa Catarina
2008 - 2010
BSc in Computer Science
Federal University of Santa Maria
2004 - 2008
- flake8-scrapy: A Flake8 plugin to catch common issues in Scrapy spiders.
- scrapy-fieldstats: A Scrapy extension to generate summaries of the scraped fields.
- Python Help: A blog where I write about Python, in Portuguese.
- DunderDoc: Another blog where I write about Python, but in English.
Open Source Contributions
- Scrapy: A fast and efficient web crawling framework.
- Parsel: The parsing engine behind Scrapy.
- Shub: A command line interface to Scrapinghub services.
- Extruct: A metadata parser for web scraping.
- Scrapy-Crawlera: Crawlera middleware for Scrapy.
- Cookiecutter Pypackage: A template generator for Python projects.
- VOC: A transpiler to convert Python into Java.