Unknown Company

Data Collection Engineer

workfromhome • Posted 4 days ago
Remote Full Time Management Occupations

To design, scale, and maintain distributed web scraping and data extraction infrastructure, the full-time remote Data Collection Engineer will build resilient data pipelines, ensure data quality through automated validation, and manage containerized workloads at scale. Key responsibilities Design and deploy high-performance, distributed web scrapers using Python and Scrapy for efficient data extraction Utilize browser scripting tools to extract data from dynamic and JavaScript-heavy websites Deploy and manage scraping workloads on Kubernetes, ensuring optimal resource allocation and fault tolerance Required qualifications 7+ years of professional software engineering experience focused on web scraping, data engineering, or distributed systems Expert-level proficiency in Python and deep experience with Scrapy and distributed scraping architectures Proven experience with browser automation tools such as Playwright, Selenium, or Puppeteer Hands-on experience with Elasticsearch for indexing and querying data Bachelor's degree in Computer Science or Engineering

Back to Job Search