Python Web Scraping Expert (JSON/API Output)

We are looking for an experienced Python web scraping expert who can extract structured data from websites and deliver it in JSON format or through a custom API.
The data should be clean, well-organized, and ready for use in applications or databases.

Key Responsibilities:

Scrape data from static and dynamic websites

Handle pagination, AJAX-loaded content, and login-protected pages (if needed)

Format extracted data into clean and structured JSON

Optionally expose the data via REST API (Flask/FastAPI)

Ensure the scraper is efficient, robust, and handles exceptions/errors

Maintain and update scrapers if website structures change

Required Skills:

Strong experience with Python and libraries like BeautifulSoup, Scrapy, or Selenium

Experience using requests, httpx, or aiohttp

Ability to deal with JavaScript-heavy sites using Selenium or Playwright

Data formatting skills (JSON, CSV, etc.)

Optional but preferred: experience with Flask or FastAPI

Knowledge of proxies, rate-limiting, and CAPTCHA handling

Clear and well-documented code practices

Attention to detail and ability to work independently

Deliverables:

Python code for scraping

JSON files or working API endpoint

Brief documentation or instructions to run the script/API

Bonus Points:

Ability to schedule scraping jobs (e.g., using cron or Celery)

Experience with storing data in a database (MongoDB, PostgreSQL, etc.)

Ability to containerize with Docker (optional)

• Strong proficiency in Python
• Experience with web scraping libraries: BeautifulSoup, Scrapy, or Selenium
• Knowledge of requests, httpx, or aiohttp for HTTP requests
• Ability to scrape both static and JavaScript-rendered content
• Familiarity with JSON data formatting and structuring
• Optional: experience with Flask or FastAPI to create APIs
• Understanding of proxies, rate-limiting, and CAPTCHA handling
• Clean, modular, and well-documented coding practices
• Ability to debug and maintain scraping scripts as websites change
• Self-motivated and able to work with minimal supervision