Bootstrapped Data Products — Here's What I Learned
External Source means this preview stays on Qubicweb while the full article lives on the publisher site.
Brief points
Key points will appear here after this read is condensed for Qubicweb. Use the source link below if you need the full article immediately.
I'm Heshan. I build data products. No VC, no team — just me, a lot of Playwright scripts, and PostgreSQL.
Two years ago I couldn't build anything that worked. Today I run three production data platforms. Here's what they are and what each one taught me.
Social Intel — socialintel.io
A social media dataset marketplace. 130+ pre-enriched datasets across 100+ platforms, plus custom dataset builds.
Every post goes through a deterministic enrichment pipeline (no LLMs) — sentiment, topics, financial signals, bot detection, 250+ fields, 25 quality checks. Export in CSV, JSON, JSONL, or Parquet.
Built with: FastAPI, PostgreSQL, Next.js 15, Playwright, VADER Lexicon, AWS S3 What I learned: A deterministic pipeline beats LLMs for reproducibility. No token costs, no schema drift.
Pulse Aggregator — pulseaggregator.com
Production news platform indexing 100,000+ articles per week from 10,000+ sources. Updated hourly.
4-tier fallback: RSS → Playwright scraper (paywalls) → full-text search → REST API. Runs on APScheduler, has been at 99.9% uptime since launch.
Built with: FastAPI, PostgreSQL, Playwright, Next.js, APScheduler What I learned: Fallback chains save you. When one source breaks, the next tier catches it. No single point of failure.
GoodQuote Scraper
Production Goodreads scraper that outputs structured CSV/JSON. Pagination, data validation, multi-page crawling.
Built with: BeautifulSoup, Python What I learned: Sometimes BeautifulSoup is all you need. Not every project needs Playwright.
FinPull — finpull.dev (Upcoming)
Financial data pipeline — OHLCV, earnings, P/E ratios, analyst ratings. Playwright + yfinance → FastAPI → PostgreSQL → dashboard.
Target: Traders, analysts, portfolio dashboards.
Built with: Playwright, yfinance, FastAPI, PostgreSQL, Next.js What I'm learning now: Financial data has worse APIs than social media. That's saying something.
The Stack That Stuck
After building all of these, here's what I actually reach for:
Production Tech Stack
| Layer | Tools |
|---|---|
| Scraping | Playwright · BeautifulSoup · Asyncio · Proxy rotation |
| Data | Pandas · NumPy · Parquet/JSONL exports |
| Backend | FastAPI · PostgreSQL · APScheduler · JWT |
| Frontend | Next.js 15 · Tailwind · TypeScript |
| Infra | Vultr · Vercel · Supabase · Docker |
freeCodeCamp Certified: Responsive Web Design (Mar 2024) · Scientific Computing with Python (Nov 2025) Everything I know about production scraping I learned by breaking things in production.
Right now I split time between Social Intel (growing, revenue-generating) and building FinPull. If you work with data at scale or have opinions about Playwright vs. pure HTTP — I'd love to hear your take.
All projects are linked on my GitHub.