DropResearch — AI-Powered Shopify Niche Research Tool

• 5 microservices built and confirmed working locally through iterative Claude Code sessions


• 4-step research pipeline validated end-to-end with real data


• 3-round niche engine with rotating category system across 10 themes and a banned niches list


• 5 markets validated in parallel with live Google data per run


• 5 parallel data sources with 7-day compound cache key in Step 2


• Sourcing brief generated in ~15 seconds with 6-8 product categories and viral angles


• Full .docx report export with 7 structured sections and disclaimer


• Google OAuth login gate with full state preservation across OAuth redirect

The Problem

Starting a Shopify dropshipping store is one of the most common entry points for people trying to build an income online — but most beginners fail at the very first step: choosing a niche. They either pick something too competitive (phone cases, electronics), too saturated (candles, soap, macrame), or something with no real supplier availability. There is no accessible tool that combines real competition data, supplier availability, and AI reasoning into a single guided flow a beginner can actually follow.

What Was Built

DropResearch is a 5-microservice AI research platform that walks a user through four validated steps — from discovering an underserved niche to receiving a complete sourcing brief with product categories, viral content angles, pricing strategy, and what to avoid. Every step uses real data: live Shopify store counts via Google, CJ Dropshipping API product availability, Google Trends, Amazon signals, and AliExpress pricing — synthesised by Groq into actionable recommendations. The session ends with a downloadable Word document (.docx) the user can annotate and share.

The 4-Step Pipeline

Step 0 — Niche Discovery: A 3-round validated engine generates 20 niche candidates using a rotating category system across 10 themes (marine hobbies, fiber arts, miniatures, outdoor survival, science hobbies, and more). Each candidate is validated with real Shopify store counts via StealthyFetcher Google searches, cross-referenced against CJ Dropshipping product availability, then ranked by Groq on opportunity score. Niches with over 20,000 Shopify stores are hard-rejected. A banned list prevents Groq from repeatedly suggesting overused niches like candle making, soap making, and macrame. The result is 10 genuinely underserved niche cards with competition labels, store counts, margin ranges, and buyer demand signals.

Step 1 — Market Validation: Five target markets (US, CA, UK, AU, NZ) are validated in parallel using StealthyFetcher Google searches. Each market returns an opportunity score, demand count, and local competitor store count. Groq ranks the results and recommends the optimal 2-3 markets with reasoning.

Step 2 — Research Lab: Five data sources fire in parallel — Google Trends via pytrends, Shopify store count, AliExpress listing count, Amazon review signals, and competitor price data. Results are cached by a compound niche and sub-niche key for 7 days. Groq synthesises everything into an evergreen score, competition level, margin range, seasonal risk assessment, and a proceed or avoid verdict.

Step 3 — Sourcing Brief: A single Groq call generates a complete sourcing brief in approximately 15 seconds. It contains a winning angle, a price strategy with supplier cost and sell price targets, 6 to 8 product categories each with a viral TikTok or Instagram hook, cost and sell and profit ranges, a supplier search term, and a difficulty rating. Three Google search links are generated per category pointing to AliExpress, CJ Dropshipping, and DHgate. A What to Avoid section lists specific product types to skip with reasons. Supplier tips and competition insight close the brief.

The Download Feature

At the end of the flow, users can download a fully formatted Word document containing the complete research session — niche overview, market recommendations, research analysis, the full sourcing brief, all product categories, and the avoid list. A repurposed Export tab in the progress bar serves as the final deliverable view, summarising the session before download.

Key Technical Decisions

Scrapling's StealthyFetcher was chosen for Google scraping because it uses a modified Firefox browser with fingerprint spoofing to bypass bot detection. This works correctly in local Docker environments but fails on Railway due to Google's immediate blocking of cloud datacenter IPs — a known infrastructure limitation that affects all cloud providers. The YouTube video for this project was recorded locally where the full pipeline runs correctly.

Step 3 was redesigned mid-build. The original design used a multi-source product discovery loop combining the CJ API and Scrapling to surface specific products. The output was too generic — wire cutters and basic household items appearing in speciality niche searches. The redesign replaced the entire loop with a single Groq call generating a structured sourcing brief. The Groq-only approach produced dramatically more specific, more actionable output. The lesson: when AI automation produces commodity results, redesign the output strategy rather than patching the pipeline.

The Build Process

Like the earlier days in the series, Day 08 was built through an iterative feedback loop — describe the problem, review what Claude Code produced, test it running locally, identify what broke or produced poor output, give specific feedback, and repeat. The niche rotation system, the login gate with state preservation, the sourcing brief redesign, and the Word document export were all changes made after initial versions proved insufficient. Claude Code handled every implementation decision. The product decisions — what the output should feel like, what counts as useful, when to scrap an approach entirely — remained with the human.

• When AI automation produces commodity output, redesign the output strategy rather than patching the pipeline. The original Step 3 used a multi-source product discovery loop combining the CJ API and Scrapling. The results were generic and low-value. Replacing the entire loop with a single Groq call producing a structured sourcing brief took 30 minutes and produced dramatically better output. The lesson applies beyond this project: complexity in a pipeline does not guarantee quality in the output.

• Scrapling StealthyFetcher works correctly in local Docker but fails immediately on Railway and other cloud providers because Google blocks datacenter IP ranges. This is not a code problem — it is a network infrastructure problem. Residential proxies are required for any Google scraping at scale on cloud infrastructure. Know this before designing a cloud product that depends on Google data.

• A rotating category system prevents repetitive AI output far more reliably than prompt variation alone. Without rotation, Groq returns the same overused niches on every run regardless of how the prompt is worded. With 10 rotating themes and a banned niches list, outputs are genuinely diverse across sessions. Structural constraints on AI generation produce more consistent variety than prompt engineering alone.

• State preservation across OAuth redirects requires an explicit save-before and restore-after pattern using localStorage. Supabase Google OAuth redirects away from the app and back — any in-memory state is lost. The fix is to save the pending niche name to localStorage before calling signInWithGoogle(), then check and restore it inside the onAuthStateChange SIGNED_IN handler. Without this, users who are not yet logged in lose their niche selection after authenticating.

• CJ Dropshipping API v2 silently rate-limits parallel requests. The limit is 1 request per second — sequential calls with 1100ms delays between them are required. Parallel calls appear to succeed but return empty product arrays with no error. Always test API rate limits under production conditions before designing a parallel processing flow around them.

• A banned niches list is as important as a good generation prompt. Without it, LLMs will suggest the same high-profile niches on every run regardless of temperature or category instructions. The banned list is the structural enforcement that makes the generation prompt work as intended.