URL Scraping

Adding items by hand is tedious. GiftWrapt lets you paste a product URL and tries its best to pull back the title, photo, price, and a few other fields automatically. Results vary by site, but even a partial fill saves typing - whatever doesn’t come back, you fill in manually.

How It Feels

1. Click Add item on a list.
2. Paste a URL (Amazon, Etsy, an indie shop, anywhere).
3. The form fills itself in. Edit anything you want; save.

The scrape runs as a live progress indicator so you can see which provider is working and stop early if you don’t need to wait.

What Gets Pulled

When it works, the scraper extracts:

• Title - the product name.
• Photo(s) - the main product image, plus any extras you can swap between.
• Price and currency - parsed from the page.
• Vendor - derived from the URL itself (the domain), not from the page content, so this one is reliable even when the rest of the scrape comes back empty.

If a field is missing, just fill it in. The form never clobbers fields you’ve already typed in - once you touch a field, the scraper leaves it alone.

Best Effort, Not Guaranteed

Some sites change their markup faster than any parser keeps up with, and some pages simply don’t expose the fields in a form we can read. A partial fill is normal. Fill in the rest by hand and move on.

Harder Pages

Plenty of product pages don’t yield much to a plain HTTP fetch: the content is assembled in the browser, or the markup shifts often enough that a simple parse comes back thin. When the basic fetch doesn’t produce a usable result, GiftWrapt can fall through a configurable chain of additional providers:

• Headless rendering (self-hosted Browserless, hosted Browserbase, or a self-hosted GiftWrapt Scraper gateway) for JavaScript-heavy pages that need a real browser before there is anything to parse.
• FlareSolverr, for pages that only return content after an interstitial resolves.
• ScrapFly, a hosted rendering service, for pages the lighter providers cannot render.
• A custom HTTP endpoint, if you run your own renderer and want GiftWrapt to call it.
• An optional AI extractor that reads the fetched HTML and pulls out fields when structured parsing comes back empty.

Operators configure these through the admin scraping settings. Pages the cheap providers handle never escalate to the paid ones, so a typical deployment spends very little.

If none of them produce a result, you’ll see a clean failure and the form will prompt you to fill the fields in by hand.

For Self-Hosters

The scraping pipeline is fully configurable per deployment:

• Add multiple providers and arrange them into tiers, 1 through 5. Everything in a tier runs in parallel; the next tier only runs if the merged result from the previous one fell below the quality threshold.
• Each provider has its own timeout, secret fields (encrypted at rest), and admin-controlled enable/disable.
• A small always-on fetch provider runs first on every scrape; everything else is opt-in.
• A per-URL cache dedups repeat scrapes for a configurable TTL.
• SSRF protection is built in: the fetcher refuses to call into private IP ranges, and re-checks on every redirect hop.

See the contributor reference for the full architecture diagram and per-provider configuration recipes.

Related

Scraping (Admin Config) Pipeline architecture, provider types, tiers, and tuning.

Items What the scraper actually fills in on each item row.