Skip to content

Commit ffaa509

Browse files
committed
feat(google): 19 product APIs — initial Google SDK + Scrapingdog parity
First commit landing the `src/google` module as a tracked tree. Ships the full Google Scraper surface plus the 7 Scrapingdog parity items we added server-side in scrape-badger/scrapebadger#135: - `client.google.search` — web SERP, with optional `ai_overview: true` that chases Google's deferred AI Overview page_token. - `client.google.maps` — search, place, reviews, photos, posts. - `client.google.news` — search, topics, trending. - `client.google.hotels` — search, details. - `client.google.trends` — interest, regions, related, trending, and **topic `autocomplete`** (categorized Knowledge Graph entities). - `client.google.jobs`, `shopping` (with merchant URL click enrichment), `patents`. - `client.google.scholar` — search, **profiles**, **author**, **authorCitation**, **cite** (MLA / APA / Chicago / Harvard / Vancouver formats with BibTeX / RIS / EndNote / RefWorks export links). - `client.google.autocomplete`, `images`, `videos`, `finance`, `aiMode`, `lens`. - `client.google.local` — **new.** Local Pack business listings for SERP queries with local intent (`tbm=lcl`), driven by `q + location/uule` instead of a place_id. - `client.google.shorts` — **new.** Short-form vertical video results (YouTube Shorts, TikTok, Reels) via Google's Shorts SERP mode (`udm=39`). - `client.google.flights` — **new.** One-way, round-trip, and multi-city flight search with passenger config, cabin class, stops filter, and max-price. Returns `best_flights`, `other_flights`, `price_insights`, and per-offer carbon emissions. - `client.google.products` — immersive product detail. **19 Google product APIs in total**, matching the monorepo `/v1/google/*` routes shipped in e823f1a. Fully typed, ESM+CJS dual-publish, accessible as `client.google.<product>`. New exported param types: `LocalSearchParams`, `ShortsSearchParams`, `FlightsSearchParams`, `FlightsTripType`, `FlightsTravelClass`, `FlightsStopsFilter`, `ScholarProfilesParams`, `ScholarAuthorParams`, `ScholarAuthorCitationParams`, `ScholarCiteParams`, `TrendsAutocompleteParams`. Version: 0.5.1 -> 0.6.0. README + CHANGELOG updated. 161 vitest tests pass (26 google + 135 other), tsup build clean, eslint + tsc green.
1 parent e49021c commit ffaa509

29 files changed

Lines changed: 2238 additions & 3 deletions

CHANGELOG.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,26 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [0.6.0] - 2026-04-11
9+
10+
### Added — Google Scrapingdog parity (refs scrape-badger/scrapebadger#135)
11+
12+
Three new Google product sub-clients and deeper Scholar / Trends / Search surface:
13+
14+
- **`client.google.local`** — Local Pack business listings ranked for a SERP query (`tbm=lcl`). Driven by `q + location/uule`, returns ratings, reviews, addresses, phone numbers, and GPS coordinates. Complementary to the Maps API.
15+
- **`client.google.shorts`** — Short-form vertical video results (YouTube Shorts, TikTok, Facebook Reels) via Google's Shorts SERP mode (`udm=39`).
16+
- **`client.google.flights`** — One-way, round-trip, and multi-city flight search with passenger config, cabin class, stops filter, and max-price. Returns `best_flights`, `other_flights`, `price_insights`, and per-offer carbon emissions.
17+
- **`client.google.scholar.profiles({ mauthors })`** — Author profile search by name with `after_author` / `before_author` pagination.
18+
- **`client.google.scholar.author({ author_id })`** — Full author profile (articles, citation stats, co-authors).
19+
- **`client.google.scholar.authorCitation({ author_id })`** — Citations-per-year chart for a Scholar author.
20+
- **`client.google.scholar.cite({ q })`** — MLA / APA / Chicago / Harvard / Vancouver citation formats plus export links (BibTeX / RIS / EndNote / RefWorks).
21+
- **`client.google.trends.autocomplete({ q })`** — Categorized Knowledge Graph topic entities (`mid`, `type`, link) for a query prefix. Distinct from Google Search autocomplete.
22+
- **`client.google.search.search({ ..., ai_overview: true })`** — Optional flag that chases Google's deferred AI Overview `page_token` with a follow-up fetch and merges the result into `ai_overview`. Adds ~1s and 1 credit only when the SERP actually defers the overview.
23+
24+
New exported param types: `LocalSearchParams`, `ShortsSearchParams`, `FlightsSearchParams`, `FlightsTripType`, `FlightsTravelClass`, `FlightsStopsFilter`, `ScholarProfilesParams`, `ScholarAuthorParams`, `ScholarAuthorCitationParams`, `ScholarCiteParams`, `TrendsAutocompleteParams`.
25+
26+
The Google product roster is now **19** (was 16).
27+
828
## [0.2.0] - 2026-03-05
929

1030
### Added

README.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
<a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT"></a>
1313
</p>
1414

15-
The official Node.js/TypeScript client library for the [ScrapeBadger](https://scrapebadger.com) API.
15+
The official Node.js/TypeScript client library for the [ScrapeBadger](https://scrapebadger.com) API — Twitter, Google, Vinted, and general web scraping.
1616

1717
## Features
1818

@@ -22,6 +22,10 @@ The official Node.js/TypeScript client library for the [ScrapeBadger](https://sc
2222
- **Smart Rate Limiting** - Reads API headers and throttles pagination automatically
2323
- **Resilient Retries** - Exponential backoff with colored console warnings
2424
- **Typed Exceptions** - Distinct error classes for every failure scenario
25+
- **37+ Twitter endpoints** - Tweets, users, lists, communities, trends, geo, real-time streams
26+
- **19 Google product APIs** - Search (with optional deferred AI Overview follow-up), Maps, News, Hotels, Trends (incl. topic autocomplete), Jobs, Shopping (+ merchant URL enrichment), Patents, Scholar (search + profiles + author + author citation + cite formats), Images, Videos, Finance, AI Mode, Lens, **Local Pack**, **Shorts**, **Flights**, Products
27+
- **Vinted scraping** - Search items, item details, user profiles, brands, colors, markets
28+
- **Web scraping** - Anti-bot bypass, JS rendering, and AI data extraction
2529

2630
## Installation
2731

@@ -73,6 +77,7 @@ const client = new ScrapeBadger();
7377
|-----|-------------|---------------|
7478
| **Web Scraping** | Scrape any website with JS rendering, anti-bot bypass, and AI extraction | [Web Scraping Guide](docs/web-scraping.md) |
7579
| **Twitter** | 37+ endpoints for tweets, users, lists, communities, trends, and real-time streams | [Twitter Guide](docs/twitter.md) |
80+
| **Google** | 19 products — Search, Maps, News, Hotels, Trends, Jobs, Shopping, Patents, Scholar, Images, Videos, Finance, AI Mode, Lens, Autocomplete, Local, Shorts, Flights, Products | [Google Guide](docs/google.md) |
7681
| **Vinted** | Search items, get details, user profiles, and reference data across all Vinted markets | [Vinted Guide](docs/vinted.md) |
7782

7883
## Error Handling

0 commit comments

Comments
 (0)