1. Basic Scrape (Non-JS, Fast)
High-performance scraping with Chrome TLS fingerprint, no JavaScript:
Write to /tmp/scrapeninja_request.json:
```json
{
"url": "https://example.com"
}
```
Then run:
```bash
bash -c 'curl -s -X POST "https://scrapeninja.p.rapidapi.com/scrape" --header "Content-Type: application/json" --header "X-RapidAPI-Key: ${SCRAPENINJA_API_KEY}" -d @/tmp/scrapeninja_request.json' | jq '{status: .info.statusCode, url: .info.finalUrl, bodyLength: (.body | length)}'
```
With custom headers and retries:
Write to /tmp/scrapeninja_request.json:
```json
{
"url": "https://example.com",
"headers": ["Accept-Language: en-US"],
"retryNum": 3,
"timeout": 15
}
```
Then run:
```bash
bash -c 'curl -s -X POST "https://scrapeninja.p.rapidapi.com/scrape" --header "Content-Type: application/json" --header "X-RapidAPI-Key: ${SCRAPENINJA_API_KEY}" -d @/tmp/scrapeninja_request.json'
```
2. Scrape with JavaScript Rendering
For JavaScript-heavy sites (React, Vue, etc.):
Write to /tmp/scrapeninja_request.json:
```json
{
"url": "https://example.com",
"waitForSelector": "h1",
"timeout": 20
}
```
Then run:
```bash
bash -c 'curl -s -X POST "https://scrapeninja.p.rapidapi.com/scrape-js" --header "Content-Type: application/json" --header "X-RapidAPI-Key: ${SCRAPENINJA_API_KEY}" -d @/tmp/scrapeninja_request.json' | jq '{status: .info.statusCode, bodyLength: (.body | length)}'
```
With screenshot:
Write to /tmp/scrapeninja_request.json:
```json
{
"url": "https://example.com",
"screenshot": true
}
```
Then run:
```bash
# Get screenshot URL from response
bash -c 'curl -s -X POST "https://scrapeninja.p.rapidapi.com/scrape-js" --header "Content-Type: application/json" --header "X-RapidAPI-Key: ${SCRAPENINJA_API_KEY}" -d @/tmp/scrapeninja_request.json' | jq -r '.info.screenshot'
```
3. Geo-Based Proxy Selection
Use proxies from specific regions:
Write to /tmp/scrapeninja_request.json:
```json
{
"url": "https://example.com",
"geo": "eu"
}
```
Then run:
```bash
bash -c 'curl -s -X POST "https://scrapeninja.p.rapidapi.com/scrape" --header "Content-Type: application/json" --header "X-RapidAPI-Key: ${SCRAPENINJA_API_KEY}" -d @/tmp/scrapeninja_request.json' | jq .info
```
Available geos: us, eu, br (Brazil), fr (France), de (Germany), 4g-eu
4. Smart Retries
Retry on specific HTTP status codes or text patterns:
Write to /tmp/scrapeninja_request.json:
```json
{
"url": "https://example.com",
"retryNum": 3,
"statusNotExpected": [403, 429, 503],
"textNotExpected": ["captcha", "Access Denied"]
}
```
Then run:
```bash
bash -c 'curl -s -X POST "https://scrapeninja.p.rapidapi.com/scrape" --header "Content-Type: application/json" --header "X-RapidAPI-Key: ${SCRAPENINJA_API_KEY}" -d @/tmp/scrapeninja_request.json'
```
5. Extract Data with Cheerio
Extract structured JSON using Cheerio extractor functions:
Write to /tmp/scrapeninja_request.json:
```json
{
"url": "https://news.ycombinator.com",
"extractor": "function(input, cheerio) { let $ = cheerio.load(input); return $(\".titleline > a\").slice(0,5).map((i,el) => ({title: $(el).text(), url: $(el).attr(\"href\")})).get(); }"
}
```
Then run:
```bash
bash -c 'curl -s -X POST "https://scrapeninja.p.rapidapi.com/scrape" --header "Content-Type: application/json" --header "X-RapidAPI-Key: ${SCRAPENINJA_API_KEY}" -d @/tmp/scrapeninja_request.json' | jq '.extractor'
```
6. Intercept AJAX Requests
Capture XHR/fetch responses:
Write to /tmp/scrapeninja_request.json:
```json
{
"url": "https://example.com",
"catchAjaxHeadersUrlMask": "api/data"
}
```
Then run:
```bash
bash -c 'curl -s -X POST "https://scrapeninja.p.rapidapi.com/scrape-js" --header "Content-Type: application/json" --header "X-RapidAPI-Key: ${SCRAPENINJA_API_KEY}" -d @/tmp/scrapeninja_request.json' | jq '.info.catchedAjax'
```
7. Block Resources for Speed
Speed up JS rendering by blocking images and media:
Write to /tmp/scrapeninja_request.json:
```json
{
"url": "https://example.com",
"blockImages": true,
"blockMedia": true
}
```
Then run:
```bash
bash -c 'curl -s -X POST "https://scrapeninja.p.rapidapi.com/scrape-js" --header "Content-Type: application/json" --header "X-RapidAPI-Key: ${SCRAPENINJA_API_KEY}" -d @/tmp/scrapeninja_request.json'
```
---
|