Spider Cloud

Spider Cloud integration (Data, Developer Tools). Imported from open-connector.

API key4 toolshttps://api.spider.cloud

GET

spider_get_credits

Get the available credits for the connected Spider Cloud account.

/data/credits

POST

spider_scrape

Scrape one public URL with Spider Cloud and return its JSON response.

/scrape

11 parameters

Param	Type	Required	Description
url	string	required	The public URL Spider Cloud should scrape.
request	string	optional	The retrieval engine Spider Cloud should use.
return_format	string	optional	The JSON-compatible page content format to return.
readability	boolean	optional	Whether Spider Cloud should apply readability processing.
metadata	boolean	optional	Whether Spider Cloud should include page metadata.
return_page_links	boolean	optional	Whether Spider Cloud should include links found on the page.
filter_output_main_only	boolean	optional	Whether Spider Cloud should keep only main-page content.
respect_robots	boolean	optional	Whether Spider Cloud should respect the target site's robots rules.
cache	boolean	optional	Whether Spider Cloud may use a cached response.
request_timeout	integer	optional	The upstream page request timeout in milliseconds.
delay	integer	optional	The delay in milliseconds before Spider Cloud captures page content.

POST

spider_search

Search the web with Spider Cloud and optionally fetch result-page content.

/search

9 parameters

Param	Type	Required	Description
search	string	required	The web search query.
search_limit	integer	optional	The maximum number of search results to retrieve.
fetch_page_content	boolean	optional	Whether Spider Cloud should fetch content from each search result.
location	string	optional	The location name used to localize search results.
country	string	optional	The country code used to localize search results.
language	string	optional	The language code used to localize search results.
page	integer	optional	The 1-indexed search result page to retrieve.
request	string	optional	The retrieval engine Spider Cloud should use.
return_format	string	optional	The JSON-compatible page content format to return.

POST

spider_get_links

Extract links from one public URL with Spider Cloud.

/links

5 parameters

Param	Type	Required	Description
url	string	required	The public URL whose links Spider Cloud should extract.
request	string	optional	The retrieval engine Spider Cloud should use.
respect_robots	boolean	optional	Whether Spider Cloud should respect the target site's robots rules.
cache	boolean	optional	Whether Spider Cloud may use a cached response.
request_timeout	integer	optional	The upstream page request timeout in milliseconds.