44
55This NodeJS module is designed to scrape and parse Google results using [ SERP API] ( https://serpapi.com ) . Feel free to fork this repository to add more backends.
66
7+ This Ruby Gem is meant to scrape and parse Google results using [ SerpApi] ( https://serpapi.com ) .
8+ The following services are provided:
9+ * [ Search API] ( https://serpapi.com/search-api )
10+ * [ Location API] ( https://serpapi.com/locations-api )
11+ * [ Search Archive API] ( https://serpapi.com/search-archive-api )
12+ * [ Account API] ( https://serpapi.com/account-api )
13+
14+ Serp API provides a [ script builder] ( https://serpapi.com/demo ) to get you started quickly.
15+
716[ The full documentation is available here.] ( https://serpapi.com/search-api )
817
918## Requirement
@@ -14,46 +23,71 @@ This NodeJS module is designed to scrape and parse Google results using [SERP AP
1423
1524## Installation
1625
26+ NPM 7+
27+
1728``` bash
1829$ npm install google-search-results-nodejs
1930```
2031
21- [ link - npm - google-search-results-nodejs ] ( https://www.npmjs.com/package/google-search-results-nodejs )
32+ [ Link to npm package ] ( https://www.npmjs.com/package/google-search-results-nodejs )
2233
23- ## Simple Example
34+ ## Quick start
2435
2536``` javascript
26- var gsr = require (' google-search-results-nodejs' )
27- let serp = new gsr .GoogleSearchResults (" Your Private Key" )
28- serp .json ({
37+ const GSR = require (' google-search-results-nodejs' )
38+ const client = new GSR .GoogleSearchResults (" Your Private Key" )
39+ client .json ({
2940 q: " Coffee" ,
3041 location: " Austin, TX"
3142}, (result ) => {
3243 console .log (result)
3344})
3445 ```
35-
36- ## Set SERP API key
37-
46+ This example runs a search about "coffee" using your secret api key.
47+
48+ The Serp API service (backend)
49+ - searches on Google using the client: q = "coffee"
50+ - parses the messy HTML responses
51+ - return a standardizes JSON response
52+ The Ruby class GoogleSearchResults
53+ - Format the request to Serp API server
54+ - Execute GET http request
55+ - Parse JSON into Ruby Hash using JSON standard library provided by Ruby
56+ Et voila..
57+
58+
59+ ## Example
60+ * [ How to set SERP API key] ( #how-to-set-serp-api-key )
61+ * [ Search API capability] ( #search-api-capability )
62+ * [ Example by specification] ( #example-by-specification )
63+ * [ Location API] ( #location-api )
64+ * [ Search Archive API] ( #search-archive-api )
65+ * [ Account API] ( #account-api )
66+ * [ Promise and callback] ( #Promise-and-callback )
67+ * [ Coding style] ( #coding-style )
68+
69+ ### How to set SERP API key
70+ The Serp API key can be set globally using a singleton pattern.
3871``` javascript
39- var gsr = require (' google-search-results-nodejs' )
40- let serp = new gsr .GoogleSearchResults (" Your Private Key" )
72+ const GSR = require (' google-search-results-nodejs' )
73+ let client = new GSR .GoogleSearchResults (" Your Private Key" )
4174```
42- Or
75+
76+ The Serp API key can be provided for each request
4377``` javascript
44- var gsr = require (' google-search-results-nodejs' )
45- let serp = new gsr.GoogleSearchResults ()
46- let result = serp .json ({
47- q: " Coffee" ,
48- location: " Austin, TX" ,
78+ const GSR = require (' google-search-results-nodejs' )
79+ let client = new GSR.GoogleSearchResults ()
80+ let result = client .json ({
81+ api_key: " Your private key" ,
82+ q: " Coffee" , // search query
83+ location: " Austin, TX" , // location
4984}, (data ) => {
5085 console .log (data)
51- }, " Your Private Key " )
86+ })
5287```
53- ## Example with all params and all outputs
88+
89+ ### Search API capability
5490``` javascript
55- var gsr = require (' google-search-results-nodejs' )
56- let serp = new gsr.GoogleSearchResults ()
5791query_params = {
5892 q: " query" ,
5993 google_domain: " Google Domain" ,
@@ -64,29 +98,158 @@ query_params = {
6498 safe: " Safe Search Flag" ,
6599 num: " Number of Results" ,
66100 start: " Pagination Offset" ,
67- serp_api_key: " Your SERP API Key"
101+ api_key: " Your SERP API Key" , // https://serpapi.com/dashboard
102+ tbm: " nws|isch|shop" ,
103+ tbs: " custom to be search criteria" ,
104+ async: true | false , // allow async query
105+ output: " json|html" , // output format
68106}
69107
70- callback = function (data ) {
108+ const GSR = require (' google-search-results-nodejs' )
109+ const client = new GSR.GoogleSearchResults ()
110+
111+ // create a callback
112+ callback = (data ) => {
71113 console .log (data)
72114}
73115
74116// Show result as JSON
75- serp .json (query_params, callback)
117+ client .json (query_params, callback)
76118
77119// Show result as HTML file
78- serp .html (query_params, callback)
120+ client .html (query_params, callback)
121+ ```
122+
123+ (the full documentation)[ https://serpapi.com/search-api ]
124+
125+ see below for more hands on examples.
126+
127+ ### Example by specification
128+
129+ We love true open source, continuous integration and Test Drive Development (TDD).
130+ We are using RSpec to test [ our infrastructure around the clock] ( https://travis-ci.org/serpapi/google-search-results-ruby ) to achieve the best QoS (Quality Of Service).
131+
132+ The directory test/ includes specification/examples.
133+
134+ Set your api key.
135+ ``` bash
136+ export API_KEY=" your secret key"
137+ ```
138+
139+ Run all tests
140+ ``` npm test ```
141+
142+ ### Location API
143+ ``` javascript
144+ const client = new GSR.GoogleSearchResults (api_key)
145+ client .location (" Austin" , 3 , (data ) => {
146+ console .log (data)
147+ })
148+ ```
149+
150+ it prints the first 3 location matching Austin (Texas, Texas, Rochester)
151+ ``` javascript
152+ [ { id: ' 585069bdee19ad271e9bc072' ,
153+ google_id: 200635 ,
154+ google_parent_id: 21176 ,
155+ name: ' Austin, TX' ,
156+ canonical_name: ' Austin,TX,Texas,United States' ,
157+ country_code: ' US' ,
158+ target_type: ' DMA Region' ,
159+ reach: 5560000 ,
160+ gps: [ - 97.7430608 , 30.267153 ],
161+ keys: [ ' austin' , ' tx' , ' texas' , ' united' , ' states' ] },
162+ ... ]
163+ ```
164+
165+ ### Search Archive API
166+
167+ The first search result returns a search_id which can be provided to get the search result from the archive.
168+ ``` javascript
169+ var client = new GSR.GoogleSearchResults (api_key)
170+ client .json ({q: " Coffee" , location: " Portland" }, (search_result ) => {
171+ // search in archive for the search just returned
172+ client .search_archive (search_result .search_metadata .id , (archived_search ) => {
173+ console .log (archived_search)
174+ })
175+ })
176+ ```
177+
178+ it prints the search from the archive.
179+
180+ ### Account API
181+ ``` javascript
182+ const client = new GSR.GoogleSearchResults (api_key)
183+ client .account ((data ) => {
184+ console .log (data)
185+ })
186+ ```
187+ it prints your account information.
188+
189+ ## Promise and callback
190+
191+ This API was developped using basic callback to handle response.
192+ And exception are just throw away with interceptor.
193+
194+ if you want to take advantage of the promise to block the request.
195+ here is how I will do.
196+ ``` javascript
197+ const util = require (' util' )
198+
199+ function getJson (parameter , resolve , reject ) {
200+ const client = new gsr.GoogleSearchResults (api_key)
201+ try {
202+ client .json (parameter, resolve)
203+ } catch (e) {
204+ reject (e)
205+ }
206+ }
207+
208+ const blockFn = util .promisify (getJson)
209+ blockFn[util .promisify .custom ](parameter).then ((data ) => {
210+ expect (data .local_results [0 ].title .length ).toBeGreaterThan (5 )
211+ done ()
212+ }).catch ((error ) => {
213+ console .error (error)
214+ done ()
215+ })
79216```
80217
81- This service supports Google Images, News, Shopping.
218+ Reference:
219+ * test: test/ExampleSpec.js
220+ * documentation: https://nodejs.org/docs/latest-v8.x/api/util.html#util_util_promisify_original
221+
222+ ## Coding style
223+
224+ This API is using callback to run in non-blocking code.
225+ Here we are trying to follow the spirit of NodeJS.
226+ For reference you can read this article:
227+ * https://nodejs.org/en/docs/guides/blocking-vs-non-blocking/
228+ * https://nodejs.org/en/docs/guides/dont-block-the-event-loop/
229+
230+ For pratical example, you can see the test located under test/.
231+
232+ ## Conclusion
233+ Serp API supports Google Images, News, Shopping and more..
82234To enable a type of search, the field tbm (to be matched) must be set to:
83235
84236 * isch: Google Images API.
85237 * nws: Google News API.
86238 * shop: Google Shopping API.
87239 * any other Google service should work out of the box.
88- * (no tbm parameter): regular Google Search.
240+ * (no tbm parameter): regular Google client.
241+
242+ The field ` tbs ` allows to customize the search even more.
89243
90244[ The full documentation is available here.] ( https://serpapi.com/search-api )
91245
92- For pratical example, you can see the test located under test/.
246+ ## Contributing
247+
248+ Contributions are welcome, feel to submit a pull request!
249+
250+ To run the tests:
251+
252+ ``` bash
253+ export API_KEY=" your api key"
254+ make test
255+ ```
0 commit comments