Thursday, February 14, 2008

Hacking Google Suggest (complete) with JavaScript Remoting

Google Suggest is an feature of the Google Search Engine User Interface and Server that creates suggestions (auto-completes) your search keywords.

This is performed using AJAX. Google will send the first few letters or word(s) you type into the Search Input back to the Google Server using the XMLHttpRequest Object (or a similar AJAX method)

When I type in the search keywords "javascript". I can watch the XMLHttpRequests created by Google Search using Firefox's Firebug extension. Google creates 5 XMLHttpRequests, each one a few letters more then the previous.

The Google Suggest XMLHttpRequests

  • http://www.google.com/complete/search?hl=en&client=suggest&js=true&q=ja - does not complete
  • http://www.google.com/complete/search?hl=en&client=suggest&js=true&q=java - completes and returns:
    window.google.ac.Suggest_apply(frameElement, "java", new Array(2, "java download", "21,600,000 results"
    
    , "java api", "9,000,000 results", "java runtime", "2,510,000 results", "java.com", "1,350,000 results"
    
    , "java update", "11,500,000 results", "javascript tutorial", "1,490,000 results", "java string", "3
    
    ,920,000 results", "java runtime environment", "894,000 results", "javanoid", "71,000 results", "java
    
     virtual machine", "2,050,000 results"), new Array(""));
  • http://www.google.com/complete/search?hl=en&client=suggest&js=true&q=javasc - completes and returns:
    window.google.ac.Suggest_apply(frameElement, "javasc", new Array(2, "javascript tutorial", "1,490,000
    
     results", "javascript substring", "120,000 results", "javascript redirect", "291,000 results", "javascript
    
     download", "20,300,000 results", "javascript replace", "283,000 results", "javascript settimeout", "198
    
    ,000 results", "javascript split", "251,000 results", "javascript indexof", "130,000 results", "javascript
    
     switch", "1,150,000 results", "javascript string replace", "153,000 results"), new Array(""));
    
  • http://www.google.com/complete/search?hl=en&client=suggest&js=true&q=javascri - does not complete
  • http://www.google.com/complete/search?hl=en&client=suggest&js=true&q=javascript - completes and returns:
    window.google.ac.Suggest_apply(frameElement, "javascript", new Array(2, "javascript tutorial", "1,490
    
    ,000 results", "javascript substring", "120,000 results", "javascript redirect", "291,000 results", "javascript
    
     download", "20,300,000 results", "javascript replace", "283,000 results", "javascript settimeout", "198
    
    ,000 results", "javascript split", "251,000 results", "javascript indexof", "130,000 results", "javascript
    
     switch", "1,150,000 results", "javascript string replace", "153,000 results"), new Array(""));
    

As you can see, each response is a JavaScript function call. Google Search makes the XMLHttpRequest call asynchronously as you type your search query, and aborts the last XMLHTTPRequest if you type more than 2 to 3 letters. Each XMLHttpRequest result is a call to the method: window.google.ac.Suggest_apply(). What what can do is create this method in our JavaScript, and wait for Google Suggest to call it with the suggested keywords and their "weight" as parameters.

Hacking Google Suggest for our own use

Now the fun part. What we do is make a HTTP request to the Google URL http://www.google.com/complete/search?hl=en&client=suggest&js=true&q={q}, where q is our keyword. Google will then return the Google Search Suggestions for that keyword.

Try it by clicking: http://www.google.com/complete/search?hl=en&client=suggest&js=true&q=javascript

Now that we know how to get Google Suggest results from the Google server, we can implement it with JavaScript. Due to the XMLHttpRequest Same Domain Policy, we cannot use the XMLHttpRequest Object. However, the results of each Google Suggest query is javascript, so we can use JavaScript Remoting.

Here is an example:

Try our Google Suggest Hack yourself. View source to see how it works.

2 comments:

Christopher Clarke said...

Wow nice! thanks for that that's really interesting. I was trying to read the code on my own but I guess I'm not advance enough to understand all of it yet.

Craig said...

Nice. Now if I can find someone that uses this technique to query a database rather than Google Search.