BlogPulse offers a trend search very similar to Google Trends but specifically targeted at blogs.
Here is the blog trend for the words Joomla, Drupal and Wordpress over the last 6 months.
This shows the percentage of the mentions of each word, Joomla, Drupal and Wordpress in blogs.
Just came across Lively.com which developed by Google, creates a virtual 3D world similar to SecondLife.
I haven't tested out Lively yet, since I'm running Ubuntu and it only supports Windows at the moment. In one of their blog posts however we can gather that Lively is well integrated with the Web. They have widgets that allows visitors - with Lively software installed - to jump into a Lively room embedded in a webpage.
By contrast I believe SecondLife offer APIs that offers a REST interface as well as other network level interfacing. Here is a SecondLife Facebook Application.
I'm wondering if Google will offer the same level of integration with their Lively 3D world. Would make fun Mashups.
Google recently released the AJAX Libraries API which allows you to load the popular JavaScript libraries from Google Servers. The benefits of this outlined in the description of the API.
The AJAX Libraries API is a content distribution network and loading architecture for the most popular open source JavaScript libraries. By using the google.load() method, your application has high speed, globally available access to a growing list of the most popular JavaScript open source libraries.
I was thinking of using it for a current project that would use JS heavily, however, since the project used a CMS (Joomla) the main concern for me was really how many times MooTools would be loaded. Joomla uses a PHP based plugin system (which registers observers of events triggered during Joomla code execution) and the loading of JavaScript by multiple plugins can be redundant as there is no central way of knowing which JavaScript library has already been loaded, nor is there a central repository for JavaScript libraries within Joomla.
MooTools is the preferred library for Joomla and in some cases it is loaded 2 or even 3 times redundantly. I did not want our extension to add to that mess. To solve the problem I would test for the existence of MooTools, if (typeof(MooTools) == 'undefined') and load it from Google only if it wasn't available. Now this would have worked well, however, I would have to add the JavaScript for AJAX Libraries API and it would only be loading 1 script, "MooTools", when I also had about 3-4 other custom libraries that I wanted loaded.
Now I thought, why don't I develop a JavaScript loader just like the Google AJAX Libraries API Loader. Should be just a simple function to append a Script element to the document head. So I started with:
function loadJS(src) {
var script = document.createElement('script');
script.src = src;
script.type = 'text/javascript';
timer = setInterval(closure(this, function(script) {
if (document.getElementsByTagName('head')[0]) {
clearTimeout(timer);
document.getElementsByTagName('head')[0].appendChild(script);
}
}, [script]), 50);
}
function closure(obj, fn, params) {
return function() {
fn.apply(obj, params);
};
}
The function loadJS would try to attach a script element to the document head, each 50 milliseconds until it succeeded.
This works but there is no way of knowing when the JavaScript file was fully loaded. Normally, the way to figure out if a JS file has finished loading from the remote server, is to have the JS file invoke a callback function on the Client JavaScript (aka: JavaScript Remoting). This however means you have to build a callback function into each JavaScript file, which is not what I wanted.
So to fix this problem I though I'd add another Interval with setInterval() to detect when the remote JS file had finished loading by testing a condition that exits when the file has completed. eg: for MooTools it would mean that the Object window.MooTools existed.
So I went about writing a JavaScript library for this, with a somewhat elaborate API, with JS libraries registering their "load condition test" and allowing their remote loading, about 1 wasted hour, (well not wasted if you learn something) only to realize that this wouldn't work for the purpose either. The reason is that it broke the window.onload functionality. Some remote files would load before the window.onload event (cached ones) and others after. This made the JavaScript already written to rely on window.onload fail.
Last Resort, how did Google Do it? I had noted earlier that if you load a JavaScript file with Google's API the file would always load before the window.onload method fired. Here is the simple test: (In the debug output, the google callback always fired first).
google.load("prototype", "1");
google.load("jquery", "1");
google.load("mootools", "1");
google.setOnLoadCallback(function() {
addLoad(function() {
debug('google.setOnLoadCallback - window.onload');
});
debug('google.setOnLoadCallback')
});
addLoad(function() {
debug('window.onload');
});
debug('end scripts');
I had to take a look at the source code for Google's AJAX Libraries API which is: http://www.google.com/jsapi to see how they achieved this.
It never occurred to me that you could force the browser to load your JavaScript before the window.onload event so I was a bit baffled. Browsing through their source code I came upon what I was looking for:
function q(a,b,c){if(c){var d;if(a=="script"){d=document.createElement("script");d.type="text/javascript";d.src=b}else if(a=="css"){d=document.createElement("link");d.type="text/css";d.href=b;d.rel="stylesheet"}var e=document.getElementsByTagName("head")[0];if(!e){e=document.body.parentNode.appendChild(document.createElement("head"))}e.appendChild(d)}else{if(a=="script"){document.write('<script src="'+b+'" type="text/javascript"><\/script>')}else if(a=="css"){document.write('<link href="'+b+'" type="text/css" rel="stylesheet"></link>'
)}}}
The code has been minified, so its a bit hard to read. Basically its the same as any javascript remoting code you'd find on the net, the but the part that jumps out is:
var e=document.getElementsByTagName("head")[0];
if(!e){e=document.body.parentNode.appendChild(document.createElement("head"))}
e.appendChild(d)
Notice how it will create a head Node and append it to the parentNode of the document body if the document head head does not exist yet.
Now that forces the browser to load the JavaScript right then, no matter what. Now following that method you can load remote JavaScript files dynamically and just used the regular old window.onload event or "domready" event and the files will be available.
Apparently this won't work on all browsers, since Google's code also has the alternative:
document.write('<script src="'+b+'" type="text/javascript"><\/script>')
with a bit of testing, you could discern which browsers worked with which and use that. I'd imagine that the latest browsers would accept the dom method and older ones would need the document.write
So my JavaScript file loading function became:
function loadJS(src) {
var script = document.createElement('script');
script.src = src;
script.type = 'text/javascript';
var head = document.getElementsByTagName('head')[0];
if (!head) {
head = document.body.parentNode.appendChild(document.createElement('head'));
}
head.appendChild(script);
}
Anyways, I finally got my JavaScript library loader working just as I liked, thanks to the good work done by Google with the AJAX Libraries API.
The Google trends for websites, which was released by Google 3 days ago, is really something to check out if you're interested in comparing website metrics between different websites and across geographical locations.
Alexa and Compete offer similar services. One of the features that stands out with Google trends is that it displays related websites that visitors to the website being viewed visit in descending order of visitors. This related websites are also filtered when filtering down to single geographic locations. This makes Google trends for websites quite a bit more powerful for your research. It allows you to view a websites competitors in each geographic location or allows link building and search engine optimization for a website for each geographic location.
Search Engine Land and Matt Cutts also blogged about the new google trends for websites.
Firefox seems to be giving Google some pretty revealing information in regular updates, even when you're not visiting Google.
Opening up Wireshark (formely Ethereal) and examining network traffic going through my machine I've noticed time and time again, HTTP requests to Google even when Firefox is sitting idle. I attributed this to having Gmail open, thinking it must be one of those Javascript Remoting calls or XMLHttpRequests going on in the background.
Today I noticed it again, and realized that I only had a page on my local web server open. It definitely could not be sending HTTP requests to Google.
Filtering out just the single TCP request revealed this:
GET /safebrowsing/update?client=navclient-auto-ffox&appver=2.0.0.13&version=goog-white-domain:1:30,goog-white-url:1:371,goog-black-url:1:20001,goog-black-enchash:1:48465 HTTP/1.1 Host: sb.google.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Cookie: SID=<snip>; MPRF=<snip>; NID=9=<snip>; PREF=ID=<snip>:TM=<snip>:LM=<snip>:DV=<snip>:GM=<snip>:IG=<snip>:S=<snip>; rememberme=<snip> Cache-Control: max-age=0 HTTP/1.1 200 OK Content-Type: text/html; charset=UTF-8 Cache-Control: public,max-age=600 Content-Encoding: gzip Server: TrustRank Frontend Content-Length: 2766 Date: Thu, 10 Apr 2008 15:19:52 GMT ..........MW...H.]....G.B...R-.$..1. .Z%&.........%u.X.....q...>...+.....5I. ..........4...}S. 0...Y"sPV..e..."...x...9.n..Xq..v..<.. G..^..)..i.?.=.o. <snip>
What happens is every now and then, Firefox Polls Google with a HTTP Request for updates on phishing sites, which is really nice. Google probably has the largest database on phishing sites, and it is nice that they are "helping" us out by keeping our Firefox browser updated on the latest phishing sites being found in its extensive indexes.
Now if we look at the HTTP request, nothing fancy, just the HTTP request headers, HTTP Response with Gzip encoded body. (If you decode the Gzip encoded body it is easier to see its data on the latest Phishing sites - the GET URI suggest that too.
The fun part is in the Cookies:
Cookie: SID=<snip>; MPRF=<snip>; NID=9=<snip>; PREF=ID=<snip>:TM=<snip>:LM=<snip>:DV=<snip>:GM=<snip>:IG=<snip>:S=<snip>; rememberme=<snip>I've snipped the actual values of course. The SID we can safely assume is the google Application level session id due to its characteristics including its name, size of the hash etc. among others. What bugs me is that this SID ties this HTTP request to my Information in their database, my name, address, bank account etc. etc. The ID Cookie most likely holds non-authenticated session ID (one of those will). You can think of it as everything you have done on *.google.com, and now even when you're not on google, until you delete the cookies.
Note: the cookies aren't valid only on the host: sb.google.com, they are valid on google.com also as they are sent to .google.com. Which means any of the domains *.google.com will trigger Firefox to send the cookies. The cookies are set similar to below:
PREF=ID=<snip>:TM=<snip>:LM=<snip>:S=<snip>; expires=Sat, 10-Apr-2010 17:22:38 GMT; path=/; domain=.google.comNotice when the cookies expire, in 2 years from now. So if you never delete your cookies, your activity is tracked for the next 2 years directly with these cookies.
With all of that said, let me stress that I'm not trying to sound any conspiracy theories here. It may very well be some technical limitation or a simple oversight. After all, Google already knows what you search for, what and who you e-mail, who you chat with and what you chat about, who you socialize with, what your social life looks like, what files are stored on your computer, what documents and spreadsheets you work on, what you blog about, what pictures you share, what you shop for, what newsgroups you read, what current events you keep up with, how you run your website, what stocks you monitor, what books you like to read, and, of course, what newsfeeds you read.A similar find, Is Firefox/Google Spying on Your News Feeds? (Update), notes that at the time of the writing of that article, 2006, the cookies were being kept till 2038, nice...
Now why would google need to know all this just to update Firefox with phishing site information? I tried to think of a possible reasoning for this, but just couldn't. The closest I could think of is if Google could tell in advance using your SID what phishing sites you would most likely visit, only send the needed ones to Firefox. Whooowee.... if they could do that, then they definitely know too much.
There is no valid reason to send your SID, PREF, ID, and other information to Google just to update Firefox with phishing information. The HTTP request is initiated by Firefox, not the user. The update does not have anything to do with the users Google account or the Google Session.
Now in addition to everything else they know about you and me, they have Firefox updating them even when you're not on Google. Letting Google know when you're online. Now I'm a bit of a night owl, I don't want Google to know that... opps too late.
Lets hope this bug gets fixed soon...
Google Suggest is an feature of the Google Search Engine User Interface and Server that creates suggestions (auto-completes) your search keywords.
This is performed using AJAX. Google will send the first few letters or word(s) you type into the Search Input back to the Google Server using the XMLHttpRequest Object (or a similar AJAX method)
When I type in the search keywords "javascript". I can watch the XMLHttpRequests created by Google Search using Firefox's Firebug extension. Google creates 5 XMLHttpRequests, each one a few letters more then the previous.
window.google.ac.Suggest_apply(frameElement, "java", new Array(2, "java download", "21,600,000 results"
, "java api", "9,000,000 results", "java runtime", "2,510,000 results", "java.com", "1,350,000 results"
, "java update", "11,500,000 results", "javascript tutorial", "1,490,000 results", "java string", "3
,920,000 results", "java runtime environment", "894,000 results", "javanoid", "71,000 results", "java
virtual machine", "2,050,000 results"), new Array(""));
window.google.ac.Suggest_apply(frameElement, "javasc", new Array(2, "javascript tutorial", "1,490,000
results", "javascript substring", "120,000 results", "javascript redirect", "291,000 results", "javascript
download", "20,300,000 results", "javascript replace", "283,000 results", "javascript settimeout", "198
,000 results", "javascript split", "251,000 results", "javascript indexof", "130,000 results", "javascript
switch", "1,150,000 results", "javascript string replace", "153,000 results"), new Array(""));
window.google.ac.Suggest_apply(frameElement, "javascript", new Array(2, "javascript tutorial", "1,490
,000 results", "javascript substring", "120,000 results", "javascript redirect", "291,000 results", "javascript
download", "20,300,000 results", "javascript replace", "283,000 results", "javascript settimeout", "198
,000 results", "javascript split", "251,000 results", "javascript indexof", "130,000 results", "javascript
switch", "1,150,000 results", "javascript string replace", "153,000 results"), new Array(""));
As you can see, each response is a JavaScript function call. Google Search makes the XMLHttpRequest call asynchronously as you type your search query, and aborts the last XMLHTTPRequest if you type more than 2 to 3 letters. Each XMLHttpRequest result is a call to the method: window.google.ac.Suggest_apply(). What what can do is create this method in our JavaScript, and wait for Google Suggest to call it with the suggested keywords and their "weight" as parameters.
Now the fun part. What we do is make a HTTP request to the Google URL http://www.google.com/complete/search?hl=en&client=suggest&js=true&q={q}, where q is our keyword. Google will then return the Google Search Suggestions for that keyword.
Try it by clicking: http://www.google.com/complete/search?hl=en&client=suggest&js=true&q=javascript
Now that we know how to get Google Suggest results from the Google server, we can implement it with JavaScript. Due to the XMLHttpRequest Same Domain Policy, we cannot use the XMLHttpRequest Object. However, the results of each Google Suggest query is javascript, so we can use JavaScript Remoting.
Here is an example: