Monday, June 23, 2008

Google Trends for websites

The Google trends for websites, which was released by Google 3 days ago, is really something to check out if you're interested in comparing website metrics between different websites and across geographical locations.

Alexa and Compete offer similar services. One of the features that stands out with Google trends is that it displays related websites that visitors to the website being viewed visit in descending order of visitors. This related websites are also filtered when filtering down to single geographic locations. This makes Google trends for websites quite a bit more powerful for your research. It allows you to view a websites competitors in each geographic location or allows link building and search engine optimization for a website for each geographic location.

Joomla vs Wordpress on Google Trends for websites

Joomla vs Wordpress on Alexa Website Analytics

Joomla vs Wordpress on Compete Website Analytics

Search Engine Land and Matt Cutts also blogged about the new google trends for websites.

Sunday, June 22, 2008

Online Application Development Platforms and Services

Everyone is offering Application development as a service!

Appjet

AppJet is the easiest way to create a web app. Just type some code into a box, and we'll host your app on our servers.
http://appjet.com/

AppPad

AppPad provides a place to create web applications completely in HTML and Javascript. AppPad gives you:
http://apppad.com/

Bungee Connect

Bungee Connect is the most comprehensive Platform-as-a-Service (PaaS) — significantly reducing the complexities, time and costs required to build and deliver web applications.
http://www.bungeelabs.com/

CogHead

Coghead is a 100% web-based system that allows knowledge workers to create their own custom business applications. There’s never any software to install or servers to maintain. Just think it, build it and share it!
http://www.coghead.com/

Google App Engine

Google App Engine enables you to build web applications on the same scalable systems that power Google applications.
http://code.google.com/appengine/

I was going to add more but got bored after reaching G. Now I'm just waiting for that meta web application development environment that integrates all the above with a RESTful API. lol...

Sunday, June 15, 2008

JavaScript Cross Window Communication via Cookies

Using JavaScript we can communicate between browser windows given that we have a reference to each window. When a new window is created the the JavaScript method window.open() it returns a reference to the new window. The child window, also has a reference to the parent window that created it via the window.opener window object. These references allow the two windows to communicate with and manipulate each other.

There are times however, when we need to communicate with an open window for which there is no window object reference. A typical example of this is when a popup window is created, then the parent window is reloaded. When reloading the parent, all JavaScript objects are "trashed", along with the reference to the open popup window. Here is where cookies come into play - they are not trashed.

The problem with cookies is that it only saves strings, so you can't write a reference to a window object directly to a cookie, since serializing the reference is not possible. However, since both the child window and the parent window are able to read and write cookies, then they have a medium for which they can communicate. Even if the medium only allows strings.

To demonstrate how communicating between windows with cookies would work, lets assume we want to open a window, and then close it a few seconds later.

var win = window.open('child.html');
setTimeout(function() { win.close(); }, 5000);
The code will open a child window, and close it after 5 seconds using the reference to the child window and the method close(). However if we didn't have a reference for some reason, we would not be able to invoke the close method. So lets see how it could be done with cookies:
window.open('child.html');
setTimeout(function() { setCookie('child', 'close'); }, 5000);
Here we open a window but do not save a reference. Then after 5 seconds we write 'close', to the cookie named 'child' (using the pseudo setCookie() function). This does not do anything by itself, but if the child window was expecting the cookie, it could close itself when it read 'close'. Lets assume the following JS is in child.html.
// child.html
setInterval(function() { getCookie('child') == 'close' ? this.close() : ''; }, 500);
This would check the cookie every half a seconds and close the window if the cookie read 'close'.

Using this method we can send any commands to any open windows and have them execute it without having a reference to that window.

Friday, June 6, 2008

Blocking Advertisements with a Hosts file, Apache and PHP

The Hosts file is located at /etc/hosts on Linux and %SystemRoot%\system32\drivers\etc\ on Windows XP and Vista. It maps host names to IP addresses and takes precedence over the DNS server. So if you add an entry in your hosts file:

207.68.172.246 google.com 
Then every time you type google.com you will be taken to msn.com instead, since 207.68.172.246 is the IP address of msn.com.

Knowing this, you can point any domain, to an IP address of choice using the Hosts file. Therefore, we can use it to block any domains that hosts unwanted advertising or malware.

Modifying your Hosts File to block Advertisements and Malware

There are many sites offering host files which block advertisments and malware. I use the one on http://www.mvps.org/winhelp2002/hosts.htm.
Here is the txt version of the hosts file: http://www.mvps.org/winhelp2002/hosts.txt

Here is an example of what the entries look like, there the list contains a lot more, about 18, 000 entries at this time.

# [Misc A - Z]
127.0.0.1  ad.a8.net
127.0.0.1  asy.a8ww.net
127.0.0.1  www.abx4.com #[Adware.ABXToolbar]
127.0.0.1  acezip.net #[SiteAdvisor.acezip.net]
127.0.0.1  www.acezip.net #[Win32/Adware.180Solutions]
127.0.0.1  phpadsnew.abac.com
127.0.0.1  a.abnad.net
127.0.0.1  b.abnad.net
127.0.0.1  c.abnad.net #[eTrust.Tracking.Cookie]
127.0.0.1  d.abnad.net
127.0.0.1  e.abnad.net
127.0.0.1  t.abnad.net
127.0.0.1  banners.absolpublisher.com
127.0.0.1  tracking.absolstats.com
127.0.0.1  adv.abv.bg
127.0.0.1  bimg.abv.bg
127.0.0.1  www2.a-counter.kiev.ua
127.0.0.1  accuserveadsystem.com
127.0.0.1  www.accuserveadsystem.com
127.0.0.1  gtb5.acecounter.com
127.0.0.1  gtcc1.acecounter.com
127.0.0.1  gtp1.acecounter.com #[eTrust.Tracking.Cookie]
127.0.0.1  acestats.com
127.0.0.1  www.acestats.com
127.0.0.1  achmedia.com
127.0.0.1  ads.active.com
127.0.0.1  am1.activemeter.com
127.0.0.1  www.activemeter.com #[eTrust.Tracking.Cookie]
127.0.0.1  ads.activepower.net
127.0.0.1  stat.active24stats.nl #[eTrust.Tracking.Cookie]
127.0.0.1  web.acumenpi.com #[AdvertPro]
127.0.0.1  ad.ad24.ru
127.0.0.1  at.ad2click.nl
127.0.0.1  cms.ad2click.nl
127.0.0.1  banner.ad.nu
127.0.0.1  ad-up.com
127.0.0.1  www.ad-up.com
You will need to download the txt file and append the entries to your hosts file.

Now once the hosts file is in effect, when you browse any website in firefox or IE or any other browser, 99% of the advertisements will not be displayed.

Setting up Apache to display a custom page or message for blocked Advertisements and Malware

Each entry in the hosts file blocks unwanted sites by resolving their domain name to 127.0.0.1 which is the IP reserved for looping back to your own IP. So all the requests for advertising sites will instead be made back to your IP. The problem with this is because there is no website on your localhost, then the browser will display an error in place of the ads.

If you're a web developer, you'll likely have a version of Apache or some other HTTP server running on your localhost. So you'll likely get a 404 error in place of the ads. You can resolve this by adding a virtual host entry into your httpd.conf file that will display a custom page instead of the 404.

To resolve this you can set up a virtual host to catch all requests made to your Apache server, for the blocked hosts. Assuming you always access your local server via the URL http://localhost/ then you probably don't need the other host possibilities on 127.0.0.1. So your virtual host could look something like:

<VirtualHost 127.0.0.1>
ServerAdmin webmaster@adblock
DocumentRoot /var/www/adblock/
ErrorDocument 404 /404.html
ErrorLog /etc/log/adblock/error.log
TransferLog /etc/log/adblock/access.log
</VirtualHost>
This will catch all requests made to 127.0.0.1. The requests will most likely have a path that doesn't exist in your file structure in /var/www/adblock/ so it will generate a 404 error. You therefore need a custom 404 document which is defined in ErrorDocument 404 /404.html. This can have the simple line, "ad or malware blocked" or something on those lines.

Now localhost also resolves to 127.0.0.1 so you will need to make sure you have a virtual host for the host localhost.

The other thing you could do instead of setting up a virtual host, and it may be simpler, is create a custom 404 document for your current setup. You can do this via a directive directly in httpd.conf like: ErrorDocument 404 /404.php. Notice that it is a PHP so you can use some PHP code to customize the error message. What you'll want is to have the PHP detect if the request was for a blocked site, and if so show your message: "site blocked", but show the regular 404 page for your actual website.

How you detect if the request is from a one of the blocked hosts is by comparing the requested host with the list of hosts in your hosts file that are blocked. The host requested is in the $_SERVER['SERVER_NAME'] variable. Since the list of blocked hosts is large and you probably do not want to read all of those with your php script each time an advertisement is blocked, you can apply the reverse comparison - if the requested host is not in the list of your valid hosts, then it is a blocked host. An example:

// our valid hosts
$valid_hosts = array('localhost', 'my.host.joe', 'my.other.host.peter');
// check if the requested host is a valid one
if (!in_array($_SERVER['SERVER_NAME'], $valid_hosts)) {
    echo 'ad or malware blocked'; // display message in place of blocked ad
} else {
    include('/404.html'); // display regular 404 page
}
Now, when you visit those websites with pesky advertisements and popups, you get a neat little line saying "ad or malware blocked" in the place of those ads.

Thursday, June 5, 2008

Synchronizing Date and Time in different Timezones with PHP, MySQL and JavaScript

How do you show the correct date and time (timestamp) to users in differnet time zones? In theory it should be simple:

Save your your date and time with the correct timezone

The date and time with timezone is a timestamp. Though implementations differ, timestamps basically contain the same information (date/time and timezone).The timezone can be explicitly recorded in the timestamp (eg: 2005-10-30 T 10:45 UTC) or implicitly taken from the context in which the timestamp was generated or recorded (eg: unix timestamp is dependent on timezone of server generating the timestamp).

Something as simple as saving a timestamp in mysql with PHP can be not so simple due to the difference in the timestamp representation in the two languages.

The PHP timestamp is defined as the number of seconds since the Unix Epoch (January 1 1970 00:00:00 GMT) while the mysql timestamp is a representation of the present date and time in the format YYYY-MM-DD HH:MM:SS.

If you save the timestamp as a mysql timestamp field, then the timestamp is saved as UTC, however, when you access the timestamp it is converted to the timezone set in the mysql server, so basically you don't get to see the stored UTC version of the timestamp. If you save it as a PHP timestamp in a varchar or unsigned int field then it would be subject to the the PHP servers timezone. So in essence both the MySQL and PHP timestamp a dependent on the timezone of their respective servers.

Whichever format you save it in, just remember that both the PHP and MySQL timestamps reference the timezone on the server they are saved on, PHP during generation of the timestamp, and mysql during retrieval.

Retrieve the timezone of the user to whom you will display the date and time to

The easy way to do this is ask the user what their timezone is. You see this used in many registration forms on websites as well as many open source forums, CMSs, blog software etc. However, not every user will even bother giving you the correct timezone.

You can use JavaScript if it is available on the browser:

var tz = (new Date().getTimezoneOffset()/60)*(-1);
This depends on the users computer's clock, so if it is set wrong, then you will get a wrong result. Time is relative however, so if a user wants to be a few hours behind, let them be.

You can use the users IP address to assume their geographic location and thus their timezone. This can be done server side and thus is not dependent on the users browser having JavaScript. Using the IP to determine timezone is dependent on the accuracy of your IP geocoding service you use. Here is an example using the hostip.info API for geocoding and earthtools.org for lat/long conversion to timezone.

<?php
/**
* Retrieves the Timezone by the IP address
* @param String $IP (optional) IP address or remote client IP address is used
*/
function getTimeZoneByIP($IP = false) {

 // timezone
 $timezone = false;

 // users IP
 $IP = $IP ? $IP : $_SERVER['REMOTE_ADDR'];

 // retrieve geocoded data from http://hostip.info/ API in plain text
 if ($geodata = file('http://api.hostip.info/get_html.php?ip='.$IP.'&position=true')) {
  // create an associative array from the data
  $geoarr = array();
  foreach($geodata as $line) {
   list($name, $value) = explode(': ', $line);
   $geoarr[$name] = $value;
  }
  
  // retrieve lat and lon values
  $lat = trim($geoarr['Latitude']);
  $lon = trim($geoarr['Longitude']);
  
  if (strlen($lat) > 0 && strlen($lon) > 0) {
   // pass this lat and long to http://www.earthtools.org/ API to get Timezone Offset in xml
   $tz_xml = file_get_contents('http://www.earthtools.org/timezone-1.1/'.$lat.'/'.$lon);
   // lets parse out the timezone offset from the xml using regex
   if (preg_match("/<offset>([^<]+)<\/offset>/i", $tz_xml, $match)) {
    $timezone = $match[1];
   }
  }

 }
 return $timezone;
}
?>

You can also use a combination of the three in order to correlate the data and get a better guess of the timezone.

Calculate the difference in hours between the saved date and time and the users date and time

Now that we have the timestamp and the users timezone, we just need to adjust the timestamp to their timezone. First we need to calculate the difference between the timezone the timestamp is saved in, as the users timezone.

$user_tz_offset = $tz_user - $tz_timestamp; 
where $user_tz_offset is how far ahead or behind in hours the user timezone is from the timestamps timezone.

Add the difference in hours to the saved date and time and display

Now we have all we need to show the correct time to the user based on their timezone. Example in pseudo code:

$user_tz_offset = $tz_user - $tz_timestamp; 
$users_timestamp = $timestamp + $user_tz_offset;

Wednesday, June 4, 2008

PHP Code Performance Profiling on Ubuntu

A few options for code profiling in PHP:

XDebug

XDebug consists of a PHP extension, and requires an XDebug client to view the output generated by XDebug. It writes profiling data by default to /tmp. What we want to do is install XDebug, enable profiling and view the files generated using Kcachegrind, which is an XDebug client that comes default with Ubuntu.

To install XDebug on Ubuntu is as simple as the command:

sudo apt-get install php5-xdebug
This assumes you installed Apache2 and PHP5 using aptitude. Otherwise you'd probably want to follow one of these instructions on installing XDebug:
http://ubuntuforums.org/showthread.php?t=525257
http://2bits.com/articles/setting-up-xdebug-dbgp-for-php-on-debian-ubuntu.html
http://www.apaddedcell.com/easy-php-debugging-ubuntu-using-xdebug-and-vim

Once you've installed xdebug you will need to enable php code profiling by setting the xdebug.profiler_enable setting to 1 in php.ini. You can view the php.ini settings using the command:

php -r 'phpinfo();'
To narrow down to just the xdebug settings use:
php -r 'phpinfo();' | grep xdebug
Note: php --info | xdebug will work also.

If you used the simple sudo apt-get install php5-xdebug to install xdebug, then it should have automatically created an ini file: /etc/php5/conf.d/xdebug.ini which is included with the php.ini.

Edit the php.ini file or /etc/php5/conf.d/xdebug.ini and add the line:

xdebug.profiler_enable=1
Example: sudo gedit /etc/php5/conf.d/xdebug.ini

After saving this you will need to restart Apache in order to reload the php settings.

sudo /etc/init.d/apache2 restart

You will then need an xdebug client to display the profiling information. Kcachegrind is installed by default in Ubuntu. Open Kcachegrind kcachegrind & and use it to open a file generated by xdebug. This will be found in the directory specified in php.ini for the directive xdebug.trace_output_dir which defaults to /tmp. The files generated by Xdebug are prefixed with cachegrind.out.. So you can view a list of these files using the command ls /tmp | grep cachegrind.out.

How to interpret the Xdebug profiling information displayed in Kcachegrind is described at: http://www.xdebug.org/docs/profiler under Analysing Profiles.

Benchmark

Benchmark is a Pear package. Documentation is found at: http://pear.php.net/package/Benchmark/docs/latest/Benchmark/Benchmark_Profiler.html

A simple usage would be:

// load class and instantiate and instance
require_once 'Benchmark/Profiler.php';
$profiler = new Benchmark_Profiler();

// start profiling
$profiler->start();

// do some stuff
myFunction();

// stop
$profiler->stop();

// display directly in PHP output
$profiler->display();

myFunction could look something like:
function myFunction() {
     global $profiler;

     $profiler->enterSection('myFunction');

     //do something
     $profiler->leaveSection('myFunction');

     return;
 }

Benchmark allows you to do profiling directly in your PHP script. One of the disadvantages of XDebug is that you have to enable profiling in PHP.ini and it would start creating profiling dump files in /tmp. These files over time could get very large if you forget to turn off XDebug. Unfortunately you can't set the XDebug profiling directive, xdebug.profiler_enable directly in PHP with ini_set().

Benchmark would seem like a bit more work compared to XDebug, however, if working with large code bases. The advantage of being able to apply it to a single file however makes it ideal for profiling live php websites. You could easily track down a problem occuring on a live site without having to recreate and simulate the live site and data in a development environment.

For example you could have Benchmark only display profiling data if requested via the URL:

// load class and instantiate and instance
require_once 'Benchmark/Profiler.php';
$profiler = new Benchmark_Profiler();

// start profiling
$profiler->start();

// do some stuff
myFunction();

// stop
$profiler->stop();

// only display if requested by me
if (isset($_GET['debug'])) $profiler->display();

I haven't got around to the other two alternatives for PHP code profiling. I will update this entry if I do.

Tuesday, June 3, 2008

Ringo Ends Service

It looks like Ringo, which is a popular social network, has decided to end its service. Here is part of the email I received 2 hours ago.

Dear Ringo member,

After much consideration we have decided to end the Ringo service.

As of June 30, 2008 the Ringo service is ending and you will no longer have access to your Ringo account. 

Twitter is faster at getting updated with news like this. A search through google won't reveal any news of this since it is only 2 or so hours old, however, a search through twitter got quite a lot of results. http://twittier.com/?q=ringo

The originating server of the email seems to check out. It originates from the tickle.com which is authoritative for ringo's mail transfer. (I consider all email guilty until proven innocent)

Is this the first death of a social network? I believe so, at least a major one.

It appears ringo.com hasn't been growing at all for the past year, if not slowly dying.

Farewell Ringo.