Thursday, January 19, 2012

Get Wikipedia blackout back online via hosts file work around

Wikipedia is on blackout for 24 hours. The blackout code is delivered via JS from the domain meta.wikimedia.org.

To get wikipedia back online during their blackout append "127.0.0.1 meta.wikimedia.org" to your hosts file.

The line:

127.0.0.1 meta.wikimedia.org

will make the domain  meta.wikimedia.org resolve to your loopback interface for IPv4 (your local machine). It will thus NOT render the HTTP response delivering the JavaScript code that will blackout the wikipedia page. Thus you can use wikipedia as normal.

If you're not familiar with what a hosts file is, here is a short overview. http://onwebdevelopment.blogspot.com/2008/06/blocking-advertisements-with-hosts-file.html
Or look here on how to edit your hosts file: http://www.windowsreference.com/windows-7/edit-hosts-file-in-windows-7-windows-vista/

These are for windows, however if you're on Linux then it's a lot simpler - like most things Linux.

echo "127.0.0.1 meta.wikimedia.org" >> /etc/hosts



You need to be root to do this, so either do "sudo" or "su root" or "sudo su" etc.

Other workarounds to wikipedia blackout

If you have foxyproxy you can also use that to block meta.wikimedia.org specifically by specifying that it proxy to some blackhole.

If you use a proxy configuration script, that would also work. Modify your script to proxy meta.wikimedia.org to your favorite blackhole.With a proxy script you can match just the JavaScript file URL, so it is more specific.

There is a number of other ways to block meta.wikimedia.org and they should all deliver the same results. 

If you're interested in the JS file delivering the blackout code then with firebug or chrome, inspect the source and then go to "Network" tab. You should see two HTTP requests for meta.wikimedia.org. The second one is delivering the JavaScript file.You can also use Wireshark to inspect the network traffic and create a Wireshark filter for meta.wikimedia.org. 



Friday, January 13, 2012

Where is httpd.conf - The Apache Configuration File

Are you tired of searching for the Apache Configuration File, httpd.conf?

To find the location of httpd.conf run the following shell command:

httpd -V

Note the -V option is capitalized. Which will give a response similar to:

Server version: Apache/2.2.21 (Unix)
Server built:   Nov 10 2011 19:22:21
Cpanel::Easy::Apache v3.7.1 rev9999
Server's Module Magic Number: 20051115:30
Server loaded:  APR 1.4.5, APR-Util 1.3.12
Compiled using: APR 1.4.5, APR-Util 1.3.12
Architecture:   32-bit
Server MPM:     Event
  threaded:     yes (fixed thread count)
    forked:     yes (variable process count)
Server compiled with....
 -D APACHE_MPM_DIR="server/mpm/experimental/event"
 -D APR_HAS_SENDFILE
 -D APR_HAS_MMAP
 -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
 -D APR_USE_SYSVSEM_SERIALIZE
 -D APR_USE_PTHREAD_SERIALIZE
 -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
 -D APR_HAS_OTHER_CHILD
 -D AP_HAVE_RELIABLE_PIPED_LOGS
 -D DYNAMIC_MODULE_LIMIT=128
 -D HTTPD_ROOT="/usr/local/apache"
 -D SUEXEC_BIN="/usr/local/apache/bin/suexec"
 -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
 -D DEFAULT_ERRORLOG="logs/error_log"
 -D AP_TYPES_CONFIG_FILE="conf/mime.types"
 -D SERVER_CONFIG_FILE="conf/httpd.conf"

The path to httpd.conf can be seen as the HTTPD_ROOT/SERVER_CONFIG_FILE which is /usr/local/apache/conf/httpd.conf in this example.

I've always been using find / -name 'httpd.conf" but this is much faster.

Once you've found httpd.conf you probably want to find the other files being included in the configuration. To do this you can search the httpd.conf file for the "Include" directives.

grep -E "^Include" /usr/local/apache/conf/httpd.conf 

The -E option gives extended regular expression so you can use "^Include" where the "^" character matches the beginning of the line. So only lines beginning with "Include" are returned.

You could follow each included file and recursively follow every included file to get all the included files but this is a good start. Something I also find myself needing is to look at all the configuration files in the apache configuration directory. ie:

find /usr/local/apache/conf -name '*.conf'

That returns every file ending in .conf.

Hope that helps.

Saturday, December 10, 2011

Chrome Socks Proxy using SSH Tunnel

If you’re familiar with using a SOCKS proxy while browsing the internet you may have found out that Chrome will not work with a SOCKS5 proxy such as created when you create a SSH tunnel.

This is because chrome assumes the SOCKS proxy is a SOCKS4 proxy when it is SOCKS5.

A workaround is to have chrome read the proxy configuration from a script.
The configuration script is a javascript file with a function called FindProxyForUrl() that will be called for every HTTP URL to proxy.
Here is an example:


/**
* .pac files are for automated proxy configuration
* This is a fix for chrome to use SOCKS5 as it assumes SOCKS4
*/
function FindProxyForURL(url, host)
{
// no proxy for localhost
if (host.match('localhost') return false;
// proxy for other hosts
return "SOCKS5 localhost:8111";
}


Make sure your port number is the port that the socks proxy is bound to.

Save this script as ssh-tunnel.pac or similar and in chrome go to:
Options -> Under the Hood -> Network -> Change Proxy Settings

Chrome uses the same network settings as IE. So you will see the system window open and choose:
LAN settings -> Use Automatic Configuration Script

Enter the path to the ssh-tunnel.pac file you created.
Now reload the webpage in Chrome.

Creating an SSH tunnel.
If you’re not familiar with creating an SSH Socks5 proxy visit:
Windows SSH as Proxy
Linux SSH as Proxy


Tuesday, December 6, 2011

PHP Command Line Telnet Client

A while ago I wrote a single line PHP Command line client.

while (1) {  fputs(STDOUT, "\n\-PHP$ "); eval(trim(fgets(STDIN))); }

Recently I needed to test an XMPP server and found out that Windows7 does not have telnet enabled by default. Usually I'd just use putty as it supports telnet also but wondered if I could just do this from the command line via PHP. Well here it is, a PHP command line telnet client.

echo "PHP Telnet Client. (c) 2011 Fiji Web Design, http://www.fijiwebdesign.com.\n";

$opts = getopt("h:p:") or die("Invalid options. Please supply -h [host] -p [port]");

$host = $opts['h'];
$port = $opts['p'];

$fp = fsockopen($host, $port) or die("Could not connect to host ($host) on port ($port)");
echo "Connected to server...\n";

stream_set_blocking($fp, 0);

while (1) {

  $input = (fgets(STDIN));
  fwrite($fp, $input) or die('Could not write to server');
  sleep(1); // let server respond

  $out = '';
  while($buf = fread($fp, 2028)) {
    $out .= $buf;
  }

  if ($out != '') echo $out;

}


Now that isn't one line like the PHP command line client. Also, if you are on windows and a version of PHP lower then 5.3, you will not have the getopt() function. To solve this here is a substitute for getopt().

if (!function_exists('getopt')) {

  function getopt($opts) {
      $argv = $_SERVER["argv"];
      $result = false;
      $opts_array = explode(':', $opts);
      foreach($opts_array as $opt) {
          $key = array_search('-' . $opt, $argv);
          if($key && !in_array($argv[$key+1], $opts_array)) {
              $result[$opt] = trim($argv[$key+1]);
          } elseif($key) {
              $result[$opt] = '';
          }
      }
      return $result;
  }

}

Save the PHP code to a file, I call it telnet.php. Then open the shell and navigate to the directory which has telnet.php and type in:

php telnet.php -h [hostname] -p [port]

For example:

php telnet.php -h google.com -p 80

This will open a connection to google.com on the http port. Then you can type in your HTTP headers:

GET / HTTP/1.1
HOST: google.com

Then press enter twice, because HTTP requires that you send a newline to terminate the HTTP headers. Google.com should respond with the headers and HTML of the Google website.

You can telnet into any listening TCP port so for instance you can test XMPP servers.

php telnet.php -h talk.google.com -p 5222

Then send your XMPP stanzas.

Or even telnet into an email server and send or retrieve emails.

Notes


The socket connection to the server your are telneting to is currently non-blocking while the read from STDIN is currently blocking. This causes the response from the server to not show until you hit enter (send \n) a few times. It would be better design to have both streams non-blocking and do a socket_select() call but the current works for now and does not require socket_select() support which I believe requires php built with sockets support. The current implementation should work without that support but does use up a lot of CPU with the while(1) loop.

Sending email from PHP on Windows

I'm assuming you've installed Xampp or WAMP on your local windows machine. If not please visit the XAMPP website. WAMP and XAMPP will install the full windows equivalent of the LAMP (Linux Apache MySQL PHP/Perl/Python) stack on your Windows Machine, and offer a GUI to manage it. Here I'll talk about sending email from windows specifically on XAMPP, but you can follow this for WAMP or your own PHP setup.

I'm also assuming you're trying to send email through the PHP function "mail()".

Unlike Linux, Windows is not distributed with a default mail transfer agent (MTA). Linux flavors usually come with sendmail, which acts as a local MTA.

In order to have PHP send email, your windows machine should be able to send email, and have PHP configured to send email through it. So first you need to get your machine to send email.

You can either install an MTA locally (not recommended since sending emails is a complicated by spam detection mechanisms built into the protocol).
http://www.google.com/Top/Computers/Software/Internet/Servers/Mail/
http://en.wikipedia.org/wiki/List_of_mail_servers

Or use a free MTA such as hotmail.com or gmail.com. You need an account.

To use a free MTA, you need to specify the MTA host, port, user and password. Luckily there are also sendmail like programs for windows and one is included with Xampp that makes this easy.

You will just need to configure PHP to use the sendmail binary provided by Xampp, to send emails. To do this edit the PHP configuration file, php.ini. You can find this in xampp by opening the control panel, clicking "explore" and going to the folder "php". The full path should be something like:

c:/xampp/php/php.ini

Look for:
;sendmail_path = "\"C:\xampp\sendmail\sendmail.exe\" -t"

and remove the ";" from the beginning of that line, to enable that configuration directive.

Now your PHP is configured to use the sendmail program that comes with Xampp for win.

You now need to configure your sendmail program to send email to your SMTP server.

So open up the sendmail.ini in the "sendmail" folder.
c:\xampp\sendmail\sendmail.ini

Create a new account configuration. Example for gmail:

# Gmail example
account Gmail
tls on
tls_certcheck off
host smtp.gmail.com
from myuser@gmail.com
auth on
user myuser@gmail.com
password mypassword

Substitute myuser and mypassword for your details.

Now you need to make this account the default by editing the last line in the file to:

# Set a default account
account default : Gmail

You will then need to restart the apache service. You can do this from the Xampp control panel. (stop/start). This reloads the configuration for PHP.

Now you should be able to send email.

Monday, October 13, 2008

JavaScript XMLHttpRequest Wrapper

Here is my version of the XMLHttpRequest Wrapper, Open Source, and available on Google Code. Enjoy.

Here is a simple example of a http request with the lib:

new fiji.xhr('get', 'echo.php?param=value', function(xhr) {
   if (this.readyState == 4 && this.status == 200) {
      alert(this.responseText);
   }
}).send();

The readystate callback handler executes in the scope of the XHR Instance. So you can use the this keyword to reference the XHR object instance. This behavior is the same as the specifications for the callback scope in the W3C XMLHttpRequest Specs. You can also receive a reference to the XHR library instance which is passed as the first parameter to the callback. In the case above it would be xhr. This allows you to attach further references or objects to the XHR library instance that would be persisted for the duration of the XHR call. For example, a request ID.

Tuesday, August 26, 2008

Quoting Strings in SQLite with PHP

Unlike MySQL, SQLite follows the quoting standards in SQL strictly and does not understand the backslash \ as an escape character. SQLite only understands escaping a single quote with another single quote.

For example, if you receive the input data 'cheeky ' string' and use the PHP function addslahes() to escape literal characters in the string then you will get 'cheeky \' string' which according to SQLite is not escaped properly. You need to escape the string so that it looks like 'cheeky '' string'.

If you have magic_quotes turned on then you are in even more trouble. This PHP setting escapes all HTTP variables received by PHP with an equivalent of addslshes(). So the correct way to escape strings in SQLite would be:

function sqlite_quote_string($str) {
 if (get_magic_quotes_gpc()) {
  $str = stripslashes($str);
 }
 return sqlite_escape_string($str);
}
This will remove the escape characters added by the magic_quotes setting, and escape strings with SQLites sqlite_escape_string() function which correctly escapes the string with '.

Creating a custom SQLite Function in PHP

SQLite is available in PHP5 either by compiling PHP5 with SQLite support or enabling the SQLite extension dynamically from the PHP configuration (PHP.ini). A distinct feature of SQLite is that it is an embedded database, and thus offers some features a Server/Hosted database such as the popular MySQL database doesn't.

Creating Custom Functions in SQLite

One of the really cool features of SQLite in PHP is that you can create custom PHP functions, that will be called by SQLite in your queries. Thus you can extend the SQLite functions using PHP.

A custom Regexp function for SQLite in PHP

// create a regex match function for sqlite
sqlite_create_function($db, 'REGEX_MATCH', 'sqlite_regex_match', 2);
function sqlite_regex_match($str, $regex) {
 if (preg_match($regex, $str, $matches)) {
  return $matches[0];
 }
 return false;
}
The above PHP code will create a custom function called REGEX_MATCH for the SQLite connection referenced by $db. The REGEX_MATCH SQLite function is implemented by the sqlite_regex_match user function we define in PHP.

Here is an example query that makes use of the custom function we created. Notice that in the SQLite query, we call our custom function REGEX_MATCH:

$query = 'SELECT REGEX_MATCH(link, \'|http://[^/]+/|i\') AS domain, link, COUNT(link) AS total'
 .' FROM links WHERE domain != 0'
 .' GROUP BY domain'
 .' LIMIT 10';
$result = sqlite_query($db, $query);
This will make SQLite call the PHP function sqlite_regex_match for each database table row that is goes over when performing the select query, sending it the link field value as the first parameter, and the regular expression string as the second parameter. PHP will then process the function and return its results to SQLite, which continues to the next table row.

Custom Functions in SQLite compared to MySQL

In comparison with MySQL, you cannot create a custom function in PHP that mysql will use. MySQL allows creation of custom functions, but they have to be written in MySQL. Thus you cannot extend MySQL's query functionality with PHP.

I believe the reason for this is simply because having a callback function called on the client, by the database, over a Client-Server model for each row that has to be processed would be just inefficient. Imaging processing 100,000 rows in a MySQL database and having MySQL make a callback to PHP over a TCP connection, the overhead of sending the data back and forth for the callback would be way too much.
With an embedded database like SQLite, this isn't the case since making the actual communication between the language and the embedded database does not pose such a high overhead.

Monday, August 25, 2008

PHP Email Address validation through SMTP

Here is a PHP class written for PHP4 and PHP5 that will validate email addresses by querying the SMTP (Simple Mail Transfer Protocol) server. This is meant to complement validation of the syntax of the email address, which should be used before validating the email via SMTP, which is more resource and time consuming.

Update: Sept 8, 2008

The class has been updated to work with Windows MTA's such as Hotmail and many other fixes have been made. See changes. The class will no longer get you blacklisted by Hotmail due to improper HELO procedure.

Update: Sept 10, 2008

Window Support Added through Net_DNS (pear DNS class). Added support for validating multiple emails on the same domain through a single Socket. Improved the Email Parsing to support literal @ signs.

Update: Sept 29, 2008

The code for this project has been moved to Google Code. The latest source can be grabbed from SVN.

Update: Nov 22, 2008

SMTP Email Validation Class has been added to the Yii PHP Framework. http://www.yiiframework.com/. Yii is a high-performance component-based PHP framework for developing large-scale Web applications.

<?php
 
 /**
 * Validate Email Addresses Via SMTP
 * This queries the SMTP server to see if the email address is accepted.
 * @copyright http://creativecommons.org/licenses/by/2.0/ - Please keep this comment intact
 * @author gabe@fijiwebdesign.com
 * @contributers adnan@barakatdesigns.net
 * @version 0.1a
 */
class SMTP_validateEmail {

 /**
  * PHP Socket resource to remote MTA
  * @var resource $sock 
  */
 var $sock;

 /**
  * Current User being validated
  */
 var $user;
 /**
  * Current domain where user is being validated
  */
 var $domain;
 /**
  * List of domains to validate users on
  */
 var $domains;
 /**
  * SMTP Port
  */
 var $port = 25;
 /**
  * Maximum Connection Time to an MTA 
  */
 var $max_conn_time = 30;
 /**
  * Maximum time to read from socket
  */
 var $max_read_time = 5;
 
 /**
  * username of sender
  */
 var $from_user = 'user';
 /**
  * Host Name of sender
  */
 var $from_domain = 'localhost';
 
 /**
  * Nameservers to use when make DNS query for MX entries
  * @var Array $nameservers 
  */
 var $nameservers = array(
 '192.168.0.1'
);
 
 var $debug = false;

 /**
  * Initializes the Class
  * @return SMTP_validateEmail Instance
  * @param $email Array[optional] List of Emails to Validate
  * @param $sender String[optional] Email of validator
  */
 function SMTP_validateEmail($emails = false, $sender = false) {
  if ($emails) {
   $this->setEmails($emails);
  }
  if ($sender) {
   $this->setSenderEmail($sender);
  }
 }
 
 function _parseEmail($email) {
  $parts = explode('@', $email);
 $domain = array_pop($parts);
 $user= implode('@', $parts);
 return array($user, $domain);
 }
 
 /**
  * Set the Emails to validate
  * @param $emails Array List of Emails
  */
 function setEmails($emails) {
  foreach($emails as $email) {
  list($user, $domain) = $this->_parseEmail($email);
  if (!isset($this->domains[$domain])) {
    $this->domains[$domain] = array();
  }
  $this->domains[$domain][] = $user;
 }
 }
 
 /**
  * Set the Email of the sender/validator
  * @param $email String
  */
 function setSenderEmail($email) {
 $parts = $this->_parseEmail($email);
 $this->from_user = $parts[0];
 $this->from_domain = $parts[1];
 }
 
 /**
 * Validate Email Addresses
 * @param String $emails Emails to validate (recipient emails)
 * @param String $sender Sender's Email
 * @return Array Associative List of Emails and their validation results
 */
 function validate($emails = false, $sender = false) {
  
  $results = array();

  if ($emails) {
   $this->setEmails($emails);
  }
  if ($sender) {
   $this->setSenderEmail($sender);
  }

  // query the MTAs on each Domain
  foreach($this->domains as $domain=>$users) {
   
  $mxs = array();
  
   // retrieve SMTP Server via MX query on domain
   list($hosts, $mxweights) = $this->queryMX($domain);

   // retrieve MX priorities
   for($n=0; $n < count($hosts); $n++){
    $mxs[$hosts[$n]] = $mxweights[$n];
   }
   asort($mxs);
 
   // last fallback is the original domain
   array_push($mxs, $this->domain);
   
   $this->debug(print_r($mxs, 1));
   
   $timeout = $this->max_conn_time/count($hosts);
    
   // try each host
   while(list($host) = each($mxs)) {
    // connect to SMTP server
    $this->debug("try $host:$this->port\n");
    if ($this->sock = fsockopen($host, $this->port, $errno, $errstr, (float) $timeout)) {
     stream_set_timeout($this->sock, $this->max_read_time);
     break;
    }
   }
  
   // did we get a TCP socket
   if ($this->sock) {
    $reply = fread($this->sock, 2082);
    $this->debug("<<<\n$reply");
    
    preg_match('/^([0-9]{3}) /ims', $reply, $matches);
    $code = isset($matches[1]) ? $matches[1] : '';
 
    if($code != '220') {
     // MTA gave an error...
     foreach($users as $user) {
      $results[$user.'@'.$domain] = false;
  }
  continue;
    }

    // say helo
    $this->send("HELO ".$this->from_domain);
    // tell of sender
    $this->send("MAIL FROM: <".$this->from_user.'@'.$this->from_domain.">");
    
    // ask for each recepient on this domain
    foreach($users as $user) {
    
     // ask of recepient
     $reply = $this->send("RCPT TO: <".$user.'@'.$domain.">");
     
      // get code and msg from response
     preg_match('/^([0-9]{3}) /ims', $reply, $matches);
     $code = isset($matches[1]) ? $matches[1] : '';
  
     if ($code == '250') {
      // you received 250 so the email address was accepted
      $results[$user.'@'.$domain] = true;
     } elseif ($code == '451' || $code == '452') {
   // you received 451 so the email address was greylisted (or some temporary error occured on the MTA) - so assume is ok
   $results[$user.'@'.$domain] = true;
     } else {
      $results[$user.'@'.$domain] = false;
     }
    
    }
    
    // quit
    $this->send("quit");
    // close socket
    fclose($this->sock);
   
   }
  }
 return $results;
 }


 function send($msg) {
  fwrite($this->sock, $msg."\r\n");

  $reply = fread($this->sock, 2082);

  $this->debug(">>>\n$msg\n");
  $this->debug("<<<\n$reply");
  
  return $reply;
 }
 
 /**
  * Query DNS server for MX entries
  * @return 
  */
 function queryMX($domain) {
  $hosts = array();
 $mxweights = array();
  if (function_exists('getmxrr')) {
   getmxrr($domain, $hosts, $mxweights);
  } else {
   // windows, we need Net_DNS
  require_once 'Net/DNS.php';

  $resolver = new Net_DNS_Resolver();
  $resolver->debug = $this->debug;
  // nameservers to query
  $resolver->nameservers = $this->nameservers;
  $resp = $resolver->query($domain, 'MX');
  if ($resp) {
   foreach($resp->answer as $answer) {
    $hosts[] = $answer->exchange;
    $mxweights[] = $answer->preference;
   }
  }
  
  }
 return array($hosts, $mxweights);
 }
 
 /**
  * Simple function to replicate PHP 5 behaviour. http://php.net/microtime
  */
 function microtime_float() {
  list($usec, $sec) = explode(" ", microtime());
  return ((float)$usec + (float)$sec);
 }

 function debug($str) {
  if ($this->debug) {
   echo htmlentities($str);
  }
 }

}

 
?>

Using the PHP SMTP Email Address Validation Class

Example Usage:

// the email to validate
$email = 'joe@gmail.com';
// an optional sender
$sender = 'user@example.com';
// instantiate the class
$SMTP_Valid = new SMTP_validateEmail();
// do the validation
$result = $SMTP_Valid->validate($email, $sender);
// view results
var_dump($result);
echo $email.' is '.($result ? 'valid' : 'invalid')."\n";

// send email? 
if ($result) {
  //mail(...);
}

Code Status

This is a very basic, and alpha version of this php class. I just wrote it to demonstrate an example. There are a few limitations. One, it is not optimized. Each email you verify will create a new MX DNS query and a new TCP connection to the SMTP server. The DNS query and TCP socket is not cached for the next query at all, even if they are to the same host or the same SMTP server.
Second, this will only work on Linux. Windwos does not have the DNS function needed. You could replace the DNS queries with the Pear Net_DNS Library if you need it on Windows.

Limitations of verifying via SMTP

Not all SMTP servers are configured to let you know that an email address does not exist on the server. If the SMTP server does respond with an "OK", it does not mean that the email address exists. It just means that the SMTP server will accept the email address and not bounce it. What it does with the actual email is different. It may deliver it to the recipient, or it may just send it to a blackhole.
If you get an invalid response from the SMTP server however, you can be pretty sure your email will bounce if you actually send it.
You should also NOT use this class to try and guess emails, for spamming purposes. You will quickly get blacklisted on Spamhaus or a similar list.

Good uses of verifying via SMTP

If you have forms such as registration forms, where users enter their email addresses. It may be a good idea to first check the syntax of the email address, to see if it is valid as per the SMTP protocol specifications. Then if it is valid, you may want to verify that the email will be accepted (will not bounce). This can allow you to notify the user of a problem with their email address, in case they made a typo, knowingly entered an invalid email. This could increase the number of successful registrations.

How it works

If you're interested in how it works, it is quite simple. The class will first take an email, and separate it to the user and host portions. The host portion, tells us which domain to send the email to. However, a domain may have an SMTP server on a different domain so we retrieve a list of SMTP servers that are available for the domain by doing a DNS query of type MX on that domain. We receive a list of SMTP servers, so we iterate through each trying to make a connection. Once connected, we send SMTP commands to the SMTP server, first saying "HELO", then setting our sender, then our recipient. If the recipient is rejected, we know an actual sending of an email will fail. Thus, we close the TCP connection to the SMTP server and quit.

Thursday, August 21, 2008

XSS (Cross Site Scripting) and stealing passwords

XSS (Cross Site Scripting) would be viewed by most web developers as the stealing of users session cookies by injecting JavaScript into a web page through URL. You do not associate it with stealing passwords, but worse then stealing session cookies, it can steal a users username and password directly from the browser.

Many users choose to have the browser remember their login credentials. So when ever they visit a login form, their username and password fields are pre-populated by the browser. Now if there is an XSS vulnerability on that login page, then a remote attacker can successfully retrieve the users username and password.

Hello World in XSS

You have a page that has an XSS vulnerability. Let say a website has a PHP page, mypage.php with the code:

<?php

// the variable is returned raw to the browser
echo $_GET['name'];

?>
Because the variable $_GET['name'] is not encoded into HTML entities, or stripped of HTML, it has an XSS vulnerability. Now all an attacker has to do is create a URL that a victim will click, that exploits the vulnerability.
mypage.php?name=%3Cscript%3Ealert(document.cookie);%3C/script%3E
This basically will make PHP write <script>alert(document.cookie);</script> onto the page, which displays a modal dialog with the value of the saved cookies for that domain.

How Does stealing passwords with XSS work?

The example above displays the cookies on the domain the webpage is on. Now imagine the same page has a login form, and the user chose to have their passwords remembered by the browser. Lets say the PHP page looks like this:

<?php

// the variable is returned raw to the browser
echo $_GET['name'];

?>

<form action="login.php">
<input type="text" name="username" />
<input type="password" name="password" />
<input type="submit" value="Login" />
</form>

Now an attacker just needs to craft a URL that retrieves the username and password. Here is an example that retrieves the password:
mypage.php?name=%3Cscript%3Ewindow.onload=function(){alert(document.forms[0].password);}%3C/script%3E

As you can see, it is just a normal XSS exploit, except it is applied to the username and password populated by the browser after the window.onload event.

Password stealing XSS vs Session Cookie stealing XSS

Well, they are both suck from a developers perspective. According to Wikipedia, 70% or so of websites are vulnerable to XSS attacks.

As a developer, I've always thought of XSS as an exploit on a users session, just as CSRF/XSRF (Cross Site Request Forgery), which requires an active session. Now, as you can see, XSS of the type described does NOT require an active session. The user does not have to be logged into the site. They could have logged out 10 years ago, but as long as the browser remembers their login credentials, the XSS exploit can steal those login credentials.

Due to its ability to be executed without having the user logged into a website, this exploit should be regarded worse then session based XSS.

Proof of Concept

Fill in the form below with dummy values and click the "Login" button.

Login Form
Username:
Password:

Now return to the same page, to simulate logging out. Now click the Exploit. This will simulate an XSS exploit on this page, and alert the saved password.

I've set up a proof of concept based on an actual XSS exploit here: http://xss-password.appjet.net/.

Preventing Stealing Passwords via XSS

The only way I can think of right now is to give your username and password fields unique names so that the browser does not remember their values. In PHP you can do this with the time() function. eg:

<input type="password" name="pass[<?php echo sha1(time().rand().'secret'); ?>]" />
The unique names prevents the browser from remembering the password field. This should work universally in all browsers.

Sunday, July 27, 2008

OpenOffice Modal Dialog "JRE Required"

Was trying to edit a chart in OpenOffice Writer on Ubuntu today when it popped up a modal dialog box saying that it needed Java Runtime Environment, JRE, to complete the task (not in those words).

Now even after using the Ubuntu package manager, Aptitude, to install the required software OpenOffice still hangs on the modal dialog, making it impossible to close Open Office or do anything else with it. Only solution is to kill the Open Office process, which is called soffice. ie: pkill soffice.bin. First annoyance I've had with OpenOffice, which apart from this 'lil quirk is the best Document Publisher/Editor - even better then the Microsoft Office Suite.

It appears this bug is already addressed http://qa.openoffice.org/issues/show_bug.cgi?id=74940.

Friday, July 11, 2008

BlogPluse Trends

BlogPulse offers a trend search very similar to Google Trends but specifically targeted at blogs.

Here is the blog trend for the words Joomla, Drupal and Wordpress over the last 6 months.
This shows the percentage of the mentions of each word, Joomla, Drupal and Wordpress in blogs.

Lively - Googles 3D Virtual World

Just came across Lively.com which developed by Google, creates a virtual 3D world similar to SecondLife.

I haven't tested out Lively yet, since I'm running Ubuntu and it only supports Windows at the moment. In one of their blog posts however we can gather that Lively is well integrated with the Web. They have widgets that allows visitors - with Lively software installed - to jump into a Lively room embedded in a webpage.

By contrast I believe SecondLife offer APIs that offers a REST interface as well as other network level interfacing. Here is a SecondLife Facebook Application.

I'm wondering if Google will offer the same level of integration with their Lively 3D world. Would make fun Mashups.

Friday, July 4, 2008

Chaining Functions in JavaScript

A lil snippet I borrowed from the Google AJAX libraries API:

chain = function(args) {
    return function() {
     for(var i = 0; i < args.length; i++) {
      args[i]();
     }
    }
   };
You're probably familiar with the multiple window onload method written by Simon Willison which creates a closure of functions within functions in order to chain them.
Here is the same functionality using the chain function defined above.
function addLoad(fn) {
    window.onload = typeof(window.onload) != 'function' ? fn : chain([onload, fn]);
};
Chain can also be namespaced to the Function.prototype if thats how you like your JS.
Function.prototype.chain = function(args) {
    args.push(this);
    return function() {
     for(var i = 0; i < args.length; i++) {
      args[i]();
     }
    }
   };
So the multiple window onload function would be:
window.addLoad = function(fn) {
    window.onload = typeof(window.onload) != 'function' 
    ? fn : window.onload.chain([fn]);
   };

Hacking Google Loader and the AJAX Libraries API

Google recently released the AJAX Libraries API which allows you to load the popular JavaScript libraries from Google Servers. The benefits of this outlined in the description of the API.

The AJAX Libraries API is a content distribution network and loading architecture for the most popular open source JavaScript libraries. By using the google.load() method, your application has high speed, globally available access to a growing list of the most popular JavaScript open source libraries.

I was thinking of using it for a current project that would use JS heavily, however, since the project used a CMS (Joomla) the main concern for me was really how many times MooTools would be loaded. Joomla uses a PHP based plugin system (which registers observers of events triggered during Joomla code execution) and the loading of JavaScript by multiple plugins can be redundant as there is no central way of knowing which JavaScript library has already been loaded, nor is there a central repository for JavaScript libraries within Joomla.

MooTools is the preferred library for Joomla and in some cases it is loaded 2 or even 3 times redundantly. I did not want our extension to add to that mess. To solve the problem I would test for the existence of MooTools, if (typeof(MooTools) == 'undefined') and load it from Google only if it wasn't available. Now this would have worked well, however, I would have to add the JavaScript for AJAX Libraries API and it would only be loading 1 script, "MooTools", when I also had about 3-4 other custom libraries that I wanted loaded.

Now I thought, why don't I develop a JavaScript loader just like the Google AJAX Libraries API Loader. Should be just a simple function to append a Script element to the document head. So I started with:

function loadJS(src) {
    var script = document.createElement('script');
    script.src = src;
    script.type = 'text/javascript';
    timer = setInterval(closure(this, function(script) {
     if (document.getElementsByTagName('head')[0]) {
      clearTimeout(timer);
      document.getElementsByTagName('head')[0].appendChild(script);
     }
    }, [script]), 50);
   }
function closure(obj, fn, params) {
    return function() {
     fn.apply(obj, params);
    };
   }
The function loadJS would try to attach a script element to the document head, each 50 milliseconds until it succeeded.

This works but there is no way of knowing when the JavaScript file was fully loaded. Normally, the way to figure out if a JS file has finished loading from the remote server, is to have the JS file invoke a callback function on the Client JavaScript (aka: JavaScript Remoting). This however means you have to build a callback function into each JavaScript file, which is not what I wanted.

So to fix this problem I though I'd add another Interval with setInterval() to detect when the remote JS file had finished loading by testing a condition that exits when the file has completed. eg: for MooTools it would mean that the Object window.MooTools existed.

So I went about writing a JavaScript library for this, with a somewhat elaborate API, with JS libraries registering their "load condition test" and allowing their remote loading, about 1 wasted hour, (well not wasted if you learn something) only to realize that this wouldn't work for the purpose either. The reason is that it broke the window.onload functionality. Some remote files would load before the window.onload event (cached ones) and others after. This made the JavaScript already written to rely on window.onload fail.

Last Resort, how did Google Do it? I had noted earlier that if you load a JavaScript file with Google's API the file would always load before the window.onload method fired. Here is the simple test: (In the debug output, the google callback always fired first).

google.load("prototype", "1");
   google.load("jquery", "1");
   google.load("mootools", "1");
   google.setOnLoadCallback(function() {
    addLoad(function() {
     debug('google.setOnLoadCallback - window.onload');
    });
    debug('google.setOnLoadCallback')
   });
   addLoad(function() {
    debug('window.onload');
   });
   debug('end scripts');
I had to take a look at the source code for Google's AJAX Libraries API which is: http://www.google.com/jsapi to see how they achieved this.

It never occurred to me that you could force the browser to load your JavaScript before the window.onload event so I was a bit baffled. Browsing through their source code I came upon what I was looking for:

function q(a,b,c){if(c){var d;if(a=="script"){d=document.createElement("script");d.type="text/javascript";d.src=b}else if(a=="css"){d=document.createElement("link");d.type="text/css";d.href=b;d.rel="stylesheet"}var e=document.getElementsByTagName("head")[0];if(!e){e=document.body.parentNode.appendChild(document.createElement("head"))}e.appendChild(d)}else{if(a=="script"){document.write('<script src="'+b+'" type="text/javascript"><\/script>')}else if(a=="css"){document.write('<link href="'+b+'" type="text/css" rel="stylesheet"></link>'
)}}}
The code has been minified, so its a bit hard to read. Basically its the same as any javascript remoting code you'd find on the net, the but the part that jumps out is:
var e=document.getElementsByTagName("head")[0];
if(!e){e=document.body.parentNode.appendChild(document.createElement("head"))}
e.appendChild(d)
Notice how it will create a head Node and append it to the parentNode of the document body if the document head head does not exist yet.

Now that forces the browser to load the JavaScript right then, no matter what. Now following that method you can load remote JavaScript files dynamically and just used the regular old window.onload event or "domready" event and the files will be available.

Apparently this won't work on all browsers, since Google's code also has the alternative:

document.write('<script src="'+b+'" type="text/javascript"><\/script>')
with a bit of testing, you could discern which browsers worked with which and use that. I'd imagine that the latest browsers would accept the dom method and older ones would need the document.write

So my JavaScript file loading function became:

function loadJS(src) {
    var script = document.createElement('script');
    script.src = src;
    script.type = 'text/javascript';
    var head = document.getElementsByTagName('head')[0];
    if (!head) {
     head = document.body.parentNode.appendChild(document.createElement('head'));
    }
    head.appendChild(script);
    
   }

Anyways, I finally got my JavaScript library loader working just as I liked, thanks to the good work done by Google with the AJAX Libraries API.

Secure HTTP over SSH proxy with Linux

In an previous post I made I detailed how to create a secure your browser's HTTP communications by tunneling the HTTP session over an SSH proxy using Putty.

Putty is what you would use if you use a Windows desktop. If you're on a Linux Desktop you do not need Putty since you should have OpenSSH with the distribution you use.

Doing a man ssh on your Linux Desktop should give you the manual on how to use your SSH client:

SSH(1)                                                         BSD General Commands Manual                                                         SSH(1)

NAME
     ssh - OpenSSH SSH client (remote login program)

SYNOPSIS
     ssh [-1246AaCfgKkMNnqsTtVvXxY] [-b bind_address] [-c cipher_spec] [-D  [bind_address:]port] [-e escape_char] [-F configfile] [-i identity_file] [-L
         [bind_address:]port:host:hostport] [-l login_name] [-m mac_spec] [-O ctl_cmd] [-o option] [-p port] [-R  [bind_address:]port:host:hostport]
         [-S ctl_path] [-w local_tun[:remote_tun]] [user@]hostname [command]

... etc ...
The synopsis gives you the format of the command and the options that can be used with the ssh command. Of interest is the -D option. This allows you to bind the SSH session to a local address and port. Below is the part of the manual explaining the D option:
     -D [bind_address:]port
             Specifies a local “dynamic” application-level port forwarding.  This works by allocating a socket to listen to port on the local side,
             optionally bound to the specified bind_address.  Whenever a connection is made to this port, the connection is forwarded over the secure
             channel, and the application protocol is then used to determine where to connect to from the remote machine.  Currently the SOCKS4 and
             SOCKS5 protocols are supported, and ssh will act as a SOCKS server.  Only root can forward privileged ports.  Dynamic port forwardings can
             also be specified in the configuration file.

             IPv6 addresses can be specified with an alternative syntax: [bind_address/]port or by enclosing the address in square brackets.  Only the
             superuser can forward privileged ports.  By default, the local port is bound in accordance with the GatewayPorts setting.  However, an
             explicit bind_address may be used to bind the connection to a specific address.  The bind_address of “localhost” indicates that the listen‐
             ing port be bound for local use only, while an empty address or ‘*’ indicates that the port should be available from all interfaces.
Basically it means that you can start an SSH session using the OpenSSH client with a command such as:
ssh -D localhost:8000 user@example.com
and it will create a SOCKS proxy on port 8000 that will tunnel your HTTP connection over SSH to the server at example.com under the username user.

Now you can configure your applications that access the internet to use the secure HTTP tunnel you've created to your remote SSH server. The applications are not limited to web browsers, you can configure your Instant Messenger, Skype, Games etc. to use the socks proxy, as long at the communication protocol is supported.

Configuring Firefox to use the Socks Proxy

  • Tools -> Options -> Advanced -> Network
  • Under Connection click on the Settings button
  • Choose Manual Proxy configuration, and SOCKS v5
  • Fill in localhost for the host, and 8000 (or the port number you used) for the port
  • Click OK and reload the page

Now what you can do is have the the ssh session start up when you start your desktop. Thats if you want to use your secure tunnel every time you use Firefox or whatever program you have configured to use it. On Ubuntu (Debian) you'd add a shell script to your home directory.
Example:

#!/bin/sh
ssh -D localhost:8000 user@example.com
That should start up the ssh connection and create the socks proxy when you log in. The other alternative is to create a launcher and use ssh -D localhost:8000 user@example.com as the command, allowing you to launch the proxy whenever you need.

You can also set up an ssh key for authentication instead of having to log in. This is detailed in other posts: http://pkeck.myweb.uga.edu/ssh/ and http://sial.org/howto/openssh/publickey-auth/. This allows you to use the proxy transparently in the background without having to start it and log in.

For Firefox you can switch between proxy and direct connection using the switchproxy extension.

Disclaimer: Please note that it is your responsibility to use the information in this article within the legal laws of your country. Some countries do not allow encryption of internet traffic, therefore you SHOULD NOT use this resource if you live in such a country. I provide this information without warranty and free of charge and will not be held accountable for any damages lost due to its use.. etc etc.

Monday, June 23, 2008

Google Trends for websites

The Google trends for websites, which was released by Google 3 days ago, is really something to check out if you're interested in comparing website metrics between different websites and across geographical locations.

Alexa and Compete offer similar services. One of the features that stands out with Google trends is that it displays related websites that visitors to the website being viewed visit in descending order of visitors. This related websites are also filtered when filtering down to single geographic locations. This makes Google trends for websites quite a bit more powerful for your research. It allows you to view a websites competitors in each geographic location or allows link building and search engine optimization for a website for each geographic location.

Joomla vs Wordpress on Google Trends for websites

Joomla vs Wordpress on Alexa Website Analytics

Joomla vs Wordpress on Compete Website Analytics

Search Engine Land and Matt Cutts also blogged about the new google trends for websites.

Sunday, June 22, 2008

Online Application Development Platforms and Services

Everyone is offering Application development as a service!

Appjet

AppJet is the easiest way to create a web app. Just type some code into a box, and we'll host your app on our servers.
http://appjet.com/

AppPad

AppPad provides a place to create web applications completely in HTML and Javascript. AppPad gives you:
http://apppad.com/

Bungee Connect

Bungee Connect is the most comprehensive Platform-as-a-Service (PaaS) — significantly reducing the complexities, time and costs required to build and deliver web applications.
http://www.bungeelabs.com/

CogHead

Coghead is a 100% web-based system that allows knowledge workers to create their own custom business applications. There’s never any software to install or servers to maintain. Just think it, build it and share it!
http://www.coghead.com/

Google App Engine

Google App Engine enables you to build web applications on the same scalable systems that power Google applications.
http://code.google.com/appengine/

I was going to add more but got bored after reaching G. Now I'm just waiting for that meta web application development environment that integrates all the above with a RESTful API. lol...

Sunday, June 15, 2008

JavaScript Cross Window Communication via Cookies

Using JavaScript we can communicate between browser windows given that we have a reference to each window. When a new window is created the the JavaScript method window.open() it returns a reference to the new window. The child window, also has a reference to the parent window that created it via the window.opener window object. These references allow the two windows to communicate with and manipulate each other.

There are times however, when we need to communicate with an open window for which there is no window object reference. A typical example of this is when a popup window is created, then the parent window is reloaded. When reloading the parent, all JavaScript objects are "trashed", along with the reference to the open popup window. Here is where cookies come into play - they are not trashed.

The problem with cookies is that it only saves strings, so you can't write a reference to a window object directly to a cookie, since serializing the reference is not possible. However, since both the child window and the parent window are able to read and write cookies, then they have a medium for which they can communicate. Even if the medium only allows strings.

To demonstrate how communicating between windows with cookies would work, lets assume we want to open a window, and then close it a few seconds later.

var win = window.open('child.html');
setTimeout(function() { win.close(); }, 5000);
The code will open a child window, and close it after 5 seconds using the reference to the child window and the method close(). However if we didn't have a reference for some reason, we would not be able to invoke the close method. So lets see how it could be done with cookies:
window.open('child.html');
setTimeout(function() { setCookie('child', 'close'); }, 5000);
Here we open a window but do not save a reference. Then after 5 seconds we write 'close', to the cookie named 'child' (using the pseudo setCookie() function). This does not do anything by itself, but if the child window was expecting the cookie, it could close itself when it read 'close'. Lets assume the following JS is in child.html.
// child.html
setInterval(function() { getCookie('child') == 'close' ? this.close() : ''; }, 500);
This would check the cookie every half a seconds and close the window if the cookie read 'close'.

Using this method we can send any commands to any open windows and have them execute it without having a reference to that window.

Friday, June 6, 2008

Blocking Advertisements with a Hosts file, Apache and PHP

The Hosts file is located at /etc/hosts on Linux and %SystemRoot%\system32\drivers\etc\ on Windows XP and Vista. It maps host names to IP addresses and takes precedence over the DNS server. So if you add an entry in your hosts file:

207.68.172.246 google.com 
Then every time you type google.com you will be taken to msn.com instead, since 207.68.172.246 is the IP address of msn.com.

Knowing this, you can point any domain, to an IP address of choice using the Hosts file. Therefore, we can use it to block any domains that hosts unwanted advertising or malware.

Modifying your Hosts File to block Advertisements and Malware

There are many sites offering host files which block advertisments and malware. I use the one on http://www.mvps.org/winhelp2002/hosts.htm.
Here is the txt version of the hosts file: http://www.mvps.org/winhelp2002/hosts.txt

Here is an example of what the entries look like, there the list contains a lot more, about 18, 000 entries at this time.

# [Misc A - Z]
127.0.0.1  ad.a8.net
127.0.0.1  asy.a8ww.net
127.0.0.1  www.abx4.com #[Adware.ABXToolbar]
127.0.0.1  acezip.net #[SiteAdvisor.acezip.net]
127.0.0.1  www.acezip.net #[Win32/Adware.180Solutions]
127.0.0.1  phpadsnew.abac.com
127.0.0.1  a.abnad.net
127.0.0.1  b.abnad.net
127.0.0.1  c.abnad.net #[eTrust.Tracking.Cookie]
127.0.0.1  d.abnad.net
127.0.0.1  e.abnad.net
127.0.0.1  t.abnad.net
127.0.0.1  banners.absolpublisher.com
127.0.0.1  tracking.absolstats.com
127.0.0.1  adv.abv.bg
127.0.0.1  bimg.abv.bg
127.0.0.1  www2.a-counter.kiev.ua
127.0.0.1  accuserveadsystem.com
127.0.0.1  www.accuserveadsystem.com
127.0.0.1  gtb5.acecounter.com
127.0.0.1  gtcc1.acecounter.com
127.0.0.1  gtp1.acecounter.com #[eTrust.Tracking.Cookie]
127.0.0.1  acestats.com
127.0.0.1  www.acestats.com
127.0.0.1  achmedia.com
127.0.0.1  ads.active.com
127.0.0.1  am1.activemeter.com
127.0.0.1  www.activemeter.com #[eTrust.Tracking.Cookie]
127.0.0.1  ads.activepower.net
127.0.0.1  stat.active24stats.nl #[eTrust.Tracking.Cookie]
127.0.0.1  web.acumenpi.com #[AdvertPro]
127.0.0.1  ad.ad24.ru
127.0.0.1  at.ad2click.nl
127.0.0.1  cms.ad2click.nl
127.0.0.1  banner.ad.nu
127.0.0.1  ad-up.com
127.0.0.1  www.ad-up.com
You will need to download the txt file and append the entries to your hosts file.

Now once the hosts file is in effect, when you browse any website in firefox or IE or any other browser, 99% of the advertisements will not be displayed.

Setting up Apache to display a custom page or message for blocked Advertisements and Malware

Each entry in the hosts file blocks unwanted sites by resolving their domain name to 127.0.0.1 which is the IP reserved for looping back to your own IP. So all the requests for advertising sites will instead be made back to your IP. The problem with this is because there is no website on your localhost, then the browser will display an error in place of the ads.

If you're a web developer, you'll likely have a version of Apache or some other HTTP server running on your localhost. So you'll likely get a 404 error in place of the ads. You can resolve this by adding a virtual host entry into your httpd.conf file that will display a custom page instead of the 404.

To resolve this you can set up a virtual host to catch all requests made to your Apache server, for the blocked hosts. Assuming you always access your local server via the URL http://localhost/ then you probably don't need the other host possibilities on 127.0.0.1. So your virtual host could look something like:

<VirtualHost 127.0.0.1>
ServerAdmin webmaster@adblock
DocumentRoot /var/www/adblock/
ErrorDocument 404 /404.html
ErrorLog /etc/log/adblock/error.log
TransferLog /etc/log/adblock/access.log
</VirtualHost>
This will catch all requests made to 127.0.0.1. The requests will most likely have a path that doesn't exist in your file structure in /var/www/adblock/ so it will generate a 404 error. You therefore need a custom 404 document which is defined in ErrorDocument 404 /404.html. This can have the simple line, "ad or malware blocked" or something on those lines.

Now localhost also resolves to 127.0.0.1 so you will need to make sure you have a virtual host for the host localhost.

The other thing you could do instead of setting up a virtual host, and it may be simpler, is create a custom 404 document for your current setup. You can do this via a directive directly in httpd.conf like: ErrorDocument 404 /404.php. Notice that it is a PHP so you can use some PHP code to customize the error message. What you'll want is to have the PHP detect if the request was for a blocked site, and if so show your message: "site blocked", but show the regular 404 page for your actual website.

How you detect if the request is from a one of the blocked hosts is by comparing the requested host with the list of hosts in your hosts file that are blocked. The host requested is in the $_SERVER['SERVER_NAME'] variable. Since the list of blocked hosts is large and you probably do not want to read all of those with your php script each time an advertisement is blocked, you can apply the reverse comparison - if the requested host is not in the list of your valid hosts, then it is a blocked host. An example:

// our valid hosts
$valid_hosts = array('localhost', 'my.host.joe', 'my.other.host.peter');
// check if the requested host is a valid one
if (!in_array($_SERVER['SERVER_NAME'], $valid_hosts)) {
    echo 'ad or malware blocked'; // display message in place of blocked ad
} else {
    include('/404.html'); // display regular 404 page
}
Now, when you visit those websites with pesky advertisements and popups, you get a neat little line saying "ad or malware blocked" in the place of those ads.