Reversing Counter Strike Source: nephew cheating and owning me

I have finally managed to persuade my nephew to start blogging about the stuff he is doing - reversing games, cheating and thus owning me totally those few times we do sit down and play together.

It is a good posting in my opinion

Reversing Valve's Source Engine functions: Getting weapon information without using Valve's SDK

SugarCrm: Continous / automated installation of packages

I needed to find a way to faster go from a bug being reported "Development Complete" to it being installed on a test installation for the testers so they could begin testing. We are not hosting the entire sugarcrm site in SVN and also deployment to the final production system is done by a third party so all we can so is send them the package zip files. So it needed to follow the Module Loader principle.

This script is run by cron at an timed interval (I am using 10 minutes).

Disclaimer
The script could be cleaned up by extracting the svn username and password into configuration variables.

The other suboptimal issue is:
When you are installing multiple packages this way and all packages are having post_install.php files defining the same functions you will get an error. The reason is that the post_install.php file currently is just a php file with named functions in it.The correct way to solve this would be to move the post_install.php functions inside a class. However that was not an option for me so I needed to handle the re-including of functions with the same name. To be able to do this I am renaming the already included functions.

#!/usr/bin/php
<?php
/**
* You must install 
* 
* yum install php-pear
* pecl install apd
* 
* and enabled in the php.ini file 
* 
* [Zend]
* zend_extension = /usr/lib/php/modules/apd.so
* 
* for this to work
*/


//make sure we dump the stuff to stderr
ini_set("error_log","php://stderr");

$svnExportDir='/mnt/';
$siteInstalledVersionStateFilesDir='/mnt/';

$svnPackages = array (
 // Define your packages here and the full SVN url
 'mypackage' => 'https://XXX/svn/mypackage/trunk',
 'mypackage2' => 'https://XXX/svn/mypackage2/trunk',
);

$preInstallFunctions = array (
 // If your package are using pre_install hooks and you are installing multiple packages and the pre_install hooks contain the same functions
 // then you need to define them here. This is hack, if the pre_install functions was defined in a class then we could just unset the class.
    "pre_install",
);

$postInstallFunctions = array (
 // If your package are using post_install hooks and you are installing multiple packages and the pre_install hooks contain the same functions
 // then you need to define them here. This is hack, if the post_install functions was defined in a class then we could just unset the class.
    "post_install",
);
 
$sugarcrmInstallationsToUpdate = array (
  0 => array(
    'name' => 'SugarCrmInstallation1 (staging)',
    'sugarInstallDir' => '/var/www/sites/staging',
    'packagesToUpdate' => array (
        0 => 'mypackage',
        1 => 'mypackage2',
    ),
  ),
);


//initialize
if(!defined('sugarEntry'))  define('sugarEntry', true);

// If you are updating more than one installation there is no need to do the svn export more than one time per
// package, just reuse that already downloaded/exported package
$packageDownloadedFromSvnRevisionLog = $siteInstalledVersionStateFilesDir . 'ci_packageDownloadedFromSvnRevision.log';
$packageVersionsDownloaded = getArrayFromFile($packageDownloadedFromSvnRevisionLog);

foreach($sugarcrmInstallationsToUpdate as $sugarCrmSite){
    chdir($sugarCrmSite['sugarInstallDir']);
    require_once('include/entryPoint.php');
    require_once('ModuleInstall/ModuleInstaller.php');
    $current_user = new User();
    $current_user->is_admin = '1';
    
    $siteInstalledVersionsStateFile = $siteInstalledVersionStateFilesDir . $sugarCrmSite['name'] . '_version';
    $revisionNumberInstalledInSite = getArrayFromFile($siteInstalledVersionsStateFile);
    
    foreach($sugarCrmSite['packagesToUpdate'] as $key=>$package){
        $packageRevisionInstalled = array_key_exists($package, $revisionNumberInstalledInSite) ?  $revisionNumberInstalledInSite[$package] : 0;

        ciLog ("Checking for newer versions in svn ...\n", $sugarCrmSite['name'],$package);
        $packageRevisionInSvn = trim(`svn info --non-interactive --username YOUR_SVN_USERNAME --password YOUR_SVN_PASSWORD $svnPackages[$package] | grep 'Last Changed Rev' | head -1 | grep -Eo "[0-9]+"`);
        
        if ($packageRevisionInstalled < $packageRevisionInSvn) {
   ciLog ("There are updates in SVN (installed: $packageRevisionInstalled, svn: $packageRevisionInSvn).\n", $sugarCrmSite['name'],$package);
            
            $packageRevisionPreviouslyDownloaded = array_key_exists($package, $packageVersionsDownloaded) ?  $packageVersionsDownloaded[$package] : 0;
            if ($packageRevisionPreviouslyDownloaded < $packageRevisionInSvn){
    ciLog ("Getting latest version from svn ...\n", $sugarCrmSite['name'],$package);

                $output = `svn export --force --non-interactive --username YOUR_SVN_USERNAME --password YOUR_SVN_PASSWORD $svnPackages[$package] $svnExportDir$package`;
                $packageVersionsDownloaded[$package] = $packageRevisionInSvn;
                saveArrayToFile($packageDownloadedFromSvnRevisionLog, $packageVersionsDownloaded);
            }
            else {
    ciLog ("(rev $packageRevisionInSvn) has previously been downloaded, using local version.\n", $sugarCrmSite['name'],$package);
            }

            //initialize the module installer
            $modInstaller = new ModuleInstaller();
            $modInstaller->silent = true;  //shuts up the javscript progress bar

            //start installation
   ciLog ("(rev $packageRevisionInSvn) Starting installation into " . $sugarCrmSite['sugarInstallDir'] . " ... \n", $sugarCrmSite['name'],$package);
   
   $preInstallFile = "$svnExportDir$package/scripts/pre_install.php";
   if(is_file($preInstallFile))
   {
    ciLog ("Including $preInstallFile.\n", $sugarCrmSite['name'],$package);
    include($preInstallFile);
    ciLog ("Executing $preInstallFile.\n", $sugarCrmSite['name'],$package);
    pre_install();
                
                // Undeclaring the pre_install functions so we can include the next pre_install file. Nasty, this is a hack. Requires PECL ADP 
                undeclareFunctions($preInstallFunctions);
   }
            $modInstaller->install($svnExportDir.$package);

   $postInstallFile = "$svnExportDir$package/scripts/post_install.php";
   if(is_file($postInstallFile))
   {
                $_REQUEST['unzip_dir'] = "$svnExportDir$package";
    ciLog ("Including $postInstallFile.\n", $sugarCrmSite['name'],$package);
    include($postInstallFile);
    ciLog ("Executing $postInstallFile.\n", $sugarCrmSite['name'],$package);
    post_install();

                // Undeclaring the post_install functions so we can include the next pre_install file. Nasty, this is a hack. Requires PECL ADP 
                undeclareFunctions($postInstallFunctions);
   }
   ciLog ("(rev $packageRevisionInSvn) Installation into " . $sugarCrmSite['sugarInstallDir'] . " is done.\n", $sugarCrmSite['name'],$package);
            
            $revisionNumberInstalledInSite[$package] = $packageRevisionInSvn;
        }
        else {
   ciLog ("No updates found (installed: $packageRevisionInstalled, svn: $packageRevisionInSvn).\n", $sugarCrmSite['name'],$package);
        }
    }
    // ciLog what package revisions that were installed
    saveArrayToFile($siteInstalledVersionsStateFile, $revisionNumberInstalledInSite);
    
    // Make sure apache owns the sugar installation
    $sugarInstallDir = $sugarCrmSite['sugarInstallDir'];
    $output = `chown -R apache:apache $sugarInstallDir`;
}

function undeclareFunctions($functions){
    foreach($functions as $function) {
       if (function_exists($function)) { rename_function($function, uniqid()); } 
    }   
}

function ciLog($message, $site, $package){
 $fullMessage = "%s %s-%s: " . $message;
 printf($fullMessage, date('m/d/Y H:i:s'), $site, $package);
}

function getArrayFromFile($filename){
    if (file_exists($filename)) {
        return unserialize(file_get_contents($filename));
    }
    else {
        return array();
    }
}

function saveArrayToFile($filename, $arrayToSave){
    $fh = fopen($filename, 'w') or die("can't open file");
    fwrite($fh, serialize($arrayToSave));
    fclose($fh);
}
?>

Setting up Centos 5.6 with PHP 5.3 on Amazon EC2 with ec2-consistent-snapshot

# Command order to get a CentOS 5.6 running with PHP 5.3 and ec2-consistent-snapshot running on AWS

# Change hostname
hostname YOUR_HOSTNAME
vi /etc/sysconfig/network
------------------------------------
HOSTNAME=YOUR_HOSTNAME
NETWORKING=yes
NETWORKING_IPV6=no
------------------------------------


#Install PHP and modules and APC
yum install php53 php53-mbstring php53-mysql php53-gd gd php53-imap php53-ldap php53-pdo php53-cli php53-pear php53-devel httpd-devel pcre-devel php-pear
pear upgrade --force Console_Getopt
pear upgrade --force pear
pear upgrade-all
pear version
pecl install apc

# Attach and mount a AWS EBS volumn (ec2-consistent-snapshot prefers xfs) 
# Make sure there is xfs support
yum install kmod-xfs xfsdump xfsprogs dmapi
mkfs.xfs  /dev/sdf
echo "/dev/sdf /vol xfs defaults,noatime 0 0" >> /etc/fstab
mkdir /vol
mount /vol

# Mysql
yum install mysql-server
mkdir /vol/mysql
chown -R mysql:mysql /vol/mysql
vi /etc/my.cnf

# Change the my.cnf file to 
------------------------------------
[mysqld]
datadir=/vol/mysql
user=mysql
log-warnings=2
log-error=/var/log/mysqld.log
default-storage-engine=innodb
skip-bdb
socket=/var/lib/mysql/mysql.sock

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
------------------------------------

service mysqld start
mysqladmin -u root password YOUR_PASSWORD
chkconfig mysqld on


# ec2-consistent-snapshot
perl -MCPAN -e shell
cpan>o conf prerequisites_policy follow
cpan>o conf commit
perl -MCPAN -e "install(q{Bundle::CPAN})"
perl -MCPAN -e "install Net::Amazon::EC2"
yum -y install perl-DBD-MySQL
yum -y install 'perl(File::Slurp)'
yum -y install 'perl(DBI)'
yum -y install 'perl(Net::SSLeay)'
yum -y install 'perl(IO::Socket::SSL)'
yum -y install 'perl(LWP::Protocol::https)'
wget http://bazaar.launchpad.net/~alestic/ec2-consistent-snapshot/trunk/download/head:/ec2consistentsnapsho-20090928015038-9m9x0fc4yoy54g4j-1/ec2-consistent-snapshot
mkdir /home/ec2/bin/
mv ec2-consistent-snapshot /home/ec2/bin/
chmod 700 /home/ec2/bin/ec2-consistent-snapshot

# Create / EBS snapshot script
cd /root
vi snapshot_ebs_root.sh
------------------------------------
#!/bin/sh

PATH=/usr/kerberos/sbin:/usr/kerberos/bin:/home/ec2/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/ho
me/ec2/bin:/root/bin

description="`hostname`-rootdrive-`date +%Y-%m-%d-%H-%M-%S`"

ec2-consistent-snapshot --aws-access-key-id=YOUR_ACCESS_KEY --aws-secret-access-key=YOUR_SECRET_ACCESS_KEY \
--description=$description --region=eu-west-1 YOUR_VOLUME_ID
------------------------------------
chmod 700 snapshot_ebs_root.sh


# Create /vol EBS snapshot script
vi ./snapshot_ebs_vol.sh
------------------------------------
#!/bin/sh

PATH=/usr/kerberos/sbin:/usr/kerberos/bin:/home/ec2/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/ho
me/ec2/bin:/root/bin

description="`hostname`-rootdrive-`date +%Y-%m-%d-%H-%M-%S`"

ec2-consistent-snapshot --aws-access-key-id=YOUR_ACCESS_KEY --aws-secret-access-key=YOUR_SECRET_ACCESS_KEY \
--mysql-username=root --mysql-password=YOUR_DB_ROOT_PASSWORD --mysql-host=localhost --description=$description --mysql --region=eu-west-1 \
--freeze-filesystem='/vol' YOUR_VOLUME_ID
------------------------------------
chmod 700 snapshot_ebs_vol.sh


#Set the cronschedule for EBS snapshots
crontab -e
------------------------------------
54 6-23 * * * /root/snapshot_ebs_vol.sh > /dev/null 2>&1
59 22 * * * /root/snapshot_ebs_root.sh > /dev/null 2>&1
------------------------------------

# Change httpd DocumentRoot to /vol/www/default
vi /etc/httpd/conf/httpd.conf
mkdir /vol/www/default
mkdir /vol/www/sites
chown -R apache:apache /vol/www
chkconfig httpd on
service httpd start

#Set server timezone
cd /etc
ln -sf /usr/share/zoneinfo/CET localtime

Split domain with Exchange 2007 as Primary and Zimbra 5.0.15+ as secondary

Introduction

This document describes methods and procedures for setting up a split-domain configuration for a Microsoft Exchange 2007 server and a Zimbra Collaboration Suite (ZCS) server running 5.0.15 or higher. While multiple configurations work, this document will describe the split domain configuration with Exchange as the primary and ZCS as the secondary mail servers.


This is a Exchange 2007 compatible document based on this Exchange 2003 document Split domain with Exchange as Primary SAMPLE.

Credits to Syed Moiz Ali Shah for converting the Exchange 2003 document into an Exchange 2007 compatible one.

 

Microsoft Exchange Server 2007 Configurations:-

To use the Exchange Management Console to create an internal relay domain:

1- Open the Exchange Management Console. In the console tree, expand Organization Configuration, select Hub Transport, and then in the work pane, click the Accepted Domains tab.





2- In the action pane, click New Accepted Domain…. The New Accepted Domain wizard appears.



3- On the New Accepted Domain page, complete the following fields:
Name: Use this field to identify the accepted domain in the user interface. You can type any name that you want. We recommend that you select a meaningful name that helps you easily identify the purpose of this accepted domain. For example, you may want to use a name that identifies this as a subsidiary domain or as a hosted domain. You must use a unique name for each accepted domain.
Accepted Domain: Use this field to identify the SMTP namespace for which the Exchange organization will accept e-mail messages. You can use a wildcard character to accept messages for a domain and all its subdomains. For example, you can type *. zm-train.com to set zm-train.com and all its subdomains as accepted domains.



4- After you complete these fields on the New Accepted Domain page, select the following option: Internal Relay Domain. E-mail is relayed to an e-mail server in another Active Directory forest in the organization.



5- Click New.



6- On the Completion page, click Finish.

To use the Exchange Management Console to create a Send connector to route e-mail to the shared domain

1- Open the Exchange Management Console. In the console tree, expand Organization Configuration, select Hub Transport, and then in the work pane, click the Send Connectors tab.



2- In the action pane, click New Send Connector. The New SMTP Send Connector wizard starts.



3- On the Introduction page, follow these steps:
In the Name: field, type a meaningful name for this connector. This name is used to identify the connector.
In the Select the intended use for this connector: field, Select “Internet Select this usage type if the e-mail system with which Exchange 2007 shares an address space is a third-party e-mail system.”



4- Click Next.



5- On the Address space page, click Add. In the Add Address Space dialog box, enter the domain name to which this connector will send mail, for example: *.zm-train.com. You may select the Include all subdomains check box to use this connector to send e-mail to all subdomains of the address space. When you are finished, click OK. Click Next.



6- On the Network settings page, select Route all mail through the following smart hosts. Click Add.



7- In the Add Smart Host dialog box, select IP Address or Fully qualified domain name (FQDN) to specify how to locate the smart host. If you select IP Address, enter the IP address of the smart host. If you select Fully qualified domain name (FQDN), enter the FQDN of the smart host. The sending server must be able to resolve the FQDN. When you are finished, click OK. To add more smart hosts, click Add, and repeat this step. When you are finished, click Next.



8- On the Smart host security settings page, select the method as following;
None



9- Click Next.



10- On the Source Server page, click Add to add a source server. By default, the Hub Transport server that you are currently working on is listed as a source server. In the Select Hub Transport servers and Edge Subscriptions dialog box, select the Hub Transport servers that will be used as the source server for sending messages to the shared address space. When you finish adding source servers, click OK. Click Next.



11- On the New Connector page, review the configuration summary for the connector. If you want to modify the settings, click Back. To create the Send connector by using the settings in the configuration summary, click New.



12- On the Completion page, click Finish.

Zimbra Configurations:-

Zimbra Server As Secondary

The configuration elements documented here have been completed for zm-train.com and are shown here for documentation purposes.
Zimbra Server
Hostname is my-zimbra-server.zm-train.com
mail domain = zm-train.com


1- The secondary MTA must accept mail for accounts that are hosted on the secondary. The first two commands (in combination) tell the Zimbra postfix to accept all addresses in the @zm-train.com domain as valid addresses.


$ zmprov md zm-train.com zimbraMailCatchAllAddress @zm-train.com
$ zmprov md zm-train.com zimbraMailCatchAllForwardingAddress @zm-train.com


2- But must forward all other mail for accounts on this domain to the primary system
This third command establishes default mail routing for the domain. Any users that do not exist on the Zimbra system will have their mail routed according to this rule.


$ zmprov md zm-train.com zimbraMailTransport smtp:my-exchange-server.zm-train.com:25


3- On the Zimbra server, we will turn off DNS lookups and internet wide message routing from the secondary host and route all mail through the primary. In the case of Relay mail to Primary with:


$ zmprov mcf zimbraMtaRelayHost my-exchange-server.zm-train.com:25


4- Turn off DNS lookups with:


$ zmprov mcf zimbraMtaDnsLookupsEnabled FALSE


5- After configuration changes, restart services/server if needed.


$ postfix stop
$ postfix start

Testing

Be sure to exercise each of the Use cases described earlier to verify mail routing is occurring as expected.

HOWTO-Run a local Linux server for PHP development on a Windows PC

Download and install
http://www.vmware.com/products/player/

Download the CentOS 5.5 iso from
http://vault.centos.org/5.5/isos/i386/CentOS-5.5-i386-netinstall.iso

Create a new LInux virtual machine in vmware player
mount the iso

Start the virtual machine
Install CentOS chose as minimal install as possible

when you are done follow the blog posting at
http://kenneththorman.blogspot.com/2011/05/installing-sugarcrm-from-scratch-on.html

Setup and configure samba
http://technicians-blog.kingcomputer.com.au/setup-samba-on-linux-centos-server/

Get the IP address of the linux machine (if this is a vm that is directly attached to the LAN then it is a good idea to change the DHCP lease to a static one if possible or give it a static IP address)

get the ip with ifconfig

Make an entry in the hosts file on windows

XXXXXXXXXX linuxdev.local

Make sure that any interfering firewalls (windows firewall or iptabels does not interfere)

Map the created network share on Windows

HOWTO - Empty SugarCrm ModuleLoader install history

Sometimes after installing many packages on a development installation of SugarCrm the Module Loader will take a long time to load. To regain the freshly installed SugarCrm speed of ModuleLoader you need to clean out the temp installation files (they will still be installed). You can use the commands below to clean it up to make it more responsive again. (This is working on 5.1.0b of SugarCrm, newer versions might behave differently).

Please note that you do lose the list of installed packages.

rm -f /<SUGARCRM_INSTALL_DIR>/cache/upload/*.zip
rm -fr /<SUGARCRM_INSTALL_DIR>/cache/upload/upgrades/module/*
rm -fr /<SUGARCRM_INSTALL_DIR>/cache/upload/upgrades/temp/*
# truncate table upgrade_history in instance database

Installing SugarCrm from scratch on CentOS 5.X

This posting outlines the Linux commands needed to install SugarCrm on a RHEL/CentOS 5.5 standard installation. This posting assumes that you are installing the database server on one server and the webserver on another server.

Before beginning I would like to change the SELinux configuration of the servers.


Since RHEL/CentOS out of the box has SELinux enabled i recommend changing the setting from enforcing to permissive. This will log all security breaches in a log and then you can later reconfigure SELinux by using the audit2allow script.
So for now we will permanently set SELinux to permissive.
In the /etc/sysconfig/selinux file you will something like the following

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=enforcing
...

Change SELINUX=enforcing to SELINUX=permissive and save the file.
Then run the following command

setenforce 0

First let us install the Apache webserver (or make sure it is updated) as well as php and the php modules needed to run SugarCRM.
It is important to note that Sugarcrm 5.1 only supports the standard PHP package with Centos 5.X (PHP 5.1.6 and NOT PHP 5.3)

yum install httpd
yum install php
yum install php-mbstring php-mysql php-gd gd php-imap php-ldap
yum install html2ps

We need to configure the iptables firewall to allow incomming trafic to HTTP/HTTPS and I will leave that as an exercise to the reader.


Now lets us install the APC PHP opcode cache. This is going to speed up the execution on the server. However please note that this can have some sideeffects when running SugarCRM. In particular if you are going to do a lot of customizations using Studio or ModuleBuilder. For that reason my preference is to only install and enable this on production sites. First lets us install the needed dependecies.

yum install php-pear php-devel httpd-devel pcre-devel

Now if you are running your server behind a proxy you need to configure pear to use that proxy otherwise the installation will fail. If you are not behind a proxy please ignore this. The syntax is pear config-set http_proxy http://myproxy.yourdomain.com:your_proxy_port/ and a example would be

pear config-set http_proxy http://myproxy.yourdomain.com:8080/

Installing APC

pecl install apc 

We need to configure the /etc/php.ini file, the below values are the values that I prefer to run with, you might have different requirements, please make sure you understand what you are changing and why.

memory_limit = 512M
max_execution_time = 300
error_reporting = E_ALL & ~E_NOTICE
post_max_size = 64M
upload_max_filesize = 64M

At the end of the php.ini file add the following lines. These lines tells PHP that we want to import the APC extension we installed above (on production servers). However we disable this APC module by default. This allows us to override and turn it on in a VirtualHost .htaccess file. This way we can have a test site running without APC and a production site running with APC on the same server (depending on your environment and expected production system load, this might be a good/bad idea).

extension=apc.so
[apc]
apc.cache_by_default=0 # disable by default

The default configuration of Apache when installed on RHEL/CentOS is to disable namebased virtual hosts, thus we need to enable this. (this is not strictly required since if you only plan on running one and one site only on this server you can do without, but I prefer to use namebased virtual hosts. So let us enable this in the /etc/httpd/conf/httpd.conf file. Find the following line and change as specified below.

#NameVirtualHost *:80

is changed to

NameVirtualHost *:80

Now we need to create a virtual host configuration file. I have a convention of naming the configuration file based on the DNS name of the site. The configuration file is placed in /etc/httpd/conf.d/DNS_NAME_OF_SITE.conf.

That would for a http://www.somewhere.com end up as a file named www.somewhere.com.conf.

Depending on your server setup you might want to use different values than what I have shown below, but the below configuration achives a pretty good Google PageSpeed score and also for Yahoo YSlow. However the caching is somewhat aggressive so you might have requirements that requires you to change the below values.

<VirtualHost *:80 >
  ServerName "HOSTNAME"
  DocumentRoot /var/www/sites/HOSTNAME
  DirectoryIndex index.html index.php
  ErrorLog logs/HOSTNAME-error_log
  CustomLog logs/HOSTNAME-access_log common

  <Directory  "/var/www/sites/HOSTNAME">
    SetOutputFilter DEFLATE
    AddDefaultCharset UTF-8

    AllowOverRide All
    Options FollowSymlinks

        # Enable ETag
        FileETag MTime Size

        # Set expiration header
        ExpiresActive on
        ExpiresDefault "access plus 1 year"
        ExpiresByType image/gif "access plus 1 month"
        ExpiresByType image/png "access plus 1 month"
        ExpiresByType image/jpeg "access plus 1 month"
        ExpiresByType text/css "access plus 1 month"
        ExpiresByType text/javascript "access plus 1 month"
        ExpiresByType text/js "access plus 1 month"

        # Compress some text file types
        AddOutputFilterByType DEFLATE text/html text/css text/xml application/x-javascript text/javascript text/js

        # Deactivate compression for buggy browsers
        BrowserMatch ^Mozilla/4 gzip-only-text/html
        BrowserMatch ^Mozilla/4\.0[678] no-gzip
        BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
  </Directory>
</VirtualHost>

We need to install java on the server since the preconfigured java does not work with ZuckerReports. This link was the latest version at the time of writing this posting. Please note that it is the jdk we are installing.

wget http://download.oracle.com/otn-pub/java/jdk/6u25-b06/jdk-6u25-linux-x64.bin
chmod 755 jdk-6u25-linux-x64.bin
./jdk-6u25-linux-x64.bin

Now we need to change the default java used by the system (please note the output of the java versions

java --version 
unlink /usr/bin/java
ln -s /usr/java/jdk1.6.0_04/bin/java /usr/bin/java
java -version

Now we will leave the webserver and move to the database server. First we need to install the mysql server.

yum install mysql-server

Before starting the mysql server I prefer to change the datadir location to a separate mount point in the Linux system rather than the default under /var/lib/mysql. That is done by editing the /etc/my.cnf file and changing the value of the datadir variable.

datadir=/<MY_CUSTOM_DATADIR_DIRECTORY_PATH>/mysql

# an example 
datadir=/data/mysql

Now start the mysql server and set it to autostart when the Linux server is started.

service mysqld start
chkconfig mysqld on

Now let us change a few more parameters in the my.cnf file (some of these parameter can have a breaking effect if you change them before the first start of the mysql server. That is why we are waiting and doing the configuration in 2 steps).
Change the /etc/my.cnf file and add the following variables under the [mysqld] region.

log-warnings=2
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
default-storage-engine=innodb

restart the mysqld server.

service mysqld restart

now we need to set a password for the root user.

mysqladmin -u root -p password '<MY_ROOT_PASSWORD>'

Create a database for the SugarCRM installation.

mysql -uroot -p<MY_ROOT_PASSWORD>
create database sugarcrm;

Create users and grant permissions to the users that need it.
Assuming that we want to allow access from localhost, and from remote Apache server

CREATE USER 'sugarcrm_user'@'localhost' IDENTIFIED BY 'mypass';
GRANT ALL ON sugarcrm.* TO 'sugarcrm_user'@'localhost'; 
CREATE USER 'sugarcrm_user'@'127.0.0.1' IDENTIFIED BY 'mypass';
GRANT ALL ON sugarcrm.* TO 'sugarcrm_user'@'127.0.0.1';
CREATE USER 'sugarcrm_user'@'<APACHE_SERVER_IP_ADDRESS>' IDENTIFIED BY 'mypass';
GRANT ALL ON sugarcrm.* TO 'sugarcrm_user'@'<APACHE_SERVER_IP_ADDRESS>'; 

Now if you plan on installing custom built modules into SugarCrm where the installer package contains MySQL trigger code that needs to be executed by the sugarcrm_user database user. MySQL 5.0 contains a less than optimal permission implementation. So to be able to install/create triggers the user needs SUPER permission. And since the default database version that is installed in RHEL/CentOS 5.5 at the time of writing this blog post is 5.0.77 we need to grant SUPER to our database user.

GRANT SUPER on *.* to 'sugarcrm_user'@'localhost';
GRANT SUPER on *.* to 'sugarcrm_user'@'127.0.0.1';
GRANT SUPER on *.* to 'sugarcrm_user'@'<APACHE_SERVER_IP_ADDRESS>';

Now we need to configure the iptables firewall to allow incomming trafic to MySQL.

iptables -A INPUT -i eth0 -s <APACHE_SERVER_IP_ADDRESS> -p tcp --destination-port 3306 -j ACCEPT
service iptables save

Now we are ready to install SugarCrm either using the installer from SugarCrm (please follow the official installation guide) or cloning an existing site

- tar the site

tar -czf mysite.somewhere.com.tar.gz mysite.somewhere.com

dump the staging sugarcrm mysql database (changing the tables to InnoDB and remove DEFINER statements from the dump)

mysqldump -h<HOSTNAME> -u<USERNAME> -p<PASSWORD> --triggers --routines <DATABASE> | perl -pe 's/MyISAM/InnoDB/' |  perl -pe 's{(/\*!\d+ )\s*definer\s*=\s*\S+}{$1}i' > mysite.somewhere.com.sql
gzip mysite.somewhere.com.sql

Move the tar site dump and mysql dump to the production server and unzip/untar the site dump

wget http://my.staging.server/mysite.somewhere.com.tar.gz
wget http://my.staging.server/mysite.somewhere.com.sql.gz
tar -xf mysite.somewhere.com.tar.gz
mv mysite.somewhere.com /var/www/sites/HOSTNAME  #(HOSTNAME as defined in the Apache virtualhost conf file)
chown -R apache:apache /var/www/sites/HOSTNAME  #(HOSTNAME as defined in the Apache virtualhost conf file)

#Secure directories
chmod -R 644 *
find . -type d | xargs chmod 755

chmod 664 config.php
chmod 775 ./custom 
chmod 775 ./data/upload 
chmod 775 ./data/upload 
chmod 775 ./cache 
chmod 775 ./cache/images 
chmod 775 ./cache/import 
chmod 775 ./cache/layout 
chmod 775 ./cache/pdf 
chmod 775 ./cache/upload 
chmod 775 ./cache/xml 
chmod 775 ./modules

install the mysql database dump

gunzip mysite.somewhere.com.sql.gz
mysql -h<DATABASEHOSTNAME> -usugarcrm_user -pmypass -D sugarcrm < mysite.somewhere.com.sql

edit the /var/www/sites/HOSTNAME/config.php file and make sure the following variables are updated with your own values

'dbconfig' =>
  array (
    'db_host_name' => '',
    'db_host_instance' => 'SQLEXPRESS',
    'db_user_name' => '',
    'db_password' => '',
    'db_name' => '',
    'db_type' => 'mysql',
'host_name' => '',
'site_url' => 'http://',
'unique_key' => '',

unique_key should be unique for each different server.

IMPORTANT!: Edit and update the /SITE_DIRECTORY/.htaccess file to match the destination DNS sitename.

If this is a production server enable APC

comment the following line
#RedirectMatch /modules/(.*)/(.*).php http://XXXXX/index.php

add the following line
php_flag apc.cache_by_default On

After running the site for a while, tweak your MySQL config file using MySQLTuner

wget --no-check-certificate https://raw.github.com/rackerhacker/MySQLTuner-perl/master/mysqltuner.pl

perl mysqltuner.pl

Knockout JS: Helping you build dynamic JavaScript UIs with MVVM and ASP.NET

Very nice take on how to make a responsive client side experience in JavaScript with minimal code. I am impressed. Cannot wait to play with this.

Knockout JS: Helping you build dynamic JavaScript UIs with MVVM and ASP.NET

The Dreyfus Model of Skills Acquisition (skill levels/taxonomy)

This is a copy of a posting originally found at http://blog.bruceabernethy.com/post/The-Dreyfus-Model-of-Skills-Acquisition.aspx however since that original posting/entire blog seems to have disappeared and as I actively use/find the taxonomy defined in this specific blog posting great I am reposting it here. All credits go to Bruce Abernethy.

UPDATE: It seems that Bruce Abernethy's blog is back up again, with a new nice design. The link is http://blog.bruceabernethy.com/myblog/2007/08/the-dreyfus-model-of-skills-acquisition/ (thank you for notifying me David). I will leave the posting here due to the few comments it has inspired, but recommend anyone to go to the original authors blog posting.

Regards
Kenneth Thorman

OK, this post will be a little “heady” for a Saturday morning but I want to reference this topic for some other work I am going to do this fall. I am planning to do some non-trivial work to get some resources and links online to help kids get into doing projects and learning about things like physics, space science, and robotics/electronics – things if you know me, you know I’ve been into for a long time. The problem with the way kids learn about these topics nowadays is that it is mostly abstract, in theories, reading, or just “on paper.” To really learn more than trivia and facts about these topics you really need to “do” things with the science, not just learn things “about” them.

There are several articles or models from people that really affect the way I am going to approach things, and have impacted my thinking about the approach to learning things. I’ll hit each of them separately and then pull them all together later as kind of an approach for what I am going to try. None of these articles or their ideas should really come as a surprise to anyone – they make sense when you read them. But the authors do a great job of describing things in a depth and detail that I never could.

The first article and model is the Dreyfus Model – which is based originally on a report by Hubert Dreyfus and Stuart Dreyfus in the early 80s called “A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition.” I’ll try to link to the study itself which is in a PDF file (scanned in mirror 783k) – don’t read it yet, just follow along here for a minute.

This study was prepared initially for the Air Force Office of Scientific Research for the training of pilots, but its model and purpose has been used effectively for training everyone from pilots, to nurses, to chess players, foreign language learners, and even computer programmers (Google “Dreyfus Model” http://www.google.com/search?q=dreyfus+model and will see lots of different results of people using it (and some people arguing against it)). I like it, so I am going to use it.

To keep this to as few words as possible, Dreyfus identifies five to seven stages of learning a new skill or domain:

• Novice,
• Advanced Beginner,
• Competent,
• Proficient,
• Expert, and
• Master.

I don’t know about you but I find that many people use terms like “Competent” and “Expert” in describing their skills but really have no objective model for putting them in one category or another. Some people move themselves from not knowing anything about a particular skill up to “Competent” just by reading a book about the topic. Others call themselves an “Expert” after completing one project using a particular skill or technology. I am reminded of an episode of “Monk” where the detective Adrian Monk is going on a boat and someone asks him if he can swim. He says he knows how to swim, and even produces a card from his wallet certifying him as a swimmer (from a correspondence course), but also admits he has never actually been in the water to try out his skills but thinks he’ll do fine. Some people are that way with technology and other domains where they have studied it in books and on the Internet but have never really “done” real-world things to use what they have learned. You learn by doing things, you don’t (can’t) really learn before you do something.

What this model does is two important things: (1) helps better define what these stages of learning mean and (2) explains a workable model for moving people from one stage to the next.

Briefly, the stages:

Novice

A novice is all about following rules – specific rules, without context or modification. You don’t need to “think” you just need to “do”. A rule is absolute, and must never be violated. The main thing to do here is to get experience following directions and doing the new skill. You can follow the instructions on a box of cake mix and hopefully produce a decent cake. All you are responsible for is following directions.
“To improve, the novice needs monitoring, either by self-observation or instructional feedback, so as to bring his behavior more and more completely into conformity with the rule.”

Advanced Beginner

Still rules based, but rules start to have situational conditions. In one situation you use one rule, in other situations you use another. The advanced beginner needs to be able to identify the limited need to selectively apply different rules. So if you want a chocolate cake, follow the chocolate rule(s), if you want a vanilla cake, follow the other rule(s). If you are over 5,000ft of altitude you will need to alter the amount of some ingredients. This is still a recipe, but has a few decision points. Again, follow the different “branches” of instructions and you should be fine. It is easy to see how this could collapse into a large Novice category, but it is a step before the much larger step to Competence.

Competent

You realize that your skill or domain is more complex than a series of rules and branches. You start to see patterns and principles (or aspects) rather than a discrete set of rules – rules become “rules of thumb”. You are lead more by your experience and active decision-making than by strictly following rules. What is developed now are guidelines that help direct competent individuals at a higher level. You now are accountable for your decisions as you are not following the strict rules and context of the previous stages. You’ve made a lot of cakes and have a number of recipes. When asked to make a cake of a different type you pull from experience the best way to put a new cake together. If the new cake doesn’t work out, you are responsible. This is the critical tipping point for most people when learning a new skill – and why most people never really become “competent” in most things they learn. Here you either need to decide to just “follow the rules” or spend the time to get fully involved with and take responsibility.
“Competence comes only after considerable experience actually coping with real situations …”

Proficient

At this point your understanding of your skill or domain has become more of an instinct or intuition. You will do and try things because it just seems like the right thing to do (and you will most often be right). Instead of a discrete set of different parts you can perceive a complete system. A large amount of real-world experience will show you that there are often multiple competing solutions to a specific problem and you have a “gut feeling” about which is correct. “Calculation and rational analysis seem to disappear”. Will quickly know “what” needs to be done and then formulate how to do it. Proficiency is developed by exposure to a “wide variety of typical whole situations.”

Expert

At this point you are not solving problems or making conscious decisions about things, you just “do” and it works. “Optimal performance becomes second nature.” People may ask you why you decided to do things “that way” and you may not know how to explain to them the 10 steps necessary to get from “A” to “B” because to you it was really just one step. Forcing an expert to detail the steps necessary before proceeding will often cause them to fail or second-guess. Here you think of grandma getting up at 6:00am and making biscuits from scratch for many, many years. She doesn’t measure, time, or probably even think about baking – she just does it, and it works. Very few people will attain this level in a particular skill or domain. Some estimates say 10-15 years in a particular area is required.
An Expert has experience that “is so vast that normally each specific situation immediately dictates an intuitively appropriate action.”

Master

Mastery is mostly about style. A Master of something is really just an “Expert on a roll.” Sometimes you may have witnessed someone or spent time with someone who is so good at something, and gets so caught up in doing it, that you can’t help but feel that you are watching a genius at work. I’d also say a Master is an Expert who can look back and put themselves in a Novice’s shoes and create the rules, and do the monitoring/mentoring necessary to help them move forward. If you have met a Master you remember them – by name – they are rare and you would do well to spend as much time with them as possible. An Expert basketball player could be excellent at execution and without formal thought just picture the ball going through the hoop (and it does). But Michael Jordan could do it with such style, grace and physics-defying ease that you just had to stop everything and watch him when he was “in the groove.”
A Master “is capable of experiencing moments of intense absorption in his work, during which his performance transcends even its usual high level.”

The Point

If you’ve read this far, thanks, and I’ll get to the point (1,000+ words later) – I believe that too many people today are learning just enough to be considered an “Advanced Beginner” in the vast majority of topics. Painfully this is increasingly true in science, math and technology. If you know enough to “pass the test” then that is all you need to know. The No Child Left Behind law was designed to keep people accountable for learning. But instead of having students be able to become Competent or even Proficient in a handful of skills, we have instead created a system where we have students stuck at “Advanced Beginner” in many more subjects – and perhaps unable to move forward. We are teaching kids that it is enough to know “about” things, but not actually “do” things.
So what can we do about it? We need to catalog and create a list of resources that will help kids get from Novice to Competent in science, math, and technology - things that will get them active in doing projects and making the connections necessary to move ahead. A lofty goal, but attainable. Many people are seeing similar voids and doing things about it. Linking and teaming with them will be key in our success.

More on this to come …..
Bruce

p.s. I need a good label for "Level 0" - before Novice - before you've ever really started learning about a particular topic.  "Ignorant" is cold, as is "Empty" - "Unaware" sounds judgmental.  If you haven't heard of Newton's Laws of Motion it most likely isn't your fault, but you aren't quite a "Novice" yet in its study.  Something positive like "ready" or "waiting"  or "willing" - ideas are welcome.

Mark Seeman: Dependency Injection in .NET

I started reading Mark Seeman's book "Dependency Injection in .NET" yesterday evening. So far I have been through the first 4 chapters, and I find the mix of culinary hint and anecdotes a fresh and welcome take on writing factual development / programming books.

From what I have read so far I can highly recommend this book from a technical perspective.

For anyone interested in designing decoupled software and especially .NET developers this is a MUST read. Many of the concepts and techniques described in the book are relevant  for most other programming languages as well.

ASP.NET MVC 3 Dependency Injection using Windsor Castle, Singleton Lifestyle and object not set to an instance

Dependency Injection and Inversion of Control is an increasingly common and popular design principle in ASP.NET MVC.

However for the uninitiated (and even for those how have worked with it for some time) you can run into unexpected errors, especially when mixing Lifestyles for different Components on larger projects where you have multiple container installers/initializers.

I will here present a very obvious case that nevertheless had me and another developer spending a few hours trying to figure out what was wrong. This bug presented here are especially difficult to track down when you are doing frequent builds on a development machine and that build process causes the IIS application pool(s) to recycle, effectively resetting the IoC container.

I will be using Castle Windsor as the IoC (Inversion of Control) container in this posting.

The zipped solution can be downloaded here IocLifestyles.zip

First of all we will need to start with a new standard empty ASP.MVC NET 3.0 site/project and add a reference to Castle.Windsor. This reference can be added using NuGet or by downloading the source or binary libraries at the Castle Windsor website.

Secondly we need to tell ASP.NET MVC that we do not want to use the DefaultControllerFactory to create controller instances but that we want to use our own which in turn uses Windsor to provide the dependencies the various controllers need.

I do this in the Global.asax file by adding the following lines in the Application_Start method.

// Using WindsorControllerFactory rather then DependencyResolver
// http://bradwilson.typepad.com/blog/2010/07/service-location-pt1-introduction.html (read comments)
// http://bradwilson.typepad.com/blog/2010/10/service-location-pt5-idependencyresolver.html (read comments)
// http://bradwilson.typepad.com/blog/2010/10/service-location-pt10-controller-activator.html (read comments)
// http://mikehadlow.blogspot.com/2011/02/mvc-30-idependencyresolver-interface-is.html
// http://kozmic.pl/2010/08/19/must-windsor-track-my-components/
ControllerBuilder.Current.SetControllerFactory(new WindsorControllerFactory(applicationWideWindsorContainer));

// Initialize / install components in container
applicationWideWindsorContainer.Install(new WindsorInstaller());

As well as the the field

WindsorContainer applicationWideWindsorContainer = new WindsorContainer();

This leads to our Global.asax files looking like this

using System.Web.Mvc;
using System.Web.Routing;
using Castle.Windsor;

namespace IocLifestyles
{
 // Note: For instructions on enabling IIS6 or IIS7 classic mode, 
 // visit http://go.microsoft.com/?LinkId=9394801

 public class MvcApplication : System.Web.HttpApplication
 {
  public static void RegisterGlobalFilters(GlobalFilterCollection filters)
  {
   filters.Add(new HandleErrorAttribute());
  }

  public static void RegisterRoutes(RouteCollection routes)
  {
   routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

   routes.MapRoute(
     "Default", // Route name
     "{controller}/{action}/{id}", // URL with parameters
     new { controller = "Default", action = "Index", id = UrlParameter.Optional } // Parameter defaults
   );

  }

  WindsorContainer applicationWideWindsorContainer = new WindsorContainer();

  protected void Application_Start()
  {
   AreaRegistration.RegisterAllAreas();

   RegisterGlobalFilters(GlobalFilters.Filters);
   RegisterRoutes(RouteTable.Routes);

   // Using WindsorControllerFactory rather then DependencyResolver
   // http://bradwilson.typepad.com/blog/2010/07/service-location-pt1-introduction.html (read comments)
   // http://bradwilson.typepad.com/blog/2010/10/service-location-pt5-idependencyresolver.html (read comments)
   // http://bradwilson.typepad.com/blog/2010/10/service-location-pt10-controller-activator.html (read comments)
   // http://mikehadlow.blogspot.com/2011/02/mvc-30-idependencyresolver-interface-is.html
   // http://kozmic.pl/2010/08/19/must-windsor-track-my-components/
   ControllerBuilder.Current.SetControllerFactory(new WindsorControllerFactory(applicationWideWindsorContainer));

   // Initialize / install components in container
   applicationWideWindsorContainer.Install(new WindsorInstaller());
  }
 }
}

We now need to look closer at 2 classes, namely the WindsorControllerFactory and the WindsorInstaller.

The responsibility of the WindsorControllerFactory is to provide all the objects to the controller which it depends on to perform it's logic. This is passed in to the controllers constructor. This is done using Constructor Injection (you can see a lot of good tips, thoughts and recommendations at Mark Seeman's blog).

using System;
using System.Web.Mvc;
using System.Web.Routing;
using Castle.Windsor;

namespace IocLifestyles
{
 public class WindsorControllerFactory : DefaultControllerFactory
 {
  private readonly IWindsorContainer windsorContainer;

  public WindsorControllerFactory(IWindsorContainer windsorContainer)
  {
   this.windsorContainer = windsorContainer;
  }

  protected override IController GetControllerInstance(RequestContext requestContext, Type controllerType)
  {
   return windsorContainer.Resolve(controllerType) as IController;
  }

  public override void ReleaseController(IController controller)
  {
   var disposableController = controller as IDisposable;
   if (disposableController != null)
   {
    disposableController.Dispose();
   }

   windsorContainer.Release(controller);
  }
 }
}

Now we need to register the classes which we are going to use through out our application with the IoC container and we do this by implementing the IWindsorInstaller interface.

Please note the Lifestyles for the different components, since this is the key to the bug I am going to illustrate.

using System.Web.Mvc;
using Castle.Facilities.FactorySupport;
using Castle.MicroKernel.Registration;
using Castle.MicroKernel.SubSystems.Configuration;
using Castle.Windsor;
using System.Web;

namespace IocLifestyles
{
 public class WindsorInstaller : IWindsorInstaller
 {
  public void Install(IWindsorContainer container, IConfigurationStore store)
  {
   // Register all controllers from this assembly
   container.Register(
    AllTypes.FromThisAssembly()
    .BasedOn<Controller>()
    .Configure(c => c.LifeStyle.PerWebRequest)
   );

   // Register HttpContext(Base) and HttpRequest(Base) so it automagically can be injected using IoC
   container.AddFacility<FactorySupportFacility>();
   container.Register(Component.For<HttpRequestBase>().LifeStyle.PerWebRequest
      .UsingFactoryMethod(() => new HttpRequestWrapper(HttpContext.Current.Request)));
   container.Register(Component.For<HttpContextBase>().LifeStyle.PerWebRequest
     .UsingFactoryMethod(() => new HttpContextWrapper(HttpContext.Current)));

   // Respository and Service registrations
   container.Register(Component.For<ISomeRepository>().ImplementedBy<SomeRepository>());
   container.Register(Component.For<ISessionValidator>().ImplementedBy<SessionValidator>().LifeStyle.PerWebRequest);
  }
 }
}

Looking closer at the 2 classes that we have registered in the WindsorInstaller: SomeRepository and SessionValidator.

namespace IocLifestyles
{
 public interface ISomeRepository
 {
  SessionToken GetSessionToken();
 }

 public class SomeRepository : ISomeRepository
 {
  private readonly ISessionValidator sessionValidator;

  public SomeRepository(ISessionValidator sessionValidator)
  {
   this.sessionValidator = sessionValidator;
  }

  public SessionToken GetSessionToken()
  {
   return sessionValidator.ValidateSession();
  }
 }
}

using System.Net;
using System.Web;

namespace IocLifestyles
{
 public interface ISessionValidator
 {
  SessionToken ValidateSession();
 }
 public class SessionValidator : ISessionValidator
 {
  private readonly HttpContextBase httpContextBase;

  public SessionValidator(HttpContextBase httpContextBase)
  {
   this.httpContextBase = httpContextBase;
  }

  public SessionToken ValidateSession()
  {
   // Do some validation here
   var sessionToken = new SessionToken
             {
              IpAddress = IPAddress.Parse(httpContextBase.Request.UserHostAddress),
              IsValid = true
             };

   return sessionToken;
  }
 }
}

The final 2 pieces needed to actually see something in the browser is the DefaultController

using System.Web.Mvc;

namespace IocLifestyles.Controllers
{
 public class DefaultController : Controller
 {
  private readonly ISomeRepository someRepository;

  // Constructor Injection
  public DefaultController(ISomeRepository someRepository)
  {
   this.someRepository = someRepository;
  }

  public ActionResult Index()
  {
   ViewData.Model = someRepository.GetSessionToken();
   return View();
  }
 }
}

and the Views/Default/Index.cshtml view.

@model IocLifestyles.SessionToken
@{
 ViewBag.Title = "Index";
}
<h2>
 SessionToken IpAddress: @Model.IpAddress.ToString()</h2>

This is a rather contrived example, but please bear with me for the sake of demonstration purposes.

What is interesting however is what happens when we run the application the first time and contrast this to what happens when we run it a second time without rebuilding the code.

First time:

Second and subsequent times:

Playing around with this you will notice that you can display the page one time after a build, this is giving us an indication of what is wrong.

Below you can see the full stack trace with all details

Server Error in '/' Application.
Object reference not set to an instance of an object.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.NullReferenceException: Object reference not set to an instance of an object.

Source Error:

Line 20:   {
Line 21:    // Do some validation here
Line 22:    var sessionToken = new SessionToken
Line 23:              {
Line 24:               IpAddress = IPAddress.Parse(httpContextBase.Request.UserHostAddress),


Source File: D:\Users\kst\Visual Studio 2010\projects\IocLifestyles\IocLifestyles\SessionValidator.cs    Line: 22

Stack Trace:

[NullReferenceException: Object reference not set to an instance of an object.]
   Microsoft.VisualStudio.WebHost.Connection.get_RemoteIP() +0
   Microsoft.VisualStudio.WebHost.Request.GetRemoteAddress() +65
   System.Web.HttpRequestWrapper.get_UserHostAddress() +22
   IocLifestyles.SessionValidator.ValidateSession() in D:\Users\kst\Visual Studio 2010\projects\IocLifestyles\IocLifestyles\SessionValidator.cs:22
   IocLifestyles.SomeRepository.GetSessionToken() in D:\Users\kst\Visual Studio 2010\projects\IocLifestyles\IocLifestyles\SomeRepository.cs:19
   IocLifestyles.Controllers.DefaultController.Index() in D:\Users\kst\Visual Studio 2010\projects\IocLifestyles\IocLifestyles\Controllers\DefaultController.cs:17
   lambda_method(Closure , ControllerBase , Object[] ) +96
   System.Web.Mvc.ActionMethodDispatcher.Execute(ControllerBase controller, Object[] parameters) +17
   System.Web.Mvc.ReflectedActionDescriptor.Execute(ControllerContext controllerContext, IDictionary`2 parameters) +208
   System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(ControllerContext controllerContext, ActionDescriptor actionDescriptor, IDictionary`2 parameters) +27
   System.Web.Mvc.<>c__DisplayClass15.<InvokeActionMethodWithFilters>b__12() +55
   System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodFilter(IActionFilter filter, ActionExecutingContext preContext, Func`1 continuation) +263
   System.Web.Mvc.<>c__DisplayClass17.<InvokeActionMethodWithFilters>b__14() +19
   System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodWithFilters(ControllerContext controllerContext, IList`1 filters, ActionDescriptor actionDescriptor, IDictionary`2 parameters) +191
   System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName) +343
   System.Web.Mvc.Controller.ExecuteCore() +116
   System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext) +97
   System.Web.Mvc.ControllerBase.System.Web.Mvc.IController.Execute(RequestContext requestContext) +10
   System.Web.Mvc.<>c__DisplayClassb.<BeginProcessRequest>b__5() +37
   System.Web.Mvc.Async.<>c__DisplayClass1.<MakeVoidDelegate>b__0() +21
   System.Web.Mvc.Async.<>c__DisplayClass8`1.<BeginSynchronous>b__7(IAsyncResult _) +12
   System.Web.Mvc.Async.WrappedAsyncResult`1.End() +62
   System.Web.Mvc.<>c__DisplayClasse.<EndProcessRequest>b__d() +50
   System.Web.Mvc.SecurityUtil.<GetCallInAppTrustThunk>b__0(Action f) +7
   System.Web.Mvc.SecurityUtil.ProcessInApplicationTrust(Action action) +22
   System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult) +60
   System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAsyncResult result) +9
   System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +8862285
   System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +184


Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.0.30319.208

The reason this is failing is the mixing of Lifestyles in the WindsorInstaller. Lets bring up that particular code again.

using System.Web.Mvc;
using Castle.Facilities.FactorySupport;
using Castle.MicroKernel.Registration;
using Castle.MicroKernel.SubSystems.Configuration;
using Castle.Windsor;
using System.Web;

namespace IocLifestyles
{
 public class WindsorInstaller : IWindsorInstaller
 {
  public void Install(IWindsorContainer container, IConfigurationStore store)
  {
   // Register all controllers from this assembly
   container.Register(
    AllTypes.FromThisAssembly()
    .BasedOn<Controller>()
    .Configure(c => c.LifeStyle.PerWebRequest)
   );

   // Register HttpContext(Base) and HttpRequest(Base) so it automagically can be injected using IoC
   container.AddFacility<FactorySupportFacility>();
   container.Register(Component.For<HttpRequestBase>().LifeStyle.PerWebRequest
      .UsingFactoryMethod(() => new HttpRequestWrapper(HttpContext.Current.Request)));
   container.Register(Component.For<HttpContextBase>().LifeStyle.PerWebRequest
     .UsingFactoryMethod(() => new HttpContextWrapper(HttpContext.Current)));

   // Respository and Service registrations
   container.Register(Component.For<ISomeRepository>().ImplementedBy<SomeRepository>());
   container.Register(Component.For<ISessionValidator>().ImplementedBy<SessionValidator>().LifeStyle.PerWebRequest);
  }
 }
}

The default lifestyle is Singleton. Omitting the lifestyle on ISomeRepository will register it as a Lifestyle.Singleton. SomeRepository will be instantiated one time by Windsor and then cached internally for future use. This means that even though all the other registered Components are Lifestyle.PerWebRequest, ISomeRepository will not be able to benefit from this.

There are no warnings that helps you if you mix Lifestyles in your installers, and especially in larger projects, this can be a challenge when you start tweaking for memory/performance reasons.
You need to know how all the dependencies work, and how they are used, to be able tweak Lifestyles for components. If you do not have this knowledge the risk is very high that you will introduce bugs. My suggestion especially if you are new to ASP.NET MVC leave Lifestyles to Lifestyle.PerWebRequest for all components, unless it is very obvious that this is a real singleton component. Only start tweaking Lifestyles if performance or memory consumption becomes a problem.

Changing the installer to the following solves the problem.

using System.Web.Mvc;
using Castle.Facilities.FactorySupport;
using Castle.MicroKernel.Registration;
using Castle.MicroKernel.SubSystems.Configuration;
using Castle.Windsor;
using System.Web;

namespace IocLifestyles
{
 public class WindsorInstaller : IWindsorInstaller
 {
  public void Install(IWindsorContainer container, IConfigurationStore store)
  {
   // Register all controllers from this assembly
   container.Register(
    AllTypes.FromThisAssembly()
    .BasedOn<Controller>()
    .Configure(c => c.LifeStyle.PerWebRequest)
   );

   // Register HttpContext(Base) and HttpRequest(Base) so it automagically can be injected using IoC
   container.AddFacility<FactorySupportFacility>();
   container.Register(Component.For<HttpRequestBase>().LifeStyle.PerWebRequest
      .UsingFactoryMethod(() => new HttpRequestWrapper(HttpContext.Current.Request)));
   container.Register(Component.For<HttpContextBase>().LifeStyle.PerWebRequest
     .UsingFactoryMethod(() => new HttpContextWrapper(HttpContext.Current)));

   // Respository and Service registrations
   container.Register(Component.For<ISomeRepository>().ImplementedBy<SomeRepository>().LifeStyle.PerWebRequest);
   container.Register(Component.For<ISessionValidator>().ImplementedBy<SessionValidator>().LifeStyle.PerWebRequest);
  }
 }
}

Is throw null and throw new NullReferenceException() the same?

A few times I have seen the use of 

throw null;

where a more verbosely inclined programmer would have written 

throw new NullReferenceException();

So I set out to find out if these 2 uses indeed are equivalent.
Starting out with the code for our class, the matching generated MSIL and finally the stack trace.

using System;
namespace NullRefernceException
{
 public class Class1
 {
  public void DoSomething()
  {
   throw null;
  }
 }
}

.method public hidebysig instance void  DoSomething() cil managed
{
  // Code size       2 (0x2)
  .maxstack  8
  IL_0000:  ldnull
  IL_0001:  throw
} // end of method Class1::DoSomething

System.NullReferenceException: Object reference not set to an instance of an object.
   at NullRefernceException.Class1.DoSomething() in D:\Users\kst\Visual Studio 2010\projects\NullRefernceException\NullRefernceException\Class1.cs:line 8
   at NullRefernceException.Class1Tests.DoSomething_TestDriveNullReferenceException_ExceptionThrown() in D:\Users\kst\Visual Studio 2010\projects\NullRefernceException\NullRefernceException\Class1Tests.cs:line 15

And now for the version of the code that is using throw new NullReferenceException

using System;
namespace NullRefernceException
{
 public class Class1
 {
  public void DoSomething()
  {
   throw new NullReferenceException();
  }
 }
}

.method public hidebysig instance void  DoSomething() cil managed
{
  // Code size       6 (0x6)
  .maxstack  8
  IL_0000:  newobj     instance void [mscorlib]System.NullReferenceException::.ctor()
  IL_0005:  throw
} // end of method Class1::DoSomething

System.NullReferenceException: Object reference not set to an instance of an object.
   at NullRefernceException.Class1.DoSomething() in D:\Users\kst\Visual Studio 2010\projects\NullRefernceException\NullRefernceException\Class1.cs:line 8
   at NullRefernceException.Class1Tests.DoSomething_TestDriveNullReferenceException_ExceptionThrown() in D:\Users\kst\Visual Studio 2010\projects\NullRefernceException\NullRefernceException\Class1Tests.cs:line 15

And the test used to drive

using System;
using NUnit.Framework;

namespace NullRefernceException
{
 [TestFixture]
 public class Class1Tests
 {
  [Test]
  public void DoSomething_TestDriveNullReferenceException_ExceptionThrown()
  {
   var class1 = new Class1();
   try
   {
    class1.DoSomething();
   }
   catch (Exception exception)
   {
    Console.WriteLine(exception);
   }
  }
 }
}

So from a stacktrace point of view it does not look like there is a difference but in IL there is a difference.

After going through this exercise in code I flexed my Google Fu powers :) and found more information at this link on Stackoverflow - Why does C# allow you to 'throw null'?

IIS administration using C# - multiple websites text search and replace in physicalpath

I recently found myself in a situation where I had several different web applications that where branches/tagged/deployed as a unit. When working with several different feature branches this leads to either having quite a few individual sites in IIS or updating the configurations of the existing IIS sites when switching to another branch. Since the setup and configuration of these sites was non-trivial, I ended up wondering if I could have one setup in IIS and then just bulk update the web applications physical path using a search and replace functionality.


Initially I tried vbs, but then I found this nice dll Microsoft.Web.Administration, which allows you to perform IIS administration from C#/.NET.


We have the following file system layout

Looking at the IIS administration tool we are currently pointing to branch1.





Running the program and bulk updating one or more site's physical paths.

Looking in the IIS administration again we see that the physical path of the site has been updated.

The source code below is pretty self explanatory. (Please note that I have kept the sources to the bare minimum to illustrate working with the Web.Administration API, rather than doing null checks, array length checks, try catches ...)

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using Microsoft.Web.Administration;

namespace UpdateIisApplicationPaths
{
 public partial class Form1 : Form
 {
  private readonly ServerManager serverManager = new ServerManager();
  public Form1()
  {
   InitializeComponent();
  }

  private void Form1_Load(object sender, EventArgs e)
  {
   foreach (var site in serverManager.Sites)
   {
    websitesListBox.Items.Add(site.Name);
   }  
  }

  private void commitChangesButton_Click(object sender, EventArgs e)
  {
   Cursor.Current = Cursors.WaitCursor;
   foreach (var sitename in websitesListBox.SelectedItems)
   {
    var physicalPath = serverManager.Sites[(string) sitename].Applications[0].VirtualDirectories[0].PhysicalPath;
    physicalPath = physicalPath.Replace(oldPhysicalSitePathTextBox.Text, newPhysicalSitePathTextBox.Text);
    serverManager.Sites[(string) sitename].Applications[0].VirtualDirectories[0].PhysicalPath = physicalPath;
   }
   serverManager.CommitChanges();
   Cursor.Current = Cursors.Default;
  }

  private void websitesListBox_SelectedIndexChanged(object sender, EventArgs e)
  {
   if (websitesListBox.SelectedItems.Count > 0)
   {
    oldPhysicalSitePathTextBox.Text =
     serverManager.Sites[(string) websitesListBox.SelectedItems[0]].Applications[0].VirtualDirectories[0].PhysicalPath;
   }
  }
 }
}

WCF REST Exception handling deserializing with XmlSerializer

As an alternative to using the DataContractSerializer you can use the XmlSerializer in your WCF services. Assuming that you have a special need that cannot be covered by the DataContractSerializer here is a sample of how to catch and handle WCF REST faults on the client side. This provides an alternative to using the DataContractSerializer which is used in this post WCF REST Exception handling. The WCF REST Exception handling post sets the scene for this post.

Dan Rigsbyb have a great blog posting where he compares the DataContractSerializer with the XmlSerializer, it is located here XmlSerializer vs DataContractSerializer: Serialization in Wcf.
The advantage of this version of the WcfRestExceptionHelper class is that it allows for somewhat simpler client code (compared to the version in  WCF REST Exception handling). 
To mix things up a little bit I have chosen to use the XmlSerializer instead of the DataContractSerializer. Without further ado.

WCF REST client code:

var factory = new ChannelFactory<IService1Wrapper>("Service1WrapperREST");var proxy = factory.CreateChannel();

try
{
 var serviceResult = proxy.GetProductById("1");

 // Do something with result
}
catch (Exception exceptionThrownByRestWcfCall)
{
 var serviceResult = WcfRestExceptionHelper<SampleItem[], SampleError>.HandleRestServiceError(exceptionThrownByRestWcfCall);

 // Do something with result, let higher levels in the callstack handle possible real exceptions
}
finally
{
 ((IDisposable)proxy).Dispose();
}

The WcfRestExceptionHelper class

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Net;
using System.ServiceModel;
using System.Text;
using System.Xml.Serialization;

namespace WcfRestClient.WcfErrors
{
 public static class WcfRestExceptionHelper<TServiceResult, TServiceFault> where TServiceFault : class
 {
  private static IDictionary<Type, XmlSerializer> cachedSerializers = new Dictionary<Type, XmlSerializer>();

  public static TServiceResult HandleRestServiceError(Exception exception)
  {
   if (exception == null) throw new ArgumentNullException("exception");

   // REST uses the HTTP procol status codes to communicate errors that happens on the service side.
   // This means if we have a teller service and you need to supply username and password to login
   // and you do not supply the password, a possible scenario is that you get a 400 - Bad request.
   // However it is still possible that the expected type is returned so it would have been possible 
   // to process the response - instead it will manifest as a ProtocolException on the client side.
   var protocolException = exception as ProtocolException;
   if (protocolException != null)
   {
    var webException = protocolException.InnerException as WebException;
    if (webException != null)
    {
     var responseStream = webException.Response.GetResponseStream();
     if (responseStream != null)
     {
      try
      {
       // Debugging code to be able to see the reponse in clear text
       //SeeResponseAsClearText(responseStream);

       // Try to deserialize the returned XML to the expected result type (TServiceResult)
       return (TServiceResult) GetSerializer(typeof(TServiceResult)).Deserialize(responseStream);
      }
      catch (InvalidOperationException serializationException)
      {
       // This happens if we try to deserialize the responseStream to type TServiceResult
       // when an error occured on the service side. An service side error serialized object 
       // is not deserializable into a TServiceResult

       // Reset responseStream to beginning and deserialize to a TServiceError instead
       responseStream.Seek(0, SeekOrigin.Begin);

       var serviceFault = (TServiceFault)GetSerializer(typeof(TServiceFault)).Deserialize(responseStream);
       throw new WcfRestServiceException<TServiceFault>() { ServiceFault = serviceFault };
      }
     }
    }
   }

   // Don't know how to handle this exception
   throw exception;
  }
  
  /// <summary>
  /// Based on the knowledge of how the XmlSerializer work, I found it safest to explicitly implement my own caching mechanism.
  /// 
  /// From MSDN:
  /// To increase performance, the XML serialization infrastructure dynamically generates assemblies to serialize and 
  /// deserialize specified types. The infrastructure finds and reuses those assemblies. This behavior occurs only when 
  /// using the following constructors:
  /// 
  /// XmlSerializer.XmlSerializer(Type)
  /// XmlSerializer.XmlSerializer(Type, String)
  /// 
  /// If you use any of the other constructors, multiple versions of the same assembly are generated and never unloaded, 
  /// which results in a memory leak and poor performance. The easiest solution is to use one of the previously mentioned 
  /// two constructors. Otherwise, you must cache the assemblies.
  /// 
  /// </summary>
  /// <param name="classSpecificSerializer"></param>
  /// <returns></returns>
  private static XmlSerializer GetSerializer(Type classSpecificSerializer)
  {
   if (!cachedSerializers.ContainsKey(classSpecificSerializer))
   {
    cachedSerializers.Add(classSpecificSerializer, new XmlSerializer(classSpecificSerializer));
   }
   return cachedSerializers[classSpecificSerializer];
  }

  /// <summary>
  /// Debugging helper method in case there are problems with the deserialization
  /// </summary>
  /// <param name="responseStream"></param>
  private static void SeeResponseAsClearText(Stream responseStream)
  {
   var responseStreamLength = responseStream.Length;
   var buffer = new byte[responseStreamLength];
   var x = responseStream.Read(buffer, 0, Convert.ToInt32(responseStreamLength));
   var enc = new UTF8Encoding();
   var response = enc.GetString(buffer);
   Debug.WriteLine(response);
   responseStream.Seek(0, SeekOrigin.Begin);
  }
 }
}

The generic exception that we can throw

using System;
namespace WcfRestClient.WcfErrors
{
 public class WcfRestServiceException<TServiceError> : Exception where TServiceError : class
 {
  public WcfRestServiceException() : base("An service error occured. Please inspect the ServiceError property for details.")
  {
  }
  
  public TServiceError ServiceFault { get; set; }
 }
}