Monday, September 19, 2011

SugarCrm: Continous / automated installation of packages


I needed to find a way to faster go from a bug being reported "Development Complete" to it being installed on a test installation for the testers so they could begin testing. We are not hosting the entire sugarcrm site in SVN and also deployment to the final production system is done by a third party so all we can so is send them the package zip files. So it needed to follow the Module Loader principle.

This script is run by cron at an timed interval (I am using 10 minutes).

Disclaimer
The script could be cleaned up by extracting the svn username and password into configuration variables.

The other suboptimal issue is:
When you are installing multiple packages this way and all packages are having post_install.php files defining the same functions you will get an error. The reason is that the post_install.php file currently is just a php file with named functions in it.The correct way to solve this would be to move the post_install.php functions inside a class. However that was not an option for me so I needed to handle the re-including of functions with the same name. To be able to do this I am renaming the already included functions.


#!/usr/bin/php
<?php
/**
* You must install 
* 
* yum install php-pear
* pecl install apd
* 
* and enabled in the php.ini file 
* 
* [Zend]
* zend_extension = /usr/lib/php/modules/apd.so
* 
* for this to work
*/


//make sure we dump the stuff to stderr
ini_set("error_log","php://stderr");

$svnExportDir='/mnt/';
$siteInstalledVersionStateFilesDir='/mnt/';

$svnPackages = array (
 // Define your packages here and the full SVN url
 'mypackage' => 'https://XXX/svn/mypackage/trunk',
 'mypackage2' => 'https://XXX/svn/mypackage2/trunk',
);

$preInstallFunctions = array (
 // If your package are using pre_install hooks and you are installing multiple packages and the pre_install hooks contain the same functions
 // then you need to define them here. This is hack, if the pre_install functions was defined in a class then we could just unset the class.
    "pre_install",
);

$postInstallFunctions = array (
 // If your package are using post_install hooks and you are installing multiple packages and the pre_install hooks contain the same functions
 // then you need to define them here. This is hack, if the post_install functions was defined in a class then we could just unset the class.
    "post_install",
);
 
$sugarcrmInstallationsToUpdate = array (
  0 => array(
    'name' => 'SugarCrmInstallation1 (staging)',
    'sugarInstallDir' => '/var/www/sites/staging',
    'packagesToUpdate' => array (
        0 => 'mypackage',
        1 => 'mypackage2',
    ),
  ),
);


//initialize
if(!defined('sugarEntry'))  define('sugarEntry', true);

// If you are updating more than one installation there is no need to do the svn export more than one time per
// package, just reuse that already downloaded/exported package
$packageDownloadedFromSvnRevisionLog = $siteInstalledVersionStateFilesDir . 'ci_packageDownloadedFromSvnRevision.log';
$packageVersionsDownloaded = getArrayFromFile($packageDownloadedFromSvnRevisionLog);

foreach($sugarcrmInstallationsToUpdate as $sugarCrmSite){
    chdir($sugarCrmSite['sugarInstallDir']);
    require_once('include/entryPoint.php');
    require_once('ModuleInstall/ModuleInstaller.php');
    $current_user = new User();
    $current_user->is_admin = '1';
    
    $siteInstalledVersionsStateFile = $siteInstalledVersionStateFilesDir . $sugarCrmSite['name'] . '_version';
    $revisionNumberInstalledInSite = getArrayFromFile($siteInstalledVersionsStateFile);
    
    foreach($sugarCrmSite['packagesToUpdate'] as $key=>$package){
        $packageRevisionInstalled = array_key_exists($package, $revisionNumberInstalledInSite) ?  $revisionNumberInstalledInSite[$package] : 0;

        ciLog ("Checking for newer versions in svn ...\n", $sugarCrmSite['name'],$package);
        $packageRevisionInSvn = trim(`svn info --non-interactive --username YOUR_SVN_USERNAME --password YOUR_SVN_PASSWORD $svnPackages[$package] | grep 'Last Changed Rev' | head -1 | grep -Eo "[0-9]+"`);
        
        if ($packageRevisionInstalled < $packageRevisionInSvn) {
   ciLog ("There are updates in SVN (installed: $packageRevisionInstalled, svn: $packageRevisionInSvn).\n", $sugarCrmSite['name'],$package);
            
            $packageRevisionPreviouslyDownloaded = array_key_exists($package, $packageVersionsDownloaded) ?  $packageVersionsDownloaded[$package] : 0;
            if ($packageRevisionPreviouslyDownloaded < $packageRevisionInSvn){
    ciLog ("Getting latest version from svn ...\n", $sugarCrmSite['name'],$package);

                $output = `svn export --force --non-interactive --username YOUR_SVN_USERNAME --password YOUR_SVN_PASSWORD $svnPackages[$package] $svnExportDir$package`;
                $packageVersionsDownloaded[$package] = $packageRevisionInSvn;
                saveArrayToFile($packageDownloadedFromSvnRevisionLog, $packageVersionsDownloaded);
            }
            else {
    ciLog ("(rev $packageRevisionInSvn) has previously been downloaded, using local version.\n", $sugarCrmSite['name'],$package);
            }

            //initialize the module installer
            $modInstaller = new ModuleInstaller();
            $modInstaller->silent = true;  //shuts up the javscript progress bar

            //start installation
   ciLog ("(rev $packageRevisionInSvn) Starting installation into " . $sugarCrmSite['sugarInstallDir'] . " ... \n", $sugarCrmSite['name'],$package);
   
   $preInstallFile = "$svnExportDir$package/scripts/pre_install.php";
   if(is_file($preInstallFile))
   {
    ciLog ("Including $preInstallFile.\n", $sugarCrmSite['name'],$package);
    include($preInstallFile);
    ciLog ("Executing $preInstallFile.\n", $sugarCrmSite['name'],$package);
    pre_install();
                
                // Undeclaring the pre_install functions so we can include the next pre_install file. Nasty, this is a hack. Requires PECL ADP 
                undeclareFunctions($preInstallFunctions);
   }
            $modInstaller->install($svnExportDir.$package);

   $postInstallFile = "$svnExportDir$package/scripts/post_install.php";
   if(is_file($postInstallFile))
   {
                $_REQUEST['unzip_dir'] = "$svnExportDir$package";
    ciLog ("Including $postInstallFile.\n", $sugarCrmSite['name'],$package);
    include($postInstallFile);
    ciLog ("Executing $postInstallFile.\n", $sugarCrmSite['name'],$package);
    post_install();

                // Undeclaring the post_install functions so we can include the next pre_install file. Nasty, this is a hack. Requires PECL ADP 
                undeclareFunctions($postInstallFunctions);
   }
   ciLog ("(rev $packageRevisionInSvn) Installation into " . $sugarCrmSite['sugarInstallDir'] . " is done.\n", $sugarCrmSite['name'],$package);
            
            $revisionNumberInstalledInSite[$package] = $packageRevisionInSvn;
        }
        else {
   ciLog ("No updates found (installed: $packageRevisionInstalled, svn: $packageRevisionInSvn).\n", $sugarCrmSite['name'],$package);
        }
    }
    // ciLog what package revisions that were installed
    saveArrayToFile($siteInstalledVersionsStateFile, $revisionNumberInstalledInSite);
    
    // Make sure apache owns the sugar installation
    $sugarInstallDir = $sugarCrmSite['sugarInstallDir'];
    $output = `chown -R apache:apache $sugarInstallDir`;
}

function undeclareFunctions($functions){
    foreach($functions as $function) {
       if (function_exists($function)) { rename_function($function, uniqid()); } 
    }   
}

function ciLog($message, $site, $package){
 $fullMessage = "%s %s-%s: " . $message;
 printf($fullMessage, date('m/d/Y H:i:s'), $site, $package);
}

function getArrayFromFile($filename){
    if (file_exists($filename)) {
        return unserialize(file_get_contents($filename));
    }
    else {
        return array();
    }
}

function saveArrayToFile($filename, $arrayToSave){
    $fh = fopen($filename, 'w') or die("can't open file");
    fwrite($fh, serialize($arrayToSave));
    fclose($fh);
}
?>

2 comments:

Unknown said...

Kenneth, I think that this is extremely useful.

If I understand correctly, you are deploying packages automatically.

I could really use to talk to you, can you contact me at michaelwjoyner{at}gmail[dot]com.

I have a couple of simple questions.

Kenneth Thorman said...

Hi Michael

I prefer to answer the questions here so others might benefit from any answer that I might be able to give.

Can you post your questions here?

Regards
Kenneth