Monday, May 30, 2011

HOWTO - Empty SugarCrm ModuleLoader install history


Sometimes after installing many packages on a development installation of SugarCrm the Module Loader will take a long time to load. To regain the freshly installed SugarCrm speed of ModuleLoader you need to clean out the temp installation files (they will still be installed). You can use the commands below to clean it up to make it more responsive again. (This is working on 5.1.0b of SugarCrm, newer versions might behave differently).

Please note that you do lose the list of installed packages.

rm -f /<SUGARCRM_INSTALL_DIR>/cache/upload/*.zip
rm -fr /<SUGARCRM_INSTALL_DIR>/cache/upload/upgrades/module/*
rm -fr /<SUGARCRM_INSTALL_DIR>/cache/upload/upgrades/temp/*
# truncate table upgrade_history in instance database

Tuesday, May 24, 2011

Installing SugarCrm from scratch on CentOS 5.X

This posting outlines the Linux commands needed to install SugarCrm on a RHEL/CentOS 5.5 standard installation. This posting assumes that you are installing the database server on one server and the webserver on another server.

Before beginning I would like to change the SELinux configuration of the servers.


Since RHEL/CentOS out of the box has SELinux enabled i recommend changing the setting from enforcing to permissive. This will log all security breaches in a log and then you can later reconfigure SELinux by using the audit2allow script.
So for now we will permanently set SELinux to permissive.
In the /etc/sysconfig/selinux file you will something like the following

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=enforcing
...

Change SELINUX=enforcing to SELINUX=permissive and save the file.
Then run the following command

setenforce 0

First let us install the Apache webserver (or make sure it is updated) as well as php and the php modules needed to run SugarCRM.
It is important to note that Sugarcrm 5.1 only supports the standard PHP package with Centos 5.X (PHP 5.1.6 and NOT PHP 5.3)

yum install httpd
yum install php
yum install php-mbstring php-mysql php-gd gd php-imap php-ldap
yum install html2ps

We need to configure the iptables firewall to allow incomming trafic to HTTP/HTTPS and I will leave that as an exercise to the reader.


Now lets us install the APC PHP opcode cache. This is going to speed up the execution on the server. However please note that this can have some sideeffects when running SugarCRM. In particular if you are going to do a lot of customizations using Studio or ModuleBuilder. For that reason my preference is to only install and enable this on production sites. First lets us install the needed dependecies.

yum install php-pear php-devel httpd-devel pcre-devel

Now if you are running your server behind a proxy you need to configure pear to use that proxy otherwise the installation will fail. If you are not behind a proxy please ignore this. The syntax is pear config-set http_proxy http://myproxy.yourdomain.com:your_proxy_port/ and a example would be

pear config-set http_proxy http://myproxy.yourdomain.com:8080/

Installing APC

pecl install apc 

We need to configure the /etc/php.ini file, the below values are the values that I prefer to run with, you might have different requirements, please make sure you understand what you are changing and why.

memory_limit = 512M
max_execution_time = 300
error_reporting = E_ALL & ~E_NOTICE
post_max_size = 64M
upload_max_filesize = 64M

At the end of the php.ini file add the following lines. These lines tells PHP that we want to import the APC extension we installed above (on production servers). However we disable this APC module by default. This allows us to override and turn it on in a VirtualHost .htaccess file. This way we can have a test site running without APC and a production site running with APC on the same server (depending on your environment and expected production system load, this might be a good/bad idea).

extension=apc.so
[apc]
apc.cache_by_default=0 # disable by default

The default configuration of Apache when installed on RHEL/CentOS is to disable namebased virtual hosts, thus we need to enable this. (this is not strictly required since if you only plan on running one and one site only on this server you can do without, but I prefer to use namebased virtual hosts. So let us enable this in the /etc/httpd/conf/httpd.conf file. Find the following line and change as specified below.

#NameVirtualHost *:80

is changed to

NameVirtualHost *:80

Now we need to create a virtual host configuration file. I have a convention of naming the configuration file based on the DNS name of the site. The configuration file is placed in /etc/httpd/conf.d/DNS_NAME_OF_SITE.conf.

That would for a http://www.somewhere.com end up as a file named www.somewhere.com.conf.

Depending on your server setup you might want to use different values than what I have shown below, but the below configuration achives a pretty good Google PageSpeed score and also for Yahoo YSlow. However the caching is somewhat aggressive so you might have requirements that requires you to change the below values.

<VirtualHost *:80 >
  ServerName "HOSTNAME"
  DocumentRoot /var/www/sites/HOSTNAME
  DirectoryIndex index.html index.php
  ErrorLog logs/HOSTNAME-error_log
  CustomLog logs/HOSTNAME-access_log common

  <Directory  "/var/www/sites/HOSTNAME">
    SetOutputFilter DEFLATE
    AddDefaultCharset UTF-8

    AllowOverRide All
    Options FollowSymlinks

        # Enable ETag
        FileETag MTime Size

        # Set expiration header
        ExpiresActive on
        ExpiresDefault "access plus 1 year"
        ExpiresByType image/gif "access plus 1 month"
        ExpiresByType image/png "access plus 1 month"
        ExpiresByType image/jpeg "access plus 1 month"
        ExpiresByType text/css "access plus 1 month"
        ExpiresByType text/javascript "access plus 1 month"
        ExpiresByType text/js "access plus 1 month"

        # Compress some text file types
        AddOutputFilterByType DEFLATE text/html text/css text/xml application/x-javascript text/javascript text/js

        # Deactivate compression for buggy browsers
        BrowserMatch ^Mozilla/4 gzip-only-text/html
        BrowserMatch ^Mozilla/4\.0[678] no-gzip
        BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
  </Directory>
</VirtualHost>

We need to install java on the server since the preconfigured java does not work with ZuckerReports. This link was the latest version at the time of writing this posting. Please note that it is the jdk we are installing.

wget http://download.oracle.com/otn-pub/java/jdk/6u25-b06/jdk-6u25-linux-x64.bin
chmod 755 jdk-6u25-linux-x64.bin
./jdk-6u25-linux-x64.bin

Now we need to change the default java used by the system (please note the output of the java versions

java --version 
unlink /usr/bin/java
ln -s /usr/java/jdk1.6.0_04/bin/java /usr/bin/java
java -version

Now we will leave the webserver and move to the database server. First we need to install the mysql server.


yum install mysql-server

Before starting the mysql server I prefer to change the datadir location to a separate mount point in the Linux system rather than the default under /var/lib/mysql. That is done by editing the /etc/my.cnf file and changing the value of the datadir variable.

datadir=/<MY_CUSTOM_DATADIR_DIRECTORY_PATH>/mysql

# an example 
datadir=/data/mysql

Now start the mysql server and set it to autostart when the Linux server is started.

service mysqld start
chkconfig mysqld on

Now let us change a few more parameters in the my.cnf file (some of these parameter can have a breaking effect if you change them before the first start of the mysql server. That is why we are waiting and doing the configuration in 2 steps).
Change the /etc/my.cnf file and add the following variables under the [mysqld] region.

log-warnings=2
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
default-storage-engine=innodb

restart the mysqld server.

service mysqld restart

now we need to set a password for the root user.

mysqladmin -u root -p password '<MY_ROOT_PASSWORD>'
Create a database for the SugarCRM installation.

mysql -uroot -p<MY_ROOT_PASSWORD>
create database sugarcrm;

Create users and grant permissions to the users that need it.
Assuming that we want to allow access from localhost, and from remote Apache server

CREATE USER 'sugarcrm_user'@'localhost' IDENTIFIED BY 'mypass';
GRANT ALL ON sugarcrm.* TO 'sugarcrm_user'@'localhost'; 
CREATE USER 'sugarcrm_user'@'127.0.0.1' IDENTIFIED BY 'mypass';
GRANT ALL ON sugarcrm.* TO 'sugarcrm_user'@'127.0.0.1';
CREATE USER 'sugarcrm_user'@'<APACHE_SERVER_IP_ADDRESS>' IDENTIFIED BY 'mypass';
GRANT ALL ON sugarcrm.* TO 'sugarcrm_user'@'<APACHE_SERVER_IP_ADDRESS>'; 

Now if you plan on installing custom built modules into SugarCrm where the installer package contains MySQL trigger code that needs to be executed by the sugarcrm_user database user. MySQL 5.0 contains a less than optimal permission implementation. So to be able to install/create triggers the user needs SUPER permission. And since the default database version that is installed in RHEL/CentOS 5.5 at the time of writing this blog post is 5.0.77 we need to grant SUPER to our database user.

GRANT SUPER on *.* to 'sugarcrm_user'@'localhost';
GRANT SUPER on *.* to 'sugarcrm_user'@'127.0.0.1';
GRANT SUPER on *.* to 'sugarcrm_user'@'<APACHE_SERVER_IP_ADDRESS>';

Now we need to configure the iptables firewall to allow incomming trafic to MySQL.

iptables -A INPUT -i eth0 -s <APACHE_SERVER_IP_ADDRESS> -p tcp --destination-port 3306 -j ACCEPT
service iptables save

Now we are ready to install SugarCrm either using the installer from SugarCrm (please follow the official installation guide) or cloning an existing site

- tar the site

tar -czf mysite.somewhere.com.tar.gz mysite.somewhere.com

dump the staging sugarcrm mysql database (changing the tables to InnoDB and remove DEFINER statements from the dump)

mysqldump -h<HOSTNAME> -u<USERNAME> -p<PASSWORD> --triggers --routines <DATABASE> | perl -pe 's/MyISAM/InnoDB/' |  perl -pe 's{(/\*!\d+ )\s*definer\s*=\s*\S+}{$1}i' > mysite.somewhere.com.sql
gzip mysite.somewhere.com.sql

Move the tar site dump and mysql dump to the production server and unzip/untar the site dump

wget http://my.staging.server/mysite.somewhere.com.tar.gz
wget http://my.staging.server/mysite.somewhere.com.sql.gz
tar -xf mysite.somewhere.com.tar.gz
mv mysite.somewhere.com /var/www/sites/HOSTNAME  #(HOSTNAME as defined in the Apache virtualhost conf file)
chown -R apache:apache /var/www/sites/HOSTNAME  #(HOSTNAME as defined in the Apache virtualhost conf file)

#Secure directories
chmod -R 644 *
find . -type d | xargs chmod 755

chmod 664 config.php
chmod 775 ./custom 
chmod 775 ./data/upload 
chmod 775 ./data/upload 
chmod 775 ./cache 
chmod 775 ./cache/images 
chmod 775 ./cache/import 
chmod 775 ./cache/layout 
chmod 775 ./cache/pdf 
chmod 775 ./cache/upload 
chmod 775 ./cache/xml 
chmod 775 ./modules

install the mysql database dump

gunzip mysite.somewhere.com.sql.gz
mysql -h<DATABASEHOSTNAME> -usugarcrm_user -pmypass -D sugarcrm < mysite.somewhere.com.sql
edit the /var/www/sites/HOSTNAME/config.php file and make sure the following variables are updated with your own values
'dbconfig' =>
  array (
    'db_host_name' => '',
    'db_host_instance' => 'SQLEXPRESS',
    'db_user_name' => '',
    'db_password' => '',
    'db_name' => '',
    'db_type' => 'mysql',
'host_name' => '',
'site_url' => 'http://',
'unique_key' => '',
unique_key should be unique for each different server.

IMPORTANT!: Edit and update the /SITE_DIRECTORY/.htaccess file to match the destination DNS sitename.

If this is a production server enable APC

comment the following line
#RedirectMatch /modules/(.*)/(.*).php http://XXXXX/index.php

add the following line
php_flag apc.cache_by_default On
After running the site for a while, tweak your MySQL config file using MySQLTuner
wget --no-check-certificate https://raw.github.com/rackerhacker/MySQLTuner-perl/master/mysqltuner.pl

perl mysqltuner.pl