Showing posts with label SugarCrm. Show all posts
Showing posts with label SugarCrm. Show all posts

Monday, June 04, 2012

SugarCrm 6.3.1: Enhancing Importer - allow importing of related module data

Often when importing data into SugarCrm the built-in importer at a module level is working great. However if you have a need to import data that is actually modeled in SugarCrm as being 2 modules (tables) with a many-many relationship in between then the built-in importer comes up short.

The full patch can be found at the bottom of this posting. This is a non upgradesafe change. Apply at your own risk.

Let us assume that we have the following entity relational model in SugarCrm that has been configured in SugarCrm Studio or in SugarCrm Module Builder.


Module Builder

The Sugar Module Builder enables users to build custom modules from scratch or combine existing or custom objects into a brand new CRM module. Developers can leverage existing Sugar Objects such as Contacts, Accounts, Documents, Cases, and Opportunities to build a new module or create their own custom objects from scratch to form a new module. Users can build an unlimited number of custom modules, which interoperate seamlessly with Reporting, Workflow, and Sugar Studio tools. Upgrades for custom modules are fully supported. Building new modules allows developers to extend Sugar beyond the typical CRM functions and optimize Sugar for any xRM (any Relationship Management) function.

Positive Impacts

  • Create custom modules to track information critical to your business
  • Use pre-defined objects or create custom objects for the new module
  • Share or charge for custom objects on SugarForge and Sugar Exchange.

Sugar Studio

Sugar Studio is the starting place for an administrator to configure the way information is presented in Sugar. Administrators can use Sugar Studio to create and add custom fields, hide fields that are not relevant, and use the extensive customization capabilities of Sugar Logic to create calculated, dependent, and related fields. Sugar Studio is a very simple but powerful WYSIWYG interface that administrators and developers use to configure Sugar to complement a company’s existing business processes.

Positive Impacts

  •  Rearrange the order of fields according to your company’s requirements
  •  Hide fields that are not relevant to your business process
  •  Create and add new custom fields
  •  Calculate variable values based on the value of other fields like opportunity amount or expected close date
  •  Present fields only when necessary using dependent fields


If importing has been enabled for the Skill and the Language module when these modules were created then we can import Skill or Languages as standalone data. The Contact module is a built-in SugarCrm module and it has importing enabled but can also only import contact standalone data. 




There are a few challenges importing related data into SugarCrm:
  1. We need to handle relationships and splitting data into the correct modules.
  2. We need to handle the fact that the id column might not contain the actual id of the data but that another column might do instead.

Without the 2 challenges solved we will not be able to import the below data the way I would prefer.


What we really want in the database after importing the above data is.
  • 2 contacts: John and Jane
  • 3 skills: English literature, European history and PHP
  • 4 languages: English, German, Danish and Swedish 
  • as well as the many-many relations needed so we can recreate the data pictured above

The 2 above mentioned challenges have been solved the following ways:

#1: In the importer step 3 we have added all the "main module's" (Contacts in this case) related modules' fields in the module field dropdown. This way you can map any data  you want into a related module's field.

#2: A primary key field selector has been added on Step 2 in the import if you have selected "Create new records and update existing records" on Step 1 in the importer.




Lets look at the proposed solution.

Step 1: the importer has not been modified visually and looks as below.

(Please note the support for saving an import configuration is working with the new importer functionality).



Step 2: We have added a new field that allows you to select the field that is to be used as "id". It defaults to the standard importer behavior which is using the id column.



Step 3: We have added a marker for the field used as primary id ( purple box below ). Please note the text in the dropdown marked with red boxes. The field selected in the dropdown is prefixex with a module name (<modulename>.<fieldname>) this indicates that it is a field from a related module you want to map to.

As you can see from the screenshot below, the module field dropdown now contains all fields from the main module (Contacts) (main module fields are not prefixed with the module name), as well as the related modules fields (skill_Skill, and lang_Language, these fields are prefixed with the module name).



Note: Do not map any value to a related module's "ID" field. This is handled behind the scenes.


Step 4: No visual changes has happened here



There are quite a few code changes to add this new functionality into the importer and you can see the Subversion patch file below.
Index: include/database/DBHelper.php
===================================================================
--- include/database/DBHelper.php (revision 73)
+++ include/database/DBHelper.php (working copy)
@@ -1185,8 +1185,8 @@
                          $before_value=(float)$bean->fetched_row[$field];
                          $after_value=(float)$bean->$field;
                      } else {
-                         $before_value=$bean->fetched_row[$field];
-                         $after_value=$bean->$field;
+                         $before_value=$bean->fetched_row[$field];   
+                         $after_value=@$bean->$field;
                    }
 
      //Because of bug #25078(sqlserver haven't 'date' type, trim extra "00:00:00" when insert into *_cstm table). so when we read the audit datetime field from sqlserver, we have to replace the extra "00:00:00" again.
Index: modules/Import/Importer.php
===================================================================
--- modules/Import/Importer.php (revision 73)
+++ modules/Import/Importer.php (working copy)
@@ -59,6 +59,7 @@
      */
     private $importColumns;
 
+ private $importRelatedColumns;
     /**
      * @var importSource
      */
@@ -104,9 +105,11 @@
         //Get the default user currency
         $this->defaultUserCurrency = new Currency();
         $this->defaultUserCurrency->retrieve('-99');
-
+        
         //Get our import column definitions
         $this->importColumns = $this->getImportColumns();
+  $this->importRelatedColumns = $this->getRelatedImportColumns();
+
         $this->isUpdateOnly = ( isset($_REQUEST['import_type']) && $_REQUEST['import_type'] == 'update' );
     }
 
@@ -132,7 +135,7 @@
     protected function importRow($row)
     {
         global $sugar_config, $mod_strings;
-
+  global $beanList,$beanFiles;
         $focus = clone $this->bean;
         $focus->unPopulateDefaultValues();
         $focus->save_from_post = false;
@@ -140,6 +143,11 @@
         $this->ifs->createdBeans = array();
         $this->importSource->resetRowErrorCounter();
         $do_save = true;
+        
+        $primary_key = (isset($_REQUEST['primary_key']) && !empty($_REQUEST['primary_key'])) ? $_REQUEST['primary_key'] : false;
+        if(!$primary_key){
+            $primary_key = 'id';
+        }
 
         for ( $fieldNum = 0; $fieldNum < $_REQUEST['columncount']; $fieldNum++ )
         {
@@ -322,46 +330,58 @@
                 return;
             }
         }
-
+        
+        $focus_array = array();
+        $newRecord_array = array();
+        
         // if the id was specified
         $newRecord = true;
-        if ( !empty($focus->id) )
+        if ( !empty($focus->$primary_key) )
         {
-            $focus->id = $this->_convertId($focus->id);
-
+            if($primary_key=='id')
+                $focus->id = $this->_convertId($focus->$primary_key);
             // check if it already exists
-            $query = "SELECT * FROM {$focus->table_name} WHERE id='".$focus->db->quote($focus->id)."'";
+            $join = '';
+            $_def = $focus->getFieldDefinition($primary_key);
+            if($_def['source'] == 'custom_fields') $join = "JOIN {$focus->table_name}_cstm ON id_c = id";
+            $query = "SELECT * FROM {$focus->table_name} $join WHERE $primary_key='".$focus->db->quote($focus->$primary_key)."'";
             $result = $focus->db->query($query)
             or sugar_die("Error selecting sugarbean: ");
 
-            $dbrow = $focus->db->fetchByAssoc($result);
+            while($dbrow = $focus->db->fetchByAssoc($result)){
 
-            if (isset ($dbrow['id']) && $dbrow['id'] != -1)
+                if (isset ($dbrow[$primary_key]) && $dbrow[$primary_key] != -1)
             {
-                // if it exists but was deleted, just remove it
-                if (isset ($dbrow['deleted']) && $dbrow['deleted'] == 1 && $this->isUpdateOnly ==false)
+                    $focus->id = $dbrow['id'];
+                    // if it exists but was deleted, just remove it                                         //skiping deleted rows
+                    if (isset ($dbrow['deleted']) && $dbrow['deleted'] == 1 && $this->isUpdateOnly == false /*&& $primary_key=='id'*/)
                 {
                     $this->removeDeletedBean($focus);
                     $focus->new_with_id = true;
                 }
                 else
                 {
+                        //skiping deleted rows
+                        if(isset ($dbrow['deleted']) && $dbrow['deleted'] == 1 ) continue;
                     if( ! $this->isUpdateOnly )
                     {
-                        $this->importSource->writeError($mod_strings['LBL_ID_EXISTS_ALREADY'],'ID',$focus->id);
+                            $this->importSource->writeError($mod_strings['LBL_ID_EXISTS_ALREADY'],$primary_key,$focus->$primary_key);
                         $this->_undoCreatedBeans($this->ifs->createdBeans);
                         return;
                     }
-
+                        
                     $clonedBean = $this->cloneExistingBean($focus);
                     if($clonedBean === FALSE)
                     {
-                        $this->importSource->writeError($mod_strings['LBL_RECORD_CANNOT_BE_UPDATED'],'ID',$focus->id);
+                            $this->importSource->writeError($mod_strings['LBL_RECORD_CANNOT_BE_UPDATED'],$primary_key,$focus->$primary_key);
                         $this->_undoCreatedBeans($this->ifs->createdBeans);
                         return;
                     }
                     else
                     {
+                            $focus_array[] = clone $clonedBean;
+                            $newRecord_array[] = FALSE;
+                            
                         $focus = $clonedBean;
                         $newRecord = FALSE;
                     }
@@ -369,19 +389,100 @@
             }
             else
             {
+                    if($focus->id){
                 $focus->new_with_id = true;
+                    }else{
+                        $focus->new_with_id = false;
+                        $focus_array[] = $this->cloneExistingBean($focus);
+                        $newRecord_array[] = true;
             }
         }
-
+            }
+        }
+        if(count($focus_array)==0){
+            $focus_array[] = $focus;
+            $newRecord_array[] = $newRecord;
+        }
+        for($focus_i=0,$focus_count=count($focus_array);$focus_i<$focus_count;$focus_i++){
+            $focus = $focus_array[$focus_i];
+            $newRecord = $newRecord_array[$focus_i];
         if ($do_save)
         {
             $this->saveImportBean($focus, $newRecord);
+       $mainModuleId = $focus->id;
+       //Save records for related module
+       foreach($this->importRelatedColumns as $k=>$v)
+       {
+         $exploedFields = explode("$",$v);
+         $relatedModule[] =  $exploedFields[0];
+         $relatedModuleField[$exploedFields[0]][] = $exploedFields[1];
+       }
+                if(isset($relatedModule) && is_array($relatedModule))
+       $relatedModules = array_unique($relatedModule);
+                if(isset($relatedModule) && is_array($relatedModule))
+                    foreach($relatedModuleField as $modName => $fldsArr)
+        {
+         $haveValue = 0;
+         $relatedModuleFile = $beanList[$modName];
+         include_once($beanFiles[$relatedModuleFile]);
+         $relatedModuleObj = new $relatedModuleFile();
+         $whereArr = array();
+                        $is_custom = false;
+         foreach($fldsArr as $k=>$fldName)
+         {
+          $fld = $modName."$".$fldName;
+          $fieldNo = array_search($fld,$this->importRelatedColumns);
+          if($row[$fieldNo]!="")
+          {
+           $haveValue++;
+           $relatedModuleObj->$fldName = $row[$fieldNo];
+           $whereArr[] = $fldName ."='". $row[$fieldNo]."'";
+                                if(@$relatedModuleObj->field_defs[$fldName]['source'] == 'custom_fields')  $is_custom = true;
+          }
+         }
+         if($haveValue)
+         {
+          $where = "";
+          $where = implode(" AND ",$whereArr);
+          $sql = "select id from ".$relatedModuleObj->table_name;
+          if($is_custom) $sql .= ' inner join '.$relatedModuleObj->table_name.'_cstm on id=id_c';
+                            $sql .= " where ".$where." and deleted=0";
+                            $result2 = $relatedModuleObj->db->query($sql);
+          $rowQuery = $relatedModuleObj->db->fetchByAssoc($result2);
+          if(count($rowQuery)==0 || empty($rowQuery))
+          {
+           $this->saveImportBean($relatedModuleObj, $newRecord); 
+           $relatedId = $relatedModuleObj->id;
+          }
+          else
+          {
+           $relatedId = $rowQuery['id'];
+          }
+          $rel_table = strtolower($relatedModuleObj->table_name);
+                            $foc_table = strtolower($focus->table_name);
+                            if(strlen($rel_table)>16) $rel_table = substr($rel_table,0,11).substr($rel_table,-5);
+                            if(strlen($foc_table)>16) $foc_table = substr($foc_table,0,11).substr($foc_table,-5);
+                            $rel_field = $foc_table.'_'.$rel_table;
+                            if(!$focus->load_relationship($rel_field)){       
+                                $rel_field = $rel_table.'_'.$foc_table;
+                                if(!$focus->load_relationship($rel_field)){
+                                    $GLOBALS['log']->error("SugarBean.load_relationships, relationship not found.");
+                                    $rel_field = false;
+                                }
+                            }
+                            if($rel_field){
+                                $focus->$rel_field->add($relatedId,array());
+                            }
+                             $focus->save();
+         }
+        }
+                   
             // Update the created/updated counter
             $this->importSource->markRowAsImported($newRecord);
         }
         else
             $this->_undoCreatedBeans($this->ifs->createdBeans);
-
+        }
         unset($defaultRowValue);
 
     }
@@ -544,11 +645,15 @@
         
         $firstrow    = unserialize(base64_decode($_REQUEST['firstrow']));
         $mappingValsArr = $this->importColumns;
-        $mapping_file = new ImportMap();
+        $mappingRelatedArr = $this->importRelatedColumns;
+        $mappArr = $mappingRelatedArr+$mappingValsArr;
+        $mapping_file = new ImportMap();     
+        
+        
         if ( isset($_REQUEST['has_header']) && $_REQUEST['has_header'] == 'on')
         {
             $header_to_field = array ();
-            foreach ($this->importColumns as $pos => $field_name)
+            foreach ($mappArr as $pos => $field_name)
             {
                 if (isset($firstrow[$pos]) && isset($field_name))
                 {
@@ -557,6 +662,8 @@
             }
 
             $mappingValsArr = $header_to_field;
+        }      else{
+         $mappingValsArr =    $mappArr;
         }
         //get array of values to save for duplicate and locale settings
         $advMapping = $this->retrieveAdvancedMapping();
@@ -679,7 +786,26 @@
 
         return $importColumns;
     }
+ protected function getRelatedImportColumns()
+    {
+        $importRelatedColumns = array();
+        foreach ($_REQUEST as $name => $value)
+        {
+            // only look for var names that start with "fieldNum"
+            if (strncasecmp($name, "colnum_", 7) != 0)
+                continue;
 
+            // pull out the column position for this field name
+            $pos = substr($name, 7);
+
+            if ( strpos($value,"$") )
+            {
+                // now mark that we've seen this field
+                $importRelatedColumns[$pos] = $value;
+            }
+        }
+        return $importRelatedColumns;
+    }
     protected function getFieldSanitizer()
     {
         $ifs = new ImportFieldSanitize();
Index: modules/Import/tpls/confirm.tpl
===================================================================
--- modules/Import/tpls/confirm.tpl (revision 73)
+++ modules/Import/tpls/confirm.tpl (working copy)
@@ -170,8 +170,30 @@
         </tr>
         <tr>
             <td colspan="2"><div class="hr" style="margin-top: 0px;"></div></td>
+        </tr>    
+        {if $TYPE != 'import'}
+        <tr>
+            <td colspan="2"><h3>Select primaty field&nbsp;{sugar_help text="It will update the row with the field equal to the value of the csv row"}</h3></td>
         </tr>
         <tr>
+            <td colspan="2">
+                <select name="primary_key">
+                    {foreach from=$SAMPLE_ROWS item=row name=row}
+                        {foreach from=$row item=value}
+                            {if $smarty.foreach.row.first}
+                                {if $HAS_HEADER}
+                                    <option value="{$value}"{if $value|lower == 'id'} selected='selected' {/if}>{$value}</option>
+                                {/if}
+                            {/if}
+                        {/foreach}
+                    {/foreach}
+                </select>
+            </td>
+        </tr>
+        {else}
+            <input name="primary_key" value="" type="hidden" />
+        {/if}                                                                                                 
+        <tr>
             <td colspan="2"><h3>{$MOD.LBL_THIRD_PARTY_CSV_SOURCES}&nbsp;{sugar_help text=$MOD.LBL_THIRD_PARTY_CSV_SOURCES_HELP}</h3></td>
         </tr>
         <tr>
Index: modules/Import/tpls/dupcheck.tpl
===================================================================
--- modules/Import/tpls/dupcheck.tpl (revision 73)
+++ modules/Import/tpls/dupcheck.tpl (working copy)
@@ -84,6 +84,7 @@
 <input type="hidden" id="enabled_dupes" name="enabled_dupes" value="">
 <input type="hidden" id="disabled_dupes" name="disabled_dupes" value="">
 <input type="hidden" id="current_step" name="current_step" value="{$CURRENT_STEP}">
+<input type="hidden" name="primary_key" value="{$primary_field}">
 
    <div class="hr"></div>
     <div>
Index: modules/Import/tpls/step3.tpl
===================================================================
--- modules/Import/tpls/step3.tpl (revision 73)
+++ modules/Import/tpls/step3.tpl (working copy)
@@ -118,10 +118,34 @@
 {/if}
 <tr>
     {if $HAS_HEADER == 'on'}
-    <td id="row_{$smarty.foreach.rows.index}_header">{$item.cell1}</td>
+    <td id="row_{$smarty.foreach.rows.index}_header">
+        {if $primary_field==$item.cell1}
+            <b>{$item.cell1}
+            <small>*Primary Field</small>
+            </b>
+            <input type="hidden" name="primary_key" value="{$primary_field}" id="primary_key" />
+            <script>
+                {literal}
+                (function(obj, evType, fn){ 
+                    if (obj.addEventListener){ 
+                       obj.addEventListener(evType, fn, false); 
+                       return true; 
+                     } else if (obj.attachEvent){ 
+                        var r = obj.attachEvent("on"+evType, fn); 
+                        return r; 
+                     } else { 
+                        return false; 
+                     } 
+                })(window,'load',function(){document.getElementById('primary_key').value=document.getElementById('primary_key_list').value})
+                {/literal}
+            </script>
+        {else}
+            {$item.cell1}
     {/if}
+    </td>
+    {/if}
     <td valign="top" align="left" id="row_{$smarty.foreach.rows.index}_col_0">
-        <select class='fixedwidth' name="colnum_{$smarty.foreach.rows.index}">
+        <select class='fixedwidth' name="colnum_{$smarty.foreach.rows.index}"{if $primary_field==$item.cell1} id="primary_key_list" onblur="document.getElementById('primary_key').value = this.value" {/if}>
             <option value="-1">{$MOD.LBL_DONT_MAP}</option>
             {$item.field_choices}
         </select>
Index: modules/Import/views/view.confirm.php
===================================================================
--- modules/Import/views/view.confirm.php (revision 73)
+++ modules/Import/views/view.confirm.php (working copy)
@@ -63,7 +63,7 @@
         global $sugar_config, $locale;
         
         $this->ss->assign("IMPORT_MODULE", $_REQUEST['import_module']);
-        $this->ss->assign("TYPE",( !empty($_REQUEST['type']) ? $_REQUEST['type'] : "import" ));
+        $this->ss->assign("TYPE",( !empty($_REQUEST['type']) ? $_REQUEST['type'] : ( !empty($_REQUEST['import_type'])?$_REQUEST['import_type'] : "import") ));
         $this->ss->assign("SOURCE_ID", $_REQUEST['source_id']);
 
         $this->instruction = 'LBL_SELECT_PROPERTY_INSTRUCTION';
Index: modules/Import/views/view.dupcheck.php
===================================================================
--- modules/Import/views/view.dupcheck.php (revision 73)
+++ modules/Import/views/view.dupcheck.php (working copy)
@@ -73,6 +73,7 @@
         $this->ss->assign("IMPORT_MODULE", $_REQUEST['import_module']);
         $this->ss->assign("CURRENT_STEP", $this->currentStep);
         $this->ss->assign("JAVASCRIPT", $this->_getJS());
+        $this->ss->assign("primary_field", @$_REQUEST['primary_key'] ? $_REQUEST['primary_key'] : 'id' );
 
         $content = $this->ss->fetch('modules/Import/tpls/dupcheck.tpl');
         $this->ss->assign("CONTENT", $content);
Index: modules/Import/views/view.last.php
===================================================================
--- modules/Import/views/view.last.php (revision 73)
+++ modules/Import/views/view.last.php (working copy)
@@ -109,6 +109,8 @@
         $this->ss->assign("errorrecordsFile",ImportCacheFiles::getErrorRecordsWithoutErrorFileName());
         $this->ss->assign("dupeFile",ImportCacheFiles::getDuplicateFileName());
         
+        $this->ss->assign("primary_field", @$_REQUEST['primary_key'] ? $_REQUEST['primary_key'] : 'id' );
+        
         if ( $this->bean->object_name == "Prospect" )
         {
          $this->ss->assign("PROSPECTLISTBUTTON", $this->_addToProspectListButton());
Index: modules/Import/views/view.step3.php
===================================================================
--- modules/Import/views/view.step3.php (revision 73)
+++ modules/Import/views/view.step3.php (working copy)
@@ -63,7 +63,7 @@
   public function display()
     {
         global $mod_strings, $app_strings, $current_user, $sugar_config, $app_list_strings, $locale;
-        
+        global $beanList,$beanFiles;
         $this->ss->assign("IMPORT_MODULE", $_REQUEST['import_module']);
         $has_header = ( isset( $_REQUEST['has_header']) ? 1 : 0 );
         $sugar_config['import_max_records_per_file'] = ( empty($sugar_config['import_max_records_per_file']) ? 1000 : $sugar_config['import_max_records_per_file'] );
@@ -200,7 +200,91 @@
         if (!empty($importColumns)) {
             $column_sel_from_req = true;
         }
-
+        foreach($this->bean->field_defs as $field)
+        {
+            if(0 == strcmp($field['type'],'link') && (!empty($field['module']) || !empty($field['relationship'])))
+            {
+                if(!empty($field['module']))
+                {
+                    $related_module = $field['module'];
+                }
+                elseif(!empty($field['relationship']))
+                {
+                    require_once("data/Relationships/RelationshipFactory.php");
+                    $test = SugarRelationshipFactory::getInstance()->getRelationship($field['relationship']);
+                    if($test->def['relationships'][$field['relationship']]['lhs_module'] == $this->bean->module_dir)
+                    {
+                        $relatedMods[] = $test->def['relationships'][$field['relationship']]['rhs_module'];
+                    }
+                    else
+                    {
+                        $relatedMods[] = $test->def['relationships'][$field['relationship']]['lhs_module'];
+                    }
+                }
+            }
+        }
+        //add extra related fields
+        foreach($relatedMods as $key=>$relatedModule)
+        {
+                if(array_key_exists($relatedModule,$beanList))
+                {
+                $relatedModuleFile = $beanList[$relatedModule];
+                include_once($beanFiles[$relatedModuleFile]);
+                $relatedModuleObj = new $relatedModuleFile();
+                $fieldsRel  = $relatedModuleObj->get_importable_fields();
+                $relModuleStrings = return_module_language($current_language, $relatedModuleObj->module_dir);
+                foreach ( $fieldsRel as $fieldname => $properties ) {
+                    if($properties['relationship'])
+                    continue;
+                    // get field name
+                    if (!empty($relModuleStrings['LBL_EXPORT_'.strtoupper($fieldname)]) )
+                    {
+                         $displayname = str_replace(":","", $relModuleStrings['LBL_EXPORT_'.strtoupper($fieldname)] );
+                    }
+                    else if (!empty ($properties['vname']))
+                    {
+                        $displayname = str_replace(":","",translate($properties['vname'] ,$relatedModuleObj->module_dir));
+                    }
+                    else
+                        $displayname = str_replace(":","",translate($properties['name'] ,$relatedModuleObj->module_dir));
+                    
+                    $selected = '';   
+                    if ($column_sel_from_req && isset($importColumns[$field_count])) {
+                        if ($fieldname == $importColumns[$field_count]) {
+                            //$selected = ' selected="selected" ';
+                            $defaultField = $fieldname;
+                            $mappedFields[] = $fieldname;
+                        }
+                    } else {
+                        if ( !empty($defaultValue) && !in_array($fieldname,$mappedFields)
+                                                        && !in_array($fieldname,$ignored_fields) )
+                        {
+                            if ( strtolower($fieldname) == strtolower($defaultValue)
+                                || strtolower($fieldname) == str_replace(" ","_",strtolower($defaultValue))
+                                || strtolower($displayname) == strtolower($defaultValue)
+                                || strtolower($displayname) == str_replace(" ","_",strtolower($defaultValue)) )
+                            {
+                                //$selected = ' selected="selected" ';
+                                $defaultField = $fieldname;
+                                $mappedFields[] = $fieldname;
+                            }
+                        }
+                    }  
+                    // get field type information
+                    $fieldtype = '';
+                    if ( isset($properties['type'])
+                            && isset($mod_strings['LBL_IMPORT_FIELDDEF_' . strtoupper($properties['type'])]) )
+                        $fieldtype = ' [' . $mod_strings['LBL_IMPORT_FIELDDEF_' . strtoupper($properties['type'])] . '] ';
+                    if ( isset($properties['comment']) )
+                        $fieldtype .= ' - ' . $properties['comment'];
+                        
+                    $optionsRelated[$relatedModuleObj->module_dir.'$'.$fieldname] = '<option value="'.$relatedModuleObj->module_dir.'$'.$fieldname.'" title="'. $displayname . htmlentities($fieldtype) . '"'
+                        . $selected . $req_class . '>'. $relatedModuleObj->module_dir.".". $displayname . '</option>\n';
+                }
+                
+                }
+        }
+        
         for($field_count = 0; $field_count < $ret_field_count; $field_count++) {
             // See if we have any field map matches
             $defaultValue = "";
@@ -225,7 +309,17 @@
             $defaultField = '';
             global $current_language;
       $moduleStrings = return_module_language($current_language, $this->bean->module_dir);
+            
+            $related_options_clone = $optionsRelated;
 
+            if(
+                isset($firstrow_name) &&
+                isset($field_map[$firstrow_name]) &&
+                isset($related_options_clone[$field_map[$firstrow_name]])
+            ){
+                $related_options_clone[$field_map[$firstrow_name]] = str_replace('<option ','<option selected="selected" ',$related_options_clone[$field_map[$firstrow_name]]);
+            }
+            
             foreach ( $fields as $fieldname => $properties ) {
                 // get field name
                 if (!empty($moduleStrings['LBL_EXPORT_'.strtoupper($fieldname)]) )
@@ -278,7 +372,6 @@
                 $options[$displayname.$fieldname] = '<option value="'.$fieldname.'" title="'. $displayname . htmlentities($fieldtype) . '"'
                     . $selected . $req_class . '>' . $displayname . $req_mark . '</option>\n';
             }
-
             // get default field value
             $defaultFieldHTML = '';
             if ( !empty($defaultField) ) {
@@ -306,6 +399,11 @@
             $cellOneData = isset($rows[0][$field_count]) ? $rows[0][$field_count] : '';
             $cellTwoData = isset($rows[1][$field_count]) ? $rows[1][$field_count] : '';
             $cellThreeData = isset($rows[2][$field_count]) ? $rows[2][$field_count] : '';
+            
+            if(count($optionsRelated)>0 && @$_REQUEST['primary_key'] != $firstrow_name)
+            {
+                $options = array_merge($options, $related_options_clone);
+            }
             $columns[] = array(
                 'field_choices' => implode('',$options),
                 'default_field' => $defaultFieldHTML,
@@ -382,6 +480,7 @@
 
         $this->ss->assign("COLUMNCOUNT",$ret_field_count);
         $this->ss->assign("rows",$columns);
+        $this->ss->assign("primary_field", @$_REQUEST['primary_key'] ? $_REQUEST['primary_key'] : 'id' );
 
         $this->ss->assign('datetimeformat', $GLOBALS['timedate']->get_cal_date_time_format());
 
@@ -768,4 +867,4 @@
 
 EOJAVASCRIPT;
     }
-}
+}
\ No newline at end of file

Friday, January 13, 2012

Insert, retrieve and delete a record using SugarCrm webservices from C#/WCF

SugarCrm consists of of a core framework, MVC modules and UI widgets called dashlets, as well as a lot of other functionality. Some modules like the built in CRM modules are handcrafted, to contain very specific business logic, and other are generated using a built in tool called Module Builder. 

On top of all the modules generated through module builder, SugarCrm supplies a set of generic web services, that can be used to manipulate records in those modules, provided that you have a valid username and password. This allows for integration from other systems.

I will show how you can insert, retrieve and delete a record using C# and WCF.

First a simple supporting class for throwing exceptions
using System;

namespace SugarCrmWebserviceTests
{
 public class SugarCrmException : Exception
 {
  public SugarCrmException(string errorNumber, string description) : base (description)
  {
   ErrorNumber = errorNumber;
  }

  public string ErrorNumber { get; private set; }
 }
}

Then the full integration test with supporting helper private methods
  1. insert of a record with known values
  2. retrieve the just inserted record and compare with the known values
  3. mark the record deleted (this is also an example of how to do an record update)

using System;
using System.Collections.Generic;
using System.Linq;
using System.Security.Cryptography;
using System.ServiceModel;
using System.Text;
using NUnit.Framework;
using SugarCrmWebserviceTests.SugarCrm;

namespace SugarCrmWebserviceTests
{
 public class WebserviceTests
 {
  // The internal sugarcrm name for the module (normally the same as the underlying table)
  private const string ModuleName = "Incident";
  
  // The key matches the module field name (database table column name) 
  // and the value is the value to be inserted/update into this field
  // id is a special field:
  // if id exists in the database then you will update that row
  // if you leave it empty (string.Empty) then you will insert a new row
  private readonly Dictionary<string,string> _incident = new Dictionary<string, string>
       {
        {"id", string.Empty},
        {"name", "Clark Kent"},
        {"description", "Something is rotten in the state of Denmark"},
        {"status", "Open"},
        {"category", "Question"}
       };

  private const string ServiceUrl = "http://somesugarcrminstall.somwhere.com/soap.php";
  private const string SugarCrmUsername = "YOUR_SUGAR_USERNAME";
  private const string SugarCrmPassword = "YOUR_SUGAR_PASSWORD";

  private readonly sugarsoapPortType _sugarServiceClient;
  private string SessionId { get; set; }
  private string RecordId { get; set; }

  public WebserviceTests()
  {
   _sugarServiceClient = ChannelFactory<sugarsoapPortType>.CreateChannel(new BasicHttpBinding(),
                      new EndpointAddress(ServiceUrl));
  }

  [TestFixtureSetUp]
  public void BeforeAnyTestsHaveRun()
  {
   // The session is valid until either you are logged out or after 30 minutes of inactivity
   Login();
  }

  [TestFixtureTearDown]
  public void AfterAllTestsHaveRun()
  {
   Logout();
  }

  [Test]
  public void InsertReadDelete()
  {
   InsertRecordInSugarCrmModule();
   RetrieveInsertedRecordFromSugarCrmModule();
   DeleteEntry();
  }

  private void InsertRecordInSugarCrmModule()
  {
   // arrange

   // act
   var result = _sugarServiceClient.set_entry(SessionId, ModuleName, _incident.Select(nameValue => new name_value { name = nameValue.Key, value = nameValue.Value }).ToArray());
   CheckResultForError(result);
   RecordId = result.id; 

   // assert
   Assert.AreEqual("0", result.error.number);
  }
  
  private void RetrieveInsertedRecordFromSugarCrmModule()
  {
   // arrange
   var fieldsToRetrieve = new[] { "id", "name", "description", "status", "category" };

   // act
   var result = _sugarServiceClient.get_entry(SessionId, ModuleName, RecordId, fieldsToRetrieve);
   CheckResultForError(result);

   // assert
   Assert.AreEqual(_incident["name"], GetValueFromNameValueList("name", result.entry_list[0].name_value_list));
   Assert.AreEqual(_incident["description"], GetValueFromNameValueList("description", result.entry_list[0].name_value_list));
   Assert.AreEqual(_incident["status"], GetValueFromNameValueList("status", result.entry_list[0].name_value_list));
   Assert.AreEqual(_incident["category"], GetValueFromNameValueList("category", result.entry_list[0].name_value_list));
  }
  
  private void DeleteEntry()
  {
   // arrange
   var deletedIncident = new Dictionary<string, string>
       {
        {"id", RecordId},
        {"deleted", "1"},
       };   

   // act
   var result = _sugarServiceClient.set_entry(SessionId, ModuleName, deletedIncident.Select(nameValue => new name_value { name = nameValue.Key, value = nameValue.Value }).ToArray());
   CheckResultForError(result);

   // assert
  }

  private void Login()
  {
   var sugarUserAuthentication = new user_auth { user_name = SugarCrmUsername, password = Md5Encrypt(SugarCrmPassword) };
   var sugarAuthenticatedUser = _sugarServiceClient.login(sugarUserAuthentication, "Some Application Name");
   SessionId = sugarAuthenticatedUser.id;
  }

  private void Logout()
  {
   //logout
   _sugarServiceClient.logout(SessionId);
  }

  private static string GetValueFromNameValueList(string key, IEnumerable<name_value> nameValues)
  {
   return nameValues.Where(nv => nv.name == key).ToArray()[0].value;
  }

  /// <summary>
  /// You can only call this method with objects that contain SugarCrm.error_value classes in the root 
  /// and the property accessor is called error. There is no compile time checking for this, if you pass
  /// an object that does not contain this then you will get a runtime error
  /// </summary>
  /// <param name="result">object contain SugarCrm.error_value class in the root and the property accessor is called error</param>
  private static void CheckResultForError(dynamic result)
  {
   if (result.error.number != "0" && result.error.description != "No Error")
   {
    throw new SugarCrmException(result.error.number, result.error.description);
   }
  }

  private static string Md5Encrypt(string valueString)
  {
   var ret = String.Empty;
   var md5Hasher = new MD5CryptoServiceProvider();
   var data = Encoding.ASCII.GetBytes(valueString);
   data = md5Hasher.ComputeHash(data);
   for (int i = 0; i < data.Length; i++)
   {
    ret += data[i].ToString("x2").ToLower();
   }
   return ret;
  }
 }
}

Monday, September 19, 2011

SugarCrm: Continous / automated installation of packages


I needed to find a way to faster go from a bug being reported "Development Complete" to it being installed on a test installation for the testers so they could begin testing. We are not hosting the entire sugarcrm site in SVN and also deployment to the final production system is done by a third party so all we can so is send them the package zip files. So it needed to follow the Module Loader principle.

This script is run by cron at an timed interval (I am using 10 minutes).

Disclaimer
The script could be cleaned up by extracting the svn username and password into configuration variables.

The other suboptimal issue is:
When you are installing multiple packages this way and all packages are having post_install.php files defining the same functions you will get an error. The reason is that the post_install.php file currently is just a php file with named functions in it.The correct way to solve this would be to move the post_install.php functions inside a class. However that was not an option for me so I needed to handle the re-including of functions with the same name. To be able to do this I am renaming the already included functions.


#!/usr/bin/php
<?php
/**
* You must install 
* 
* yum install php-pear
* pecl install apd
* 
* and enabled in the php.ini file 
* 
* [Zend]
* zend_extension = /usr/lib/php/modules/apd.so
* 
* for this to work
*/


//make sure we dump the stuff to stderr
ini_set("error_log","php://stderr");

$svnExportDir='/mnt/';
$siteInstalledVersionStateFilesDir='/mnt/';

$svnPackages = array (
 // Define your packages here and the full SVN url
 'mypackage' => 'https://XXX/svn/mypackage/trunk',
 'mypackage2' => 'https://XXX/svn/mypackage2/trunk',
);

$preInstallFunctions = array (
 // If your package are using pre_install hooks and you are installing multiple packages and the pre_install hooks contain the same functions
 // then you need to define them here. This is hack, if the pre_install functions was defined in a class then we could just unset the class.
    "pre_install",
);

$postInstallFunctions = array (
 // If your package are using post_install hooks and you are installing multiple packages and the pre_install hooks contain the same functions
 // then you need to define them here. This is hack, if the post_install functions was defined in a class then we could just unset the class.
    "post_install",
);
 
$sugarcrmInstallationsToUpdate = array (
  0 => array(
    'name' => 'SugarCrmInstallation1 (staging)',
    'sugarInstallDir' => '/var/www/sites/staging',
    'packagesToUpdate' => array (
        0 => 'mypackage',
        1 => 'mypackage2',
    ),
  ),
);


//initialize
if(!defined('sugarEntry'))  define('sugarEntry', true);

// If you are updating more than one installation there is no need to do the svn export more than one time per
// package, just reuse that already downloaded/exported package
$packageDownloadedFromSvnRevisionLog = $siteInstalledVersionStateFilesDir . 'ci_packageDownloadedFromSvnRevision.log';
$packageVersionsDownloaded = getArrayFromFile($packageDownloadedFromSvnRevisionLog);

foreach($sugarcrmInstallationsToUpdate as $sugarCrmSite){
    chdir($sugarCrmSite['sugarInstallDir']);
    require_once('include/entryPoint.php');
    require_once('ModuleInstall/ModuleInstaller.php');
    $current_user = new User();
    $current_user->is_admin = '1';
    
    $siteInstalledVersionsStateFile = $siteInstalledVersionStateFilesDir . $sugarCrmSite['name'] . '_version';
    $revisionNumberInstalledInSite = getArrayFromFile($siteInstalledVersionsStateFile);
    
    foreach($sugarCrmSite['packagesToUpdate'] as $key=>$package){
        $packageRevisionInstalled = array_key_exists($package, $revisionNumberInstalledInSite) ?  $revisionNumberInstalledInSite[$package] : 0;

        ciLog ("Checking for newer versions in svn ...\n", $sugarCrmSite['name'],$package);
        $packageRevisionInSvn = trim(`svn info --non-interactive --username YOUR_SVN_USERNAME --password YOUR_SVN_PASSWORD $svnPackages[$package] | grep 'Last Changed Rev' | head -1 | grep -Eo "[0-9]+"`);
        
        if ($packageRevisionInstalled < $packageRevisionInSvn) {
   ciLog ("There are updates in SVN (installed: $packageRevisionInstalled, svn: $packageRevisionInSvn).\n", $sugarCrmSite['name'],$package);
            
            $packageRevisionPreviouslyDownloaded = array_key_exists($package, $packageVersionsDownloaded) ?  $packageVersionsDownloaded[$package] : 0;
            if ($packageRevisionPreviouslyDownloaded < $packageRevisionInSvn){
    ciLog ("Getting latest version from svn ...\n", $sugarCrmSite['name'],$package);

                $output = `svn export --force --non-interactive --username YOUR_SVN_USERNAME --password YOUR_SVN_PASSWORD $svnPackages[$package] $svnExportDir$package`;
                $packageVersionsDownloaded[$package] = $packageRevisionInSvn;
                saveArrayToFile($packageDownloadedFromSvnRevisionLog, $packageVersionsDownloaded);
            }
            else {
    ciLog ("(rev $packageRevisionInSvn) has previously been downloaded, using local version.\n", $sugarCrmSite['name'],$package);
            }

            //initialize the module installer
            $modInstaller = new ModuleInstaller();
            $modInstaller->silent = true;  //shuts up the javscript progress bar

            //start installation
   ciLog ("(rev $packageRevisionInSvn) Starting installation into " . $sugarCrmSite['sugarInstallDir'] . " ... \n", $sugarCrmSite['name'],$package);
   
   $preInstallFile = "$svnExportDir$package/scripts/pre_install.php";
   if(is_file($preInstallFile))
   {
    ciLog ("Including $preInstallFile.\n", $sugarCrmSite['name'],$package);
    include($preInstallFile);
    ciLog ("Executing $preInstallFile.\n", $sugarCrmSite['name'],$package);
    pre_install();
                
                // Undeclaring the pre_install functions so we can include the next pre_install file. Nasty, this is a hack. Requires PECL ADP 
                undeclareFunctions($preInstallFunctions);
   }
            $modInstaller->install($svnExportDir.$package);

   $postInstallFile = "$svnExportDir$package/scripts/post_install.php";
   if(is_file($postInstallFile))
   {
                $_REQUEST['unzip_dir'] = "$svnExportDir$package";
    ciLog ("Including $postInstallFile.\n", $sugarCrmSite['name'],$package);
    include($postInstallFile);
    ciLog ("Executing $postInstallFile.\n", $sugarCrmSite['name'],$package);
    post_install();

                // Undeclaring the post_install functions so we can include the next pre_install file. Nasty, this is a hack. Requires PECL ADP 
                undeclareFunctions($postInstallFunctions);
   }
   ciLog ("(rev $packageRevisionInSvn) Installation into " . $sugarCrmSite['sugarInstallDir'] . " is done.\n", $sugarCrmSite['name'],$package);
            
            $revisionNumberInstalledInSite[$package] = $packageRevisionInSvn;
        }
        else {
   ciLog ("No updates found (installed: $packageRevisionInstalled, svn: $packageRevisionInSvn).\n", $sugarCrmSite['name'],$package);
        }
    }
    // ciLog what package revisions that were installed
    saveArrayToFile($siteInstalledVersionsStateFile, $revisionNumberInstalledInSite);
    
    // Make sure apache owns the sugar installation
    $sugarInstallDir = $sugarCrmSite['sugarInstallDir'];
    $output = `chown -R apache:apache $sugarInstallDir`;
}

function undeclareFunctions($functions){
    foreach($functions as $function) {
       if (function_exists($function)) { rename_function($function, uniqid()); } 
    }   
}

function ciLog($message, $site, $package){
 $fullMessage = "%s %s-%s: " . $message;
 printf($fullMessage, date('m/d/Y H:i:s'), $site, $package);
}

function getArrayFromFile($filename){
    if (file_exists($filename)) {
        return unserialize(file_get_contents($filename));
    }
    else {
        return array();
    }
}

function saveArrayToFile($filename, $arrayToSave){
    $fh = fopen($filename, 'w') or die("can't open file");
    fwrite($fh, serialize($arrayToSave));
    fclose($fh);
}
?>

Monday, May 30, 2011

HOWTO - Empty SugarCrm ModuleLoader install history


Sometimes after installing many packages on a development installation of SugarCrm the Module Loader will take a long time to load. To regain the freshly installed SugarCrm speed of ModuleLoader you need to clean out the temp installation files (they will still be installed). You can use the commands below to clean it up to make it more responsive again. (This is working on 5.1.0b of SugarCrm, newer versions might behave differently).

Please note that you do lose the list of installed packages.

rm -f /<SUGARCRM_INSTALL_DIR>/cache/upload/*.zip
rm -fr /<SUGARCRM_INSTALL_DIR>/cache/upload/upgrades/module/*
rm -fr /<SUGARCRM_INSTALL_DIR>/cache/upload/upgrades/temp/*
# truncate table upgrade_history in instance database

Tuesday, May 24, 2011

Installing SugarCrm from scratch on CentOS 5.X

This posting outlines the Linux commands needed to install SugarCrm on a RHEL/CentOS 5.5 standard installation. This posting assumes that you are installing the database server on one server and the webserver on another server.

Before beginning I would like to change the SELinux configuration of the servers.


Since RHEL/CentOS out of the box has SELinux enabled i recommend changing the setting from enforcing to permissive. This will log all security breaches in a log and then you can later reconfigure SELinux by using the audit2allow script.
So for now we will permanently set SELinux to permissive.
In the /etc/sysconfig/selinux file you will something like the following

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=enforcing
...

Change SELINUX=enforcing to SELINUX=permissive and save the file.
Then run the following command

setenforce 0

First let us install the Apache webserver (or make sure it is updated) as well as php and the php modules needed to run SugarCRM.
It is important to note that Sugarcrm 5.1 only supports the standard PHP package with Centos 5.X (PHP 5.1.6 and NOT PHP 5.3)

yum install httpd
yum install php
yum install php-mbstring php-mysql php-gd gd php-imap php-ldap
yum install html2ps

We need to configure the iptables firewall to allow incomming trafic to HTTP/HTTPS and I will leave that as an exercise to the reader.


Now lets us install the APC PHP opcode cache. This is going to speed up the execution on the server. However please note that this can have some sideeffects when running SugarCRM. In particular if you are going to do a lot of customizations using Studio or ModuleBuilder. For that reason my preference is to only install and enable this on production sites. First lets us install the needed dependecies.

yum install php-pear php-devel httpd-devel pcre-devel

Now if you are running your server behind a proxy you need to configure pear to use that proxy otherwise the installation will fail. If you are not behind a proxy please ignore this. The syntax is pear config-set http_proxy http://myproxy.yourdomain.com:your_proxy_port/ and a example would be

pear config-set http_proxy http://myproxy.yourdomain.com:8080/

Installing APC

pecl install apc 

We need to configure the /etc/php.ini file, the below values are the values that I prefer to run with, you might have different requirements, please make sure you understand what you are changing and why.

memory_limit = 512M
max_execution_time = 300
error_reporting = E_ALL & ~E_NOTICE
post_max_size = 64M
upload_max_filesize = 64M

At the end of the php.ini file add the following lines. These lines tells PHP that we want to import the APC extension we installed above (on production servers). However we disable this APC module by default. This allows us to override and turn it on in a VirtualHost .htaccess file. This way we can have a test site running without APC and a production site running with APC on the same server (depending on your environment and expected production system load, this might be a good/bad idea).

extension=apc.so
[apc]
apc.cache_by_default=0 # disable by default

The default configuration of Apache when installed on RHEL/CentOS is to disable namebased virtual hosts, thus we need to enable this. (this is not strictly required since if you only plan on running one and one site only on this server you can do without, but I prefer to use namebased virtual hosts. So let us enable this in the /etc/httpd/conf/httpd.conf file. Find the following line and change as specified below.

#NameVirtualHost *:80

is changed to

NameVirtualHost *:80

Now we need to create a virtual host configuration file. I have a convention of naming the configuration file based on the DNS name of the site. The configuration file is placed in /etc/httpd/conf.d/DNS_NAME_OF_SITE.conf.

That would for a http://www.somewhere.com end up as a file named www.somewhere.com.conf.

Depending on your server setup you might want to use different values than what I have shown below, but the below configuration achives a pretty good Google PageSpeed score and also for Yahoo YSlow. However the caching is somewhat aggressive so you might have requirements that requires you to change the below values.

<VirtualHost *:80 >
  ServerName "HOSTNAME"
  DocumentRoot /var/www/sites/HOSTNAME
  DirectoryIndex index.html index.php
  ErrorLog logs/HOSTNAME-error_log
  CustomLog logs/HOSTNAME-access_log common

  <Directory  "/var/www/sites/HOSTNAME">
    SetOutputFilter DEFLATE
    AddDefaultCharset UTF-8

    AllowOverRide All
    Options FollowSymlinks

        # Enable ETag
        FileETag MTime Size

        # Set expiration header
        ExpiresActive on
        ExpiresDefault "access plus 1 year"
        ExpiresByType image/gif "access plus 1 month"
        ExpiresByType image/png "access plus 1 month"
        ExpiresByType image/jpeg "access plus 1 month"
        ExpiresByType text/css "access plus 1 month"
        ExpiresByType text/javascript "access plus 1 month"
        ExpiresByType text/js "access plus 1 month"

        # Compress some text file types
        AddOutputFilterByType DEFLATE text/html text/css text/xml application/x-javascript text/javascript text/js

        # Deactivate compression for buggy browsers
        BrowserMatch ^Mozilla/4 gzip-only-text/html
        BrowserMatch ^Mozilla/4\.0[678] no-gzip
        BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
  </Directory>
</VirtualHost>

We need to install java on the server since the preconfigured java does not work with ZuckerReports. This link was the latest version at the time of writing this posting. Please note that it is the jdk we are installing.

wget http://download.oracle.com/otn-pub/java/jdk/6u25-b06/jdk-6u25-linux-x64.bin
chmod 755 jdk-6u25-linux-x64.bin
./jdk-6u25-linux-x64.bin

Now we need to change the default java used by the system (please note the output of the java versions

java --version 
unlink /usr/bin/java
ln -s /usr/java/jdk1.6.0_04/bin/java /usr/bin/java
java -version

Now we will leave the webserver and move to the database server. First we need to install the mysql server.


yum install mysql-server

Before starting the mysql server I prefer to change the datadir location to a separate mount point in the Linux system rather than the default under /var/lib/mysql. That is done by editing the /etc/my.cnf file and changing the value of the datadir variable.

datadir=/<MY_CUSTOM_DATADIR_DIRECTORY_PATH>/mysql

# an example 
datadir=/data/mysql

Now start the mysql server and set it to autostart when the Linux server is started.

service mysqld start
chkconfig mysqld on

Now let us change a few more parameters in the my.cnf file (some of these parameter can have a breaking effect if you change them before the first start of the mysql server. That is why we are waiting and doing the configuration in 2 steps).
Change the /etc/my.cnf file and add the following variables under the [mysqld] region.

log-warnings=2
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
default-storage-engine=innodb

restart the mysqld server.

service mysqld restart

now we need to set a password for the root user.

mysqladmin -u root -p password '<MY_ROOT_PASSWORD>'
Create a database for the SugarCRM installation.

mysql -uroot -p<MY_ROOT_PASSWORD>
create database sugarcrm;

Create users and grant permissions to the users that need it.
Assuming that we want to allow access from localhost, and from remote Apache server

CREATE USER 'sugarcrm_user'@'localhost' IDENTIFIED BY 'mypass';
GRANT ALL ON sugarcrm.* TO 'sugarcrm_user'@'localhost'; 
CREATE USER 'sugarcrm_user'@'127.0.0.1' IDENTIFIED BY 'mypass';
GRANT ALL ON sugarcrm.* TO 'sugarcrm_user'@'127.0.0.1';
CREATE USER 'sugarcrm_user'@'<APACHE_SERVER_IP_ADDRESS>' IDENTIFIED BY 'mypass';
GRANT ALL ON sugarcrm.* TO 'sugarcrm_user'@'<APACHE_SERVER_IP_ADDRESS>'; 

Now if you plan on installing custom built modules into SugarCrm where the installer package contains MySQL trigger code that needs to be executed by the sugarcrm_user database user. MySQL 5.0 contains a less than optimal permission implementation. So to be able to install/create triggers the user needs SUPER permission. And since the default database version that is installed in RHEL/CentOS 5.5 at the time of writing this blog post is 5.0.77 we need to grant SUPER to our database user.

GRANT SUPER on *.* to 'sugarcrm_user'@'localhost';
GRANT SUPER on *.* to 'sugarcrm_user'@'127.0.0.1';
GRANT SUPER on *.* to 'sugarcrm_user'@'<APACHE_SERVER_IP_ADDRESS>';

Now we need to configure the iptables firewall to allow incomming trafic to MySQL.

iptables -A INPUT -i eth0 -s <APACHE_SERVER_IP_ADDRESS> -p tcp --destination-port 3306 -j ACCEPT
service iptables save

Now we are ready to install SugarCrm either using the installer from SugarCrm (please follow the official installation guide) or cloning an existing site

- tar the site

tar -czf mysite.somewhere.com.tar.gz mysite.somewhere.com

dump the staging sugarcrm mysql database (changing the tables to InnoDB and remove DEFINER statements from the dump)

mysqldump -h<HOSTNAME> -u<USERNAME> -p<PASSWORD> --triggers --routines <DATABASE> | perl -pe 's/MyISAM/InnoDB/' |  perl -pe 's{(/\*!\d+ )\s*definer\s*=\s*\S+}{$1}i' > mysite.somewhere.com.sql
gzip mysite.somewhere.com.sql

Move the tar site dump and mysql dump to the production server and unzip/untar the site dump

wget http://my.staging.server/mysite.somewhere.com.tar.gz
wget http://my.staging.server/mysite.somewhere.com.sql.gz
tar -xf mysite.somewhere.com.tar.gz
mv mysite.somewhere.com /var/www/sites/HOSTNAME  #(HOSTNAME as defined in the Apache virtualhost conf file)
chown -R apache:apache /var/www/sites/HOSTNAME  #(HOSTNAME as defined in the Apache virtualhost conf file)

#Secure directories
chmod -R 644 *
find . -type d | xargs chmod 755

chmod 664 config.php
chmod 775 ./custom 
chmod 775 ./data/upload 
chmod 775 ./data/upload 
chmod 775 ./cache 
chmod 775 ./cache/images 
chmod 775 ./cache/import 
chmod 775 ./cache/layout 
chmod 775 ./cache/pdf 
chmod 775 ./cache/upload 
chmod 775 ./cache/xml 
chmod 775 ./modules

install the mysql database dump

gunzip mysite.somewhere.com.sql.gz
mysql -h<DATABASEHOSTNAME> -usugarcrm_user -pmypass -D sugarcrm < mysite.somewhere.com.sql
edit the /var/www/sites/HOSTNAME/config.php file and make sure the following variables are updated with your own values
'dbconfig' =>
  array (
    'db_host_name' => '',
    'db_host_instance' => 'SQLEXPRESS',
    'db_user_name' => '',
    'db_password' => '',
    'db_name' => '',
    'db_type' => 'mysql',
'host_name' => '',
'site_url' => 'http://',
'unique_key' => '',
unique_key should be unique for each different server.

IMPORTANT!: Edit and update the /SITE_DIRECTORY/.htaccess file to match the destination DNS sitename.

If this is a production server enable APC

comment the following line
#RedirectMatch /modules/(.*)/(.*).php http://XXXXX/index.php

add the following line
php_flag apc.cache_by_default On
After running the site for a while, tweak your MySQL config file using MySQLTuner
wget --no-check-certificate https://raw.github.com/rackerhacker/MySQLTuner-perl/master/mysqltuner.pl

perl mysqltuner.pl

Wednesday, November 24, 2010

SugarCrm - Change max size of default name field in module builder

The following posting and the indicated modifications to code is based on the 5.5.2 release of SugarCrm CE edition.

Objective
SugarCRM allows defining Max Size for "Text Field" field types in Module Builder. We need to add feature to define/change html display size attribute for the Text Field from Module Builder. 

The changes will affect following fields:
1.    For custom Text Field
2.    For pre-defined TextField i.e.  name

Purpose
By default the display size for the Text Field is 30 characters but it can hold up to 255 characters. When you are creating or editing a record in EditView some times if the text you are editing in a text field is long enough it becomes awkward. This problem can be avoided by increasing the display size of the TextField.

Currently the TextField for name field in EditView will be rendered as:

<input type="text" maxlength="" size="30" id="name" title="" value="" name="name" />

I would like to be able to change this when I define a new module in Module Builder.


This change is not upgrade safe.

Files need to be changed

(I have marked the hack with comments in each individual section below)

1.    /modules/DynamicFields/DynamicField.php
function addFieldObject(&$field){
        $GLOBALS['log']->debug('adding field');
        $object_name = $this->module;
        $db_name = $field->name;
        
        $fmd = new FieldsMetaData();
        $id =  $fmd->retrieve($object_name.$db_name,true, false);
        $is_update = false;
        $label = $field->label;
        if(!empty($id)){
            $is_update = true;
        }else{
            $db_name = $this->getDBName($field->name);
            $field->name = $db_name;
        }
         $this->createCustomTable();
        $fmd->id = $object_name.$db_name;
        $fmd->custom_module= $object_name;
        $fmd->name = $db_name;
        $fmd->vname = $label;
        $fmd->type = $field->type;
        $fmd->help = $field->help;
        //Hack:Allow for changing the individual textboxes html size setting
        $fmd->size = $field->size;
        //Hack:Allow for changing the individual textboxes html size setting
        if (!empty($field->len))
           $fmd->len = $field->len; // tyoung bug 15407 - was being set to $field->size so changes weren't being saved
        $fmd->required = ($field->required ? 1 : 0);
        $fmd->default_value = $field->default;
        $fmd->ext1 = $field->ext1;
        $fmd->ext2 = $field->ext2;
        $fmd->ext3 = $field->ext3;
        $fmd->ext4 = (isset($field->ext4) ? $field->ext4 : '');
        $fmd->comments = $field->comment;
        $fmd->massupdate = $field->massupdate;
        $fmd->importable = ( isset ( $field->importable ) ) ? $field->importable : null ;
        $fmd->duplicate_merge = $field->duplicate_merge;
        $fmd->audited =$field->audited;
        $fmd->reportable = ($field->reportable ? 1 : 0);
        if(!$is_update){
            $fmd->new_with_id=true;
        }
        $fmd->save();
        $this->buildCache($this->module);
        if($field){
            if(!$is_update){
                $query = $field->get_db_add_alter_table($this->bean->table_name . '_cstm');
            }else{
                $query = $field->get_db_modify_alter_table($this->bean->table_name . '_cstm');
            }
            if(!empty($query)){
                $GLOBALS['db']->query($query);
            }
        }
        return true;
    }

function addField($name,$label='', $type='Text',$max_size='255',$size='',$required_option='optional', $default_value='', $ext1='', $ext2='', $ext3='',$audited=0, $mass_update = 0 , $ext4='', $help='',$duplicate_merge=0, $comment=''){
        require_once('modules/DynamicFields/templates/Fields/TemplateField.php');
        $field = new TemplateField();
        $field->label = $label;
        if(empty($field->label)){
            $field->label = $name;
        }
        $field->name = $name;
        $field->type = $type;
        $field->len = $max_size;
        $field->required = (!empty($required_option) && $required_option != 'optional');
        $field->default = $default_value;
        $field->ext1 = $ext1;
        $field->ext2 = $ext2;
        $field->ext3 = $ext3;
        $field->ext4 = $ext4;
        $field->help = $help;
        //Hack:Allow for changing the individual textboxes html size setting
        $field->size = $size;
        //Hack:Allow for changing the individual textboxes html size setting
        $field->comments = $comment;
        $field->massupdate = $mass_update;
        $field->duplicate_merge = $duplicate_merge;
        $field->audited = $audited;
        $field->reportable = 1;
        return $this->addFieldObject($field);
    }


2.    /modules/DynamicFields/templates/Fields/TemplateField.php
class TemplateField{
    /*
        The view is the context this field will be used in
        -edit
        -list
        -detail
        -search
    */
    var $view = 'edit';
    var $name = '';
    var $vname = '';
    var $id = '';
    // Hack:Allow for changing the individual textboxes html size setting
    var $size = '';
    // Hack:Allow for changing the individual textboxes html size setting

    var $len = '255';
    var $required = false;
    var $default = null;
    var $default_value = null;

function get_field_def(){
        $array =  array(
            'required'=>$this->required,
            'source'=>'custom_fields',
            'name'=>$this->name,
            'vname'=>$this->vname,
            'type'=>$this->type,
            'massupdate'=>$this->massupdate,
            'default'=>$this->default,
            'comments'=> (isset($this->comments)) ? $this->comments : '',
            'help'=> (isset($this->help)) ?  $this->help : '',
            // Hack:Allow for changing the individual textboxes html size setting
            'size'=>$this->size,
            // Hack:Allow for changing the individual textboxes html size setting
            'importable'=>$this->importable,
            'duplicate_merge'=>$this->duplicate_merge,
            'duplicate_merge_dom_value'=>$this->duplicate_merge, //nsingh: bug #14897 The UI did not get updated according to $app_list_strings['custom_fields_merge_dup_dom'], so include an extra field for the dom and leave the original duplicate_merge logic intact.
            'audited'=>($this->audited ? 1 : 0),
            'reportable'=>($this->reportable ? 1 : 0),
            );
            if(!empty($this->len)){
                $array['len'] = $this->len;
            }
            $this->get_dup_merge_def($array);
            return $array;
    }

3.    /modules/DynamicFields/templates/Fields/Forms/varchar.tpl

{else}
        <input type='hidden' name='len' value='{$vardef.len}'>{$vardef.len}
    {/if}
    </td>
</tr>

{* Hack:Allow for changing the individual textboxes html size setting *}
<tr>
    <td class='mbLBL'>{$MOD.COLUMN_TITLE_SIZE}: </td>
    <td>
        <input type="text" name="size" value="{$vardef.size}">
    </td>
</tr>
{* Hack:Allow for changing the individual textboxes html size setting *}
{include file="modules/DynamicFields/templates/Fields/Forms/coreBottom.tpl"}

4.    /modules/EditCustomFields/EditCustomFields.php

function add_custom_field($name, $label, $data_type, $max_size,$size,
        $required_option, $default_value, $deleted, $ext1, $ext2, $ext3, $audited, $mass_update=0, $duplicate_merge=0, $reportable = true)
    {
        $module_name = $this->module_name;

        $fields_meta_data = new FieldsMetaData();
        $fields_meta_data->name = $name;
        $fields_meta_data->label = $label;
        $fields_meta_data->module = $module_name;
        $fields_meta_data->data_type = $data_type;
        $fields_meta_data->max_size = $max_size;
        //Hack:Allow for changing the individual textboxes html size setting
        $fields_meta_data->size = $size;
        //Hack:Allow for changing the individual textboxes html size setting    
        $fields_meta_data->required_option = $required_option;
        $fields_meta_data->default_value = $default_value;
        $fields_meta_data->deleted = $deleted;

5.    /modules/EditCustomFields/EditView.html

<tr><td nowrap="nowrap">{MOD.COLUMN_TITLE_DATA_TYPE}:</td><td><select name="data_type" id='data_type' onchange="typeChanged(this);" {NOEDIT}>{data_type_options}</select></td></tr>

<tr><td nowrap="nowrap">{MOD.COLUMN_TITLE_MAX_SIZE}:</td><td><input type="text" name="max_size" value="{max_size}" /></td></tr>

<!-- Hack: Allow for changing the individual textboxes html size setting -->
<tr><td nowrap="nowrap">{MOD.COLUMN_TITLE_SIZE}:</td><td><input type="text" name="size" value="{size}" /></td></tr>
<!-- Hack: Allow for changing the individual textboxes html size setting -->    

<tr><td nowrap="nowrap">{MOD.COLUMN_TITLE_REQUIRED_OPTION}:</td><td><input type="checkbox" name="required_option" value="{required_option}" {REQUIRED_CHECKED}/></td></tr>

<tr><td nowrap="nowrap">{MOD.COLUMN_TITLE_DEFAULT_VALUE}:</td><td><input type="text" name="default_value" value="{default_value}" /></td></tr>

6.    /modules/EditCustomFields/EditView.php

$xtpl->assign('NOEDIT', 'disabled');
    $xtpl->assign('custom_field_id', $focus->id);
    $xtpl->assign('name', $focus->name);
    $xtpl->assign('label', $focus->label);
    $xtpl->assign('custom_module', $focus->custom_module);
    
   $data_type_options_html = get_select_options_with_id($data_type_array,
        $focus->data_type);
    
    $xtpl->assign('data_type_options', $data_type_options_html);
    $xtpl->assign('max_size', $focus->max_size);
    //Hack:Allow for changing the individual textboxes html size setting
    $xtpl->assign('size', $focus->size);
    //Hack:Allow for changing the individual textboxes html size setting
    $xtpl->assign('required_option', $focus->required_option);
    if($focus->required_option == 'required'){
        $xtpl->assign('REQUIRED_CHECKED', 'checked');
    }
    $xtpl->assign('default_value', $focus->default_value);
    
    $xtpl ->assign('ENUM_OPTIONS', get_select_options_with_id($enum_keys, $focus->ext1));
    $xtpl->assign('ext1', $focus->ext1);
    $xtpl->assign('ext2', $focus->ext2);
    $xtpl->assign('ext3', $focus->ext3);

7.    /modules/EditCustomFields/Forms.html

{mod.COLUMN_TITLE_DATA_TYPE}<br />
<select name="data_type">{data_type_options}</select><br />
{mod.COLUMN_TITLE_MAX_SIZE}<br />
<input type="text" name="max_size" value="" /><br />
<!-- Hack: Allow for changing the individual textboxes html size setting -->
{mod.COLUMN_TITLE_SIZE}<br />
<input type="text" name="size" value="" /><br />
<!-- Hack: Allow for changing the individual textboxes html size setting -->
{mod.COLUMN_TITLE_REQUIRED_OPTION}<br />
<input type="checkbox" name="required_option" value="1" /><br />
{mod.COLUMN_TITLE_DEFAULT_VALUE}
<input type="text" name="default_value" value="" /><br />

8.    /modules/EditCustomFields/ListView.html

<td scope="col" ><a href="{ORDER_BY}data_type" class="listViewThLinkS1"
    >{MOD.COLUMN_TITLE_DATA_TYPE}{arrow_start}{data_type_arrow}{arrow_end}</a></td>
<td scope="col" ><a href="{ORDER_BY}max_size" class="listViewThLinkS1"
    >{MOD.COLUMN_TITLE_MAX_SIZE}{arrow_start}{max_size_arrow}{arrow_end}</a></td>
<!-- Hack: Allow for changing the individual textboxes html size setting -->
<td scope="col" class="listViewThS1"><a href="{ORDER_BY}size" class="listViewThLinkS1">{MOD.COLUMN_TITLE_SIZE}{arrow_start}{size_arrow}{arrow_end}</a></td>    
<!-- Hack: Allow for changing the individual textboxes html size setting -->    
<td scope="col" ><a href="{ORDER_BY}required_option" class="listViewThLinkS1"
    >{MOD.COLUMN_TITLE_REQUIRED_OPTION}{arrow_start}{required_option_arrow}{arrow_end}</a></td>
<td scope="col" ><a href="{ORDER_BY}default_value" class="listViewThLinkS1"
    >{MOD.COLUMN_TITLE_DEFAULT_VALUE}{arrow_start}{default_value_arrow}{arrow_end}</a></td>
<td scope="col" >&nbsp;</td>

9.    /modules/EditCustomFields/Popup.html

tr>
<td nowrap="nowrap">{mod.COLUMN_TITLE_DATA_TYPE}</td>
<td><select name="data_type">{data_type_options}</select></td>
</tr>

<tr>
<td nowrap="nowrap">{mod.COLUMN_TITLE_MAX_SIZE}</td>
<td><input type="text" name="max_size" value="{max_size}" /></td>
</tr>
<!-- Hack: Allow for changing the individual textboxes html size setting -->
<tr>
<td nowrap="nowrap">{mod.COLUMN_TITLE_SIZE}</td>
<td><input type="text" name="size" value="{size}" /></td>
</tr>
<!-- Hack: Allow for changing the individual textboxes html size setting -->

<tr>
<td nowrap="nowrap">{mod.COLUMN_TITLE_REQUIRED_OPTION}</td>
<td><input type="checkbox" name="required_option" value="{required_option}" /></td>
</tr>

10.    /modules/EditCustomFields/Save.php

if(isset($_REQUEST['ext2'])){        
    $ext2 = $_REQUEST['ext2'];
}
$ext3 = '';
if(isset($_REQUEST['ext3'])){        
    $ext3 = $_REQUEST['ext3'];
}
$max_size = '255';
if(isset($_REQUEST['max_size'])){        
    $max_size = $_REQUEST['max_size'];
}
//Hack:Allow for changing the individual textboxes html size setting
if(isset($_REQUEST['size'])){        
    $size = $_REQUEST['size'];
}
//Hack:Allow for changing the individual textboxes html size setting
$required_opt = 'optional';
if(isset($_REQUEST['required_option'])){
    $required_opt = 'required';
}
$default_value = '';
if(isset($_REQUEST['default_value'])){
    $default_value = $_REQUEST['default_value'];
}

11.    /modules/EditCustomFields/vardefs.php

'fields' => array (
        'id'=>array('name' =>'id', 'type' =>'varchar', 'len'=>'255', 'reportable'=>false),
        'name'=>array('name' =>'name', 'vname'=>'COLUMN_TITLE_NAME', 'type' =>'varchar', 'len'=>'255'),
        'vname'=>array('name' =>'vname' ,'type' =>'varchar','vname'=>'COLUMN_TITLE_LABEL',  'len'=>'255'),
        'comments'=>array('name' =>'comments' ,'type' =>'varchar','vname'=>'COLUMN_TITLE_LABEL',  'len'=>'255'),
        'help'=>array('name' =>'help' ,'type' =>'varchar','vname'=>'COLUMN_TITLE_LABEL',  'len'=>'255'),
        'custom_module'=>array('name' =>'custom_module',  'type' =>'varchar', 'len'=>'255', ),
        'type'=>array('name' =>'type', 'vname'=>'COLUMN_TITLE_DATA_TYPE',  'type' =>'varchar', 'len'=>'255'),
        'len'=>array('name' =>'len','vname'=>'COLUMN_TITLE_MAX_SIZE', 'type' =>'int', 'len'=>'11', 'required'=>false, 'validation' => array('type' => 'range', 'min' => 1, 'max' => 255),),
        // Hack: Allow for changing the individual textboxes html size setting
        'size'=>array('name' =>'size' ,'type' =>'varchar','vname'=>'COLUMN_TITLE_SIZE',  'len'=>'255'),
        // Hack: Allow for changing the individual textboxes html size setting
        'required'=>array('name' =>'required', 'type' =>'bool', 'default'=>'0'),
        'default_value'=>array('name' =>'default_value', 'type' =>'varchar', 'len'=>'255', ),
        'date_modified'=>array('name' =>'date_modified', 'type' =>'datetime', 'len'=>'255',),        
        'deleted'=>array('name' =>'deleted', 'type' =>'bool', 'default'=>'0', 'reportable'=>false),
        'audited'=>array('name' =>'audited', 'type' =>'bool', 'default'=>'0'),        
        'massupdate'=>array('name' =>'massupdate', 'type' =>'bool', 'default'=>'0'),    
        'duplicate_merge'=>array('name' =>'duplicate_merge', 'type' =>'short', 'default'=>'0'),  
        'reportable' => array('name'=>'reportable', 'type'=>'bool', 'default'=>'1'),

12.    /modules/EditCustomFields/language/en_us.lang.php

'LBL_REPORTABLE'=>'Reportable',
    'ERR_RESERVED_FIELD_NAME' => "Reserved Keyword",
    'ERR_SELECT_FIELD_TYPE' => 'Please Select a Field Type',
    'LBL_BTN_ADD' => 'Add',
    'LBL_BTN_EDIT' => 'Edit',
    'LBL_GENERATE_URL' => 'Generate URL',
    'LBL_DEPENDENT_CHECKBOX'=>'Dependent',
    'LBL_DEPENDENT_TRIGGER'=>'Trigger',
    'LBL_BTN_EDIT_VISIBILITY'=>'Edit Visibility',
    // Hack: Allow for changing the individual textboxes html size setting
    'COLUMN_TITLE_SIZE' => "Size",
    // Hack: Allow for changing the individual textboxes html size setting
);
?>

13.    /modules/ModuleBuilder/MB/MBField.php

class MBField{
    var $type = 'varchar';
    var $name = false;
    var $label = false;
    var $vname = false;
    //Hack:Allow for changing the individual textboxes html size setting
    var $size = false;
    //Hack:Allow for changing the individual textboxes html size setting
    var $options = false;
    var $length = false;
    var $error = '';
    var $required = false;
    var $reportable = true;
    var $default = 'MSI1';

function getFieldVardef(){
        if(empty($this->name)){
            $this->error = 'A name is required to create a field';
            return false;
        }        
        if(empty($this->label))$this->label = $this->name;
        $this->name = strtolower($this->getDBName($this->name));
        $vardef = array();
        $vardef['name']=$this->name;
        if(empty($this->vname))$this->vname = 'LBL_' . strtoupper($this->name);
        $vardef['vname'] = $this->addLabel();
        if(!empty($this->required))$vardef['required'] = $this->required;
        if(empty($this->reportable))$vardef['reportable'] = false;
        if(!empty($this->comment))$vardef['comment'] = $this->comment;
        //Hack:Allow for changing the individual textboxes html size setting
        if(!empty($this->size))$vardef['size'] = $this->size;
        //Hack:Allow for changing the individual textboxes html size setting
        if($this->default !== 'MSI1')$vardef['default'] = $this->default;
        switch($this->type){
            case 'date':
            case 'datetime':

14.    /modules/ModuleBuilder/MB/MBPackage.php

$result=$db->query("SELECT *  FROM fields_meta_data where custom_module='$value'");
                    while($row = $db->fetchByAssoc($result)){
                        $name = $row['id'];
                        foreach($row as $col=>$res){
                            switch ($col) {
                                case 'custom_module':
                                    $installdefs['custom_fields'][$name]['module'] = $res;
                                    break;
                                case 'required':
                                    $installdefs['custom_fields'][$name]['require_option'] = $res;
                                    break;
                                case 'vname':
                                    $installdefs['custom_fields'][$name]['label'] = $res;
                                    break;
                            //Hack:Allow for changing the individual textboxes html size setting
                                case 'size':
                                    $installdefs['custom_fields'][$name]['size'] = $res;
                                    break;                                    
                            //Hack:Allow for changing the individual textboxes html size setting                        
                                case 'required':
                                    $installdefs['custom_fields'][$name]['require_option'] = $res;
                                    break;
                                case 'massupdate':
                                    $installdefs['custom_fields'][$name]['mass_update'] = $res;
                                    break;

15.    /modules/ModuleBuilder/views/view.modulefield.php

if(empty($vardef['name'])){
                if(!empty($_REQUEST['type']))$vardef['type'] = $_REQUEST['type'];
                    $fv->ss->assign('hideLevel', 0);
            }else{
                if(!empty($module->mbvardefs->vardef['fields'][$vardef['name']])){
                    $fv->ss->assign('hideLevel', 1);
                }elseif(isset($vardef['custom_module'])){
                    $fv->ss->assign('hideLevel', 2);
                }else{
                    //Hack:Allow for changing the individual textboxes html size setting 
                    if($vardef['name'] == 'name'){
                        $action = 'saveField';
                         
                    } else {
                        $action = 'saveLabel';
                    }
                    //Hack:Allow for changing the individual textboxes html size setting 
                    $fv->ss->assign('hideLevel', 10); // tyoung bug 17350 - effectively mark template derived fields as readonly
                }
            }

16.    /modules/ModuleBuilder/views/view.modulefields.php

} else {
            require_once('modules/ModuleBuilder/MB/ModuleBuilder.php');
            $mb = new ModuleBuilder();
            $mb->getPackage($_REQUEST['view_package']);
            $package = $mb->packages[$_REQUEST['view_package']];

            $package->getModule($module_name);
            $this->module = $package->modules[$module_name];
            $this->loadPackageHelp($module_name);
            $this->module->getVardefs(true);
            $this->module->mbvardefs->vardefs['fields'] = array_reverse($this->module->mbvardefs->vardefs['fields'], true);
            foreach($this->module->mbvardefs->vardefs['fields'] as $k=>$v){
                if($k != $module_name)
                    $titleLBL[$k]=translate("LBL_".strtoupper($k),'ModuleBuilder');
                else{
                    $titleLBL[$k]=$k;
                }
            }
            
            //Hack:Allow for changing the individual textboxes html size setting 
            foreach($this->module->mbvardefs->vardefs['fields'][$module_name] as $k=>$v){
                foreach($titleLBL as $template){
                    if($template != $module_name){
                        $template = strtolower($template);
                        
                        //Ketty: Upgrade sugarbase to SugarCRM 5.5
                        $template = str_replace(' ', '_', $template);
                        //Ketty: Upgrade sugarbase to SugarCRM 5.5
                        
                        foreach($this->module->mbvardefs->vardefs['fields'][$template] as $key => $value){
                            if($k == $key && $key == 'name'){
                                unset($this->module->mbvardefs->vardefs['fields'][$template][$key] );
                            }
                        }
                    }                
                }                
            }
            //Hack:Allow for changing the individual textboxes html size setting 
            $this->module->mbvardefs->vardefs['fields'][$module_name] = $this->cullFields($this->module->mbvardefs->vardefs['fields'][$module_name]);
            if(file_exists($this->module->path. '/language/'.$GLOBALS['current_language'].'.lang.php')){
                include($this->module->path .'/language/'. $GLOBALS['current_language'].'.lang.php');
                $this->module->setModStrings($GLOBALS['current_language'],$mod_strings);
            }
            elseif(file_exists($this->module->path. '/language/en_us.lang.php')){
                include($this->module->path .'/language/en_us.lang.php');
                $this->module->setModStrings('en_us',$mod_strings);
            }
            $smarty->assign('title', $titleLBL);
            $smarty->assign('package', $package);
            $smarty->assign('module', $this->module);
            $smarty->assign('editLabelsMb','1'); //need to merge MB labels and studio labels. quick fix for now.

17.    /include/SugarFields/Fields/Base/EditView.tpl

{if strlen({{sugarvar key='value' string=true}}) <= 0}
{assign var="value" value={{sugarvar key='default_value' string=true}} }
{else}
{assign var="value" value={{sugarvar key='value' string=true}} }
{/if}
{* Hack:Allow for changing the individual textboxes html size setting *}  
<input type='text' name='{{sugarvar key='name'}}' id='{{sugarvar key='name'}}' 
{{if isset($displayParams.maxlength)}}maxlength='{{$displayParams.maxlength}}'
{{elseif isset($vardef.len)}}maxlength='{{$vardef.len}}'{{/if}} 
{{if !empty($displayParams.size)}}size='{{$displayParams.size}}'
{{elseif !empty($vardef.size)}}size='{{$vardef.size}}'{{/if}} 
value='{$value}' title='{{$vardef.help}}' tabindex='{{$tabindex}}' {{$displayParams.field}}>



Additional InfoYou need to execute sql statement to add column “size” in "fields_meta_data" table.
ALTER TABLE fields_meta_data  add column size varchar (255) NULL ;