High performance cachable websites web 2.0 in YiiFramework, mustache.js and icanhaz.js

Preface: 

To be able to cache webpage content as much as possible then it is important to delay merging data with the HTML markup until the last possible moment (which is in the browser). The moment you merge data with HTML then the page is always specific to the context in which it was retrieved. If this context happens to be a user context (which is almost always the case), then you cannot use this cached page for another user, which is bad for cacheability. 

Achieving this "late-merging" of data and markup can be achieved by building the skeleton of the site in very few html pages. Most normal sites could potentially be built using the following skeleton pages:

• frontpage/landingpage 
• login 
• content 

Of course the more the pages differ from each other the more of these skeleton pages you will have. The point is to isolate the common things between the pages and boil it down to the minimum amount of pages. 

These pages (one for each distinct different buildup of the page) will contain a link to a javascript file that contains all the clientside templates/markup needed to render data. These clientside templates, can be built in one of the many templating js engines out there, for instance:

• dust.js 
• handlebars.js 
• mustache.js 

(LinkedIn's review of quite a few of these template engines The client-side templating throwdown: mustache, handlebars, dust.js, and more). 


This way we can cache the clientside needed markup used to display lists and forms (since it is a static javascript file) using nginx or other caching servers. The skeleton pages can also be cached since they contain no data. The only thing remaining is CSS, javascript, images, videos and data. Everything apart from data can also be cached using caching servers. Data we will retrieve using AJAX requests as json and then rendering it clientside. 

YiiFramework: 

I am new to the Yii framework, but needed to build a web 2.0 site that should support caching of as much of the content as possible to avoid overloading the webserver re-sending the same markup over and over again just with different data embedded. As far as I could see there was not really any built-in support in Yii to do this (apart from doing it all manually). The exising Mustache extention seemed to be related to server side templating rather than clientside templating. For this reason I created my own extension. 

The goals were: 

1. Creating views in Yii clientside should be as similar as possible as creating server side views 
2. Templates should be accessible on the clientside without a lot of plumbing code 
3. Rendering templates should be clean. 

In the following I will try to argue for my decisions on how to solve the above.

- Creating views in Yii clientside should be as similar as possible as creating server side views
& -Templates should be accessible on the clientside without a lot of plumbing code

I wanted to be able to put my clientside views in what folder I though made the most sense, and since the clientside templates very much are related to the controller actions that are delivering the data (just like server side views) I wanted to be able to put them for instance in the view folders.
I wanted it to be easy to edit the templates in an IDE and one template should be self contained and not mixed up with the other templates

The templates should not incur a significant serverside overhead (currently I am still working on a better solution than the one I have found - mentioned at the end of the posting).

To achieve the above I decided that in the IDE the client side templates should be seperate files, and to be able to distinguish them from the server side templates then needed another name that supported the js templating engine that I choose (mustache.js), so the file extension ended up being .tpl (since Smarty templates are supported in my IDE).

So the challenge is how to convert disparate files on the filesystem into a single js file that is read and initialized by the browser automatically.

I need go through the filesystem recursively and look for files of a certain kind (*.mustache.tpl), index them by they location in the filesystem and their filename. Push them info a javascript array and add the code needed to initialize the templates when the browser loads the template javascript  file.

All the above is solved in the following Yii component, which also adds the needed javascripts for the template rendering clientside.

<?php
/**
 * ClientsideViews class file.
 * @author Kenneth Thorman (kenneth.thorman@appinux.com)
 * @license http://www.opensource.org/licenses/bsd-license.php New BSD License
 */

/**
 * ClientsideViews application component.
 * Used for registering ClientsideViews core functionality.
 */
class ClientsideViews extends CApplicationComponent {

    /**
     * @var boolean whether to register jQuery and the ClientsideViews JavaScript.
     */
    public $enableJS = true;

    protected $_assetsUrl;
    protected $_assetsPath;

    /**
     * Initializes the component.
     */
    public function init( ) {
        if( !Yii::getPathOfAlias( 'clientsideviews' ) )
            Yii::setPathOfAlias( 'clientsideviews', realpath( dirname( __FILE__ ).'/..' ) );
        
        $generatedTemplateFile = Yii::getPathOfAlias( 'clientsideviews.assets.javascripts' ) . DIRECTORY_SEPARATOR . 'mustache.tpl.js';
        if (!is_file($generatedTemplateFile)) {
            $this->refreshMustacheTemplates();            
        }
        
        if( $this->enableJS ) {
            $this->registerJs( );
        }
    }

    /**
     * Registers the core JavaScript plugins.
     * @since 0.9.8
     */
    public function registerJs( ) {
  
  Yii::app( )->clientScript->registerCoreScript( 'jquery' );
        $this->registerScriptFile( 'ICanHaz.min.js' );
        $this->registerScriptFile( 'mustache.tpl.js' );
    }

    /**
     * Registers a JavaScript file in the assets folder.
     * @param string $fileName the file name.
     * @param integer $position the position of the JavaScript file.
     */
    public function registerScriptFile( $fileName, $position = CClientScript::POS_END ) {
        Yii::app( )->clientScript->registerScriptFile( $this->getAssetsUrl( ).DIRECTORY_SEPARATOR.$fileName, $position );
    }

    /**
     * Returns the URL to the published assets folder.
     * @return string the URL
     */
    protected function getAssetsUrl( ) {
        if( $this->_assetsUrl == null ) {
            $assetsPath = Yii::getPathOfAlias( 'clientsideviews.assets.javascripts' );
            $this->_assetsUrl = Yii::app( )->assetManager->publish( $assetsPath, false, -1, YII_DEBUG );
        }
        return $this->_assetsUrl;
    }
 
    public function refreshMustacheTemplates()
    {
        /* 
        Find all files recursivly in the basepath/protected named mustache.tpl
        Foreach files add to js array with a name based on the directory path and filename without 
        mustache.tpl
        
        */
        $basePath = Yii::app()->basePath;
        $templates = array();
        $options=  array('fileTypes'=>array('tpl'));
        $templateFiles = CFileHelper::findFiles(realpath(Yii::app()->basePath),$options);
        foreach($templateFiles as $file){
            // stupid additional check due to the findFiles function cannot handle . seperated filenames
            if (strpos($file,'mustache') !== false) {
                $templateId = str_replace(array($basePath,DIRECTORY_SEPARATOR,'mustache.tpl','.'),array('','_','',''),$file);
                array_push($templates, array(
                    'name' => $templateId,
                    'template' => $this->stripEndLine($this->readTemplate($file)))
                );
            }
        }

        $templatesJs = "$.each(".json_encode($templates).", function (index, template) {ich.addTemplate(template.name, template.template);});";
        $this->writeTemplateFile(Yii::getPathOfAlias( 'clientsideviews.assets.javascripts' ), $templatesJs);        
 }
    
    private function writeTemplateFile($path,$fileContents)
    {
        $my_file = $path. DIRECTORY_SEPARATOR . 'mustache.tpl.js';
        $handle = fopen($my_file, 'w') or die('Cannot open file:  '.$my_file);
        fwrite($handle, $fileContents);
        fclose($handle);
    }
        
    private function readTemplate($file)
    {

        $handle = fopen($file, 'r');
        $data = fread($handle,filesize($file));
        fclose($handle);
        return $data;
    }
    
    private function stripEndLine($template)
    {
        $output = str_replace(array("\r\n", "\r"), "\n", $template);
        $lines = explode("\n", $output);
        $new_lines = array();

        foreach ($lines as $i => $line) {
            if(!empty($line))
                $new_lines[] = trim($line);
        }
        return implode($new_lines);        
    }
 
}

By initilizing the ICanHaz javascript Mustache template wrapper in the mustache.tpl.js file then the templates are ready for use in the browser without manually having to register the templates.

$.each(
[
{"name":"_views_meeting_meetingList",
"template":"<table class='responsive'><th>Meeting ID<\/th><th>Meeting Name<\/th><th>Create Time<\/th><th>Running<\/th>{{#meetings}}<tr><td>{{meetingID}}<\/td><td>{{meetingName}}<\/td><td>{{createTime}}<\/td><td>{{running}}<\/td><\/tr>{{\/meetings}}<\/table>"}
], 
function (index, template) {ich.addTemplate(template.name, template.template);}
);

3. Rendering templates should be clean.

What now remains is the actual code that is rendering data from a controller call with the clientside template.

The controller that is returning some data in json format

public function actionMeetingList()
 {
        ...

        //show all meetings
        $meetings=$bbb->getMeetings();
        echo json_encode($meetings) ;
 }

And finally the clientside code responsible for the merging of data and the clientside templates.

<div id="meeting_list"></div>


<script type="text/javascript">
    $(document).ready(function () {
        /* 
        Getting data and rendering in template
        */
        $.getJSON('/meeting/meetinglist', function (meetings) {
            renderedTemplate = ich._views_meeting_meetingList({'meetings': meetings});
            $('#user_list').append(renderedTemplate);
        });
    });
</script>

A final note, currently to avoid a full traversal of the filesystem on every request, I have added a check in the function init() in the ClientsideViews Yii component. This checks if the file exists on the file system. This is a tradeoff between my limited knowledge of Yii and how to better implement this in the framework, avoiding traversing the filesystem (very slow) for each request and manually being able to trigger a refresh by deleting the file located under /assets/javascripts/mustache.tpl.js. I am sure someone with more knowledge of the framework know the right way to hook this up to automate the refreshing when the files change. This however works for my purpose. I have created a github repository that is available at Yii ClientsideViews extension. I have attempted to create a new extension on the YiiFramework website, but was not able to since apparently I am "too new" on the site.