Showing posts with label REST. Show all posts
Showing posts with label REST. Show all posts

Monday, November 26, 2012

High performance cachable websites web 2.0 in YiiFramework, mustache.js and icanhaz.js

Preface: 

To be able to cache webpage content as much as possible then it is important to delay merging data with the HTML markup until the last possible moment (which is in the browser). The moment you merge data with HTML then the page is always specific to the context in which it was retrieved. If this context happens to be a user context (which is almost always the case), then you cannot use this cached page for another user, which is bad for cacheability. 

Achieving this "late-merging" of data and markup can be achieved by building the skeleton of the site in very few html pages. Most normal sites could potentially be built using the following skeleton pages:

  • frontpage/landingpage 
  • login 
  • content 
Of course the more the pages differ from each other the more of these skeleton pages you will have. The point is to isolate the common things between the pages and boil it down to the minimum amount of pages. 

These pages (one for each distinct different buildup of the page) will contain a link to a javascript file that contains all the clientside templates/markup needed to render data. These clientside templates, can be built in one of the many templating js engines out there, for instance:

(LinkedIn's review of quite a few of these template engines The client-side templating throwdown: mustache, handlebars, dust.js, and more). 


This way we can cache the clientside needed markup used to display lists and forms (since it is a static javascript file) using nginx or other caching servers. The skeleton pages can also be cached since they contain no data. The only thing remaining is CSS, javascript, images, videos and data. Everything apart from data can also be cached using caching servers. Data we will retrieve using AJAX requests as json and then rendering it clientside. 

YiiFramework: 

I am new to the Yii framework, but needed to build a web 2.0 site that should support caching of as much of the content as possible to avoid overloading the webserver re-sending the same markup over and over again just with different data embedded. As far as I could see there was not really any built-in support in Yii to do this (apart from doing it all manually). The exising Mustache extention seemed to be related to server side templating rather than clientside templating. For this reason I created my own extension. 

The goals were: 
  1. Creating views in Yii clientside should be as similar as possible as creating server side views 
  2. Templates should be accessible on the clientside without a lot of plumbing code 
  3. Rendering templates should be clean. 

In the following I will try to argue for my decisions on how to solve the above.

- Creating views in Yii clientside should be as similar as possible as creating server side views
&
-Templates should be accessible on the clientside without a lot of plumbing code

I wanted to be able to put my clientside views in what folder I though made the most sense, and since the clientside templates very much are related to the controller actions that are delivering the data (just like server side views) I wanted to be able to put them for instance in the view folders.
I wanted it to be easy to edit the templates in an IDE and one template should be self contained and not mixed up with the other templates

The templates should not incur a significant serverside overhead (currently I am still working on a better solution than the one I have found - mentioned at the end of the posting).

To achieve the above I decided that in the IDE the client side templates should be seperate files, and to be able to distinguish them from the server side templates then needed another name that supported the js templating engine that I choose (mustache.js), so the file extension ended up being .tpl (since Smarty templates are supported in my IDE).

So the challenge is how to convert disparate files on the filesystem into a single js file that is read and initialized by the browser automatically.

I need go through the filesystem recursively and look for files of a certain kind (*.mustache.tpl), index them by they location in the filesystem and their filename. Push them info a javascript array and add the code needed to initialize the templates when the browser loads the template javascript  file.

All the above is solved in the following Yii component, which also adds the needed javascripts for the template rendering clientside.


<?php
/**
 * ClientsideViews class file.
 * @author Kenneth Thorman (kenneth.thorman@appinux.com)
 * @license http://www.opensource.org/licenses/bsd-license.php New BSD License
 */

/**
 * ClientsideViews application component.
 * Used for registering ClientsideViews core functionality.
 */
class ClientsideViews extends CApplicationComponent {

    /**
     * @var boolean whether to register jQuery and the ClientsideViews JavaScript.
     */
    public $enableJS = true;

    protected $_assetsUrl;
    protected $_assetsPath;

    /**
     * Initializes the component.
     */
    public function init( ) {
        if( !Yii::getPathOfAlias( 'clientsideviews' ) )
            Yii::setPathOfAlias( 'clientsideviews', realpath( dirname( __FILE__ ).'/..' ) );
        
        $generatedTemplateFile = Yii::getPathOfAlias( 'clientsideviews.assets.javascripts' ) . DIRECTORY_SEPARATOR . 'mustache.tpl.js';
        if (!is_file($generatedTemplateFile)) {
            $this->refreshMustacheTemplates();            
        }
        
        if( $this->enableJS ) {
            $this->registerJs( );
        }
    }

    /**
     * Registers the core JavaScript plugins.
     * @since 0.9.8
     */
    public function registerJs( ) {
  
  Yii::app( )->clientScript->registerCoreScript( 'jquery' );
        $this->registerScriptFile( 'ICanHaz.min.js' );
        $this->registerScriptFile( 'mustache.tpl.js' );
    }

    /**
     * Registers a JavaScript file in the assets folder.
     * @param string $fileName the file name.
     * @param integer $position the position of the JavaScript file.
     */
    public function registerScriptFile( $fileName, $position = CClientScript::POS_END ) {
        Yii::app( )->clientScript->registerScriptFile( $this->getAssetsUrl( ).DIRECTORY_SEPARATOR.$fileName, $position );
    }

    /**
     * Returns the URL to the published assets folder.
     * @return string the URL
     */
    protected function getAssetsUrl( ) {
        if( $this->_assetsUrl == null ) {
            $assetsPath = Yii::getPathOfAlias( 'clientsideviews.assets.javascripts' );
            $this->_assetsUrl = Yii::app( )->assetManager->publish( $assetsPath, false, -1, YII_DEBUG );
        }
        return $this->_assetsUrl;
    }
 
    public function refreshMustacheTemplates()
    {
        /* 
        Find all files recursivly in the basepath/protected named mustache.tpl
        Foreach files add to js array with a name based on the directory path and filename without 
        mustache.tpl
        
        */
        $basePath = Yii::app()->basePath;
        $templates = array();
        $options=  array('fileTypes'=>array('tpl'));
        $templateFiles = CFileHelper::findFiles(realpath(Yii::app()->basePath),$options);
        foreach($templateFiles as $file){
            // stupid additional check due to the findFiles function cannot handle . seperated filenames
            if (strpos($file,'mustache') !== false) {
                $templateId = str_replace(array($basePath,DIRECTORY_SEPARATOR,'mustache.tpl','.'),array('','_','',''),$file);
                array_push($templates, array(
                    'name' => $templateId,
                    'template' => $this->stripEndLine($this->readTemplate($file)))
                );
            }
        }

        $templatesJs = "$.each(".json_encode($templates).", function (index, template) {ich.addTemplate(template.name, template.template);});";
        $this->writeTemplateFile(Yii::getPathOfAlias( 'clientsideviews.assets.javascripts' ), $templatesJs);        
 }
    
    private function writeTemplateFile($path,$fileContents)
    {
        $my_file = $path. DIRECTORY_SEPARATOR . 'mustache.tpl.js';
        $handle = fopen($my_file, 'w') or die('Cannot open file:  '.$my_file);
        fwrite($handle, $fileContents);
        fclose($handle);
    }
        
    private function readTemplate($file)
    {

        $handle = fopen($file, 'r');
        $data = fread($handle,filesize($file));
        fclose($handle);
        return $data;
    }
    
    private function stripEndLine($template)
    {
        $output = str_replace(array("\r\n", "\r"), "\n", $template);
        $lines = explode("\n", $output);
        $new_lines = array();

        foreach ($lines as $i => $line) {
            if(!empty($line))
                $new_lines[] = trim($line);
        }
        return implode($new_lines);        
    }
 
}

By initilizing the ICanHaz javascript Mustache template wrapper in the mustache.tpl.js file then the templates are ready for use in the browser without manually having to register the templates.


$.each(
[
{"name":"_views_meeting_meetingList",
"template":"<table class='responsive'><th>Meeting ID<\/th><th>Meeting Name<\/th><th>Create Time<\/th><th>Running<\/th>{{#meetings}}<tr><td>{{meetingID}}<\/td><td>{{meetingName}}<\/td><td>{{createTime}}<\/td><td>{{running}}<\/td><\/tr>{{\/meetings}}<\/table>"}
], 
function (index, template) {ich.addTemplate(template.name, template.template);}
);


3. Rendering templates should be clean.

What now remains is the actual code that is rendering data from a controller call with the clientside template.

The controller that is returning some data in json format

public function actionMeetingList()
 {
        ...

        //show all meetings
        $meetings=$bbb->getMeetings();
        echo json_encode($meetings) ;
 }


And finally the clientside code responsible for the merging of data and the clientside templates.


<div id="meeting_list"></div>


<script type="text/javascript">
    $(document).ready(function () {
        /* 
        Getting data and rendering in template
        */
        $.getJSON('/meeting/meetinglist', function (meetings) {
            renderedTemplate = ich._views_meeting_meetingList({'meetings': meetings});
            $('#user_list').append(renderedTemplate);
        });
    });
</script>

A final note, currently to avoid a full traversal of the filesystem on every request, I have added a check in the function init() in the ClientsideViews Yii component. This checks if the file exists on the file system. This is a tradeoff between my limited knowledge of Yii and how to better implement this in the framework, avoiding traversing the filesystem (very slow) for each request and manually being able to trigger a refresh by deleting the file located under /assets/javascripts/mustache.tpl.js. I am sure someone with more knowledge of the framework know the right way to hook this up to automate the refreshing when the files change. This however works for my purpose. I have created a github repository that is available at Yii ClientsideViews extension. I have attempted to create a new extension on the YiiFramework website, but was not able to since apparently I am "too new" on the site.

Sunday, February 27, 2011

WCF REST Exception handling deserializing with XmlSerializer

As an alternative to using the DataContractSerializer you can use the XmlSerializer in your WCF services. Assuming that you have a special need that cannot be covered by the DataContractSerializer here is a sample of how to catch and handle WCF REST faults on the client side. This provides an alternative to using the DataContractSerializer which is used in this post WCF REST Exception handlingThe WCF REST Exception handling post sets the scene for this post.

Dan Rigsbyb have a great blog posting where he compares the DataContractSerializer with the XmlSerializer, it is located here XmlSerializer vs DataContractSerializer: Serialization in Wcf.
The advantage of this version of the WcfRestExceptionHelper class is that it allows for somewhat simpler client code (compared to the version in  WCF REST Exception handling). 
To mix things up a little bit I have chosen to use the XmlSerializer instead of the DataContractSerializerWithout further ado.

WCF REST client code:

var factory = new ChannelFactory<IService1Wrapper>("Service1WrapperREST");var proxy = factory.CreateChannel();

try
{
 var serviceResult = proxy.GetProductById("1");

 // Do something with result
}
catch (Exception exceptionThrownByRestWcfCall)
{
 var serviceResult = WcfRestExceptionHelper<SampleItem[], SampleError>.HandleRestServiceError(exceptionThrownByRestWcfCall);

 // Do something with result, let higher levels in the callstack handle possible real exceptions
}
finally
{
 ((IDisposable)proxy).Dispose();
}

The WcfRestExceptionHelper class

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Net;
using System.ServiceModel;
using System.Text;
using System.Xml.Serialization;

namespace WcfRestClient.WcfErrors
{
 public static class WcfRestExceptionHelper<TServiceResult, TServiceFault> where TServiceFault : class
 {
  private static IDictionary<Type, XmlSerializer> cachedSerializers = new Dictionary<Type, XmlSerializer>();

  public static TServiceResult HandleRestServiceError(Exception exception)
  {
   if (exception == null) throw new ArgumentNullException("exception");

   // REST uses the HTTP procol status codes to communicate errors that happens on the service side.
   // This means if we have a teller service and you need to supply username and password to login
   // and you do not supply the password, a possible scenario is that you get a 400 - Bad request.
   // However it is still possible that the expected type is returned so it would have been possible 
   // to process the response - instead it will manifest as a ProtocolException on the client side.
   var protocolException = exception as ProtocolException;
   if (protocolException != null)
   {
    var webException = protocolException.InnerException as WebException;
    if (webException != null)
    {
     var responseStream = webException.Response.GetResponseStream();
     if (responseStream != null)
     {
      try
      {
       // Debugging code to be able to see the reponse in clear text
       //SeeResponseAsClearText(responseStream);

       // Try to deserialize the returned XML to the expected result type (TServiceResult)
       return (TServiceResult) GetSerializer(typeof(TServiceResult)).Deserialize(responseStream);
      }
      catch (InvalidOperationException serializationException)
      {
       // This happens if we try to deserialize the responseStream to type TServiceResult
       // when an error occured on the service side. An service side error serialized object 
       // is not deserializable into a TServiceResult

       // Reset responseStream to beginning and deserialize to a TServiceError instead
       responseStream.Seek(0, SeekOrigin.Begin);

       var serviceFault = (TServiceFault)GetSerializer(typeof(TServiceFault)).Deserialize(responseStream);
       throw new WcfRestServiceException<TServiceFault>() { ServiceFault = serviceFault };
      }
     }
    }
   }

   // Don't know how to handle this exception
   throw exception;
  }
  
  /// <summary>
  /// Based on the knowledge of how the XmlSerializer work, I found it safest to explicitly implement my own caching mechanism.
  /// 
  /// From MSDN:
  /// To increase performance, the XML serialization infrastructure dynamically generates assemblies to serialize and 
  /// deserialize specified types. The infrastructure finds and reuses those assemblies. This behavior occurs only when 
  /// using the following constructors:
  /// 
  /// XmlSerializer.XmlSerializer(Type)
  /// XmlSerializer.XmlSerializer(Type, String)
  /// 
  /// If you use any of the other constructors, multiple versions of the same assembly are generated and never unloaded, 
  /// which results in a memory leak and poor performance. The easiest solution is to use one of the previously mentioned 
  /// two constructors. Otherwise, you must cache the assemblies.
  /// 
  /// </summary>
  /// <param name="classSpecificSerializer"></param>
  /// <returns></returns>
  private static XmlSerializer GetSerializer(Type classSpecificSerializer)
  {
   if (!cachedSerializers.ContainsKey(classSpecificSerializer))
   {
    cachedSerializers.Add(classSpecificSerializer, new XmlSerializer(classSpecificSerializer));
   }
   return cachedSerializers[classSpecificSerializer];
  }

  /// <summary>
  /// Debugging helper method in case there are problems with the deserialization
  /// </summary>
  /// <param name="responseStream"></param>
  private static void SeeResponseAsClearText(Stream responseStream)
  {
   var responseStreamLength = responseStream.Length;
   var buffer = new byte[responseStreamLength];
   var x = responseStream.Read(buffer, 0, Convert.ToInt32(responseStreamLength));
   var enc = new UTF8Encoding();
   var response = enc.GetString(buffer);
   Debug.WriteLine(response);
   responseStream.Seek(0, SeekOrigin.Begin);
  }
 }
}
The generic exception that we can throw

using System;
namespace WcfRestClient.WcfErrors
{
 public class WcfRestServiceException<TServiceError> : Exception where TServiceError : class
 {
  public WcfRestServiceException() : base("An service error occured. Please inspect the ServiceError property for details.")
  {
  }
  
  public TServiceError ServiceFault { get; set; }
 }
}

Friday, February 18, 2011

WCF REST Internal Microsoft HttpWebResponse validation rules

The posting WCF REST Exception handling deals with handling ProtocolExceptions thrown by Microsoft's WCF REST implementation. This specific ProtocolException/Exception is thrown based on rules located in various methods in the following classes/methods in the .NET framework.

private bool CompleteGetResponse(IAsyncResult result);
 
Declaring Type: System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelAsyncRequest 
Assembly: System.ServiceModel, Version=4.0.0.0 

 
public Message WaitForReply(TimeSpan timeout);
 
Declaring Type: System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest 
Assembly: System.ServiceModel, Version=4.0.0.0 
Then you can trace the code into the various static utility methods such as

Declaring Type: System.ServiceModel.Channels.HttpChannelUtilities 
Assembly: System.ServiceModel, Version=4.0.0.0 


public static HttpWebResponse ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason);
 
public static HttpInput ValidateRequestReplyResponse(HttpWebRequest request, HttpWebResponse response, HttpChannelFactory factory, WebException responseException, ChannelBinding channelBinding);

public static Exception CreateNullReferenceResponseException(NullReferenceException nullReferenceException);

Monday, February 07, 2011

WCF REST Exception handling

WCF normally offers all the extensibility you need, but recently when implementing a WCF REST client in .NET 4.0, I found that it seems like there is at least a little inconsistency in how WCF REST handles service faults compared to other WCF implementations.


Ground rules:
In WCF it is not recommended to throw .NET exceptions since they most likely are not known and recognized in non .NET clients. so instead we are using FaultException<TDetail> and WebFaultException. The below quote is taken from the MSDN WebFaultException documentation.
When using a WCF REST endpoint (WebHttpBinding and WebHttpBehavior or WebScriptEnablingBehavior) the HTTP status code on the response is set accordingly. However, WebFaultException can be used with non-REST endpoints and behaves like a regular FaultException.

When using a WCF REST endpoint, the response format of the serialized fault is determined in the same way as a non-fault response. For more information about WCF REST formatting, see WCF REST Formatting.
Thoughts and implementation 
My intial take on how to handle service side generated faults/errors was to use the IClientMessageInspector & Message Inspectors for fault handling. However nothing seemed to work. I got a ProtocolException that was thrown from deep in the WCF REST implementation. The exception was thrown by the underlying WCF REST implementation before the IClientMessageInspector.AfterReceiveReply method was invoked. After spending some time in Reflector with a fellow developer it indeed seemed like there are no extensibility points you can hook into. As soon as a WCF REST service throws a WebFaultException with a fault indicating http response code it will trigger a ProtocolException on the WCF REST client side.


So it seems like you have to solve this with wrapping the proxy calls in a try catch block and then extract the InnerException from the ProtocolException and convert this from a Stream ... etc, etc, etc (i.e. repetitive plumbing code) . Since this plumbing code will be the same for most if not all clients, I created a helper class that should work for any service.


The code that is used here is building on code that has been explained in the following blog postings:
  1. WCF REST service with custom http header check in .NET 4
  2. WCF REST client using custom http headers
The full solution can be downloaded here WcfRestClientExceptionHandling.7z.

Try running the client form with the default app.config and then try changing
<customHttpHeaders>
 <headers>
  <add key="MyCustomHttpHeader" value="some_value"></add>
  <add key="MyCustomHttpHeader2" value="yet_another_value"></add>
 </headers>
</customHttpHeaders>
to the following
<customHttpHeaders>
 <headers>
 </headers>
</customHttpHeaders>
First let us define our service.
using System.Net;
using System.ServiceModel;
using System.ServiceModel.Activation;
using System.ServiceModel.Web;
using Common;

namespace WcfRestService1
{
 // Start the service and browse to http://<machine_name>:<port>/Service1/help to view the service's generated help page
 // NOTE: By default, a new instance of the service is created for each call; change the InstanceContextMode to Single if you want
 // a single instance of the service to process all calls. 
 [ServiceContract]
 [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
 [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
 public class Service1
 {
  [WebGet(UriTemplate = "{id}")]
  public SampleItem[] GetByProductId(string id)
  {
   if (WebOperationContext.Current.IncomingRequest.Headers["MyCustomHttpHeader"] == null) 
     throw new WebFaultException<SampleError>(new SampleError(){ErrorCode = "1223456789", Message = "The custom httpheader 'MyCustomHttpHeader' has not been set."}, HttpStatusCode.BadRequest);
   
   return new[] { new SampleItem() { Id = 1, StringValue = "Product1" } };
  }
 }
}
Looking through the code we see that there are 2 possible objects returned (ignoring all the potential unhandled exceptions). The 2 possible objects are SampleItem, SampleError.

using System.Runtime.Serialization;

namespace Common
{
 [DataContract]
 public class SampleError
 {
  [DataMember]
  public string Message { get; set; }
  [DataMember]
  public string ErrorCode { get; set; }
 }
}

using System.Runtime.Serialization;

namespace Common
{
 [DataContract]
 public class SampleItem
 {
  [DataMember]
  public int Id { get; set; }
  [DataMember]
  public string StringValue { get; set; }
 }
}
I will not get into the additional classes used to add custom http headers for the requests as they are explained in the previous indicated postings. Suffice to say that you can add and modify the headers included in the call to the WCF service by editing the customHeaders/headers element in the app.config file.

<behaviors>
 <endpointBehaviors>
  <behavior name="CustomHeaderBehavior">
   <customHttpHeaders>
    <headers>
     <add key="MyCustomHttpHeader" value="some_value"></add>
     <add key="MyCustomHttpHeader2" value="yet_another_value"></add>
    </headers>
   </customHttpHeaders>
   <webHttp/>
  </behavior>
 </endpointBehaviors>
</behaviors>
Very briefly looking at the interface used to describe the service on the client side

using System.ServiceModel;
using System.ServiceModel.Web;
using Common;

namespace WcfRestClient
{
 [ServiceContract]
 public interface IService1Wrapper
 {
  [OperationContract]
  [WebGet(
   BodyStyle = WebMessageBodyStyle.Bare,
   ResponseFormat = WebMessageFormat.Xml,
   UriTemplate = "/Service1/GetByProductId?id={id}")]
  SampleItem[] GetProductById(string id);
 }
}
Now we will look at the client code. There are a few different outcomes of a service call.


Handled by proposed helper class:
  1. Everything is OK and the call returns the expected type deserialized on our/client end
  2. Everything is not OK and the service throws a FaultException and the http status code is one of the fault status codes. This will result in the current WCF REST implementation to raise a ProtocolException. The proposed code will try to handle that.
  3. Any other Exceptions thrown when trying to handle the result or fault from the service can be caught and handled or rethrown.

Not Handled by proposed helper class:
  1. Inconsistent use of http status codes and raising FaultExceptions. (Everything is not OK and the service throws a FaultException which needs to be handled on our client end. If this is combined with a http status code of 200/OK then we expect a TServiceResult but we are getting a TServiceFault. This will throw a SerializationException in the HandleRestServiceError() mehotd. The solution proposed here will not be able to handle that case. However one could argue that that is a problem at the service level.)
using System;
using System.ServiceModel;
using System.Windows.Forms;
using Common;
using WcfRestClient.WcfErrors;

namespace WcfRestClient
{
 public partial class Form1 : Form
 {
  public Form1()
  {
   InitializeComponent();
  }

  private void button1_Click(object sender, EventArgs e)
  {
   var factory = new ChannelFactory<IService1Wrapper>("Service1WrapperREST");
   var proxy = factory.CreateChannel();

   try
   {
    // This call can return one of the following object as a serialized response
    // If successful -> SampleItem[]
    // If error      -> SampleError
    // We can only handle SampleItem[]. If a combination of
    // a FaultException thrown + http status code OK then this code will 
    // throw a SerializationException
    var serviceResult = proxy.GetProductById("1");

    // In this case the serviceResult is returning the expected type
    HandleServiceResult(serviceResult);
   }
   catch (Exception exceptionThrownByRestWcfCall)
   {
    // There are a few different ways you can write the code below
    // Lambda expression with anonymous method
    // serviceResult =>
    //  {
    //    MessageBox.Show("ServiceResult:" + serviceResult[0].StringValue);
    //  },
    //
    // Lambda expression with named method
    // serviceResult => HandleServiceResult(serviceResult)
    //
    // .NET 4.0 MethodGroup shorthand
    // HandleServiceResult
    //
    // MethodGroup shorthand is the shortest / clearest I think but I am 
    // opting for the Lambda expression to keep the coding style the same.
    WcfRestExceptionHelper<SampleItem[], SampleError>.HandleRestServiceError(
     exceptionThrownByRestWcfCall,
     serviceResult =>
      {
       HandleServiceResult(serviceResult);
      },
     serviceFault =>
      {
       MessageBox.Show(serviceFault.Message, "ServiceFaultHandler");
      },
     exception =>
      {
       MessageBox.Show(exception.ToString(), "ExceptionHandler");
      }
     );
   }
   finally
   {
    ((IDisposable)proxy).Dispose();
   }
  }

  private static void HandleServiceResult(SampleItem[] serviceResult)
  {
   MessageBox.Show(serviceResult[0].StringValue, "ServiceResult:");
  }
 }
}
The interesting part is the following code

try
{
 var serviceResult = proxy.GetProductById("1");
 HandleServiceResult(serviceResult);
}
catch (Exception exceptionThrownByRestWcfCall)
{
 WcfRestExceptionHelper<SampleItem[], SampleError>.HandleRestServiceError(
  exceptionThrownByRestWcfCall,
  serviceResult =>
   {
    HandleServiceResult(serviceResult);
   },
  serviceFault =>
   {
    MessageBox.Show(serviceFault.Message, "ServiceFaultHandler");
   },
  exception =>
   {
    MessageBox.Show(exception.ToString(), "ExceptionHandler");
   }
  );
}
That is a lot of code for a handling the responses from a simple service call, rewriting the code a bit (removing bracketed approach in the lambdas, omitting the optional exceptionHandler Action delegate in the call, as well as using a method group shorthand we can squeeze the code down to a fairly long (unreadable) one-liner.

try
{
 var serviceResult = proxy.GetProductById("1");
 HandleServiceResult(serviceResult);
}
catch (Exception exceptionThrownByRestWcfCall)
{
 WcfRestExceptionHelper<SampleItem[], SampleError>.HandleRestServiceError(exceptionThrownByRestWcfCall, HandleServiceResult, serviceFault => MessageBox.Show(serviceFault.Message, "ServiceFaultHandler"));
}
This allows us to put the actual handling code here where the service is called, without most of the plumbing code. The implementation of HandleRestServiceError() can be seen below.
public static void HandleRestServiceError(Exception exception, Action<TServiceResult> serviceResultHandler, Action<TServiceFault> serviceFaultHandler = null, Action<Exception> exceptionHandler = null)
{
 var serviceResultOrServiceFaultHandled = false;

 if (exception == null) throw new ArgumentNullException("exception");
 if (serviceResultHandler == null) throw new ArgumentNullException("serviceResultHandler");

 // REST uses the HTTP procol status codes to communicate errors that happens on the service side.
 // This means if we have a teller service and you need to supply username and password to login
 // and you do not supply the password, a possible scenario is that you get a 400 - Bad request.
 // However it is still possible that the expected type is returned so it would have been possible 
 // to process the response - instead it will manifest as a ProtocolException on the client side.
 var protocolException = exception as ProtocolException;
 if (protocolException != null)
 {
  var webException = protocolException.InnerException as WebException;
  if (webException != null)
  {
   var responseStream = webException.Response.GetResponseStream();
   if (responseStream != null)
   {
    try
    {
     // Debugging code to be able to see the reponse in clear text
     //SeeResponseAsClearText(responseStream);

     // Try to deserialize the returned XML to the expected result type (TServiceResult)
     var response = (TServiceResult) GetSerializer(typeof(TServiceResult)).ReadObject(responseStream);
     serviceResultHandler(response);
     serviceResultOrServiceFaultHandled = true;
    }
    catch (SerializationException serializationException)
    {
     // This happens if we try to deserialize the responseStream to type TServiceResult
     // when an error occured on the service side. An service side error serialized object 
     // is not deserializable into a TServiceResult

     // Reset responseStream to beginning and deserialize to a TServiceError instead
     responseStream.Seek(0, SeekOrigin.Begin);

     var serviceFault = (TServiceFault) GetSerializer(typeof(TServiceFault)).ReadObject(responseStream);

     if (serviceFaultHandler != null && serviceFault != null)
     {
      serviceFaultHandler(serviceFault);
      serviceResultOrServiceFaultHandled = true;
     }
     else if (serviceFaultHandler == null && serviceFault != null)
     {
      throw new WcfServiceException<TServiceFault>() { ServiceFault = serviceFault };
     }
    }
   }
  }
 }

 // If we have not handled the serviceResult or the serviceFault then we have to pass it on to the exceptionHandler delegate
 if (!serviceResultOrServiceFaultHandled && exceptionHandler != null)
 {
  exceptionHandler(exception);
 }
 else if (!serviceResultOrServiceFaultHandled && exceptionHandler == null)
 {
  // Unable to handle and no exceptionHandler passed in throw exception to be handled at a higher level
  throw exception;
 }
}
After spending some time on this posting and coding it I found this forum post How to return a FaultException from a WCF client that calls a REST API. It does seem to come to a similar conclusion.

Tuesday, February 01, 2011

WCF REST client using custom http headers

This deals with using WCF as a framework to consume REST based services (where the services not necessarily are written in .NET). This is a follow up post of WCF REST service with custom http header check in .NET 4.


REST based services are to a higher degree than SOAP or other services based on the basic HTTP protocol mechanisms. This will also often mean that you are using http headers to relay information, as well as possibly cookies and also utilizing the different available request methods as per RFC for Hypertext Transfer Protocol -- HTTP/1.1 (Method Definitions).


Let us assume that we have the following REST based service.

using System.Net;
using System.Collections.Generic;
using System.ServiceModel;
using System.ServiceModel.Activation;
using System.ServiceModel.Web;
using Common;

namespace WcfRestService1
{
 // Start the service and browse to http://<machine_name>:<port>/Service1/help to view the service's generated help page
 // NOTE: By default, a new instance of the service is created for each call; change the InstanceContextMode to Single if you want
 // a single instance of the service to process all calls. 
 [ServiceContract]
 [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
 [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
 public class Service1
 {
  [WebGet(UriTemplate = "{id}")]
  public SampleItem[] GetProductById(string id)
  {
   if (WebOperationContext.Current.IncomingRequest.Headers["MyCustomHttpHeader"] == null) 
     throw new WebFaultException<string>("The custom httpheader 'MyCustomHttpHeader' has not been set.", HttpStatusCode.BadRequest);
   
   return new[] { new SampleItem() { Id = 1, StringValue = "Product1" } };
  }
 }
}


This service requires a http header named "MyCustomHttpHeader" is part of the request. The service does currently no care what values are sent with this header, but it needs to be part of the request.

We have defined the DataContract for SampleItem in a Common project.


using System.Runtime.Serialization;

namespace Common
{
 [DataContract]
 public class SampleItem
 {
  [DataMember]
  public int Id { get; set; }
  [DataMember]
  public string StringValue { get; set; }
 }
}


On the client side we have defined an interface that describes our service, so its methods can be accessed by client code.


using System.ServiceModel;
using System.ServiceModel.Web;
using Common;

namespace CRTest
{
 [ServiceContract]
 public interface IService1Wrapper
 {
  [OperationContract]
  [WebGet(
   BodyStyle = WebMessageBodyStyle.Bare,
   ResponseFormat = WebMessageFormat.Xml,
   UriTemplate = "/Service1/GetByProductId?id={id}")]
  SampleItem[] GetProductById(string id);
 }
}



Finally we have some client code that is trying to use our service.


using System;
using System.ServiceModel;
using System.Windows.Forms;

namespace CRTest
{
 public partial class Form1 : Form
 {
  public Form1()
  {
   InitializeComponent();
  }

  private void button1_Click(object sender, EventArgs e)
  {
   var factory = new ChannelFactory<IService1Wrapper>("Service1WrapperREST");
   var proxy = factory.CreateChannel();
   var response = proxy.GetProductById("1");
   ((IDisposable)proxy).Dispose();
  }
 }
}


There is nothing in this code that indicates that custom headers are added, and indeed there are 3 more classes that needs to be presented and we also need to see the app.config file to get the full picture.

First let us have a look at the app.config file.



<?xml version="1.0"?>
<configuration>
 <startup>
  <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
 </startup>
 <system.serviceModel>
  <client>
   <endpoint address="http://localhost:62510"
        binding="webHttpBinding"
        contract="CRTest.IService1Wrapper"
        name="Service1WrapperREST"
        behaviorConfiguration="CustomHeaderBehavior" />
  </client>
  <behaviors>
   <endpointBehaviors>
    <behavior name="CustomHeaderBehavior">
     <customHttpHeaders>
      <headers>
       <add key="MyCustomHttpHeader" value="some_value"></add>
       <add key="MyCustomHttpHeader2" value="yet_another_value"></add>
      </headers>
     </customHttpHeaders>
     <webHttp/>
    </behavior>
   </endpointBehaviors>
  </behaviors>
  <extensions>
   <behaviorExtensions>
    <add name="customHttpHeaders" type="CRTest.CustomHeaderBehaviorExtensionElement, CRTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
   </behaviorExtensions>
  </extensions>
 </system.serviceModel>
</configuration>


Looking at the system.servicemodel/client/endpoint we see nothing unusual in the first 4 lines, but then I have added a behaviorConfiguration named CustomHeaderBehavior.

When we continue down through the config file and at system.servicemodel/behaviors/endpointBehaviors/behavior name="CustomHeaderBehavior" we have an element named customHttpHeaders which basically define all the custom headers that we want to include in our REST request.

Finally in the config file we see that a behaviorExtensions has been added to add support for these new customHttpHeaders xml elements in our configuration file.

Let us start with the CRTest.CustomHeaderBehaviorExtensionElement class.


using System;
using System.Collections.Generic;
using System.Configuration;
using System.Linq;
using System.ServiceModel.Configuration;

namespace CRTest
{
 public class CustomHeaderBehaviorExtensionElement : BehaviorExtensionElement 
 {
  protected override object CreateBehavior()
  {
   Dictionary<string, string> customHeaders = null;
   if (CustomHeaders != null)
   {
    customHeaders = CustomHeaders.AllKeys.ToDictionary(key => key, key => CustomHeaders[key].Value);
   }
   return new CustomHeaderBehavior(customHeaders);
  }

  public override Type BehaviorType
  {
   get { return typeof (CustomHeaderBehavior); }
  }

  [ConfigurationProperty("headers", IsRequired = true)]
  [ConfigurationCollection(typeof(KeyValueConfigurationCollection))]
  public KeyValueConfigurationCollection CustomHeaders
  {
   get
   {
    return (KeyValueConfigurationCollection) base["headers"];
   }
  }
 }
}


Starting from the bottom of the class and moving up we see the CustomHeaders property which is tied to the headers XML element in the config file. Since we have used the XML attribues key/value we are returning a KeyValueConfigurationCollection. This collection is rather specific to configuration and config files and I do not want that being passed around in the application.

That is why I in the CreateBehavior() converts it into a Dictionary<string, string> customHeaders.

Then I return a new instance of the CustomHeaderBehavior which as an argument in the constructor takes the custom headers as a Dictionary<string, string>.


using System.Collections.Generic;
using System.ServiceModel.Channels;
using System.ServiceModel.Description;
using System.ServiceModel.Dispatcher;

namespace CRTest
{
 public class CustomHeaderBehavior : IEndpointBehavior
 {
  private readonly IDictionary<string, string> customHttpHeaders;

  public CustomHeaderBehavior(IDictionary<string, string> customHttpHeaders)
  {
   this.customHttpHeaders = customHttpHeaders;
  }

  public void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime)
  {
   var customHeaderMessageInspector = new CustomHeaderMessageInspector(customHttpHeaders);
   clientRuntime.MessageInspectors.Add(customHeaderMessageInspector);
  }

  public void AddBindingParameters(ServiceEndpoint endpoint, BindingParameterCollection bindingParameters)
  {
  }

  public void ApplyDispatchBehavior(ServiceEndpoint endpoint, EndpointDispatcher endpointDispatcher)
  {
  }

  public void Validate(ServiceEndpoint endpoint)
  {
  }
 }
}


In ApplyClientBehavior() we add a MessageInspector of type CustomHeaderMessageInspector and this is the class that is actually adding those custom headers to the request.


using System;
using System.Collections.Generic;
using System.ServiceModel;
using System.ServiceModel.Channels;
using System.ServiceModel.Dispatcher;

namespace CRTest
{
 public class CustomHeaderMessageInspector : IClientMessageInspector
 {
  private readonly IDictionary<string, string> customHttpHeaders;

  public CustomHeaderMessageInspector(IDictionary<string,string> customHttpHeaders)
  {
   if (customHttpHeaders == null) throw new ArgumentNullException("customHttpHeaders");
   this.customHttpHeaders = customHttpHeaders;
  }

  public object BeforeSendRequest(ref Message request, IClientChannel channel)
  {
   // Making sure we have a HttpRequestMessageProperty
   HttpRequestMessageProperty httpRequestMessageProperty;
   if (request.Properties.ContainsKey(HttpRequestMessageProperty.Name))
   {
    httpRequestMessageProperty = request.Properties[HttpRequestMessageProperty.Name] as HttpRequestMessageProperty;
    if (httpRequestMessageProperty == null)
    {
     httpRequestMessageProperty = new HttpRequestMessageProperty();
     request.Properties.Add(HttpRequestMessageProperty.Name, httpRequestMessageProperty);
    }
   }
   else
   {
    httpRequestMessageProperty = new HttpRequestMessageProperty();
    request.Properties.Add(HttpRequestMessageProperty.Name, httpRequestMessageProperty);
   }

   // Add custom headers to the WCF request
   foreach (var header in customHttpHeaders)
   {
    httpRequestMessageProperty.Headers.Add(header.Key, header.Value);
   }
   
   return null;
  }

  public void AfterReceiveReply(ref Message reply, object correlationState)
  {
  }
 }
}


I case you are interested in the solution file it can be downloaded here WCFCustomHeaderClient.zip.

Debugging XML deserialization issues using XmlSerializer when using REST through WCF

This url Troubleshooting Common Problems with the XmlSerializer holds some really useful information when you encounter deserialization issues when working with REST, WCF and .NET 4.0 (and XmlSerialization in general).


Saturday, January 29, 2011

WCF REST service with custom http header check in .NET 4

This is the posting I initially intended to write when I started writing WCF REST and .NET 4.0.

This posting outlines how to make a simple http header check in the WCF REST based service and depending on the the outcome of the header evaluation either return the result or throw an WebFaultException.

This WCF service is built using .NET 4.0 only without using the WCF REST starter kit (see more information here REST in Windows Communication Foundation (WCF) and here WCF REST Starter Kit Preview 2.


Start out by making a new project in Visual Studio, choose "Online Template" and select the "WCF" sub category. Select "WCF REST Service Template 40".




Change the code of the Service1.cs file to look like the following.


using System.Net;
using System.Collections.Generic;
using System.ServiceModel;
using System.ServiceModel.Activation;
using System.ServiceModel.Web;

namespace WcfRestService1
{
 // Start the service and browse to http://<machine_name>:<port>/Service1/help to view the service's generated help page
 // NOTE: By default, a new instance of the service is created for each call; change the InstanceContextMode to Single if you want
 // a single instance of the service to process all calls. 
 [ServiceContract]
 [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
 [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
 public class Service1
 {
  [WebGet(UriTemplate = "{id}")]
  public List<SampleItem> GetProductById(string id)
  {
   if (WebOperationContext.Current.IncomingRequest.Headers["MyCustomHttpHeader"] == null) 
    throw new WebFaultException<string>("The custom httpheader 'MyCustomHttpHeader' has not been set.", HttpStatusCode.BadRequest);
   
   return new List<SampleItem>() { new SampleItem() { Id = 1, StringValue = "Product1" } };
  }
 }
}

Build the solution and try to access the url http://localhost:some_random_dev_web_server_port/Service1/GetProductById?id=1





This was kind of expected, but notice how neat this is tying in with the http request return codes.

If we look at this in Fiddler we will see the following.



As expected according to our code in the service 


if (WebOperationContext.Current.IncomingRequest.Headers["MyCustomHttpHeader"] == null) 
                throw new WebFaultException<string>("The custom httpheader 'MyCustomHttpHeader' has not been set.", HttpStatusCode.BadRequest);


If we now use the Request Builder in Fiddler and add the custom http header "MyCustomHttpHeader" we will see the following result in Fiddler.




If you want to let headers flow back and forth between clients and the services your best option is to add some new classes implementing the interfaces IClientMessageInspector, IEndpointBehavior and if you want this to be configurable from the .config file you need to extend the base class BehaviorExtensionElement.

You can see a sample implementation here WCF REST client using custom http headers

WCF REST and .NET 4.0

REST  and SOAP have long been available as specifications how to access and interact with services published on the internet (or intra- /extra- net). Yahoo, Google and Amazon Web Services have long been using REST as the primary way to expose different APIs. In the .NET world we initially had SOAP based web service support built into Visual Studio (.NET / 2003 / 2005), meaning that we could point Visual Studio to a WSDL file and generate the needed code to interact with this service. You could use a REST based web service using the .NET framework by manually using a combination of HttpWebRequest, HttpWebResponse and generating supporting classes for the returned XML using xsd.exe, but there was no built in support for REST based services in the IDE.

With the revealing of the new API codenamed "Indigo" the implementation of the WS* standards into an easy and coherent programming model, as well as incorporating consuming any service no matter where it was located (In process, webservice ... ) using the same programming model, WCF was born and released with .NET 3.0.

In the WCF programming model it does not matter where the service is located. The same unchanged client code can communicate with a service located in the same app domain, across app domains but in the same process, or inter process on the same machine. The same client code can also consume and use a services on different machine either on a LAN or across the inter-, intra-, extra- net.

Juval Löwy has a great note about this in his book Programming WCF Services.

Conceptually, even in C# or VB, there are endpoints: the address is the memory address of the type’s virtual table, the binding is CLR, and the contract is the interface itself. Because in classic .NET programming you never deal with addresses or bindings, you take them for granted, and you’ve probably gotten used to equating in your mind’s eye the interface (which is merely a programming construct) with all that it takes to interface with an object. The WCF endpoint is a true interface because it contains all the information required to interface with the object. In WCF, the address and the binding are not preordained and you must specify them.
Obviously due to the physical differences in how the messages are transported from client to service there are circumstances that need to taken into account. A slow WCF service located across the world, should limit the round trips as much as possible and make sure that the OperationContract is not a "chatty one". Funny enough this applies equally well for the service located in the same app domain in the same process but for a different reason. In the remote scenario it is due to performance, in the same app domain the reason is to achieve an location agnostic service, which can be moved to a remote location only by changing the endpoint  in the configuration file. This is not enforced by the framework but is good programming practices.

Now with .NET 4.0 we have built in support for calling and consuming REST based web service. However especially if you rely on http headers to transfer custom meta information to the service you will have to dive into the extensibility points of WCF. Initially this posting was supposed to be concerning that subject specifically but seeing how it ended up being more of a history lesson of WCF, I have posted that as a separate post which is located here WCF REST service with custom http header check in .NET 4