Saturday, November 29, 2025

Signing ClickOnce Office/Excel Plugin using Azure Trusted Signing

Task

I maintain a Microsoft Office Excel plugin deployed via ClickOnce. It has run for several years without changes, but recent customer has requested some updates. As a result, I needed to revisit how to code-sign the ClickOnce application, given the certificate changes introduced in recent years.

I am no longer able to use the UI in Visual Studio IDE for ClickOnce Signing. 


Code-signing certificates are not cheap, so I wanted to explore whether Azure Trusted Signing could be used to sign ClickOnce-deployed Office plugin applications. I got it working, but finding a solution was more difficult than I expected.


Why the Code Signing Process Changed

The primary motivation was to enhance security and prevent the widespread misuse of compromised private keys, which were often easily stolen when stored as software files (like PFX) on potentially insecure development machines. The new requirements became effective on June 1, 2023. All major Certificate Authorities (CAs), such as DigiCert, Sectigo, and GlobalSign, agreed to adhere to these new rules. They now issue all new standard (OV/IV) and EV code signing certificates with the private keys generated and stored on secure hardware devices, such as USB tokens or Hardware Security Modules (HSMs), that meet specific security standards (FIPS 140-2 Level 2 or Common Criteria EAL 4+).

See GlobalSign's notice here 


Prerequisites: Azure Trusted Signing setup

Follow steps 1-7 from https://melatonin.dev/blog/code-signing-on-windows-with-azure-trusted-signing/.

I also reviewed the following blog posts that had some details that were helpful.

Proces for Signing ClickOnce Office/Excel Plugin using Azure Trusted Signing 


None of the above blog posts gave me a fully working solution since signtool.exe, which they are using, can not fully sign the ClickOnce application's manifest files (XML files), so I spent some more time searching on Google.

I ended up finding this issue "Added support for Trusted Signing" on Github - Sign CLI. Reviewing the files lead to the below steps.

  • Open Powershell as Administrator
  • Log in on the relevant Azure account where you have created the above subscription and Trusted Signing via the powershell command line

    az login


  • My output on the command line was something like the below

    Select the account you want to log in with. For more information on login with Azure CLI, see https://go.microsoft.com/fwlink/?linkid=2271136

    Retrieving tenants and subscriptions for the selection...

    [Tenant and subscription selection]

    No     Subscription name               Subscription ID                       Tenant
    -----  ------------------------------  ------------------------------------  -------------
    [1] *  ABCCodeSigningSubscription      12345678-1234-1234-1234-123456789012  ABC

    The default is marked with an *; the default tenant is 'ABC' and subscription is 'ABCCodeSigningSubscription' (12345678-1234-1234-1234-123456789012).

    Select a subscription and tenant (Type a number or Enter for no changes): 1

    Tenant: ABCTenant
    Subscription: ABCCodeSigningSubscription (12345678-1234-1234-1234-123456789012)

    [Announcements]
    With the new Azure CLI login experience, you can select the subscription you want to use more easily. Learn more about it and its configuration at https://go.microsoft.com/fwlink/?linkid=2271236

    If you encounter any problem, please open an issue at https://aka.ms/azclibug

    [Warning] The login output has been updated. Please be aware that it no longer displays the full list of available subscriptions by default.

  • Install the Sign CLI (for this blog post I used https://github.com/dotnet/sign version 0.9.1-beta.25379.1+ba6e717abf74a693f0f9c5e891c0e3ef624956b3)

    dotnet tool install --tool-path . --prerelease sign 

  • By reviewing the codechanges in https://github.com/dotnet/sign/pull/716/files and using the commandline help for trusted signing


    Description:
      Use Trusted Signing.

    Usage:
      sign code trusted-signing <file(s)>... [options]

    Arguments:
      <file(s)>  File(s) to sign.

    Options:
      -tse, --trusted-signing-endpoint <trusted-signing-endpoint> (REQUIRED)                         The Trusted Signing Account endpoint. The value must be a URI that aligns to the region that your Trusted Signing Account and Certificate Profile were created in.
      -tsa, --trusted-signing-account <trusted-signing-account> (REQUIRED)                           The Trusted Signing Account name.
      -tscp, --trusted-signing-certificate-profile <trusted-signing-certificate-profile> (REQUIRED)  The Certificate Profile name.
      -act, --azure-credential-type <azure-cli|azure-powershell|managed-identity|workload-identity>  Azure credential type that will be used. This defaults to DefaultAzureCredential.
      -mici, --managed-identity-client-id <managed-identity-client-id>                               The client id of a user assigned ManagedIdentity.
      -miri, --managed-identity-resource-id <managed-identity-resource-id>                           The resource id of a user assigned ManagedIdentity.
      -an, --application-name <application-name>                                                     Application name (ClickOnce).
      -d, --description <description>                                                                Description of the signing certificate.
      -u, --description-url <description-url>                                                        Description URL of the signing certificate.
      -b, --base-directory <base-directory>                                                          Base directory for files.  Overrides the current working directory. [default: D:\]
      -o, --output <output>                                                                          Output file or directory. If omitted, input files will be overwritten.
      -pn, --publisher-name <publisher-name>                                                         Publisher name (ClickOnce).
      -fl, --file-list <file-list>                                                                   Path to file containing paths of files to sign or to exclude from signing.
      -rc, --recurse-containers                                                                      Sign container contents. [default: True]
      -fd, --file-digest <file-digest>                                                               Digest algorithm to hash files with. Allowed values are 'sha256', 'sha384', and 'sha512'. [default: SHA256]
      -t, --timestamp-url <timestamp-url>                                                            RFC 3161 timestamp server URL. [default: http://timestamp.acs.microsoft.com/]
      -td, --timestamp-digest <timestamp-digest>                                                     Digest algorithm for the RFC 3161 timestamp server. Allowed values are sha256, sha384, and sha512. [default: SHA256]
      -m, --max-concurrency <max-concurrency>                                                        Maximum concurrency. [default: 4]
      -v, --verbosity <Critical|Debug|Error|Information|None|Trace|Warning>                          Sets the verbosity level. Allowed values are 'none', 'critical', 'error', 'warning', 'information', 'debug', and 'trace'.
                                                                                                     [default: Warning]
      -?, -h, --help                                                                                 Show help and usage information

  • I ended up with the following command

    & ./sign code trusted-signing "D:\ClickOncePublishDirectory\ABCApp.vsto" -tse https://weu.codesigning.azure.net -tsa ABCCodeSigning -tscp ABCCodeSigningCertificate -an ABCApp -pn ABC -t "http://timestamp.acs.microsoft.com" -v debug 

  • I had problems signing the ClickOnce initially due to having quite a few older versions under the "Application Files" directory, once I removed the older versions and only had the lastest build it worked fine

I  hope this helps other people that may be facing a similar task.



Saturday, November 14, 2020

Why does the medical community in general not seem more interested in early detection?

Background

For a number of years I have been investigating and working on how to use IT/IOT/mobile devices to reduce the amount of time that passes, from when a frail/elderly or person with a chronic condition becomes ill, to the appropriate reaction/treatment can be started.
This process has almost in all cases historically been a reactive process. The person in question needed to become objectively visibly ill before someone would/could react. The history of medical diagnosis is a long and interesting one and have varied over time historically, but in general they have moved from rather crude to more and more sophisticated. Even if this has improved greatly it is still a reactive process. 


"Things that take time"

  • The patient's condition starts exhibiting itself or worsens
  • The patient's condition has to be perceived as "bad enough" to cross the patient's "do I call the medical services"-threshold
  • Physical transport time to the health care professional
  • Potential waiting times at health care professional (waiting time at the emergency room or opening hours of a private physician )
  • The health care professional may not have prior knowledge of the patient's condition and may misdiagnose it
  • Tests are done
  • Treatments are started

Depending on many factors like the

  • the patient's condition
  • the patient's "do I call the medical services"-threshold
  • the patient's physical location in relation to the relevant health care provider
the time from the condition starts exhibiting itself or worsens to the appropriate treatment has begun, can be from a few hours (at probably very best in non life threatening conditions) to several days.

This is not optimal use of important time and this can cause the health condition of the patient to deteriorate to a lesser or larger degree.

An example: COPD
( https://en.wikipedia.org/wiki/Chronic_obstructive_pulmonary_disease )


Since
2013 we have been working with a list of objective rules 

  • some rules use fixed ranges for a given value and if you are outside of this that will trigger a yellow or a red color (as opposed to a green color - where things are as good as they can be given the conditions).
  • some uses percentages deviation from a baseline
These rules have been set fairly conservative, and then based on remote monitoring we get frequent data from the patient. This rule algorithm has more or less eliminated most of "things that take time". 

It is still a reactive process in that until the exacerbation manifests itself in a measurable way like colored sputum ( spit / mucus ) or by the lung function declining. But these are still in many cases measureable days, if not up to a week, before the patient's condition worsens to the degree that they are admitted to hospital. This early detection often makes the medical corrections needed, significantly smaller.

If this is paired with a nurse with a specialty in COPD and a medical doctor that is specialising in COPD  as a backup that monitors the patients' values and receive alarms when the situation starts to decline. This paired with emergency medication located with the patients' then in the majority of the cases you can preempt patients being admitted to the hospital.


A question 


People in general are really good at using step counters, Apple watches and other health related devices for exercise purposes and other reasons. Why have the medical community and the hospitals not pushed for a greater adoption of remote patient monitoring in general?

Wednesday, February 26, 2014

WebRTC - Current state of afairs - Native App vs HTML/Web - iOS / Android / MacOS / Windows

I received an email with an inquiry about how to approach WebRTC development in an app, and this is my experiences based on some of the functionality that we have implemented.

Depending on what your requirements are the easiest way is to do web only, but then you are currently ruling out iOS support.

In Google Chrome it works on Windows, Linux, MacOS and Android.  We have tested cross browser on Windows and it works between Chrome, Firefox (and Opera).

It is important to note that this is the full Chrome app/application not the development component "webview". I have not been able to find a webview that supports webrtc, and the new one in KitKat (Android 4.4) does not either (even though it is built on Chrome rather than the original native Android browser)

Chrome for Android supports a few features which aren't enabled in the WebView, including:
  • WebGL 3D canvas
  • WebRTC
  • WebAudio
  • Fullscreen API
  • Form validation
(https://developers.google.com/chrome/mobile/docs/webview/overview)

So it is possible to create a web-based WebRTC application that will work across Android, Windows, Linux and MacOS and at least on Windows it works across Chrome, Firefox and Opera.


IE is currently no go, although

Some people also report that IE supports WebRTC when using http://www.google.com/chromeframe
(http://stackoverflow.com/questions/15724913/which-version-of-microsoft-internet-explorer-support-webrtc)

It works pretty well across both fixed network connections like ADSL/ADSL, 3G/ADSL, 3G/3G networks but occasionally you run into some paranoid system admin that has the firewall locked down so tight that neither incoming or outgoing connections work and then you will face problems. This can occur at a higher level (i.e. ISP). ICE, STUN and TURN can only do so much.



What about native app development and support?

My thoughts on how to do this can be found at the following blog posting http://kenneththorman.blogspot.dk/2014/01/goal-cross-platform-android-ios-webrtc.html

I/we am working on both an Android and iOS solution and it will be developed on Github and everyone is free to contribute. Currently we am facing some problems on the Android version as can be seen here. The iOS version is a bit technically harder than the Android version due to Google/the WebRTC team already doing most of the hard work in their Java demo app.

So all in all web is the most doable way at the moment, but there are people that have this working on iOS using objective C and on Android using native Java.
http://ninjanetic.com/how-to-get-started-with-webrtc-and-ios-without-wasting-10-hours-of-your-life/

http://kenneththorman.blogspot.dk/2014/01/webrtc-app-c-xamarin-part-1-building.html

Wanting a cross device code base and thus creating it in C#/MVVMCross makes it harder, since you have to take mono/native (dalvik/iOS) communication into account, and may run into restrictions in the Xamarin supported API for low level native API's.


There are not many people on the internet that seem to have the skills to pull this off especially if you are aiming for both across web and native.

The skillset to pull this off involves one or more of the following depending on the signaling system you end up using and your requirements to functionality


All in all the complexity still shines through the WebRTC proposal, not because it is bad, but because it is such a diverse runtime enviroment - and it is such an amazingly complex topic. On top of this, the users perception is but this is just like a phone call this shold just work 100% of the cases. They do not see or understand that there might be a difference calling from the same device if you move from WIFI to 3G network. This is plumbing and it should just work - always - on all devices - everywhere. Auch!!! That is a pretty high expectation to fulfill.


This is still bleeding edge, I am not even sure this is out of alpha/beta stages yet - at least officially, so there are reference  implementation (Chromium i.e. Google Chrome) bugs that might bite you.


This is a super exciting area, but at the moment web seems to me the most approachable.


Another option is to buy one of the cloud based offerings that have both Java and iOS components that make this easier. They are shielding you from some of this complexity, but comes at a cost.

Neither web or native is without pitfalls, but certainly doable, if you have the skills.

Wednesday, January 22, 2014

Xamarin / Mono: Android adb tracing / debug logging

adb shell setprop debug.checkjni 1
adb shell setprop debug.mono.env MONO_LOG_LEVEL=info
adb shell setprop debug.mono.log gref,gc


And if none of the above helps:

adb shell setprop debug.mono.trace N:Your.App.Namespace

Undoing

adb shell setprop debug.checkjni 0
adb shell setprop debug.mono.env ''
adb shell setprop debug.mono.log ''
adb shell setprop debug.mono.trace ''

Monday, January 20, 2014

Goal: Cross platform (Android / iOS) WebRTC app using MVVMCross

I am attempting to port 2 Java apps that are part of the WebRTC code to Mono/C#. 
The code is available on GitHub, it is still a fairly early version but it builds (there are some problems during run-time).

Project goals
  1. make an opensource viable implementation of WebRTC for use in Mono / Xamarin / .NET on Android and iOS
  2. make an implementation of a WebRTC UIView/Activity that can be re-used across Android and iOS with as little as possible modifications
  3. be able to share as much code as possible between the different platforms by utilizing PCL (portable class libraries) and MVVMCross
  4. all possible UI logic moved to a shared portable class library using MVVMCross ViewModels
  5. move the WebRTC signaling behind to an interface located in a portable class library with an reference implementation, allowing people to reuse signaling code across platforms, while still implementing their own.
  6. in time write a new similar native interface as the JNI for Java but without JNI and directly targeting being invoked using P/Invoke / Mono / .NET allowing C#/Mono to directly interact with the C/C++ WebRTC library (see related question at Mono-JNI-Performance)
At least one of these goals are outside my comfort zone (#6), and all of them are currently pending, I believe #1 is well in progress.

Currently I am still in the progress of getting the actual WebRTC meeting working. I believe I have a problem in the converted signaling code, which I am trying to track down.



GitHub repositories
appspotdemo-mono (this is the repository that I am focusing my immediate attention on - but I have been short on time lately)

Background for the webrtc-app-mono repository:
When I built the native .so (http://kenneththorman.blogspot.dk/2014/01/webrtc-app-c-xamarin-part-1-building.html) it also build a full android apk that you can install and test. That was the app I already tried on my devices which worked the same way as apprtc.appspot.com. That that was the code I was actually looking for, and as embarrassing it is to admit it - the repo https://github.com/kenneththorman/webrtc-app-mono was me porting the "wrong" Java app code. When I found out and figured out how to work with JNI (mainly through using the Java Binding Library) I went looking in the official WebRTC code again to find the app that I tested in the Java version. This is the repo at https://github.com/kenneththorman/appspotdemo-mono. So basically having 2 repositories are proof of me not being familiar with the official WebRTC code base and not really knowing what code is which.


Any help or suggestions are welcome, please feel free to fork, create issues or communicate here and I will do my best to answer.



References

Thursday, January 09, 2014

rfc5766-turn-server one liner install on CentOS

cd /usr/src;wget https://rfc5766-turn-server.googlecode.com/files/turnserver-3.2.1.4.tar.gz; tar -xf turnserver-3.2.1.4.tar.gz; cd turnserver-3.2.1.4; ./configure; make; make test; service turnserver stop; make install; service turnserver start

Thursday, January 02, 2014

Question: Mono.Android performance: C# -> JNI wrapper -> native lib vs. C# -> Managed wrapper -> native?

I have currently ported a Java app to Mono.Android. The Java app uses a native library.

The full walk through can be read here: WebRTC app - C# / Xamarin - (C# - JNI - C/C++) - Summary and GitHub repository


Basically I have some Java code that looks like this

public native int GetVideoEngine();
Initially in my C# equivalent app I tried to use DllImport
[DllImport("libwebrtc-video-demo-jni.so")]
public static extern int Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_GetVideoEngine();
(which did not work), then I ended up wrapping the jars as Java Binding Libraries which in turn JNI'ed the needed classes, so I instead could call the generated JNI wrappers
static Delegate cb_GetVideoEngine;
#pragma warning disable 0169
static Delegate GetGetVideoEngineHandler ()
{
    if (cb_GetVideoEngine == null)
        cb_GetVideoEngine = JNINativeWrapper.CreateDelegate ((Func<IntPtr, IntPtr, int>) n_GetVideoEngine);
    return cb_GetVideoEngine;
}

static int n_GetVideoEngine (IntPtr jnienv, IntPtr native__this)
{
    global::Org.Webrtc.Videoengineapp.ViEAndroidJavaAPI __this = global::Java.Lang.Object.GetObject<global::Org.Webrtc.Videoengineapp.ViEAndroidJavaAPI> (jnienv, native__this, JniHandleOwnership.DoNotTransfer);
    return __this.VideoEngine;
}
#pragma warning restore 0169

static IntPtr id_GetVideoEngine;
public virtual int VideoEngine {
    // Metadata.xml XPath method reference: path="/api/package[@name='org.webrtc.videoengineapp']/class[@name='ViEAndroidJavaAPI']/method[@name='GetVideoEngine' and count(parameter)=0]"
    [Register ("GetVideoEngine", "()I", "GetGetVideoEngineHandler")]
    get {
        if (id_GetVideoEngine == IntPtr.Zero)
            id_GetVideoEngine = JNIEnv.GetMethodID (class_ref, "GetVideoEngine", "()I");

        if (GetType () == ThresholdType)
            return JNIEnv.CallIntMethod  (Handle, id_GetVideoEngine);
        else
            return JNIEnv.CallNonvirtualIntMethod  (Handle, ThresholdClass, id_GetVideoEngine);
    }
}

My question is related to performance:

How much slower is the C# -> JavaBindingLibrary (JAVA/JNI) -> C/C++ JNI Wrapper -> C native library than if I did unwrapped the C/C++ JNI Wrapper and rewrapped it with something like mono / cxxi or similar direct managed or manually wrote a direct callable wrapper?

Wednesday, January 01, 2014

WebRTC app - C# / Xamarin - (C# - JNI - C/C++) - Summary and GitHub repository

This posting concludes a series of in all 4 blog posting where this being the 4th.

  1. WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library
  2. WebRTC app - C# / Xamarin - Part #2 - Attempt #1 - failure to using a JNI .so file directly from C# / Mono
  3. WebRTC app - C# / Xamarin - Part #2 - Attempt #2 - success using a JNI .so file from C# / Mono
  4. WebRTC app - C# / Xamarin - (C# - JNI - C/C++) - Summary and GitHub repository
And finally the associated GitHub repository https://github.com/kenneththorman/webrtc-app-mono

WebRTC app - C# / Xamarin - Part #2 - Attempt #2 - success using a JNI .so file from C# / Mono

This is a post in a series of postings

  1. WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library
  2. WebRTC app - C# / Xamarin - Part #2 - Attempt #1 - failure to using a JNI .so file directly from C# / Mono
  3. WebRTC app - C# / Xamarin - Part #2 - Attempt #2 - success using a JNI .so file from C# / Mono
  4. WebRTC app - C# / Xamarin - (C# - JNI - C/C++) - Summary and GitHub repository
And finally the associated GitHub repository https://github.com/kenneththorman/webrtc-app-mono

In my previous posting WebRTC app - C# / Xamarin - Part #2 - Attempt #1 - failure to using a JNI .so file directly from C# / Mono I wrote
This is pushing me in a direction that I initially hoped I could avoid (mainly due to my limited knowledge in the area), JNI.
There were a few reasons that I did not prefer to use a Java Binding Library in the solution. 
  • I would have an even more mixed source code base (C# calling jar/Java which is wrapping C/C++), I would have preferred to keep it at C# wrapping C/C++.
  • The upstream build process is packaging some of the compiled java classes that I need into jar files, but not all of them so now I manually need to add a build step to the build process.
  • I am not familiar with JNI

If we look at the files that are generated during the build process explained in this posting WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library we will see that the jars that we need to build the test android app supplied with the project are located in

~/WebRTCDemo/trunk/webrtc/video_engine/test/android/libs/

namely

audio_device_module_java.jar
video_capture_module_java.jar
video_render_module_java.jar

For each of these jars we need to create a Java Bindings Library that is using the relevant  jar as an input jar (actually 2 of the jars can go into the same Java Binding Library - video_capture_module_java.jar and video_render_module_java.jar since they share the same Java package/namespace). 


(It might be possible to add them all in one Java Binding Library but I encountered problems due to Visual Studios project setting of Default Namespace, which basically was affecting the namespace of the Java classes that I needed to invoke from C#. Since each of the jars contained classes from a different Java package, the easy work around was to add a Java Binding Library for each of the jars and make sure that the Visual Studio Default namespace matched the Java package name.)

Looking at the existing Java WebRTC demo app available in the folder

~/WebRTCDemo/trunk/webrtc/video_engine/test/android/src/org/webrtc/videoengineapp

you will find

IViEAndroidCallback.java
ViEAndroidJavaAPI.java
WebRTCDemo.java

The main file is called WebRTCDemo.java and contains code like the following

...
import org.webrtc.videoengine.ViERenderer;

import java.io.File;
import java.io.IOException;
import java.net.InetAddress;
import java.net.NetworkInterface;
import java.net.SocketException;
import java.util.Enumeration;

public class WebRTCDemo extends TabActivity implements IViEAndroidCallback,
                                                       View.OnClickListener,
                                                       OnItemSelectedListener {
    private ViEAndroidJavaAPI vieAndroidAPI = null;

    // remote renderer
    private SurfaceView remoteSurfaceView = null;

    // local renderer and camera
    private SurfaceView svLocal = null;

    // channel number
    private int channel = -1;
    private int cameraId;
    private int voiceChannel = -1;
...



By looking at the code I could see there would be some problems, since the main activity is referencing 2 classes that are defined and located side by side with the main activity IViEAndroidCallback.java and ViEAndroidJavaAPI.java. In other words these are not available in a jar and especially the ViEAndroidJavaAPI.java ois the wrapper for the JNI native library so we cannot do this directly from C# according to Native library integration (posting #2).

In the beginning of this posting I outlined 3 reasons I preferred to avoid using JNI and Java Binding Libraries, one of them was
  •  The upstream build process is packaging some of the compiled java classes that I need into jar files, but not all of them so now I manually need to add a build step to the build process.
Anyhow this was fairly easy to remedy by manually creating a new jar containing the 2 files that was needed and then adding this new jar to a new Java Binding Library.
 
XXX@ubuntu:~/WebRTCDemo/trunk/webrtc/video_engine/test/android/bin/classes$
jar cvf ViEAndroidJavaAPI.jar
org/webrtc/videoengineapp/IViEAndroidCallback.class
org/webrtc/videoengineapp/ViEAndroidJavaAPI.class
added manifest
adding: org/webrtc/videoengineapp/IViEAndroidCallback.class(in = 218)
(out= 173)(deflated 20%)
adding: org/webrtc/videoengineapp/ViEAndroidJavaAPI.class(in = 2845)
(out= 1303)(deflated 54%)

Now I was able to use the native library without any exceptions occurring.

WebRTC app - C# / Xamarin - Part #2 - Attempt #1 - failure to using a JNI .so file directly from C# / Mono

This is a post in a series of postings

  1. WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library
  2. WebRTC app - C# / Xamarin - Part #2 - Attempt #1 - failure to using a JNI .so file directly from C# / Mono
  3. WebRTC app - C# / Xamarin - Part #2 - Attempt #2 - success using a JNI .so file from C# / Mono
  4. WebRTC app - C# / Xamarin - (C# - JNI - C/C++) - Summary and GitHub repository
And finally the associated GitHub repository https://github.com/kenneththorman/webrtc-app-mono

In my previous posting WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library I have showed how to build a native library that we need to use to build a WebRTC app on Xamarin / Mono.Droid.

In this posting I will take you through my struggles, subsequent failure and the next posting finally success on how to actually use this JNI native library from Mono.Android.

I started a new solution in Visual Studio 2013 and added new Android Application project. Then according to Xamarin: Using Native Libraries.
I needed to add my .so file to the location  
<project>\lib\armeabi-v7a\libwebrtc-video-demo-jni.so.

The next step that I tried was to start using DllImport statements.
using System;
using System.Runtime.InteropServices;
using Android.Content;
using Android.Util;
using Encoding = System.Text.Encoding;

namespace WebRtc
{
        public class ViEAndroidJavaAPI
        {

...
                // API Native

                [DllImport("libwebrtc-video-demo-jni.so")]
                private static extern bool NativeInit(Context context);

                // Video Engine API
                // Initialization and Termination functions
                [DllImport("libwebrtc-video-demo-jni.so")]
                public static extern int GetVideoEngine();

                [DllImport("libwebrtc-video-demo-jni.so")]
                public static extern int Init(bool enableTrace);

                [DllImport("libwebrtc-video-demo-jni.so")]
                public static extern int Terminate();
...


Trying to run my project yielded some EntryPointNotFoundException in the error log. After a bit of Google-ing I found that the method names as seen from Mono are not as you expect instead they contain the full package/class path.

Using the following command on the Ubuntu build machine
~/WebRTCDemo/trunk/webrtc/video_engine/test/android/libs/armeabi-v7a$ arm-linux-androideabi-nm -D libwebrtc-video-demo-jni.so
yielded the following output
00015358 T JNI_OnLoad
00015be4 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_AddRemoteRenderer
00016e3c T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_CreateChannel
00015f14 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_EnableNACK
00015f58 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_EnablePLI
00015e04 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_GetCameraOrientation
00015acc T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_GetCodecs
000153d4 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_GetVideoEngine
000154cc T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_Init
000153d0 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_NativeInit
00015c30 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_RemoveRemoteRenderer
00015fb0 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_SetCallback
00015ea8 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_SetExternalMediaCodecDecoderRenderer
...
So changing my code to the following made the EntryPointNotFoundException go away
using System;
using System.Runtime.InteropServices;
using Android.Content;
using Android.Util;
using Encoding = System.Text.Encoding;

namespace WebRtc
{
        public class ViEAndroidJavaAPI
        {

...
                // API Native

                [DllImport("libwebrtc-video-demo-jni.so")]
                private static extern bool Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_NativeInit(Context context);

                // Video Engine API
                // Initialization and Termination functions
                [DllImport("libwebrtc-video-demo-jni.so")]
                public static extern int Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_GetVideoEngine();

                [DllImport("libwebrtc-video-demo-jni.so")]
                public static extern int Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_Init(bool enableTrace);

                [DllImport("libwebrtc-video-demo-jni.so")]
                public static extern int Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_Terminate();

...

Now I was faced with another exception which seemed much nastier.
UNHANDLED EXCEPTION: System.Runtime.InteropServices.MarshalDirectiveException: Type Java.Lang.Object which is passed to unmanaged code must have a StructLayout attribute.

12-21 19:29:04.298 I/MonoDroid(15226): UNHANDLED EXCEPTION: System.Runtime.InteropServices.MarshalDirectiveException: Type Java.Lang.Object which is passed to unmanaged code must have a StructLayout attribute.
12-21 19:29:04.298 I/MonoDroid(15226): at (wrapper managed-to-native) WebRtc.ViEAndroidJavaAPI.Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_NativeInit (Android.Content.Context) 
12-21 19:29:04.298 I/MonoDroid(15226): at WebRtc.ViEAndroidJavaAPI..ctor (Android.Content.Context) [0x00033] in XXX\WebRtc.Mono.Droid\ViEAndroidJavaAPI.cs:30
12-21 19:29:04.298 I/MonoDroid(15226): at WebRtc.Mono.Droid.WebRTCDemo.startMain () [0x0004b] in XXX\WebRtc.Mono.Droid\WebRTCDemo.cs:533
12-21 19:29:04.298 I/MonoDroid(15226): at WebRtc.Mono.Droid.WebRTCDemo.OnCreate (Android.OS.Bundle) [0x0028e] in XXX\WebRtc.Mono.Droid\WebRTCDemo.cs:313
12-21 19:29:04.298 I/MonoDroid(15226): at Android.App.Activity.n_OnCreate_Landroid_os_Bundle_ (intptr,intptr,intptr) [0x00011] in /Users/builder/data/lanes/monodroid-mlion-monodroid-4.10.1-branch/d23a19bf/source/monodroid/src/Mono.Android/platforms/android-17/src/generated/Android.App.Activity.cs:2119
12-21 19:29:04.298 I/MonoDroid(15226): at (wrapper dynamic-method) object.705dc6ba-9c58-4bcd-a8a2-f12584a9175f (intptr,intptr,intptr)

Finally after digging I found this posting Native library integration which basically state
you cannot sanely use P/Invoke to invoke the native method. You must instead use JNI to invoke the Java-side native method.
Basically because this is Java native C/C++ interface we are to invoke, you cannot do this like normal non JNI wrapped C/C++ methods.


This is pushing me in a direction that I initially hoped I could avoid (mainly due to my limited knowledge in the area), JNI.


Later: I did some (quite a bit) reading, namely I found these links useful:

Interop with Native Libraries
Java Integration Overview
Working With JNI

In the next posting in this series I manage to invoke the native methods.

WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library

This is a post in a series of postings

  1. WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library
  2. WebRTC app - C# / Xamarin - Part #2 - Attempt #1 - failure to using a JNI .so file directly from C# / Mono
  3. WebRTC app - C# / Xamarin - Part #2 - Attempt #2 - success using a JNI .so file from C# / Mono
  4. WebRTC app - C# / Xamarin - (C# - JNI - C/C++) - Summary and GitHub repository
And finally the associated GitHub repository https://github.com/kenneththorman/webrtc-app-mono

I wish to build a cross platform app that supports WebRTC (real time communication - wikipedia article here). I would like to use Xamarin to achieve some level of code reuse between iOS and Android. There are several areas of the application that can use common code like the:
This series of blog postings will document my attempt at implementing the Android initial app (Mono.Android) and then I will attempt to move this to iOS (monotouch). Unlike some of my other postings this is a documentation project during the attempt to reach that goal.

Lets get started. 
Now building webrtc and all the associated libraries is not for the faint of heart, but luckily there are some pretty nice build tools available that does the job nicely. For both the Android and later the iOS edition we will need a C/C++ native compiled library that does all the heavy lifting with regards to audio/video rendering, decoding and encoding. So my first goal was to build a Android compatible library that I could include in my solution.

Here are some good links that got me started
http://www.webrtc.org/reference/getting-started

and then I found this little gem at Ryazantsev's blog which basically walks you through the process step by step. I already had a Ubuntu 12.04 LTS virtual machine installed and configured so the below is the console commands run on my machine. I use some slightly modified commands compared to Ryazantsev's blog. Thanks to Ryazantsev for posting this great post.


##Installing JAVA (first install default JAVA)
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update && sudo apt-get install oracle-jdk7-installer
chmod a+x jdk-6u45-linux-x64.bin
./jdk-6u45-linux-x64.bin
mkdir /usr/lib/jvm
mv jdk1.6.0_45 /usr/lib/jvm/jdk1.6.0_45

##Update alternatives of java tools
jdir=jdk1.6.0_45
sudo update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/$jdir/bin/javac 1
sudo update-alternatives --install /usr/bin/java java /usr/lib/jvm/$jdir/bin/java 1
sudo update-alternatives --install /usr/bin/javaws javaws /usr/lib/jvm/$jdir/bin/javaws 1
sudo update-alternatives --install /usr/bin/jar jar /usr/lib/jvm/$jdir/bin/jar 1

##Check alternatives and java version
sudo update-alternatives --config javac
sudo update-alternatives --config java
sudo update-alternatives --config javaws
sudo update-alternatives --config jar
ls -la /etc/alternatives/{java,javac,javaws,jar}
java -version

echo 'export JAVA_HOME=/usr/lib/jvm/jdk1.6.0_45' >> ~/.bashrc
echo 'PATH="$PATH":`pwd`/depot_tools' >> ~/.bashrc
source ~/.bashrc
printenv | grep depot_tools
 
sudo apt-get install git subversion libpulse-dev g++ pkg-config gtk+-2.0 libnss3-dev
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git

###Building WebRTC Trunk
cd ~
mkdir WebRTCDemo
cd WebRTCDemo/

gclient config https://webrtc.googlecode.com/svn/trunk
gclient sync
echo target_os = [\'android\', \'unix\'] >> .gclient
./trunk/build/install-build-deps.sh  --no-chromeos-fonts
./trunk/build/install-build-deps-android.sh
check  'sudo update-alternatives --config java ' for correct path
gclient sync
cd trunk
source ./build/android/envsetup.sh
gclient runhooks
GYP_GENERATORS=ninja ./build/gyp_chromium --depth=. all.gyp 
ninja -C out/Debug -j10 All

After downloading all the files and building the project it takes up about 4.6GB on my disk in the virtual machine. Most source files in the project will show to be a good reference when we need to start using this in a Xamarin project. The really relevant parts through is the java sources for the Java version of the app (written by the WebRTC authors) as well as the Android compatible .so file.


The compiled .so file we need is available at 
~/WebRTCDemo/trunk/webrtc/video_engine/test/android/libs/armeabi-v7a/libwebrtc-video-demo-jni.so 
The next posting will be attempt #1 in converting a Java app to a c# equivalent

Sunday, December 29, 2013

Yii Framework: Using multiple db connections in a controller method

I have a mixed database where the majority of the tables are UTF-8 encoding but there are a few tables that are required to store the data in a specific encoding that this not UTF-8. 

In Yii you are defining you global database connection in the protected/config/main.php file, and this is global for the entire web application.This is causing some challenges for me since it is very important that I deliver data bitwise compatible according to the data stored in the table. 

I was not able to find any solution to this in my first few Google attempts, so I had to dive into the code.

First I tried in the controller action method by changing the charset on the global CDbConnection directly using 

Yii::app()->db->charset='latin1';

this did not work since the database connection is already initialized at this point.

Going through the Yii Framework source code I first found CApplication->getDB(), then searching for the function getComponent() lead me to CModule->getComponent().

My next though was "If there is a getComponent(), I wonder if there is not a setComponent() function as well?" - and behold CModule->setComponent().

Armed with this knowledge it was easy to change the connection temporarily to use a new initialized database connection with a different charset.

public function loadModel($id)
{
    // Save the original database connection for later reuse
    $originalDbConnection = Yii::app()->db;
    
    // Create a new database connection based on the original database connection
    // change the charset to latin1
    $latin1DbConnection = Yii::createComponent(array(      
        'class'=>'CDbConnection',
        'connectionString' => Yii::app()->db->connectionString,
        'emulatePrepare' => Yii::app()->db->emulatePrepare,
        'username' => Yii::app()->db->username,
        'password' => Yii::app()->db->password,
        'charset' => 'latin1',
    ));

    // Set the application wide database connection for this Apache/PHP webrequest to use this special database connection
    Yii::app()->setComponent("db",$latin1DbConnection);
    
    // Do the queries against the database
    $someModel=SomeModel::model();
    $model = $someModel->findByPk($id);
    
    // Restore the original database connection 
    // note: 
    // This is actually not needed in this simple case, due to the fact that Apache/PHP rebuilds the entire web application on each request
    // Since I am not doing additional request in this logic here after returning the result everything is torn down and discarded.
    // However I have added it as part of this code for completeness and also in case someone else out there is doing more complex logic with multiple 
    // request that needs different charsets
    Yii::app()->setComponent("db",$originalDbConnection);

    if($model===null)
        throw new CHttpException(404,'The requested page does not exist.');
    return $model;
}

Sunday, September 01, 2013

Unable to change network adapter UUID on Linux

I needed to reconfigure the hardware UUID of a Linux network adapter caused by cloning a virtual machine. To do this you need to either remove the /etc/udev/rules.d/70-persistent-net.rules and let the system regenerate the file on reboot, or manually edit the file.

I decided to edit the file and manually correct the UUID. I was consistently met "readonly file" so I could not save my changes.

I found using the lsattr /etc/udev/rules.d/70-persistent-net.rules that the file had the i attribute set. This i attribute means that the file is immutable. Removing this attribute using chattr -i /etc/udev/rules.d/70-persistent-net.rules, allowed me to change the file as needed. After editing the file I set the i attribute again using chattr +i /etc/udev/rules.d/70-persistent-net.rules.

Sunday, July 28, 2013

Fixing the client IP as perceived by Apache behind Nginx reverse proxy

In my recent post Using Nginx to reverse proxy a secure apache site that is using socket.io/node.js/websockets I found that I was only getting the proxy's IP address in all of my apache logs as well as in the application tracking. This does make sense since from Apache's perspective the proxy is the actual client and not the real remote client. I found however that it was possible to get around this. Apparently there are several Apache modules that can do this, however I did not want to build from sources if I could avoid it. So I choose the one that was already available in the EPEL repository.

I already had the EPEL repository registered on the server but for those that does not you can use the following commands to register it.
rpm -Uvh http://ftp.crc.dk/fedora-epel/6/i386/epel-release-6-8.noarch.rpm
Now install the module
yum install mod_extract_forwarded
After installing the module you need to register an allowed forwarder (ie. the IP of your proxy). In all my logs the ip 127.0.0.1 was registered as the client of every request so adding that did the trick.
echo "MEFaccept 127.0.0.1" >> /etc/httpd/conf.d/mod_extract_forwarded.conf
service httpd restart
Recheck your webserver/application logs, this worked for me.

Thursday, July 25, 2013

Using Nginx to reverse proxy a secure apache site that is using socket.io/node.js/websockets

Challenge:
Firewalls can cause challenges, by blocking the ports that you want to use for websockets.

Many firewalls in use today are so called stateful firewalls and in short their function can be described as follows

Stateful firewalls only verify that a packet correlates to an existing, unclosed, connection. It tracks the state of the connection (opening, open, closing, closed) hence the name. When it detects a packet is part of an already open, authorized connection, it can short circuit all of the other rule checks and let the packet through.
(
http://stackoverflow.com/questions/1967943/will-html5-websockets-be-crippled-by-firewalls)
Most sites that are using node.js for websockets are also utilizing the library socket.io. 

A good list outlining some of the challenges when combining websockets/socket.io with various firewalls can be found at Socket.IO and firewall software.

So the question is how do you get your website running in spite of corporate firewall blocking various ports?


Possible solutions:

  1. ask the IT department to open the needed ports (in my experience hardly likely unless #2 fails)
  2. piggy-back on one of the existing established connections using http (port 80) or https (port 443) and hope the firewall is not a deep packet inspection based firewall.
 
Since I seldom have luck with suggestion number 1, I am not able to offer any guidance that is likely to help you succeed.

Suggestion number 2 however we can achieve that by letting the websockets use port 80/443 and let the website use the same port. You cannot however run 2 different services on the same port, so you cannot do this in a straightforward manner. The answer is a websocket aware reverse proxy.

There are quite a few different setups that can solve this. Below you can find 3 ways I investigated when I needed to solve this:


Setup
  • CentOS based Linux distribution
  • existing website installed as a Apache virtual host running on port 80 and 443
  • existing node.js socket.io based application running on port 8081


(setup before reverse proxy)


Please find an image of the wished for setup here below.

 
We need to do the following to achieve the reverse proxy setup
  1. Install Nginx
  2. Remove http -> https rewrite and ssl configuration from Apache as well as change Apache from listening on port 80/443 to listen on port 81
  3. Configure http -> https rewrite and reverse proxy in Nginx
  4. Change the socket.io node.js application to just use ws instead of wss (since Nginx is the SSL termination proxy now)
  5. Change the client application from using port 8081 to using 80/443

1. Install Nginx
RHEL / Centos 6: Install Nginx Using Yum Command
 


2. Remove http -> https rewrite and ssl configuration from Apache as well as change Apache from listening on port 80/443 to listen on port 81

Open the virtual hosts file for the site
vi /etc/httpd/conf.d/www_somewhere_com.conf
Change your virtual hosts file from the following 

# Catch those that are trying to access the site using http
# redirect to https
<VirtualHost *:80>
  ServerName "www.somewhere.com"
  DocumentRoot /var/www/sites/www.somewhere.com/www
  DirectoryIndex index.html index.php
  ErrorLog logs/www.somewhere.com-error_log
  CustomLog logs/www.somewhere.com-access_log common
  RewriteEngine On
  RewriteCond %{HTTPS} !=on
  RewriteRule ^(.*) https://www.somewhere.com [R,L]
</VirtualHost>

# Main site
<VirtualHost *:443>
  ServerName "www.somewhere.com"
  DocumentRoot /var/www/sites/www.somewhere.com/www
  DirectoryIndex index.html index.php
  ErrorLog logs/www.somewhere.com-error_log
  CustomLog logs/www.somewhere.com-access_log common

  SSLEngine on
  SSLCertificateFile /etc/pki/tls/certs/www_somewhere.com.crt
  SSLCertificateKeyFile /etc/pki/tls/private/www_somewhere.com.key
  SSLCertificateChainFile /etc/pki/tls/certs/Intermediate_Certificate.crt

  <Directory /var/www/sites/www.somewhere.com/www>
    Order Deny,Allow
    Allow from all
  </Directory>
</VirtualHost>

to the following

# Main site
<VirtualHost *:81>
  ServerName "www.somewhere.com"
  DocumentRoot /var/www/sites/www.somewhere.com/www
  DirectoryIndex index.html index.php
  ErrorLog logs/www.somewhere.com-error_log
  CustomLog logs/www.somewhere.com-access_log common

  <Directory /var/www/sites/www.somewhere.com/www>
    Order Deny,Allow
    Allow from all
  </Directory>
</VirtualHost>

vi /etc/httpd/conf/httpd.conf
Look for the "Listen 80" in the file and change it to "Listen 81"
Save the file


3. Configure http -> https rewrite and reverse proxy in Nginx
vi /etc/nginx/conf.d/default.conf


replace the deafult content with this


Please note that in apache config you have a seperate intermidiate certificate and the site certificate. In Nginx you need to concatenate the primary certificate file (your_domain_name.crt) and the intermediate certificate file (DigiCertCA.crt) into a single pem file by running the following command:

cat DigiCertCA.crt >> your_domain_name.crt


# Listeners
# nginx         :80         responsible for redirecting http://www.somewhere.com to https://www.somewhere.com 
# nginx         :443        responsible for handling all trafic on the https://www.somewhere.com site
#                           acts as a reverse proxy for 
#                           /socket.io/     -> port 127.0.0.1:8081  (node.js websocket server)
#                           /               -> port 127.0.0.1:81    (apache  main site)
#
# node.js   :8081   node.js websocket server
# apache    :81     apache  main site


upstream www.somewhere.com {
    server 127.0.0.1:81;
    least_conn;
}

# nginx         :80         responsible for redirecting http://www.somewhere.com to https://www.somewhere.com 
server {
    listen 80;
    server_name  www.somewhere.com;
    rewrite ^(.*) https://www.somewhere.com$1 permanent;
}

# nginx         :443        responsible for handling all trafic on the https://www.somewhere.com site
#                           acts as a reverse proxy for 
#                           /socket.io/     -> port 127.0.0.1:8081  (node.js  websocket server)
#                           /               -> port 127.0.0.1:81    (apache  main site)
server {
    listen 443 ssl;
    server_name www.somewhere.com;

    ssl_certificate     /etc/pki/tls/certs/www_somewhere.com.crt;
    ssl_certificate_key /etc/pki/tls/private/www_somewhere.com.key;
    ssl_protocols       SSLv3 TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers         RC4:HIGH:!aNULL:!MD5;

    proxy_redirect     off;
    proxy_set_header   Host             $host;
    proxy_set_header   X-Real-IP        $remote_addr;
    proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;

    # Path based websocket proxy location for nginx
    # to handle the  reverseproxying
    #
    # inital connect url: https://www.somewhere.com:8081/socket.io/1/?t=1374751734771
    # websocket url     :   wss://www.somewhere.com:8081/socket.io/1/websocket/x55Ch0B-SCACmDpW8rZd   
    location /socket.io/ {
        proxy_pass http://localhost:8081;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_read_timeout 86400;
    }   

    # Path based site proxy location for nginx
    # to handle the main apache  site
    location / {
        proxy_pass http://localhost:81;
    }   
}


4. Change the socket.io node.js application to just use ws instead of wss (since Nginx is the SSL termination proxy now)
Change you node.js socket.io application from this


var fs = require('fs');
var sslCertificate = {
    key: fs.readFileSync(config.sslCertificate.key),
    cert: fs.readFileSync(config.sslCertificate.cert),
    ca: fs.readFileSync(config.sslCertificate.ca)
};

// socket.io wss (websocket secure)
var io = require('socket.io').listen(8081, sslCertificate);
...

to the following

var fs = require('fs');
var sslCertificate = {
//    key: fs.readFileSync(config.sslCertificate.key),
//    cert: fs.readFileSync(config.sslCertificate.cert),
//    ca: fs.readFileSync(config.sslCertificate.ca)
};

// socket.io ws (websocket)
var io = require('socket.io').listen(8081, sslCertificate);
...


5. Change the client application from using port 8081 to using 80/443

Find the location in your client code where you are connecting to the websocket/socket.io application/server and change it from hardcoding the port in the url to just omitting it

So instead of (in a angular.js application / javascript single page application) doing this


var socketIO = function(data){
    // force https / wss
    var hostURL = window.location.href.replace('/'+window.location.hash, '8081').replace('http:', 'https:');

    ...
    
    try {
        var socket = io.connect(hostURL, {'sync disconnect on unload':true});
        ...
    }
    catch(e){
        console.log('Error connecting');
        ...
    }
    ...
}

do something like this
var socketIO = function(data){
    // force https / wss
    var hostURL = window.location.href.replace('/'+window.location.hash, '').replace('http:', 'https:');

    ...
    
    try {
        var socket = io.connect(hostURL, {'sync disconnect on unload':true});
        ...
    }
    catch(e){
        console.log('Error connecting');
        ...
    }
    ...
}

run the following command to check if the services are running on the correct ports

#netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name
tcp        0      0 0.0.0.0:80                  0.0.0.0:*                   LISTEN      13146/nginx
tcp        0      0 0.0.0.0:8081                0.0.0.0:*                   LISTEN      13225/node
tcp        0      0 127.0.0.1:81                0.0.0.0:*                   LISTEN      13190/httpd
tcp        0      0 0.0.0.0:443                 0.0.0.0:*                   LISTEN      13146/nginx
...

Finally some before and after pictures from chrome
Before




After