tag:blogger.com,1999:blog-216332722024-02-28T20:51:12.799+01:00it's a code thingKenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.comBlogger94125tag:blogger.com,1999:blog-21633272.post-67608166736472443412020-11-14T12:16:00.013+01:002020-11-14T12:53:19.561+01:00Why does the medical community in general not seem more interested in early detection?<h3>
<span face="Verdana, sans-serif">Background </span></h3>
<p style="text-align: left;"><span face="Verdana, sans-serif">For a number of years I have been investigating and working on how to use IT/<a href="https://en.wikipedia.org/wiki/Internet_of_things">IOT</a>/mobile devices to reduce the amount of time that passes, from when a frail/elderly or person with a <a href="https://en.wikipedia.org/wiki/Chronic_condition">chronic condition</a> becomes ill, to the appropriate reaction/treatment can be started.</span><span face="Verdana, sans-serif"><br /></span><span face="Verdana, sans-serif">This process has almost in all cases historically been a reactive process. The person in question needed to become objectively visibly ill before someone would/could react. The <a href="https://en.wikipedia.org/wiki/History_of_medical_diagnosis">history of medical diagnosis</a> is a long and interesting one and have varied over time historically, but in general they have moved from rather crude to more and more sophisticated. Even if this has improved greatly it is still a reactive process. </span></p><p style="text-align: left;"><span face="Verdana, sans-serif"></span><span face="Verdana, sans-serif"><br /></span><span face="Verdana, sans-serif"><u>"Things that take time"</u></span></p><ul><li><span face="Verdana, sans-serif">The patient's condition starts exhibiting itself or worsens</span></li><li><span face="Verdana, sans-serif">The patient's condition has to be perceived as "bad enough" to cross the patient's "do I call the medical services"-threshold</span></li><li><span face="Verdana, sans-serif">Physical transport time to the <a href="https://en.wikipedia.org/wiki/Health_care">health care</a> professional</span></li><li><span face="Verdana, sans-serif">Potential waiting times at </span><span face="Verdana, sans-serif"><span face="Verdana, sans-serif">health care professional (waiting time at </span>the emergency room or opening hours of a private physician )</span></li><li><span face="Verdana, sans-serif">The health care professional may not have prior knowledge of the patient's condition and may misdiagnose it</span></li><li><span face="Verdana, sans-serif">Tests are done</span></li><li><span face="Verdana, sans-serif">Treatments are started</span></li></ul><span face="Verdana, sans-serif"><br /></span><span face="Verdana, sans-serif">Depending on many factors like the</span><p></p><p style="text-align: left;"></p><ul style="text-align: left;"><li><span face="Verdana, sans-serif">the patient's condition</span></li><li><span face="Verdana, sans-serif">the patient's "do I call the medical services"-threshold</span></li><li><span face="Verdana, sans-serif">the patient's physical location in relation to the relevant health care provider</span></li></ul><span face="Verdana, sans-serif"></span><span face="Verdana, sans-serif">the time from the condition </span><span face="Verdana, sans-serif"><span face="Verdana, sans-serif">starts exhibiting itself or worsens </span>to the appropriate treatment has begun, can be from a few hours (at probably very best in non life threatening conditions) to several days.<br /><br />This is not optimal use of important time and this can cause the health condition of the patient to deteriorate to a lesser or larger degree.</span><p></p>
<h3>
<span face="Verdana, sans-serif">An example: <a href="https://en.wikipedia.org/wiki/Chronic_obstructive_pulmonary_disease">COPD</a><br /></span><div style="text-align: left;"><span style="font-size: medium; font-weight: 400;"><i>( https://en.wikipedia.org/wiki/Chronic_obstructive_pulmonary_disease )</i></span></div></h3>
<span face="Verdana, sans-serif"><span style="font-weight: normal;"><div><span face="Verdana, sans-serif"><span style="font-weight: normal;"><br /></span></span></div>Since</span> 2013 we have been working with a list of objective rules </span><br /><ul><li><span face="Verdana, sans-serif">some rules use fixed ranges for a given value and if you are outside of this that will trigger a yellow or a red color (as opposed to a green color - where things are as good as they can be given the conditions).</span></li>
<li><span face="Verdana, sans-serif">some uses percentages deviation from a baseline</span></li>
</ul>
<span face="Verdana, sans-serif">These rules have been set fairly conservative, and then based on remote monitoring we get frequent data from the patient. This rule algorithm has more or less eliminated most of "things that take time". </span><br />
<br />
It is still a reactive process in that until the <a href="https://en.wikipedia.org/wiki/Exacerbation">exacerbation</a> manifests itself in a measurable way like colored <a href="https://en.wikipedia.org/wiki/Sputum">sputum</a> ( spit / mucus ) or by the lung function declining. But these are still in many cases measureable days, if not up to a week, before the patient's condition worsens to the degree that they are admitted to hospital. This early detection often makes the medical corrections needed, significantly smaller.<div><br /></div><div>If this is paired with a nurse with a specialty in COPD and a medical doctor that is specialising in COPD as a backup that monitors the patients' values and receive alarms when the situation starts to decline. This paired with emergency medication located with the patients' then in the majority of the cases you can preempt patients being admitted to the hospital.<h3>
<span face="Verdana, sans-serif"></span></h3>
</div><h3 style="text-align: left;"><br /></h3><h3 style="text-align: left;">A question </h3><div><br /></div><div>People in general are really good at using step counters, Apple watches and other health related devices for exercise purposes and other reasons. Why have the medical community and the hospitals not pushed for a greater adoption of <a href="https://en.wikipedia.org/wiki/Remote_patient_monitoring">remote patient monitoring</a> in general?</div>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-80608987696229626512014-02-26T22:06:00.002+01:002014-02-26T22:10:07.069+01:00WebRTC - Current state of afairs - Native App vs HTML/Web - iOS / Android / MacOS / Windows<span style="font-family: Arial,Helvetica,sans-serif;">I received an email with an inquiry about how to approach WebRTC development in an app, and this is my experiences based on some of the functionality that we have implemented.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">Depending on what your requirements are the easiest way is to do web only, but then you are currently ruling out iOS support.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br />In Google Chrome it works on Windows, Linux, MacOS and Android. We have tested cross browser on Windows and it works between Chrome, Firefox (and Opera).<br /><br />It is important to note that this is the full Chrome app/application not the development component "webview". I have not been able to find a webview that supports webrtc, and the new one in KitKat (Android 4.4) does not either (even though it is built on Chrome rather than the original native Android browser)</span><br />
<blockquote class="tr_bq">
<span style="font-family: Arial,Helvetica,sans-serif;">Chrome for Android supports a few features which <span style="color: red;">aren't enabled</span> in the WebView, including:</span>
<br />
<ul>
<li><span style="font-family: Arial,Helvetica,sans-serif;">WebGL 3D canvas</span></li>
<li><span style="color: red;"><span style="font-family: Arial,Helvetica,sans-serif;">WebRTC</span></span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">WebAudio</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Fullscreen API</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Form validation</span></li>
</ul>
<i><span style="font-family: Arial,Helvetica,sans-serif;">(<a href="https://developers.google.com/chrome/mobile/docs/webview/overview">https://developers.google.com/chrome/mobile/docs/webview/overview</a>)</span></i></blockquote>
<span style="font-family: Arial,Helvetica,sans-serif;"></span>
<span style="font-family: Arial,Helvetica,sans-serif;"><br />So it is possible to create a web-based WebRTC application that will work across Android, Windows, Linux and MacOS and at least on Windows it works across Chrome, Firefox and Opera.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br />IE is currently no go, although </span><br />
<blockquote class="tr_bq">
<span style="font-family: Arial,Helvetica,sans-serif;">Some people also report that IE supports WebRTC when using <a href="http://www.google.com/chromeframe">http://www.google.com/chromeframe</a></span><br/>
<span style="font-family: Arial,Helvetica,sans-serif;"><i>(<a href="http://stackoverflow.com/questions/15724913/which-version-of-microsoft-internet-explorer-support-webrtc">http://stackoverflow.com/questions/15724913/which-version-of-microsoft-internet-explorer-support-webrtc</a>)</i></span></blockquote>
<span style="font-family: Arial,Helvetica,sans-serif;"><br />It works pretty well across both fixed network connections like <a href="http://en.wikipedia.org/wiki/Adsl">ADSL</a>/<a href="http://en.wikipedia.org/wiki/Adsl">ADSL</a>, <a href="http://en.wikipedia.org/wiki/3G">3G</a>/<a href="http://en.wikipedia.org/wiki/Adsl">ADSL</a>, <a href="http://en.wikipedia.org/wiki/3G">3G</a>/<a href="http://en.wikipedia.org/wiki/3G">3G</a> networks but occasionally you run into some paranoid system admin that has the firewall locked down so tight that neither incoming or outgoing connections work and then you will face problems. This can occur at a higher level (i.e. <a href="http://en.wikipedia.org/wiki/Internet_Service_Provider">ISP</a>). ICE, STUN and TURN can only do so much.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;"></span><br />
<span style="font-family: Arial,Helvetica,sans-serif;">What about native app development and support?</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br />My thoughts on how to do this can be found at the following blog posting <a href="http://kenneththorman.blogspot.dk/2014/01/goal-cross-platform-android-ios-webrtc.html">http://kenneththorman.blogspot.dk/2014/01/goal-cross-platform-android-ios-webrtc.html</a><br /><br />I/we am working on both an Android and iOS solution and it will be developed on Github and everyone is free to contribute. Currently we am facing some problems on the Android version as can be seen <a href="https://groups.google.com/forum/#!topic/discuss-webrtc/ul7Vn8brQSM">here</a>. The iOS version is a bit technically harder than the Android version due to Google/the WebRTC team already doing most of the hard work in their Java demo app.<br /><br />So all in all web is the most doable way at the moment, but there are people that have this working on iOS using objective C and on Android using native Java.<br /><a href="http://ninjanetic.com/how-to-get-started-with-webrtc-and-ios-without-wasting-10-hours-of-your-life/">http://ninjanetic.com/how-to-get-started-with-webrtc-and-ios-without-wasting-10-hours-of-your-life/</a></span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><a href="http://kenneththorman.blogspot.dk/2014/01/webrtc-app-c-xamarin-part-1-building.html">http://kenneththorman.blogspot.dk/2014/01/webrtc-app-c-xamarin-part-1-building.html</a><br /><br />Wanting a cross device code base and thus creating it in C#/MVVMCross makes it harder, since you have to take mono/native (dalvik/iOS) communication into account, and may run into restrictions in the Xamarin supported API for low level native API's.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br />There are not many people on the internet that seem to have the skills to pull this off especially if you are aiming for both across web and native.<br /><br />The skillset to pull this off involves one or more of the following depending on the signaling system you end up using and your requirements to functionality</span><br />
<ul>
<li><a href="http://en.wikipedia.org/wiki/Android_(operating_system)"><span style="font-family: Arial,Helvetica,sans-serif;">Android</span></a></li>
<li><a href="http://en.wikipedia.org/wiki/Ios"><span style="font-family: Arial,Helvetica,sans-serif;">iOS</span></a></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><a href="http://en.wikipedia.org/wiki/Session_Initiation_Protocol">SIP / Session Initiation Protocol</a></span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><a href="http://en.wikipedia.org/wiki/WebSocket">Websockets </a> </span><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: Arial,Helvetica,sans-serif;">(client / server)</span> (or maybe <a href="http://en.wikipedia.org/wiki/Xmpp">XMPP</a>) for signaling</span></li>
<li><a href="http://en.wikipedia.org/wiki/C_Sharp_(programming_language)"><span style="font-family: Arial,Helvetica,sans-serif;">C#</span></a></li>
<li><a href="https://github.com/MvvmCross/MvvmCross"><span style="font-family: Arial,Helvetica,sans-serif;">MVVMCross</span></a></li>
<li><a href="http://en.wikipedia.org/wiki/WebRTC"><span style="font-family: Arial,Helvetica,sans-serif;">WebRTC</span></a></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Firewall <a href="http://en.wikipedia.org/wiki/Hole_punching">hole punching</a> protocols <a href="http://en.wikipedia.org/wiki/Interactive_Connectivity_Establishment">ICE</a> / <a href="http://en.wikipedia.org/wiki/STUN">STUN</a> / <a href="http://en.wikipedia.org/wiki/Traversal_Using_Relays_around_NAT">TURN</a></span></li>
<li><a href="http://en.wikipedia.org/wiki/Network_topologies"><span style="font-family: Arial,Helvetica,sans-serif;">network topologies</span></a></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><a href="http://en.wikipedia.org/wiki/Firewall_(computing)">firewall</a> and protocols (UDP fastest / TCP on ports 53, 80 or 443 - the most firewall friendly)</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">low level knowledge of hardware/audio/video - ultimately the many many devices out there on the Android side where not all are equally well supported even if they are running the same android platform (<a href="https://code.google.com/p/webrtc/issues/detail?id=2315">https://code.google.com/p/webrtc/issues/detail?id=2315</a>)</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">(stun/turn server configuration)</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">(media servers for more than 4 participants or webcasts)</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">(if you want real phone integration then <a href="http://en.wikipedia.org/wiki/Asterisk_(PBX)">Asterisk</a> or similar gateways)</span></li>
</ul>
<span style="font-family: Arial,Helvetica,sans-serif;"><br />All in all the complexity still shines through the WebRTC proposal, not because it is bad, but because it is such a diverse runtime enviroment - and it is such an amazingly complex topic. On top of this, the users perception is but this is just like a phone call this shold just work 100% of the cases. They do not see or understand that there might be a difference calling from the same device if you move from WIFI to 3G network. This is plumbing and it should just work - always - on all devices - everywhere. Auch!!! That is a pretty high expectation to fulfill. </span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br />This is still bleeding edge, I am not even sure this is out of alpha/beta stages yet - at least officially, so there are reference implementation (Chromium i.e. Google Chrome) bugs that might bite you.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">This is a super exciting area, but at the moment web seems to me the most approachable.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;"></span><br />
<span style="font-family: Arial,Helvetica,sans-serif;">Another option is to buy one of the cloud based offerings that have both Java and iOS components that make this easier. They are shielding you from some of this complexity, but comes at a cost.<br /><br />Neither web or native is without pitfalls, but certainly doable, if you have the skills. </span>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-61145968729786503872014-01-22T13:52:00.000+01:002014-01-22T13:52:46.568+01:00Xamarin / Mono: Android adb tracing / debug logging<span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New",Courier,monospace;">adb shell setprop debug.checkjni 1<br />adb shell setprop debug.mono.env MONO_LOG_LEVEL=info <br />adb shell setprop debug.mono.log gref,gc</span> <br /><br />And if none of the above helps: <br /><br /><span style="font-family: "Courier New",Courier,monospace;">adb shell setprop debug.mono.trace N:Your.App.Namespace </span><br /><br />Undoing<br /><br /><span style="font-family: "Courier New",Courier,monospace;">adb shell setprop debug.checkjni 0<br />adb shell setprop debug.mono.env '' <br />adb shell setprop debug.mono.log '' <br />adb shell setprop debug.mono.trace ''</span></span>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-71393100276284357432014-01-20T13:36:00.001+01:002014-03-13T12:39:50.404+01:00Goal: Cross platform (Android / iOS) WebRTC app using MVVMCross<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
I am attempting to port 2 Java apps that are part of the WebRTC code to Mono/C#. </div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
The code is available on GitHub, it is still a fairly early version but it builds (there are some problems during run-time).</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<br /></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<b><u>Project goals</u></b></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<ol>
<li style="line-height: 17px;">make an opensource viable implementation of WebRTC for use in Mono / Xamarin / .NET on Android and iOS</li>
<li style="line-height: 17px;">make an implementation of a WebRTC UIView/Activity that can be re-used across Android and iOS with as little as possible modifications</li>
<li style="line-height: 17px;">be able to share as much code as possible between the different platforms by utilizing PCL (portable class libraries) and<span class="Apple-converted-space"> </span><a href="https://github.com/MvvmCross/MvvmCross" target="_blank">MVVMCross</a></li>
<li style="line-height: 17px;">all possible UI logic moved to a shared portable class library using MVVMCross ViewModels</li>
<li style="line-height: 17px;">move the WebRTC signaling behind to an interface located in a portable class library with an reference implementation, allowing people to reuse signaling code across platforms, while still implementing their own.</li>
<li style="line-height: 17px;">in time write a new similar native interface as the JNI for Java but without JNI and directly targeting being invoked using P/Invoke / Mono / .NET allowing C#/Mono to directly interact with the C/C++ WebRTC library (see related question at <a href="http://kenneththorman.blogspot.dk/2014/01/question-monoandroid-performance-c-jni.html" target="_blank">Mono-JNI-Performance</a>)</li>
</ol>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<i>At least one of these goals are outside my comfort zone (#6), and all of them are currently pending, I believe #1 is well in progress.</i></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<br /></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
Currently I am still in the progress of getting the actual WebRTC meeting working. I believe I have a problem in the converted signaling code, which I am trying to track down.</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<br /></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<br /></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<br /></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<u><b>GitHub repositories</b></u></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<a href="https://github.com/kenneththorman/appspotdemo-mono" target="_blank">appspotdemo-mono</a> (this is the repository that I am focusing my immediate attention on - but I have been short on time lately)</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<br /></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
(<a href="https://github.com/kenneththorman/webrtc-app-mono" target="_blank">webrtc-app-mono</a>)</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<i>Background for the webrtc-app-mono repository:</i></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<i>When I built the native .so (<a href="http://kenneththorman.blogspot.dk/2014/01/webrtc-app-c-xamarin-part-1-building.html" target="_blank">http://kenneththorman.<wbr></wbr>blogspot.dk/2014/01/webrtc-<wbr></wbr>app-c-xamarin-part-1-building.<wbr></wbr>html</a>) it also build a full android apk that you can install and test. That was the app I already tried on my devices which worked the same way as<span class="Apple-converted-space"> </span><a href="http://apprtc.appspot.com/" target="_blank">apprtc.appspot.com</a>. That that was the code I was actually looking for, and as embarrassing it is to admit it - the repo<span class="Apple-converted-space"> </span><a href="https://github.com/kenneththorman/webrtc-app-mono" target="_blank">https://github.com/<wbr></wbr>kenneththorman/webrtc-app-mono</a><span class="Apple-converted-space"> </span>was me porting the "wrong" Java app code. When I found out and figured out how to work with JNI (mainly through using the Java Binding Library) I went looking in the official WebRTC code again to find the app that I tested in the Java version. This is the repo at<span class="Apple-converted-space"> </span><a href="https://github.com/kenneththorman/appspotdemo-mono" target="_blank">https://github.com/<wbr></wbr>kenneththorman/appspotdemo-<wbr></wbr>mono</a>. So basically having 2 repositories are proof of me not being familiar with the official WebRTC code base and not really knowing what code is which.</i></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<br /></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<br /></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
Any help or suggestions are welcome, please feel free to fork, create issues or communicate here and I will do my best to answer.</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<br /></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<br /></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<br /></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<b><u>References</u></b></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<div style="border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">
Blog postings describing my finding while porting <a href="https://github.com/kenneththorman/webrtc-app-mono" target="_blank">webrtc-app-mono</a> can be found at </div>
<div style="border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">
<ol>
<li style="line-height: 17px;"><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-1-building.html" target="_blank">WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library</a></li>
<li style="line-height: 17px;"><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-2-attempt-1.html" target="_blank">WebRTC app - C# / Xamarin - Part #2 - Attempt #1 - failure to using a JNI .so file directly from C# / Mono</a></li>
<li style="line-height: 17px;"><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-2-attempt-2.html" target="_blank">WebRTC app - C# / Xamarin - Part #2 - Attempt #2 - success using a JNI .so file from C# / Mono</a></li>
<li style="line-height: 17px;"><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-c-jni-cc-summary.html" target="_blank">WebRTC app - C# / Xamarin - (C# - JNI - C/C++) - Summary and GitHub repository</a></li>
</ol>
</div>
<div style="border: 0px; margin: 0px; padding: 0px; vertical-align: baseline;">
<br /></div>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 13px;">
<a href="https://forums.xamarin.com/discussion/11589/porting-google-webrtc-app-to-c-android-and-ios-in-the-future-mvvmcross-native-lib-problem" target="_blank">https://forums.xamarin.com/<wbr></wbr>discussion/11589/porting-<wbr></wbr>google-webrtc-app-to-c-<wbr></wbr>android-and-ios-in-the-future-<wbr></wbr>mvvmcross-native-lib-problem</a></div>
Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-67631483171304644862014-01-09T22:14:00.003+01:002014-01-09T22:14:47.654+01:00rfc5766-turn-server one liner install on CentOScd /usr/src;wget https://rfc5766-turn-server.googlecode.com/files/turnserver-3.2.1.4.tar.gz; tar -xf turnserver-3.2.1.4.tar.gz; cd turnserver-3.2.1.4; ./configure; make; make test; service turnserver stop; make install; service turnserver startKenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-44025755196297431402014-01-02T16:56:00.000+01:002014-01-02T17:29:07.793+01:00Question: Mono.Android performance: C# -> JNI wrapper -> native lib vs. C# -> Managed wrapper -> native?<span style="font-family: Arial, Helvetica, sans-serif;">I have currently ported a Java app to Mono.Android. The Java app uses a native library. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />The full walk through can be read here: <a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-c-jni-cc-summary.html">WebRTC app - C# / Xamarin - (C# - JNI - C/C++) - Summary and GitHub repository</a></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />Basically I have some Java code that looks like this </span><br />
<pre class="brush: java">public native int GetVideoEngine();</pre>
<span style="font-family: Arial, Helvetica, sans-serif;">Initially in my C# equivalent app I tried to use DllImport</span><br />
<pre class="brush: csharp">[DllImport("libwebrtc-video-demo-jni.so")]
public static extern int Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_GetVideoEngine();</pre>
<span style="font-family: Arial, Helvetica, sans-serif;">(which did not work), then I ended up wrapping the jars as Java Binding Libraries which in turn JNI'ed the needed classes, so I instead could call the generated JNI wrappers</span><br />
<pre class="brush: csharp">static Delegate cb_GetVideoEngine;
#pragma warning disable 0169
static Delegate GetGetVideoEngineHandler ()
{
if (cb_GetVideoEngine == null)
cb_GetVideoEngine = JNINativeWrapper.CreateDelegate ((Func<IntPtr, IntPtr, int>) n_GetVideoEngine);
return cb_GetVideoEngine;
}
static int n_GetVideoEngine (IntPtr jnienv, IntPtr native__this)
{
global::Org.Webrtc.Videoengineapp.ViEAndroidJavaAPI __this = global::Java.Lang.Object.GetObject<global::Org.Webrtc.Videoengineapp.ViEAndroidJavaAPI> (jnienv, native__this, JniHandleOwnership.DoNotTransfer);
return __this.VideoEngine;
}
#pragma warning restore 0169
static IntPtr id_GetVideoEngine;
public virtual int VideoEngine {
// Metadata.xml XPath method reference: path="/api/package[@name='org.webrtc.videoengineapp']/class[@name='ViEAndroidJavaAPI']/method[@name='GetVideoEngine' and count(parameter)=0]"
[Register ("GetVideoEngine", "()I", "GetGetVideoEngineHandler")]
get {
if (id_GetVideoEngine == IntPtr.Zero)
id_GetVideoEngine = JNIEnv.GetMethodID (class_ref, "GetVideoEngine", "()I");
if (GetType () == ThresholdType)
return JNIEnv.CallIntMethod (Handle, id_GetVideoEngine);
else
return JNIEnv.CallNonvirtualIntMethod (Handle, ThresholdClass, id_GetVideoEngine);
}
}
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">My question is related to performance:</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">How much slower is the C# -> JavaBindingLibrary (JAVA/JNI) -> C/C++ JNI Wrapper -> C native library than if I did unwrapped the C/C++ JNI Wrapper and rewrapped it with something like <a href="https://github.com/mono/cxxi">mono / cxxi</a> or similar direct managed or manually wrote a direct callable wrapper?</span>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-85779186609097505432014-01-01T20:44:00.000+01:002014-01-02T16:41:41.730+01:00WebRTC app - C# / Xamarin - (C# - JNI - C/C++) - Summary and GitHub repository <span style="font-family: Arial,Helvetica,sans-serif;">This posting concludes a series of in all 4 blog posting where this being the 4th.<br />
</span><br />
<ol><span style="font-family: Arial,Helvetica,sans-serif;">
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-1-building.html">WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library</a></li>
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-2-attempt-1.html">WebRTC app - C# / Xamarin - Part #2 - Attempt #1 - failure to using a JNI .so file directly from C# / Mono</a></li>
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-2-attempt-2.html"> WebRTC app - C# / Xamarin - Part #2 - Attempt #2 - success using a JNI .so file from C# / Mono</a></li>
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-c-jni-cc-summary.html">WebRTC app - C# / Xamarin - (C# - JNI - C/C++) - Summary and GitHub repository</a></li>
</span></ol>
<span style="font-family: Arial,Helvetica,sans-serif;">
And finally the associated GitHub repository <a href="https://github.com/kenneththorman/webrtc-app-mono">https://github.com/kenneththorman/webrtc-app-mono</a></span>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-44056140924780282412014-01-01T20:12:00.005+01:002014-01-17T22:09:19.992+01:00WebRTC app - C# / Xamarin - Part #2 - Attempt #2 - success using a JNI .so file from C# / Mono<span style="font-family: Arial,Helvetica,sans-serif;">This is a post in a series of postings<br />
</span><br />
<ol><span style="font-family: Arial,Helvetica,sans-serif;">
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-1-building.html">WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library</a></li>
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-2-attempt-1.html">WebRTC app - C# / Xamarin - Part #2 - Attempt #1 - failure to using a JNI .so file directly from C# / Mono</a></li>
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-2-attempt-2.html"> WebRTC app - C# / Xamarin - Part #2 - Attempt #2 - success using a JNI .so file from C# / Mono</a></li>
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-c-jni-cc-summary.html">WebRTC app - C# / Xamarin - (C# - JNI - C/C++) - Summary and GitHub repository</a></li>
</span></ol>
<span style="font-family: Arial,Helvetica,sans-serif;">
And finally the associated GitHub repository <a href="https://github.com/kenneththorman/webrtc-app-mono">https://github.com/kenneththorman/webrtc-app-mono</a></span><br/><br/><span style="font-family: Arial,Helvetica,sans-serif;">In my previous posting <a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-2-attempt-1.html">WebRTC app - C# / Xamarin - Part #2 - Attempt #1 - failure to using a JNI .so file directly from C# / Mono</a> I wrote</span><br />
<blockquote class="tr_bq">
<span style="font-family: Arial,Helvetica,sans-serif;">This is pushing me in a direction that I initially hoped I could avoid (mainly due to my limited knowledge in the area), <a href="http://en.wikipedia.org/wiki/Java_Native_Interface">JNI</a>. </span></blockquote>
<span style="font-family: Arial,Helvetica,sans-serif;">There were a few reasons that I did not prefer to use a Java Binding Library in the solution. </span> <br />
<ul>
<li><span style="font-family: Arial,Helvetica,sans-serif;">I would have an even more mixed source code base (C# calling jar/Java which is wrapping C/C++), I would have preferred to keep it at C# wrapping C/C++.</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">The upstream build process is packaging some of the compiled java classes that I need into jar files, but not all of them so now I manually need to add a build step to the build process.</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">I am not familiar with JNI</span></li>
</ul>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">If we look at the files that are generated during the build process explained in this posting <a href="http://kenneththorman.blogspot.dk/2014/01/webrtc-app-c-xamarin-part-1-building.html">WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library</a> we will see that the jars that we need to build the test android app supplied with the project are located in </span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">~/WebRTCDemo/trunk/webrtc/video_engine/test/android/libs/</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">namely</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">audio_device_module_java.jar</span><br />
<span style="font-family: "Courier New",Courier,monospace;">video_capture_module_java.jar</span><br />
<span style="font-family: "Courier New",Courier,monospace;">video_render_module_java.jar</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">For each of these jars we need to create a Java Bindings Library that is using the relevant jar as an input jar (actually 2 of the jars can go into the same Java Binding Library - <span style="font-family: "Courier New",Courier,monospace;">video_capture_module_java.jar</span> and <span style="font-family: "Courier New",Courier,monospace;">video_render_module_java.jar</span> since they share the same Java package/namespace). </span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi32zMRY_eiNJNzl8xlKU2ksnvEIXOJoOjfPHzhNKtmhFlwv2B7smtJnNa2uysA5HMVwrNKXntEiFgL9e5Qsd4Uhf82eU1bXnIZOs5eoLzytRnsrrI6q2ub-SfzjeCy8PyIBypOww/s1600/webrtc_solution1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi32zMRY_eiNJNzl8xlKU2ksnvEIXOJoOjfPHzhNKtmhFlwv2B7smtJnNa2uysA5HMVwrNKXntEiFgL9e5Qsd4Uhf82eU1bXnIZOs5eoLzytRnsrrI6q2ub-SfzjeCy8PyIBypOww/s1600/webrtc_solution1.png" /></a></div>
<br />
(<span style="font-family: Arial,Helvetica,sans-serif;">It might be possible to add them all in one Java Binding Library but I encountered problems due to Visual Studios project setting of Default Namespace, which basically was affecting the namespace of the Java classes that I needed to invoke from C#. Since each of the jars contained classes from a different Java package, the easy work around was to add a Java Binding Library for each of the jars and make sure that the Visual Studio Default namespace matched the Java package name.)</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">Looking at the existing Java WebRTC demo app available in the folder</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">~/WebRTCDemo/trunk/webrtc/video_engine/test/android/src/org/webrtc/videoengineapp</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial,Helvetica,sans-serif;">you will find </span></span></span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">IViEAndroidCallback.java</span><br />
<span style="font-family: "Courier New",Courier,monospace;">ViEAndroidJavaAPI.java</span><br />
<span style="font-family: "Courier New",Courier,monospace;">WebRTCDemo.java</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">The main file is called <span style="font-family: "Courier New",Courier,monospace;">WebRTCDemo.java</span><span style="font-family: Arial,Helvetica,sans-serif;"> and contains code like the following</span><br />
</span><br />
<pre class="brush: java">...
import org.webrtc.videoengine.ViERenderer;
import java.io.File;
import java.io.IOException;
import java.net.InetAddress;
import java.net.NetworkInterface;
import java.net.SocketException;
import java.util.Enumeration;
public class WebRTCDemo extends TabActivity implements IViEAndroidCallback,
View.OnClickListener,
OnItemSelectedListener {
private ViEAndroidJavaAPI vieAndroidAPI = null;
// remote renderer
private SurfaceView remoteSurfaceView = null;
// local renderer and camera
private SurfaceView svLocal = null;
// channel number
private int channel = -1;
private int cameraId;
private int voiceChannel = -1;
...
</pre>
<br />
<br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">By looking at the code I could see there would be some problems, since the main activity is referencing 2 classes that are defined and located side by side with the main activity <span style="font-family: "Courier New",Courier,monospace;">IViEAndroidCallback.java</span> and <span style="font-family: "Courier New",Courier,monospace;">ViEAndroidJavaAPI.java. <span style="font-family: Arial, Helvetica, sans-serif;">In other words these are not available in a jar and especially the </span></span></span><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: "Courier New",Courier,monospace;">ViEAndroidJavaAPI.java </span></span></span></span></span><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial, Helvetica, sans-serif;">ois the wrapper for the JNI native library so we cannot do this directly from C# according to <a href="http://forums.xamarin.com/discussion/325/native-library-integration">Native library integration</a></span></span></span><span style="font-family: Arial,Helvetica,sans-serif;"> (posting #2).</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial,Helvetica,sans-serif;">In the beginning of this posting I outlined 3 reasons I preferred to avoid using JNI and Java Binding Libraries, one of them was</span></span></span></span></span><br />
<ul>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial,Helvetica,sans-serif;"> </span></span></span></span></span><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: Arial,Helvetica,sans-serif;">The upstream build process is
packaging some of the compiled java classes that I need into jar files,
but not all of them so now I manually need to add a build step to the
build process. </span></span></span></span></span></span></li>
</ul>
<span style="font-family: Arial,Helvetica,sans-serif;">Anyhow this was fairly easy to remedy by manually creating a new jar containing the 2 files that was needed and then adding this new jar to a new Java Binding Library.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"> </span>
<br />
<pre class="brush: bash">XXX@ubuntu:~/WebRTCDemo/trunk/webrtc/video_engine/test/android/bin/classes$
jar cvf ViEAndroidJavaAPI.jar
org/webrtc/videoengineapp/IViEAndroidCallback.class
org/webrtc/videoengineapp/ViEAndroidJavaAPI.class
added manifest
adding: org/webrtc/videoengineapp/IViEAndroidCallback.class(in = 218)
(out= 173)(deflated 20%)
adding: org/webrtc/videoengineapp/ViEAndroidJavaAPI.class(in = 2845)
(out= 1303)(deflated 54%)
</pre>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEgsyyRUuphNMflX__rUeaaSjZg9Q01GmZjUFc77HDJIcqrUV2opEq5JprKJeO1RmbXvoGKrPaLc1Xv1hgd5qDuK35QQXzwuslQIJhRbQC7w8A3vDd-V7UTM4ZtltkAulXe4UIrw/s1600/webrtc_solution2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEgsyyRUuphNMflX__rUeaaSjZg9Q01GmZjUFc77HDJIcqrUV2opEq5JprKJeO1RmbXvoGKrPaLc1Xv1hgd5qDuK35QQXzwuslQIJhRbQC7w8A3vDd-V7UTM4ZtltkAulXe4UIrw/s1600/webrtc_solution2.png" /></a></div>
<span style="font-family: Arial,Helvetica,sans-serif;">Now I was able to use the native library without any exceptions occurring.</span>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com1tag:blogger.com,1999:blog-21633272.post-12497976796671969642014-01-01T16:10:00.000+01:002014-01-17T22:08:47.763+01:00WebRTC app - C# / Xamarin - Part #2 - Attempt #1 - failure to using a JNI .so file directly from C# / Mono<span style="font-family: Arial,Helvetica,sans-serif;">This is a post in a series of postings<br />
</span><br />
<ol><span style="font-family: Arial,Helvetica,sans-serif;">
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-1-building.html">WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library</a></li>
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-2-attempt-1.html">WebRTC app - C# / Xamarin - Part #2 - Attempt #1 - failure to using a JNI .so file directly from C# / Mono</a></li>
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-2-attempt-2.html"> WebRTC app - C# / Xamarin - Part #2 - Attempt #2 - success using a JNI .so file from C# / Mono</a></li>
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-c-jni-cc-summary.html">WebRTC app - C# / Xamarin - (C# - JNI - C/C++) - Summary and GitHub repository</a></li>
</span></ol>
<span style="font-family: Arial,Helvetica,sans-serif;">
And finally the associated GitHub repository <a href="https://github.com/kenneththorman/webrtc-app-mono">https://github.com/kenneththorman/webrtc-app-mono</a></span><br/><br/>
<span style="font-family: Arial, Helvetica, sans-serif;">In my previous posting <a href="http://kenneththorman.blogspot.dk/2014/01/webrtc-app-c-xamarin-part-1-building.html">WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library</a> I have showed how to build a native library that we need to use to build a WebRTC app on Xamarin / Mono.Droid.</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">In this posting I will take you through my struggles, subsequent failure and the next posting finally success on how to actually use this JNI native library from Mono.Android.</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">I started a new solution in Visual Studio 2013 and added new Android Application project. Then according to <a href="http://docs.xamarin.com/guides/android/advanced_topics/using_native_libraries/">Xamarin: Using Native Libraries.</a></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">I needed to add my .so file to the location <span style="font-family: "Courier New", Courier, monospace;"> </span></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New", Courier, monospace;"><project>\lib\armeabi-v7a\libwebrtc-video-demo-jni.so</span>.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">The next step that I tried was to start using DllImport statements.</span><br />
<pre class="brush: csharp">using System;
using System.Runtime.InteropServices;
using Android.Content;
using Android.Util;
using Encoding = System.Text.Encoding;
namespace WebRtc
{
public class ViEAndroidJavaAPI
{
...
// API Native
[DllImport("libwebrtc-video-demo-jni.so")]
private static extern bool NativeInit(Context context);
// Video Engine API
// Initialization and Termination functions
[DllImport("libwebrtc-video-demo-jni.so")]
public static extern int GetVideoEngine();
[DllImport("libwebrtc-video-demo-jni.so")]
public static extern int Init(bool enableTrace);
[DllImport("libwebrtc-video-demo-jni.so")]
public static extern int Terminate();
...
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Trying to run my project yielded some <span style="font-family: "Courier New", Courier, monospace;"><a href="http://docs.go-mono.com/index.aspx?link=T%3ASystem.EntryPointNotFoundException">EntryPointNotFoundException</a><span style="font-family: Arial, Helvetica, sans-serif;"> in the error log. After a bit of Google-ing I found that the method names as seen from Mono are not as you expect instead they contain the full package/class path. </span></span></span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Using the following command on the Ubuntu build machine</span><br />
<pre class="brush: bash">~/WebRTCDemo/trunk/webrtc/video_engine/test/android/libs/armeabi-v7a$ arm-linux-androideabi-nm -D libwebrtc-video-demo-jni.so
</pre>
<span style="font-family: Arial, Helvetica, sans-serif;">yielded the following output</span><br />
<pre class="brush: csharp">00015358 T JNI_OnLoad
00015be4 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_AddRemoteRenderer
00016e3c T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_CreateChannel
00015f14 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_EnableNACK
00015f58 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_EnablePLI
00015e04 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_GetCameraOrientation
00015acc T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_GetCodecs
000153d4 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_GetVideoEngine
000154cc T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_Init
000153d0 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_NativeInit
00015c30 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_RemoveRemoteRenderer
00015fb0 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_SetCallback
00015ea8 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_SetExternalMediaCodecDecoderRenderer
...
</pre>
<span style="font-family: Arial, Helvetica, sans-serif;">So changing my code to the following made the <a href="http://docs.go-mono.com/index.aspx?link=T%3ASystem.EntryPointNotFoundException"><span style="font-family: "Courier New", Courier, monospace;">EntryPointNotFoundException</span></a> go away</span><br />
<pre class="brush: csharp">using System;
using System.Runtime.InteropServices;
using Android.Content;
using Android.Util;
using Encoding = System.Text.Encoding;
namespace WebRtc
{
public class ViEAndroidJavaAPI
{
...
// API Native
[DllImport("libwebrtc-video-demo-jni.so")]
private static extern bool Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_NativeInit(Context context);
// Video Engine API
// Initialization and Termination functions
[DllImport("libwebrtc-video-demo-jni.so")]
public static extern int Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_GetVideoEngine();
[DllImport("libwebrtc-video-demo-jni.so")]
public static extern int Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_Init(bool enableTrace);
[DllImport("libwebrtc-video-demo-jni.so")]
public static extern int Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_Terminate();
...
</pre>
<span style="font-family: Arial, Helvetica, sans-serif;">Now I was faced with another exception which seemed much nastier.<br /><span style="font-family: "Courier New", Courier, monospace;">UNHANDLED EXCEPTION: System.Runtime.InteropServices.MarshalDirectiveException: Type Java.Lang.Object which is passed to unmanaged code must have a StructLayout attribute.</span></span><br />
<pre class="brush: csharp">12-21 19:29:04.298 I/MonoDroid(15226): UNHANDLED EXCEPTION: System.Runtime.InteropServices.MarshalDirectiveException: Type Java.Lang.Object which is passed to unmanaged code must have a StructLayout attribute.
12-21 19:29:04.298 I/MonoDroid(15226): at (wrapper managed-to-native) WebRtc.ViEAndroidJavaAPI.Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_NativeInit (Android.Content.Context)
12-21 19:29:04.298 I/MonoDroid(15226): at WebRtc.ViEAndroidJavaAPI..ctor (Android.Content.Context) [0x00033] in XXX\WebRtc.Mono.Droid\ViEAndroidJavaAPI.cs:30
12-21 19:29:04.298 I/MonoDroid(15226): at WebRtc.Mono.Droid.WebRTCDemo.startMain () [0x0004b] in XXX\WebRtc.Mono.Droid\WebRTCDemo.cs:533
12-21 19:29:04.298 I/MonoDroid(15226): at WebRtc.Mono.Droid.WebRTCDemo.OnCreate (Android.OS.Bundle) [0x0028e] in XXX\WebRtc.Mono.Droid\WebRTCDemo.cs:313
12-21 19:29:04.298 I/MonoDroid(15226): at Android.App.Activity.n_OnCreate_Landroid_os_Bundle_ (intptr,intptr,intptr) [0x00011] in /Users/builder/data/lanes/monodroid-mlion-monodroid-4.10.1-branch/d23a19bf/source/monodroid/src/Mono.Android/platforms/android-17/src/generated/Android.App.Activity.cs:2119
12-21 19:29:04.298 I/MonoDroid(15226): at (wrapper dynamic-method) object.705dc6ba-9c58-4bcd-a8a2-f12584a9175f (intptr,intptr,intptr)
</pre>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">Finally after digging I found this posting<a href="http://forums.xamarin.com/discussion/325/native-library-integration"><span style="color: black;"> </span>Native library integration</a> which basically state </span><br />
<blockquote class="tr_bq">
<span style="font-family: Arial,Helvetica,sans-serif;">you cannot sanely use <a href="http://en.wikipedia.org/wiki/Platform_Invocation_Services">P/Invoke</a> to invoke the native method. You must instead use <a href="http://en.wikipedia.org/wiki/Java_Native_Interface">JNI</a> to invoke the Java-side native method. </span></blockquote>
<span style="font-family: Arial,Helvetica,sans-serif;">Basically because this is Java native C/C++ interface we are to invoke, you cannot do this like normal non <a href="http://en.wikipedia.org/wiki/Java_Native_Interface">JNI</a> wrapped C/C++ methods. </span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"></span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">This is pushing me in a direction that I initially hoped I could avoid (mainly due to my limited knowledge in the area), <a href="http://en.wikipedia.org/wiki/Java_Native_Interface">JNI</a>. </span><span style="font-family: Arial,Helvetica,sans-serif;"><br /></span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">Later: I did some (quite a bit) reading, namely I found these links useful:</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></span></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.mono-project.com/Interop_with_Native_Libraries">Interop with Native Libraries</a></span></span></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><a href="http://docs.xamarin.com/guides/android/advanced_topics/java_integration_overview/">Java Integration Overview</a></span></span></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New", Courier, monospace;"><span style="font-family: Arial, Helvetica, sans-serif;"><a href="http://docs.xamarin.com/guides/android/advanced_topics/java_integration_overview/working_with_jni/">Working With JNI</a></span></span></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">In the next posting in this series I manage to invoke the native methods.</span>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-32896164247563277742014-01-01T14:56:00.001+01:002014-01-17T22:09:46.018+01:00WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library<span style="font-family: Arial,Helvetica,sans-serif;">This is a post in a series of postings<br />
</span><br />
<ol><span style="font-family: Arial,Helvetica,sans-serif;">
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-1-building.html">WebRTC app - C# / Xamarin - Part #1 - Building platform native webrtc library</a></li>
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-2-attempt-1.html">WebRTC app - C# / Xamarin - Part #2 - Attempt #1 - failure to using a JNI .so file directly from C# / Mono</a></li>
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-part-2-attempt-2.html"> WebRTC app - C# / Xamarin - Part #2 - Attempt #2 - success using a JNI .so file from C# / Mono</a></li>
<li><a href="http://kenneththorman.blogspot.com/2014/01/webrtc-app-c-xamarin-c-jni-cc-summary.html">WebRTC app - C# / Xamarin - (C# - JNI - C/C++) - Summary and GitHub repository</a></li>
</span></ol>
<span style="font-family: Arial,Helvetica,sans-serif;">
And finally the associated GitHub repository <a href="https://github.com/kenneththorman/webrtc-app-mono">https://github.com/kenneththorman/webrtc-app-mono</a></span><br/><br/>
<span style="font-family: Arial, Helvetica, sans-serif;">I wish to build a cross platform app that supports <a href="http://www.webrtc.org/">WebRTC</a> (real time communication - wikipedia article <a href="http://en.wikipedia.org/wiki/WebRTC">here</a>). I would like to use <a href="http://xamarin.com/">Xamarin </a>to achieve some level of code reuse between iOS and Android. There are several areas of the application that can use common code like the: </span><br />
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif;">webrtc signaling</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">UI (ViewModel) logic if we utilize <a href="http://slodge.blogspot.co.uk/">Stuart Lodge</a>'s <a href="https://github.com/MvvmCross/MvvmCross">MVVMCross</a>.</span></li>
</ul>
<span style="font-family: Arial, Helvetica, sans-serif;">This series of blog postings will document my attempt at implementing the Android initial app (Mono.Android) and then I will attempt to move this to iOS (monotouch). Unlike some of my other postings this is a documentation project during the attempt to reach that goal. </span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Lets get started. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">Now building webrtc and all the associated libraries is not for the faint of heart, but luckily there are some pretty nice build tools available that does the job nicely. For both the Android and later the iOS edition we will need a C/C++ native compiled library that does all the heavy lifting with regards to audio/video rendering, decoding and encoding. So my first goal was to build a Android compatible library that I could include in my solution.</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Here are some good links that got me started</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.webrtc.org/reference/getting-started">http://www.webrtc.org/reference/getting-started</a></span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">and then I found this little gem at <a href="http://devryazantsev.blogspot.ru/p/building-webrtc-trunk-for-andoid-and.html">Ryazantsev's blog</a> which basically walks you through the process step by step. I already had a Ubuntu 12.04 LTS virtual machine installed and configured so the below is the console commands run on my machine. I use some slightly modified commands compared to </span><span style="font-family: Arial, Helvetica, sans-serif;"><a href="http://devryazantsev.blogspot.ru/p/building-webrtc-trunk-for-andoid-and.html">Ryazantsev's blog.</a> Thanks to Ryazantsev for posting this great post.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<br />
<pre class="brush: bash">##Installing JAVA (first install default JAVA)
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update && sudo apt-get install oracle-jdk7-installer
chmod a+x jdk-6u45-linux-x64.bin
./jdk-6u45-linux-x64.bin
mkdir /usr/lib/jvm
mv jdk1.6.0_45 /usr/lib/jvm/jdk1.6.0_45
##Update alternatives of java tools
jdir=jdk1.6.0_45
sudo update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/$jdir/bin/javac 1
sudo update-alternatives --install /usr/bin/java java /usr/lib/jvm/$jdir/bin/java 1
sudo update-alternatives --install /usr/bin/javaws javaws /usr/lib/jvm/$jdir/bin/javaws 1
sudo update-alternatives --install /usr/bin/jar jar /usr/lib/jvm/$jdir/bin/jar 1
##Check alternatives and java version
sudo update-alternatives --config javac
sudo update-alternatives --config java
sudo update-alternatives --config javaws
sudo update-alternatives --config jar
ls -la /etc/alternatives/{java,javac,javaws,jar}
java -version
echo 'export JAVA_HOME=/usr/lib/jvm/jdk1.6.0_45' >> ~/.bashrc
echo 'PATH="$PATH":`pwd`/depot_tools' >> ~/.bashrc
source ~/.bashrc
printenv | grep depot_tools
sudo apt-get install git subversion libpulse-dev g++ pkg-config gtk+-2.0 libnss3-dev
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
###Building WebRTC Trunk
cd ~
mkdir WebRTCDemo
cd WebRTCDemo/
gclient config https://webrtc.googlecode.com/svn/trunk
gclient sync
echo target_os = [\'android\', \'unix\'] >> .gclient
./trunk/build/install-build-deps.sh --no-chromeos-fonts
./trunk/build/install-build-deps-android.sh
check 'sudo update-alternatives --config java ' for correct path
gclient sync
cd trunk
source ./build/android/envsetup.sh
gclient runhooks
GYP_GENERATORS=ninja ./build/gyp_chromium --depth=. all.gyp
ninja -C out/Debug -j10 All
</pre>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /> After downloading all the files and building the project it takes up about 4.6GB on my disk in the virtual machine. Most source files in the project will show to be a good reference when we need to start using this in a Xamarin project. The really relevant parts through is the java sources for the Java version of the app (written by the WebRTC authors) as well as the Android compatible .so file.</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">The compiled .so file we need is available at </span><br />
<pre class="brush: bash">~/WebRTCDemo/trunk/webrtc/video_engine/test/android/libs/armeabi-v7a/libwebrtc-video-demo-jni.so </pre>
<span style="font-family: Arial, Helvetica, sans-serif;">The next posting will be attempt #1 in converting a Java app to a c# equivalent</span>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-32001339524286266672013-12-29T22:02:00.003+01:002014-01-01T20:16:45.613+01:00Yii Framework: Using multiple db connections in a controller method<span style="font-family: Arial,Helvetica,sans-serif;">I have a mixed database where the majority of the tables are UTF-8 encoding but there are a few tables that are required to store the data in a specific encoding that this not UTF-8. </span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">In Yii you are defining you global database connection in the protected/config/main.php file, and this is global for the entire web application.This is causing some challenges for me since it is very important that I deliver data bitwise compatible according to the data stored in the table. </span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">I was not able to find any solution to this in my first few Google attempts, so I had to dive into the code.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">First I tried in the controller action method by changing the charset on the global CDbConnection directly using </span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">Yii::app()->db->charset='latin1'; </span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">this did not work since the database connection is already initialized at this point.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">Going through the Yii Framework source code I first found <a href="http://www.yiiframework.com/doc/api/1.1/CApplication#getDb-detail">CApplication->getDB()</a>, then searching for the function getComponent() lead me to <a href="http://www.yiiframework.com/doc/api/1.1/CModule#getComponent-detail">CModule->getComponent().</a></span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">My next though was "If there is a getComponent(), I wonder if there is not a setComponent() function as well?" - and behold <a href="http://www.yiiframework.com/doc/api/1.1/CModule#setComponent-detail">CModule->setComponent()</a>.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">Armed with this knowledge it was easy to change the connection temporarily to use a new initialized database connection with a different charset.</span><br />
<br />
<pre class="brush: php">public function loadModel($id)
{
// Save the original database connection for later reuse
$originalDbConnection = Yii::app()->db;
// Create a new database connection based on the original database connection
// change the charset to latin1
$latin1DbConnection = Yii::createComponent(array(
'class'=>'CDbConnection',
'connectionString' => Yii::app()->db->connectionString,
'emulatePrepare' => Yii::app()->db->emulatePrepare,
'username' => Yii::app()->db->username,
'password' => Yii::app()->db->password,
'charset' => 'latin1',
));
// Set the application wide database connection for this Apache/PHP webrequest to use this special database connection
Yii::app()->setComponent("db",$latin1DbConnection);
// Do the queries against the database
$someModel=SomeModel::model();
$model = $someModel->findByPk($id);
// Restore the original database connection
// note:
// This is actually not needed in this simple case, due to the fact that Apache/PHP rebuilds the entire web application on each request
// Since I am not doing additional request in this logic here after returning the result everything is torn down and discarded.
// However I have added it as part of this code for completeness and also in case someone else out there is doing more complex logic with multiple
// request that needs different charsets
Yii::app()->setComponent("db",$originalDbConnection);
if($model===null)
throw new CHttpException(404,'The requested page does not exist.');
return $model;
}</pre>
Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-13713897365672779252013-09-01T10:47:00.000+02:002013-09-01T10:47:13.532+02:00Unable to change network adapter UUID on Linux<span style="font-family: Arial, Helvetica, sans-serif;">I needed to reconfigure the hardware UUID of a Linux network adapter caused by cloning a virtual machine. To do this you need to either remove the /etc/udev/rules.d/70-persistent-net.rules and let the system regenerate the file on reboot, or manually edit the file.</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">I decided to edit the file and manually correct the UUID. I was consistently met "readonly file" so I could not save my changes.</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">I found using the lsattr /etc/udev/rules.d/70-persistent-net.rules that the file had the i attribute set. This i attribute means that the file is immutable. Removing this attribute using chattr -i </span><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: Arial, Helvetica, sans-serif;">/etc/udev/rules.d/70-persistent-net.rules, allowed me to change the file as needed. After editing the file I set the i attribute again using </span></span><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: Arial, Helvetica, sans-serif;">chattr +i </span><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: Arial, Helvetica, sans-serif;">/etc/udev/rules.d/70-persistent-net.rules</span></span></span></span>.Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-66082191747075022762013-07-28T23:23:00.000+02:002014-01-21T08:11:37.355+01:00Fixing the client IP as perceived by Apache behind Nginx reverse proxy<span style="font-family: Arial, Helvetica, sans-serif;">In my recent post <a href="http://kenneththorman.blogspot.dk/2013/07/using-nginx-to-reverse-proxy-secure.html">Using Nginx to reverse proxy a secure apache site that is using socket.io/node.js/websockets</a> I found that I was only getting the proxy's IP address in all of my apache logs as well as in the application tracking. This does make sense since from Apache's perspective the proxy is the actual client and not the real remote client. I found however that it was possible to get around this. Apparently there are several Apache modules that can do this, however I did not want to build from sources if I could avoid it. So I choose the one that was already available in the EPEL repository.</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">I already had the <a href="http://fedoraproject.org/wiki/EPEL">EPEL </a>repository registered on the server but for those that does not you can use the following commands to register it.</span>
<br />
<pre class="brush: bash">rpm -Uvh http://ftp.crc.dk/fedora-epel/6/i386/epel-release-6-8.noarch.rpm
</pre>
<span style="font-family: Arial, Helvetica, sans-serif;">Now install the module</span>
<br />
<pre class="brush: bash">yum install mod_extract_forwarded
</pre>
<span style="font-family: Arial, Helvetica, sans-serif;">After installing the module you need to register an allowed forwarder (ie. the IP of your proxy). In all my logs the ip 127.0.0.1 was registered as the client of every request so adding that did the trick.</span>
<br />
<pre class="brush: bash">echo "MEFaccept 127.0.0.1" >> /etc/httpd/conf.d/mod_extract_forwarded.conf
service httpd restart
</pre>
<span style="font-family: Arial, Helvetica, sans-serif;">Recheck your webserver/application logs, this worked for me.</span>
Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-69501395206342862042013-07-25T21:34:00.000+02:002014-01-01T20:17:02.751+01:00Using Nginx to reverse proxy a secure apache site that is using socket.io/node.js/websockets<u><span style="font-family: Arial, Helvetica, sans-serif;">Challenge:</span></u><br />
<span style="font-family: Arial, Helvetica, sans-serif;">Firewalls can cause challenges, by blocking the ports that you want to use for websockets.</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Many firewalls in use today are so called <a href="https://en.wikipedia.org/wiki/Stateful_firewall">stateful firewalls</a> and in short their function can be described as follows</span><br />
<br />
<blockquote class="tr_bq">
<span style="font-family: Arial, Helvetica, sans-serif;">Stateful
firewalls only verify that a packet correlates to an existing,
unclosed, connection. It tracks the state of the connection (opening,
open, closing, closed) hence the name. When it detects a packet is part
of an already open, authorized connection, it can short circuit all of
the other rule checks and let the packet through.<br /><i>(</i></span><span style="font-family: Arial, Helvetica, sans-serif;"><i><a href="http://stackoverflow.com/questions/1967943/will-html5-websockets-be-crippled-by-firewalls">http://stackoverflow.com/questions/1967943/will-html5-websockets-be-crippled-by-firewalls</a>)</i></span></blockquote>
<span style="font-family: Arial, Helvetica, sans-serif;">Most sites that are using <a href="http://nodejs.org/">node.js</a> for websockets are also utilizing the library <a href="https://github.com/LearnBoost/socket.io">socket.io</a>. </span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">A good list outlining some of the challenges when combining websockets/socket.io with various firewalls can be found at <a href="https://github.com/LearnBoost/socket.io/wiki/Socket.IO-and-firewall-software">Socket.IO and firewall software</a>.</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">So the question is how do you get your website running in spite of corporate firewall blocking various ports?</span><br />
<br />
<br />
<u><span style="font-family: Arial, Helvetica, sans-serif;">Possible solutions:</span></u><br />
<span style="font-family: Arial,Helvetica,sans-serif;"></span><br />
<ol><span style="font-family: Arial,Helvetica,sans-serif;">
<li><span style="font-family: Arial,Helvetica,sans-serif;">ask the IT department to open the needed ports (in my experience hardly likely unless #2 fails)</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">piggy-back on one of the existing established connections using <a href="http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol">http </a>(port 80) or <a href="http://en.wikipedia.org/wiki/Https">https</a> (port 443) and hope the firewall is not a <a href="http://en.wikipedia.org/wiki/Deep_packet_inspection">deep packet inspection</a> based firewall.</span></li>
</span></ol>
<span style="font-family: Arial,Helvetica,sans-serif;">
</span>
<span style="font-family: Arial, Helvetica, sans-serif;"> </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">Since I seldom have luck with suggestion number 1, I am not able to offer any guidance that is likely to help you succeed.</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Suggestion number 2 however we can achieve that by letting the websockets use port 80/443 and let the website use the same port. You cannot however run 2 different services on the same port, so you cannot do this in a straightforward manner. The answer is a websocket aware reverse proxy.</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">There are quite a few different setups that can solve this. Below you can find 3 ways I investigated when I needed to solve this:</span><br />
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Apache acting as a reverse proxy for node.js (this was my first choice, but I did not manage to get this working with out-of-the-box available packages for the Linux distribution I am using, but you can more than likely build your own version that will work just fine <a href="http://blog.cafarelli.fr/post/2013/04/26/Backporting-Apache-support-for-websockets-reverse-proxy-(aka-getting-GateOne-to-work-behind-Apache)">Backporting Apache support for websockets reverse proxy (aka getting GateOne to work behind Apache)</a> and <a href="http://blog.alex.org.uk/2012/02/16/using-apache-websocket-to-proxy-tcp-connection/">Using apache-websocket to proxy tcp connections</a>).</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Use node.js to proxy Apache (see more here </span><span style="font-family: Arial, Helvetica, sans-serif;"><a href="https://github.com/nodejitsu/node-http-proxy">https://github.com/nodejitsu/node-http-proxy</a>). (I never managed to get this working properly either but that is more than likely due to my slightly limited exposure to node.js, and to be honest my experience with node has been a slightly turbulent one, so I feel that Apache/Nginx are the more stable options for me, that is not to say that this is not working or an great option.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;"><a href="https://en.wikipedia.org/wiki/Nginx">Nginx</a> being a reverse proxy for both apache and </span><span style="font-family: Arial, Helvetica, sans-serif;">websockets/socket.io. This is what I will describe below.</span></li>
</ul>
<br />
<br />
<u><span style="font-family: Arial, Helvetica, sans-serif;">Setup</span></u><br />
<ul>
<li><span style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.centos.org/">CentOS</a> based Linux distribution</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">existing website installed as a Apache virtual host running on port 80 and 443</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">existing node.js socket.io based application running on port 8081</span></li>
</ul>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiH21DZP9nMNyDvOgeRMAY2GHDG-2xqYPFcMDmmh4_Q6q4gsHRywsKMqtxfg_y89eFmE0zFJNuDl4cAATLmKlrUtddhjnvNTxert_ypQc7PpGuef5301dD0hQc8rnvX-NIZVahHVA/s1600/existingSetup.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiH21DZP9nMNyDvOgeRMAY2GHDG-2xqYPFcMDmmh4_Q6q4gsHRywsKMqtxfg_y89eFmE0zFJNuDl4cAATLmKlrUtddhjnvNTxert_ypQc7PpGuef5301dD0hQc8rnvX-NIZVahHVA/s1600/existingSetup.png" /></a></div>
<div style="text-align: center;">
<i><span style="font-family: Arial, Helvetica, sans-serif;">(setup before reverse proxy)</span></i></div>
<br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Please find an image of the wished for setup here below.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXs1jes_IabPy-ecCkRb3EGFww0QjHWPpovNDcaUQphlqz1f9m2XaF4zkRSyndD0u1KsiuNz7eWWalfz94FD-gNwcn3rM2-UwmgAaNjSYtxa9vwivwqwdjIT1nB-9bZ2KHjZzo8A/s1600/wantedSetup.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXs1jes_IabPy-ecCkRb3EGFww0QjHWPpovNDcaUQphlqz1f9m2XaF4zkRSyndD0u1KsiuNz7eWWalfz94FD-gNwcn3rM2-UwmgAaNjSYtxa9vwivwqwdjIT1nB-9bZ2KHjZzo8A/s1600/wantedSetup.png" /></a></div>
<span style="font-family: Arial, Helvetica, sans-serif;"> </span>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">We need to do the following to achieve the reverse proxy setup</span><br />
<ol><span style="font-family: Arial, Helvetica, sans-serif;">
<li>Install Nginx</li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Remove http -> https rewrite and ssl configuration from Apache as well as change Apache from listening on port 80/443 to listen on port 81</span></li>
<li>Configure http -> https rewrite and reverse proxy in Nginx </li>
<li>Change the socket.io node.js application to just use ws instead of wss (since Nginx is the SSL termination proxy now)</li>
<li>Change the client application from using port 8081 to using 80/443</li>
</span></ol>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;"><u>1. Install Nginx</u><br />
<a href="http://www.cyberciti.biz/faq/install-nginx-centos-rhel-6-server-rpm-using-yum-command/">RHEL / Centos 6: Install Nginx Using Yum Command</a><br /> </span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;"><u>2. Remove http -> https rewrite and ssl configuration from Apache as well as change Apache from listening on port 80/443 to listen on port 81</u></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: Arial, Helvetica, sans-serif;">Open the virtual hosts file for the site</span></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">vi /etc/httpd/conf.d/www_somewhere_com.conf</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">Change your virtual hosts file from the following </span><br />
<br />
<pre class="brush:xml"># Catch those that are trying to access the site using http
# redirect to https
<VirtualHost *:80>
ServerName "www.somewhere.com"
DocumentRoot /var/www/sites/www.somewhere.com/www
DirectoryIndex index.html index.php
ErrorLog logs/www.somewhere.com-error_log
CustomLog logs/www.somewhere.com-access_log common
RewriteEngine On
RewriteCond %{HTTPS} !=on
RewriteRule ^(.*) https://www.somewhere.com [R,L]
</VirtualHost>
# Main site
<VirtualHost *:443>
ServerName "www.somewhere.com"
DocumentRoot /var/www/sites/www.somewhere.com/www
DirectoryIndex index.html index.php
ErrorLog logs/www.somewhere.com-error_log
CustomLog logs/www.somewhere.com-access_log common
SSLEngine on
SSLCertificateFile /etc/pki/tls/certs/www_somewhere.com.crt
SSLCertificateKeyFile /etc/pki/tls/private/www_somewhere.com.key
SSLCertificateChainFile /etc/pki/tls/certs/Intermediate_Certificate.crt
<Directory /var/www/sites/www.somewhere.com/www>
Order Deny,Allow
Allow from all
</Directory>
</VirtualHost>
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">to the following</span><br />
<br />
<pre class="brush:xml"># Main site
<VirtualHost *:81>
ServerName "www.somewhere.com"
DocumentRoot /var/www/sites/www.somewhere.com/www
DirectoryIndex index.html index.php
ErrorLog logs/www.somewhere.com-error_log
CustomLog logs/www.somewhere.com-access_log common
<Directory /var/www/sites/www.somewhere.com/www>
Order Deny,Allow
Allow from all
</Directory>
</VirtualHost>
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">vi /etc/httpd/conf/httpd.conf <br />Look for the "Listen 80" in the file and change it to "Listen 81"<br />
Save the file</span>
<br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;"><u>3. Configure http -> https rewrite and reverse proxy in Nginx</u><br />
vi /etc/nginx/conf.d/default.conf</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
replace the deafult content with this</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Please note that in apache config you have a seperate intermidiate certificate and the site certificate. In Nginx you need to concatenate the primary certificate file (your_domain_name.crt) and the intermediate certificate file (DigiCertCA.crt) into a single pem file by running the following command:<br /><br />cat DigiCertCA.crt >> your_domain_name.crt</span><br />
<br />
<pre class="brush:xml"># Listeners
# nginx :80 responsible for redirecting http://www.somewhere.com to https://www.somewhere.com
# nginx :443 responsible for handling all trafic on the https://www.somewhere.com site
# acts as a reverse proxy for
# /socket.io/ -> port 127.0.0.1:8081 (node.js websocket server)
# / -> port 127.0.0.1:81 (apache main site)
#
# node.js :8081 node.js websocket server
# apache :81 apache main site
upstream www.somewhere.com {
server 127.0.0.1:81;
least_conn;
}
# nginx :80 responsible for redirecting http://www.somewhere.com to https://www.somewhere.com
server {
listen 80;
server_name www.somewhere.com;
rewrite ^(.*) https://www.somewhere.com$1 permanent;
}
# nginx :443 responsible for handling all trafic on the https://www.somewhere.com site
# acts as a reverse proxy for
# /socket.io/ -> port 127.0.0.1:8081 (node.js websocket server)
# / -> port 127.0.0.1:81 (apache main site)
server {
listen 443 ssl;
server_name www.somewhere.com;
ssl_certificate /etc/pki/tls/certs/www_somewhere.com.crt;
ssl_certificate_key /etc/pki/tls/private/www_somewhere.com.key;
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers RC4:HIGH:!aNULL:!MD5;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Path based websocket proxy location for nginx
# to handle the reverseproxying
#
# inital connect url: https://www.somewhere.com:8081/socket.io/1/?t=1374751734771
# websocket url : wss://www.somewhere.com:8081/socket.io/1/websocket/x55Ch0B-SCACmDpW8rZd
location /socket.io/ {
proxy_pass http://localhost:8081;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
}
# Path based site proxy location for nginx
# to handle the main apache site
location / {
proxy_pass http://localhost:81;
}
}
</pre>
<br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;"><u>4. Change the socket.io node.js application to just use ws instead of wss (since Nginx is the <a href="http://en.wikipedia.org/wiki/SSL_termination_proxy">SSL termination proxy</a> now)</u><br />
Change you node.js socket.io application from this
</span><br />
<br />
<pre class="brush: javascript">var fs = require('fs');
var sslCertificate = {
key: fs.readFileSync(config.sslCertificate.key),
cert: fs.readFileSync(config.sslCertificate.cert),
ca: fs.readFileSync(config.sslCertificate.ca)
};
// socket.io wss (websocket secure)
var io = require('socket.io').listen(8081, sslCertificate);
...
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">to the following</span><br />
<br />
<pre class="brush: javascript">var fs = require('fs');
var sslCertificate = {
// key: fs.readFileSync(config.sslCertificate.key),
// cert: fs.readFileSync(config.sslCertificate.cert),
// ca: fs.readFileSync(config.sslCertificate.ca)
};
// socket.io ws (websocket)
var io = require('socket.io').listen(8081, sslCertificate);
...
</pre>
<br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;"><u>5. Change the client application from using port 8081 to using 80/443</u><br />
<br />
Find the location in your client code where you are connecting to the websocket/socket.io application/server and change it from hardcoding the port in the url to just omitting it
<br /><br />
So instead of (in a <a href="http://angularjs.org/">angular.js</a> application / javascript <a href="http://en.wikipedia.org/wiki/Single-page_application">single page application</a>) doing this
</span><br />
<br />
<pre class="brush:javascript">var socketIO = function(data){
// force https / wss
var hostURL = window.location.href.replace('/'+window.location.hash, '8081').replace('http:', 'https:');
...
try {
var socket = io.connect(hostURL, {'sync disconnect on unload':true});
...
}
catch(e){
console.log('Error connecting');
...
}
...
}
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">do something like this</span>
<br />
<pre class="brush:javascript">var socketIO = function(data){
// force https / wss
var hostURL = window.location.href.replace('/'+window.location.hash, '').replace('http:', 'https:');
...
try {
var socket = io.connect(hostURL, {'sync disconnect on unload':true});
...
}
catch(e){
console.log('Error connecting');
...
}
...
}
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">run the following command to check if the services are running on the correct ports</span><br />
<br />
<pre class="brush: bash">#netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 13146/nginx
tcp 0 0 0.0.0.0:8081 0.0.0.0:* LISTEN 13225/node
tcp 0 0 127.0.0.1:81 0.0.0.0:* LISTEN 13190/httpd
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 13146/nginx
...
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">
Finally some before and after pictures from chrome
<br />Before
</span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_hnnAKg0hqlaoJaSDmX6XOiuj9N97Kc_sqf4csHnmugDfmshqbpPyjiQq8oNjRIQHvWpPLIys5P0c5Y4FJlh_BFuIC_CxHJ-ewEXcjTmY4rAY2ay6ZvG5IIpizBojAi0O9PGfeg/s1600/beforeReverseProxy2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="81" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_hnnAKg0hqlaoJaSDmX6XOiuj9N97Kc_sqf4csHnmugDfmshqbpPyjiQq8oNjRIQHvWpPLIys5P0c5Y4FJlh_BFuIC_CxHJ-ewEXcjTmY4rAY2ay6ZvG5IIpizBojAi0O9PGfeg/s640/beforeReverseProxy2.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9RPIE9_PKq9k7smoYbko9ZbacB6RY5uCGvSxTUhIornvka1i4w14aoIz8qEYAyxjU88Md1FrKwJUXTL7bqN0NSP28CBG5SKJ2x595Kd2l8iSIOkrFYX-X5v5ne-toSL_SY_8iPw/s1600/beforeReverseProxy.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="94" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9RPIE9_PKq9k7smoYbko9ZbacB6RY5uCGvSxTUhIornvka1i4w14aoIz8qEYAyxjU88Md1FrKwJUXTL7bqN0NSP28CBG5SKJ2x595Kd2l8iSIOkrFYX-X5v5ne-toSL_SY_8iPw/s640/beforeReverseProxy.png" width="640" /></a></div>
<br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">
After
</span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgwHuUlWC_qe03MCNYOm96rrf_ZQ0Uwiyb3tHImjm00e8hH3ooYEYeyzRLO29fW6QDHHF2zcxVHh67iQ-aBZMtSK5GG8lV0QY8q5IxtBTgclpuzsag78KeaiHDtk2jRg1lZDYm0Wg/s1600/afterReverseProxy2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="84" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgwHuUlWC_qe03MCNYOm96rrf_ZQ0Uwiyb3tHImjm00e8hH3ooYEYeyzRLO29fW6QDHHF2zcxVHh67iQ-aBZMtSK5GG8lV0QY8q5IxtBTgclpuzsag78KeaiHDtk2jRg1lZDYm0Wg/s640/afterReverseProxy2.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzJfoBN1bg3kdidq6NgsLiW2BeA_V92ZBYkAQGrN1FBFJGT9Te4oNIBewXKgyGm3n7hUMsCYOsCs1iZvcEi9IQvdT7ft70sp16e2LKmPJpOsWSzMZDJdok8JgAXnRi5AzQk9TMkQ/s1600/afterReverseProxy.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="106" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzJfoBN1bg3kdidq6NgsLiW2BeA_V92ZBYkAQGrN1FBFJGT9Te4oNIBewXKgyGm3n7hUMsCYOsCs1iZvcEi9IQvdT7ft70sp16e2LKmPJpOsWSzMZDJdok8JgAXnRi5AzQk9TMkQ/s640/afterReverseProxy.png" width="640" /></a></div>
<br />Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-75517374142836580602013-07-01T22:02:00.003+02:002013-07-01T22:05:48.549+02:00Compressed Node.js install on CentOS 6.4yum install -y make gcc cc gcc-c++ wget; cd /usr/src; wget http://nodejs.org/dist/v0.10.12/node-v0.10.12.tar.gz; tar zxf node-*.tar.gz; cd node-v*; ./configure; make; make install;Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-74953651422938831542013-06-19T14:04:00.001+02:002013-06-24T10:01:41.047+02:00Installing ejabberd2 on CentOS and configure a virtual host on Apache2 <span style="font-family: Arial, Helvetica, sans-serif;">1. Enable epel yum repository</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Centos 5.x</span><br />
<pre class="brush: bash">wget http://dl.fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
sudo rpm -Uvh epel-release-5*.rpm
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Centos 6.x</span><br />
<pre class="brush: bash">wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
sudo rpm -Uvh epel-release-6*.rpm
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">2. Install ejabberd2</span><br />
<pre class="brush: bash">yum --enablerepo=epel install ejabberd
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">3. Start the ejabberd2 service</span><br />
<pre class="brush: bash">service ejabberd start
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">4. Edit the virtual hosts configuration file and add</span><br />
<pre class="brush: bash"> ProxyPass /http-bind http://127.0.0.1:5280/http-bind/
ProxyPassReverse /http-bind http://127.0.0.1:5280/http-bind/
Header set Access-Control-Allow-Origin "YOUR_VIRTUAL_SERVER_HOSTNAME"
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">(please note that using "Header set Access-Control-Allow-Origin" enables <a href="http://en.wikipedia.org/wiki/Cross-origin_resource_sharing">Cross-origin resource sharing</a> and using a wildcard as origin should not normally be used)</span><br />
<br />
<blockquote class="tr_bq">
This is generally not appropriate. The only case where this is appropriate is when a page or api response is considered completely public content and it is intended to be accessible to browsable to everyone. Including any code on any site.<br />
<i>(<a href="http://en.wikipedia.org/wiki/Cross-origin_resource_sharing">http://en.wikipedia.org/wiki/Cross-origin_resource_sharing</a>)</i></blockquote>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">5. Test configuration</span><br />
<pre class="brush: bash">service httpd configtest
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">6. Reload apache configuration</span><br />
<pre class="brush: bash">service httpd reload
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Since ejabberd comes with http-bind enabled out of the box</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"> </span>
<br />
<pre>{5280, ejabberd_http, [
%%{request_handlers,
%% [
%% {["pub", "archive"], mod_http_fileserver}
%% ]},
captcha,
http_bind,
http_poll,
%%register,
web_admin
]}
</pre>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">you should now be able to access the <a href="http://xmpp.org/extensions/xep-0206.html">XMPP over BOSH (XEP-0206)</a> server in your browser looking somewhat like below.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjY2ovcr54_iT-NUyS9IO1ybUlidvDzMlfvq2HH-W_DohEY_y0m00G2zb7qA4Iw9Pj8p-QerU0aCJ-fp67gyd2R1C6RoV2yYFWiqu_GvCcMd7y0iIMrl4oj7xKERPGsx-B-j9R8kQ/s1600/ejabber_working.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjY2ovcr54_iT-NUyS9IO1ybUlidvDzMlfvq2HH-W_DohEY_y0m00G2zb7qA4Iw9Pj8p-QerU0aCJ-fp67gyd2R1C6RoV2yYFWiqu_GvCcMd7y0iIMrl4oj7xKERPGsx-B-j9R8kQ/s1600/ejabber_working.png" /></a></div>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;"> </span>
Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-59036374728226747742013-06-03T21:10:00.002+02:002013-06-24T10:02:30.235+02:00Xamarin Mono.Droid Receiving different bluetooth devices data records<span style="font-family: Arial,Helvetica,sans-serif;">When you are working with several different Bluetooth devices (from different manufacturers) you will have to be able to handle different types of data records. The official Android Bluetooth sample ported for Xamarin/Mono.Droid can be found at <a href="https://github.com/xamarin/monodroid-samples/blob/master/BluetoothChat/BluetoothChatService.cs">BluetoothChatService.cs</a></span>. <span style="font-family: Arial,Helvetica,sans-serif;">This was the code I used for inspiration when I built this app.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;">I could however not really get the code to work as I wanted and the problem was located in the following method <a href="https://github.com/xamarin/monodroid-samples/blob/master/BluetoothChat/BluetoothChatService.cs#L482">Run()</a>.</span>
<br />
<pre class="brush: csharp">public override void Run ()
{
Log.Info (TAG, "BEGIN mConnectedThread");
byte[] buffer = new byte[1024];
int bytes;
// Keep listening to the InputStream while connected
while (true) {
try {
// Read from the InputStream
bytes = mmInStream.Read (buffer, 0, buffer.Length);
// Send the obtained bytes to the UI Activity
_handler.ObtainMessage (BluetoothChat.MESSAGE_READ, bytes, -1, buffer)
.SendToTarget ();
} catch (Java.IO.IOException e) {
Log.Error (TAG, "disconnected", e);
_service.ConnectionLost ();
break;
}
}
}
</pre>
<span style="font-family: Arial,Helvetica,sans-serif;">The problem is the following call</span>
<br />
<pre class="brush: csharp">// Send the obtained bytes to the UI Activity
_handler.ObtainMessage (BluetoothChat.MESSAGE_READ, bytes, -1, buffer).SendToTarget ();
</pre>
<span style="font-family: Arial,Helvetica,sans-serif;">which might not always send back the entire device record to the handler. This is caused by the various devices implementation of how to send the data record back as a byte stream to the mobile device/phone might not finish, before the while loop is starting another iteration, as well as the <a href="http://en.wikipedia.org/wiki/Multithreading_(software)#Multithreading">multithreading</a> and the <a href="http://en.wikipedia.org/wiki/Context_switch">context switches</a> that occurs in the BluetoothChatService.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">What I discovered while testing was that for some devices it was working out of the box with the default code in the Run method above. However on some more complex devices which were sending larger data records back, it never seemed to work.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">After spending some time analyzing the problem - being new to the bluetooth scene and Mono.Droid development - I found that data records got chopped into pieces in the while(true) loop in the above Run() method. Assuming that a full data record was 65 bytes, sometimes the code would pass 13 bytes to the SendToTarget() method. The next iteration might yield 31 bytes and the last 21 bytes was yielded on the 3rd pass. </span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">The number of bytes returned on each iteration was never the same and there was no specific pattern to this. To be able to handle this for different devices sending data in different ways I needed some kind of smart buffering. I came up with the following.</span><br />
<pre class="brush: csharp">public override void Run ()
{
Log.Info (TAG, "BEGIN mConnectedThread");
byte[] buffer = new byte[1024];
int bytes;
// Keep listening to the InputStream while connected
while (true) {
try {
// Read from the InputStream
bytes = mmInStream.Read(buffer, 0, buffer.Length);
var messagePart = new byte[bytes];
Array.Copy(buffer,messagePart, bytes);
// Send the obtained bytes to the UI Activity
_handler.ObtainMessage(MESSAGE_READ, bytes, -1, messagePart)
.SendToTarget();
} catch (Java.IO.IOException e) {
Log.Error (TAG, "disconnected", e);
_service.ConnectionLost ();
break;
}
}
}
</pre>
<span style="font-family: Arial,Helvetica,sans-serif;">The new code is only a little bit different in that I am no longer sending back the buffer but instead only the partial message (messagePart).</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">In the handler that handles the read bytes I changed the implementation from what can be found here <a href="https://github.com/xamarin/monodroid-samples/blob/master/BluetoothChat/BluetoothChat.cs#L279">BluetoothChat.cs</a>.</span><br />
<pre class="brush: csharp">case MESSAGE_READ:
byte[] readBuf = (byte[])msg.Obj;
// construct a string from the valid bytes in the buffer
var readMessage = new Java.Lang.String (readBuf, 0, msg.Arg1);
bluetoothChat.conversationArrayAdapter.Add (bluetoothChat.connectedDeviceName + ": " + readMessage);
break;
</pre>
<span style="font-family: Arial,Helvetica,sans-serif;">to the following code</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"></span>
<br />
<pre class="brush: csharp">case BluetoothService.MESSAGE_READ:
byte[] readBuf = (byte[])msg.Obj;
ParseBluetoothDeviceReading(readBuf);
break;
</pre>
<span style="font-family: Arial,Helvetica,sans-serif;">and the ParseBluetoothDeviceReading() method looking like this</span><br />
<pre class="brush: csharp">public void ParseBluetoothDeviceReading (byte[] readBuffer)
{
if (readBuffer == null || readBuffer.Length <= 0)
return;
var deviceRecord = _zephyrResponseParser.ParseDeviceResponse(readBuffer);
_data.LungFunction= deviceRecord.LungFunction;
lblValue.Text = deviceRecord.LungFunction.ToString("0.00");
}</pre>
<span style="font-family: Arial,Helvetica,sans-serif;">the data record buffer is implemented in the _zephyrResponseParser which is of type IDeviceResponseParser<zephyrrecord>. The interface IDeviceResponseParser looks like the following</zephyrrecord></span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><zephyrrecord></zephyrrecord></span>
<br />
<pre class="brush: csharp">public interface IDeviceResponseParser<TDataRecord>
{
TDataRecord ParseDeviceResponse(byte[] response);
}
</pre>
<span style="font-family: Arial,Helvetica,sans-serif;">and finally the actual device specific data record parser for the zephyr device.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"></span>
<br />
<pre class="brush: csharp">using System.Collections.Generic;
namespace Droid.Devices.Zephyr
{
public class ZephyrResponseParser : IDeviceResponseParser<ZephyrRecord>
{
public const int DeviceMessageLength = 65;
private List<byte> _messageBuffer = new List<byte>();
public ZephyrRecord ParseDeviceResponse(byte[] response)
{
// Since the data (byte[]) that we are getting is retreived from a BluetoothNetworkStream there are no guarantees that we will get the full message in one go.
// For that reason we need to buffer it our selves until whe have a full message
_messageBuffer.AddRange(response);
if (_messageBuffer.Count < DeviceMessageLength)
{
// Not full message return
return new ZephyrRecord();
}
// Full message, pass on to temp so we can reset the local messageBuffer for the next
// device message
var temp = _messageBuffer.ToArray();
// Reset buffer
_messageBuffer.Clear();
// Now process the full message
response = temp;
if (response.Length >= DeviceMessageLength)
{
var deviceRecord = ExtractDataFromRawByteStream();
return new ZephyrRecord { LungFunction = deviceRecord.LungFunction };
}
return new ZephyrRecord();
}
}
...
public class ZephyrRecord
{
public float LungFunction { get; set; }
}
}
</pre>
<span style="font-family: Arial,Helvetica,sans-serif;">This has proven to work so far, without any known problems, for several different devices that are working very differently, has different manufacturers and are measuring different medical parameters.</span>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-61689841833773842592013-06-02T21:31:00.002+02:002013-06-24T10:02:57.272+02:00Android 4.0.3 and 4.0.4 bluetooth pairing needed on every connection<span style="font-family: Arial,Helvetica,sans-serif;">While developing a cross mobile device app using <a href="http://xamarin.com/">Xamarin</a>, I ran into a strange problem. Behind the cross device abstraction, that was utilized in both the Android and the iOS version of the app, the Android specific code had a problem. The problem manifested it self on one Samsung Galaxy 10.1 tablet running 4.0.3, as well as one running 4.0.4. The problem was not present on the reference "lowest common denominator tablet" (this was also a Samsung Galaxy 10.1 tablet, just running Android 3.2).</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">After some Google searches and not really coming up with anything specific, I found this link which was actually showing me the solution, although I did not know it at the time.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;"><a href="http://stackoverflow.com/questions/6483758/android-programmatically-bluetooth-pairing">Android Programmatically Bluetooth Pairing</a></span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">Our app is using a somewhat modified version of the official Android Bluetooth sample ported to monodroid, it can be found here <a href="https://github.com/xamarin/monodroid-samples/tree/master/BluetoothChat">BluetoothChat</a>.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">The problem showed out to be this line "<a href="https://github.com/xamarin/monodroid-samples/blob/8a3cefb83c2d2d8a23e3f8a2f8e5588342778db8/BluetoothChat/BluetoothChatService.cs#L397">tmp = device.CreateRfcommSocketToServiceRecord (MY_UUID);</a>"</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">changing this to </span><br />
<br />
<pre style="background: white; color: black; font-family: Consolas; font-size: 13;">tmp = device.CreateInsecureRfcommSocketToServiceRecord (MY_UUID);
</pre>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">solved the problem for 3 different devices for me.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">Some more Googling gave me the apparent reason behind this. There seems to be a bug in Android 4.0.3 and 4.0.4+ that is causing the Bluetooth pairing to be forgotten.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">This is documented amongst others on <a href="http://stackoverflow.com/questions/11082819/bluetooth-connection-on-android-ics-not-possible/14070240">Bluetooth connection on Android ICS not possible</a>.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">I spent quite some time trying to figure out what was wrong with the code, being in the mindset "I must be doing something wrong" and being a bit reluctant to start the "blamegame".</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">I hope this might help someone else out there. </span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">Below are a few links that I went through in my hunt for a solution, they are all one way or another corroborating my findings.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;"><a href="https://code.google.com/p/android/issues/detail?id=26041">ICS can't pair BT devices without PIN</a> </span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><a href="https://code.google.com/p/android/issues/detail?id=34161">Bluetooth RFCOMM Server Socket no longer connects properly to embedded device on ICS 4.0.3</a></span>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-51031341153045902192013-05-22T14:58:00.001+02:002013-06-24T10:03:24.405+02:00Runtime performance friendly AngularJs localization / translation using buildscript and Yii Framework + Invalidating the browser cache<span style="font-family: Arial,Helvetica,sans-serif;">Lately I have been working on a new web based <a href="http://en.wikipedia.org/wiki/Single-page_application">single page application</a>. The technologies and frameworks in use are:</span><br />
<ul>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><a href="http://angularjs.org/">AngularJS</a> for creating the <a href="https://en.wikipedia.org/wiki/User_interface">user interface</a>. </span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><a href="http://twitter.github.io/bootstrap/">Twitter Bootstrap</a> as starting point for <a href="https://en.wikipedia.org/wiki/Cascading_Style_Sheets">CSS</a> and to achieve initial <a href="http://en.wikipedia.org/wiki/Responsive_web_design">responsive design</a></span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><a href="http://www.yiiframework.com/">Yii Framework</a> for a <a href="http://en.wikipedia.org/wiki/LAMP_%28software_bundle%29">LAMP</a> based <a href="https://en.wikipedia.org/wiki/Representational_state_transfer">REST</a> service layer</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><a href="http://simplesamlphp.org/">SimpleSAMLphp</a> for federated authentication using <a href="http://en.wikipedia.org/wiki/SAML_2.0">SAML2.0</a> </span></li>
</ul>
<span style="font-family: Arial,Helvetica,sans-serif;">One of the design challenged were - how do we approach webpage localization?</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"></span><br />
<span style="font-family: Arial,Helvetica,sans-serif;">The following requirements were given:</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"></span><br />
<ul>
<li><span style="font-family: Arial,Helvetica,sans-serif;">It should incur as small run time overhead as possible, preferably none</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">The localizable strings should be maintained in text files to allow for external translators, without the overhead of having a full translation system installed and configured</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">The translation system should support <a href="http://www.yiiframework.com/doc/guide/1.1/en/topics.i18n#plural-forms-format">pluralization</a></span></li>
</ul>
<span style="font-family: Arial,Helvetica,sans-serif;"></span><br />
<span style="font-family: Arial,Helvetica,sans-serif;">The solution that were chosen </span><span style="font-family: Arial,Helvetica,sans-serif;">is a deployment/build step implemented in the Yii Framework that both translates and statically renders AngularJS views. The translation engine that is used is also included in the Yii Framework.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">The below image shows a module "logbook" and the views needed for some simple CRUD functionality. This image from the filesystem is from the repository (i.e. before the build script has run).</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<br />
<div class="separator">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTiwpWZFUqKxakKGRoT-HXjtO94e0zKMJ_ZEuKNtB5qA0iv0JIpGu4EelzJT-cgEojPWPYKElxSqYG1FPZu_cU7pL2CLpXnJ5IDppNqrJsDlrQx051wrEo_Vtx1JdCqbwkGn9fzw/s1600/1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTiwpWZFUqKxakKGRoT-HXjtO94e0zKMJ_ZEuKNtB5qA0iv0JIpGu4EelzJT-cgEojPWPYKElxSqYG1FPZu_cU7pL2CLpXnJ5IDppNqrJsDlrQx051wrEo_Vtx1JdCqbwkGn9fzw/s400/1.png" width="363" /></a></div>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">The image below shows how the filesystem looks after the buildscript has run (at the hosting server)</span><br />
<br />
<div class="separator">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgiq39QIzymgHw1eWCUEobBHifCzXASs9I5CtzTfTDLcavJvLPg_MyCckfwao29Hw517KmM-Lpm-pZMH6XfEyrLrdf9xFjAodMLJ-5iPnxYDjU9thpi4lsvm8jh3CSMpBh2QIzODg/s1600/2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="288" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgiq39QIzymgHw1eWCUEobBHifCzXASs9I5CtzTfTDLcavJvLPg_MyCckfwao29Hw517KmM-Lpm-pZMH6XfEyrLrdf9xFjAodMLJ-5iPnxYDjU9thpi4lsvm8jh3CSMpBh2QIzODg/s320/2.png" width="320" /></a></div>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">As we can see the build script has added a directory for each supported language with the <a href="https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes">2 letter language abbreviation </a>according to <a href="https://en.wikipedia.org/wiki/ISO_639-1">ISO 639-1</a>.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">Lets have a look at first the english and then the danish version of the actual view when it is presented on the browser screen.</span><br />
<div class="separator">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIbDahVvKln44247Vg_lIejHDL73vNNh0aKBGr3BwGDwHzRvQ9nFAdXoqIEk9tuZCyagcnZHIaVsYcwweryu_cYK9B0b8JOt4JUkbAcFWPVwWCXIwQRAEKF_tWuh0EL3Diu6NIDQ/s1600/log_edit_en.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="311" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIbDahVvKln44247Vg_lIejHDL73vNNh0aKBGr3BwGDwHzRvQ9nFAdXoqIEk9tuZCyagcnZHIaVsYcwweryu_cYK9B0b8JOt4JUkbAcFWPVwWCXIwQRAEKF_tWuh0EL3Diu6NIDQ/s400/log_edit_en.png" width="400" /></a></div>
<br />
<div class="separator">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgR3EYa74j1MsnxQkkehL9DxWRxFouYxazuPTi1Yi06wFpdxQd9aCPovoc33gR7z0INPNFavUBJDDH_wcxYSD9apoDRrFv6z3grgVakbB16sQhWFQ217A2G_iOa_FA-WycbB5UyAw/s1600/log_edit_da.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="310" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgR3EYa74j1MsnxQkkehL9DxWRxFouYxazuPTi1Yi06wFpdxQd9aCPovoc33gR7z0INPNFavUBJDDH_wcxYSD9apoDRrFv6z3grgVakbB16sQhWFQ217A2G_iOa_FA-WycbB5UyAw/s400/log_edit_da.png" width="400" /></a></div>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">Lets have a look at the html for the angular view.</span><br />
<br />
<pre class="brush: html"><div class="span12" id="logbook_edit" ng-controller="logbook.EditController">
<div id="content">
<h1><?=LogbookModule::t('Log');?></h1>
<div ng-include src="'assets/__REVNO__/app/views/'+language+'/logbook/logbook_edit_toolbar.htm'"></div>
<form id="frm_logbook_edit" data-id="{{log.id}}" class="form-horizontal">
<p class="note"><?=Yii::t('sitewide',"msg_field_with_asterisk_are_required");?></p>
<div class="control-group">
<label class="control-label required" for="logdate"><?=LogbookModule::t('Date');?></label>
<div class="controls">
<input type="text" id="logdate" name="logdate" ng-model="log.logdate" bs-datepicker data-date-format="Yii.user.preferred_date_format.toString().toLowerCase()" />
<span class="help-inline"></span>
</div>
</div>
<div class="control-group">
<label class="control-label required" for="title"><?=LogbookModule::t('Title');?></label>
<div class="controls">
<input type="text" name="title" id="title" ng-model="log.title">
<span class="help-inline"></span>
</div>
</div>
<div class="control-group">
<label class="control-label required" for="mood"><?=LogbookModule::t('Mood');?></label>
<div class="controls">
<div class="btn-group mood-button-group" ng-model="log.mood" data-toggle="buttons-radio" bs-buttons-radio>
<button type="button" value="happy" class="btn happy" style="background-image:url(assets/__REVNO__/app/img/happy-icon.png);"></button>
<button type="button" value="sad" class="btn sad" style="background-image:url(assets/__REVNO__/app/img/sad-icon.png);"></button>
<button type="button" value="angry" class="btn angry" style="background-image:url(assets/__REVNO__/app/img/angry-icon.png);"></button>
<button type="button" value="ill" class="btn ill" style="background-image:url(assets/__REVNO__/app/img/ill-icon.png);"></button>
</div>
<input type="hidden" id="mood" name="mood">
<span class="help-inline"></span>
</div>
</div>
<div class="control-group">
<label class="control-label required" for="note"><?=LogbookModule::t('Note');?></label>
<div class="controls">
<textarea cols="" rows="" id="note" name="note" ng-model="log.note"></textarea>
<span class="help-inline"></span>
</div>
</div>
<div ng-include src="'assets/__REVNO__/app/views/'+language+'/logbook/logbook_edit_toolbar.htm'"></div>
</form>
</div>
</div>
</pre>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">There are a few things that are special in this HTML code</span><br />
<ul>
<li><span style="font-family: Arial,Helvetica,sans-serif;"> "__REVNO__" placeholder which at buildscript run time is replaced by the actual revision identifier from the source code repository. This allows us to optimize the content headers, for maximum browser caching while at the same time ensuring that the end user gets the latest version when we update to a new version on the server. Controlled cache invalidation/busting</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;"> "<?=LogbookModule::t('Note');?>" is the Yii Framework translation method/function <a href="http://www.yiiframework.com/doc/api/1.1/YiiBase#t-detail">t()</a>;</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">the angular scope variable "language" that is part of the logged in users context, this is represented by an </span><span style="font-family: Arial,Helvetica,sans-serif;"><a href="https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes">2 letter language abbreviation</a>.</span></li>
</ul>
<span style="font-family: Arial,Helvetica,sans-serif;">Lets have a look at how English translation string file look (<a href="http://www.yiiframework.com/doc/api/1.1/CPhpMessageSource/">standard Yii Framework messages file</a>)</span><br />
<br />
<pre class="brush: php"><?php
return array(
'Date'=>'Date',
'Title'=>'Title',
'Mood'=>'Mood',
'Note'=>'Note',
'Log'=>'Log',
'Edit'=>'Edit',
'Delete'=>'Delete',
'Create'=>'Create',
'List'=>'List',
'Save'=>'Save',
'Logbook'=>'Logbook',
'Created_by'=>'Created by',
);
</pre>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">and then how the translated view ends up looking after running the buildscript</span><br />
<br />
<pre class="brush: html"><div class="span12" id="logbook_edit" ng-controller="logbook.EditController">
<div id="content">
<h1>Log</h1>
<div ng-include src="'assets/1932/app/views/'+language+'/logbook/logbook_edit_toolbar.htm'"></div>
<form id="frm_logbook_edit" data-id="{{log.id}}" class="form-horizontal">
<p class="note">Fields with <span class="required">*</span> are required.</p>
<div class="control-group">
<label class="control-label required" for="logdate">Date</label>
<div class="controls">
<input type="text" id="logdate" name="logdate" ng-model="log.logdate" bs-datepicker data-date-format="Yii.user.preferred_date_format.toString().toLowerCase()" />
<span class="help-inline"></span>
</div>
</div>
<div class="control-group">
<label class="control-label required" for="title">Title</label>
<div class="controls">
<input type="text" name="title" id="title" ng-model="log.title">
<span class="help-inline"></span>
</div>
</div>
<div class="control-group">
<label class="control-label required" for="mood">Mood</label>
<div class="controls">
<div class="btn-group mood-button-group" ng-model="log.mood" data-toggle="buttons-radio" bs-buttons-radio>
<button type="button" value="happy" class="btn happy" style="background-image:url(assets/1932/app/img/happy-icon.png);"></button>
<button type="button" value="sad" class="btn sad" style="background-image:url(assets/1932/app/img/sad-icon.png);"></button>
<button type="button" value="angry" class="btn angry" style="background-image:url(assets/1932/app/img/angry-icon.png);"></button>
<button type="button" value="ill" class="btn ill" style="background-image:url(assets/1932/app/img/ill-icon.png);"></button>
</div>
<input type="hidden" id="mood" name="mood">
<span class="help-inline"></span>
</div>
</div>
<div class="control-group">
<label class="control-label required" for="note">Note</label>
<div class="controls">
<textarea cols="" rows="" id="note" name="note" ng-model="log.note"></textarea>
<span class="help-inline"></span>
</div>
</div>
<div ng-include src="'assets/1932/app/views/'+language+'/logbook/logbook_edit_toolbar.htm'"></div>
</form>
</div>
</div>
</pre>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">Now on to the actual build script which is implemented as a <a href="http://www.yiiframework.com/doc/api/1.1/CConsoleCommand">Yii CConsoleCommand</a></span><br />
<br />
<ul>
</ul>
<pre class="brush: php"><?php
class BuildReleaseCommand extends CConsoleCommand
{
var $careSupportedLanguages = array ('EN','DA');
var $packageRevisionInSvn;
public function actionBuild($siteDirectory, $subversionRepoUrl='https://your_repo_url_here', $svnCheckout=0)
{
$packageRevisionInSvn = $this->getLatestRevisionNumberInSvn($subversionRepoUrl);
$this->packageRevisionInSvn = $packageRevisionInSvn;
// Delete same revsion folder to replace with newer one
`rm -rf $siteDirectory/assets/$packageRevisionInSvn`;
// Create php file to store revision number, it will be used in all php file where revision number is required
$revisionFileContent = '<?php $revision = '.$packageRevisionInSvn.'; ';
file_put_contents("$siteDirectory/revision.php", $revisionFileContent);
if($svnCheckout){
$this->checkoutLatestVersionFromSvn($subversionRepoUrl, $siteDirectory);
}
$this->insertRevisionNumberInFolderNames($siteDirectory, $packageRevisionInSvn);
// Delete all directory other then en& da in $siteDirectory/assets/{$this->packageRevisionInSvn}/app/views/
`rm -rf $siteDirectory/assets/{$this->packageRevisionInSvn}/app/views/*`;
foreach($this->careSupportedLanguages as $key=>$language) {
Yii::app()->language=strtolower($language);
$this->translateViews($siteDirectory.'/assets/__REVNO__/app/views', $siteDirectory. '/assets/'.$this->packageRevisionInSvn.'/app/views');
}
$this->insertRevisionNumberInViewUrls($siteDirectory.'/assets/'.$packageRevisionInSvn, $packageRevisionInSvn);
}
private function getLatestRevisionNumberInSvn($packageUrl) {
return trim(`svn info --non-interactive --username YOUR_USERNAME --password YOUR_PASSWORD $packageUrl | grep 'Last Changed Rev' | head -1 | grep -Eo "[0-9]+"`);
}
private function checkoutLatestVersionFromSvn($packageUrl, $svnExportDirectory){
$output = `svn export --force --non-interactive --username YOUR_USERNAME --password YOUR_PASSWORD $packageUrl $svnExportDirectory`;
$output = `cd $svnExportDir$package`;
}
/**
* Recurse through the filessystem to process all view html files
* Feature: Translation
*/
private function translateViews($srcPath, $destPath){
$ignoreFiles = array( '.', '..' );
$dh = @opendir( $srcPath );
while( false !== ( $file = readdir( $dh ) ) ){
if( !in_array( $file, $ignoreFiles) ){
if( is_dir( "$srcPath/$file" ) ){
$this->translateViews( "$srcPath/$file", "$destPath/$file" );
} else {
if (preg_match('/^.*\.(htm)$/i',$file)) {
$translatedView = $this->translateView("$srcPath/$file");
$this->saveTranslatedView("$destPath/$file", $translatedView);
}
}
}
}
closedir( $dh );
}
/**
* Actual translation of the html view file
* Feature: Translation
*/
private function translateView($fileFullPath){
$module = $this->getModuleName($fileFullPath);
$controller = new CController($module,new CWebModule($module,null));
return $controller->renderInternal($fileFullPath,null,true);
}
/**
* Save the processed view file under the ISO 639-1 2 letter language code directory
* Feature: Translation
*/
private function saveTranslatedView($fileFullPath, $translatedView) {
$translatedViewFile = str_replace('views','views/'.Yii::app()->language, $fileFullPath);
$saveDir = dirname($translatedViewFile);
if (!file_exists($saveDir)){
mkdir($saveDir,0755, true);
}
file_put_contents($translatedViewFile, $translatedView);
}
/**
* Replace the __REVNO__ placeholder in urls in the Angular views so it points to the correct revision number
* Feature: Cachebusting
*/
private function insertRevisionNumberInViewUrls($svnExportDirectory, $packageRevisionInSvn) {
$output = `find $svnExportDirectory \( -iname "*.htm" -or -iname "*.js" -or -iname "*.php" \) -print | xargs sed -i 's/__REVNO__/$packageRevisionInSvn/g'`;
}
/**
* Copy and rename the __REVNO__ folder to the actual revision number we are parsing
* Feature: Cachebusting
*/
private function insertRevisionNumberInFolderNames($svnExportDirectory, $packageRevisionInSvn) {
$output = `cp -Rf $svnExportDirectory/assets/__REVNO__ $svnExportDirectory/assets/$packageRevisionInSvn`;
}
public function getModuleName($viewPath)
{
$dirs = explode("/", $viewPath);
$passedViews = false;
$module = "";
foreach ($dirs as $key=>$dir)
{
if ($passedViews)
{
$module = $dir;
return $module;
}
if ($dir == "views") {
$passedViews = true;
continue;
}
}
}
}
?>
</pre>
<span style="font-family: Arial,Helvetica,sans-serif;">The script is commented to a fair degree, suffice to say that I am using the internal view renderer in the Yii Framework to do the heavy lifting for the translation, it knows where to find the language string files and to do the translation and pluralization. </span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">I am using the build script together with migration for auto-deployment to our test server like this:</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<br />
<pre class="brush: bash">cd /YOUR_WEBSITE_DIR; svn update --username YOUR_USERNAME --password YOUR_PASSWORD; chown -R apache:apache /YOUR_WEBSITE_DIR; protected/yiic migrate --interactive=0; protected/yiic buildrelease build --siteDirectory=/YOUR_WEBSITE_DIR/</pre>
Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-61256602692172383012013-02-06T21:41:00.001+01:002013-06-24T10:03:38.855+02:00Old HTC Desire phone not booting<span style="font-family: Arial,Helvetica,sans-serif;">I have an old HTC Desire phone that is not able to boot ... again. Last time this happened is about a year ago. Both back then and this time I had to spend the time all over since I could not remember what made it work last time. Thus the rationale for this posting. </span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">What happens is that suddenly the phone just runs out of battery and then it will not boot again.It is stuck on the white HTC screen and you cannot turn it off using the power button. To turn it off you need to pull out the battery.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">See more on a similar or the same problem here: <a href="http://forum.xda-developers.com/showthread.php?t=1634863">Phone stuck on fastboot/hboot, buttons unresponsive</a></span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">Find a available free SD card stick it into another phone and format the SDCard to FAT32 I used Linux for this since I have had problems doing this on windows for some unknown reason.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">On linux you do the following</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<br />
<ol>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Plug in your phone USB cable and wait until your OS recognizes the device</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Change to root user</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">type <span style="font-family: "Courier New",Courier,monospace;">mount <span style="font-family: Arial,Helvetica,sans-serif;">and locate the device name of the SD Card and the directory on which it is mounted (typically something like /dev/sdb1 and /media/SDCARD_ID</span></span>)</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">type <span style="font-family: "Courier New",Courier,monospace;">umount </span> /media/SDCARD_ID</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">now we are going to format the SDCARD (if you have any data on the card you will loose them) type </span><span style="font-family: "Courier New",Courier,monospace;">mkdosfs -I -F32 /dev/YOUR_FOUND_DEVICE_NAME_FROM_STEP_3</span><span style="font-family: Arial,Helvetica,sans-serif;"> <br /><br /><span style="color: orange;"><b>(please yet again note that the device name that I am refering to /dev/sdb1 might refer to a different drive on your system, when you format this drive all data will be deleted! Make sure you are pointing to the correct drive!)</b></span></span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Now you have a SDcard that is ready</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Download the RUU file (officially release ROM from the manufacturer) I am using this one <a href="http://www.shipped-roms.com/download.php?category=android&model=Desire&file=RUU_Bravo_Froyo_HTC_WWE_2.29.405.14_Radio_32.49.00.32U_5.11.05.27_release_224699_signed.exe">RUU_Bravo_Froyo_HTC_WWE_2.29.405.14_Radio_32.49.00.32U_5.11.05.27_release_224699_signed.exe</a></span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">This is the official way to install software using a Windows OS (always make sure that you virus scan exe files downloaded from the internet, and also check the MD5 checksum to make sure that it indeed is the file that you expect it to be).</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">If you phone can boot then you can re-flash on Windows by just running the exe with the phone connected by the USB cable.</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">My phone however is missing the PB99IMG.zip file and the only way that you can reinstall the phone OS is by.</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Start the exe file, and follow the following screenshots<br />
</span></li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpf2huxv9vguDanqpfSkSDtiGUFv7gNi2hBxs9enBIC8d9tZ9saeoobNy9STWn9r5fYAylioICiuihXEVchUDzKyIrueMo20xC2h4_B6-8-iwmk9L2kiP4cqhRfFC5nY9rkqB1Hw/s1600/3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="201" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpf2huxv9vguDanqpfSkSDtiGUFv7gNi2hBxs9enBIC8d9tZ9saeoobNy9STWn9r5fYAylioICiuihXEVchUDzKyIrueMo20xC2h4_B6-8-iwmk9L2kiP4cqhRfFC5nY9rkqB1Hw/s320/3.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCIt5Q_XsMMTi6qCsk2w_y-B8L-yy8BEDzi7kwC1YD7-zkaimz56DgndFhSm8khD3my9uqhdGJKpjb5TlAIu3zoZ1TmkxBYhONqVoUHg4fEba33GJWrY1Fhu8ejF-ckCwC-K_eqw/s1600/4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="201" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCIt5Q_XsMMTi6qCsk2w_y-B8L-yy8BEDzi7kwC1YD7-zkaimz56DgndFhSm8khD3my9uqhdGJKpjb5TlAIu3zoZ1TmkxBYhONqVoUHg4fEba33GJWrY1Fhu8ejF-ckCwC-K_eqw/s320/4.png" width="320" /></a></div>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Now do not go further, download and install the excellent tool <a href="http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx">Process Explorer</a> </span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Start Process Explorer and find out where the temporary files from the RUU...exe file has been extracted.<br />
</span></li>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3d2ZQdV-Rqo-SRjEko8nKdexXQFeVHlZNayhT_pITmi3wPd4_Rapz92Lq-Pf11kZcdloMS5ZArUPKd2dq2l7_ZLY2i7jvwkIX60gF84ct1ZX8eKDKRs2T8t5_MVU4D9fNbXcbOw/s1600/5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3d2ZQdV-Rqo-SRjEko8nKdexXQFeVHlZNayhT_pITmi3wPd4_Rapz92Lq-Pf11kZcdloMS5ZArUPKd2dq2l7_ZLY2i7jvwkIX60gF84ct1ZX8eKDKRs2T8t5_MVU4D9fNbXcbOw/s320/5.png" width="320" /></a></div>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Select the exe process that you have just started</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Make sure that the lower process details pane is open</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Find the directory that contains Disk1. In either this directory or one of the most recent directories in the indicated temp folder look for a file named ROM.zip</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">When you find it, copy it to the newly formatted SDCard</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Rename the file to fit you specific image (it differs from phone model to phone model. Mine is called (please note uppercase letters) PB99IMG.zip</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Eject the spare phone</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Turn it off and take out the SDCard. Put it into the broken phone</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Plug in the broken phone to a charger or to the PC USB slot</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Turn on the broken phone while pressing volume down button and then pressing the power button until you get to the HBOOT screen</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Just leave the phone and after a while it will show loading image</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">It will now ask you a number of times, if you want to flash the image and if you want to reboot the phone.</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">After rebooting you phone you should be good to go</span></li>
</ol>
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-2656475851993593282012-11-26T15:25:00.001+01:002013-06-24T10:04:31.603+02:00High performance cachable websites web 2.0 in YiiFramework, mustache.js and icanhaz.js<h2>
<span style="font-family: Arial,Helvetica,sans-serif;">Preface: </span></h2>
<span style="font-family: Arial,Helvetica,sans-serif;">To be able to cache webpage content as much as possible then it is important to delay merging data with the HTML markup until the last possible moment (which is in the browser). The moment you merge data with HTML then the page is always specific to the context in which it was retrieved. If this context happens to be a user context (which is almost always the case), then you cannot use this cached page for another user, which is bad for cacheability. </span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">Achieving this "late-merging" of data and markup can be achieved by building the skeleton of the site in very few html pages. Most normal sites could potentially be built using the following skeleton pages:</span><br />
<br />
<ul>
<li><span style="font-family: Arial,Helvetica,sans-serif;">frontpage/landingpage </span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">login </span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">content </span></li>
</ul>
<span style="font-family: Arial,Helvetica,sans-serif;">Of course the more the pages differ from each other the more of these skeleton pages you will have. The point is to isolate the common things between the pages and boil it down to the minimum amount of pages. </span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">These pages (one for each distinct different buildup of the page) will contain a link to a javascript file that contains all the clientside templates/markup needed to render data. These clientside templates, can be built in one of the many templating js engines out there, for instance:</span><br />
<br />
<ul>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><a href="https://github.com/linkedin/dustjs">dust.js</a> </span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><a href="http://handlebarsjs.com/">handlebars.js</a> </span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;"><a href="http://mustache.github.com/">mustache.js</a> </span></li>
</ul>
<span style="font-family: Arial,Helvetica,sans-serif;">(LinkedIn's review of quite a few of these template engines <a href="http://engineering.linkedin.com/frontend/client-side-templating-throwdown-mustache-handlebars-dustjs-and-more">The client-side templating throwdown: mustache, handlebars, dust.js, and more</a>). </span><br />
<br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">This way we can cache the clientside needed markup used to display lists and forms (since it is a static javascript file) using <a href="http://nginx.org/">nginx</a> or other caching servers. The skeleton pages can also be cached since they contain no data. The only thing remaining is CSS, javascript, images, videos and data. Everything apart from data can also be cached using caching servers. Data we will retrieve using <a href="http://en.wikipedia.org/wiki/Ajax_%28programming%29">AJAX</a> requests as json and then rendering it clientside. </span><br />
<br />
<h2>
<span style="font-family: Arial,Helvetica,sans-serif;">YiiFramework: </span></h2>
<span style="font-family: Arial,Helvetica,sans-serif;">I am new to the <a href="http://www.yiiframework.com/">Yii framework</a>, but needed to build a <a href="http://en.wikipedia.org/wiki/Web_2.0">web 2.0</a> site that should support caching of as much of the content as possible to avoid overloading the webserver re-sending the same markup over and over again just with different data embedded. As far as I could see there was not really any built-in support in Yii to do this (apart from doing it all manually). The exising <a href="http://www.yiiframework.com/extension/mustache/">Mustache extention</a> seemed to be related to server side templating rather than clientside templating.
For this reason I created my own extension. </span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">The goals were: </span><br />
<ol>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Creating views in Yii clientside should be as similar as possible as creating server side views </span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Templates should be accessible on the clientside without a lot of plumbing code </span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">Rendering templates should be clean. </span></li>
</ol>
<span style="font-family: Arial,Helvetica,sans-serif;"></span><br />
<span style="font-family: Arial,Helvetica,sans-serif;">In the following I will try to argue for my decisions on how to solve the above.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">- Creating views in Yii clientside should be as similar as possible as creating server side views <br />& </span><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: Arial,Helvetica,sans-serif;">-Templates should be accessible on the clientside without a lot of plumbing code</span> </span>
<br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">I wanted to be able to put my clientside views in what folder I though made the most sense, and since the clientside templates very much are related to the controller actions that are delivering the data (just like server side views) I wanted to be able to put them for instance in the view folders.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;">I wanted it to be easy to edit the templates in an IDE and one template should be self contained and not mixed up with the other templates</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">The templates should not incur a significant serverside overhead (currently I am still working on a better solution than the one I have found - mentioned at the end of the posting).</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">To achieve the above I decided that in the IDE the client side templates should be seperate files, and to be able to distinguish them from the server side templates then needed another name that supported the js templating engine that I choose (mustache.js), so the file extension ended up being .tpl (since Smarty templates are supported in my IDE).</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">So the challenge is how to convert disparate files on the filesystem into a single js file that is read and initialized by the browser automatically.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">I need go through the filesystem </span><span style="font-family: Arial,Helvetica,sans-serif;"><span style="font-family: Arial,Helvetica,sans-serif;">recursively </span>and look for files of a certain kind (*.mustache.tpl), index them by they location in the filesystem and their filename. Push them info a javascript array and add the code needed to initialize the templates when the browser loads the template javascript file.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">All the above is solved in the following Yii component, which also adds the needed javascripts for the template rendering clientside.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;"></span><br />
<pre class="brush: php"><?php
/**
* ClientsideViews class file.
* @author Kenneth Thorman (kenneth.thorman@appinux.com)
* @license http://www.opensource.org/licenses/bsd-license.php New BSD License
*/
/**
* ClientsideViews application component.
* Used for registering ClientsideViews core functionality.
*/
class ClientsideViews extends CApplicationComponent {
/**
* @var boolean whether to register jQuery and the ClientsideViews JavaScript.
*/
public $enableJS = true;
protected $_assetsUrl;
protected $_assetsPath;
/**
* Initializes the component.
*/
public function init( ) {
if( !Yii::getPathOfAlias( 'clientsideviews' ) )
Yii::setPathOfAlias( 'clientsideviews', realpath( dirname( __FILE__ ).'/..' ) );
$generatedTemplateFile = Yii::getPathOfAlias( 'clientsideviews.assets.javascripts' ) . DIRECTORY_SEPARATOR . 'mustache.tpl.js';
if (!is_file($generatedTemplateFile)) {
$this->refreshMustacheTemplates();
}
if( $this->enableJS ) {
$this->registerJs( );
}
}
/**
* Registers the core JavaScript plugins.
* @since 0.9.8
*/
public function registerJs( ) {
Yii::app( )->clientScript->registerCoreScript( 'jquery' );
$this->registerScriptFile( 'ICanHaz.min.js' );
$this->registerScriptFile( 'mustache.tpl.js' );
}
/**
* Registers a JavaScript file in the assets folder.
* @param string $fileName the file name.
* @param integer $position the position of the JavaScript file.
*/
public function registerScriptFile( $fileName, $position = CClientScript::POS_END ) {
Yii::app( )->clientScript->registerScriptFile( $this->getAssetsUrl( ).DIRECTORY_SEPARATOR.$fileName, $position );
}
/**
* Returns the URL to the published assets folder.
* @return string the URL
*/
protected function getAssetsUrl( ) {
if( $this->_assetsUrl == null ) {
$assetsPath = Yii::getPathOfAlias( 'clientsideviews.assets.javascripts' );
$this->_assetsUrl = Yii::app( )->assetManager->publish( $assetsPath, false, -1, YII_DEBUG );
}
return $this->_assetsUrl;
}
public function refreshMustacheTemplates()
{
/*
Find all files recursivly in the basepath/protected named mustache.tpl
Foreach files add to js array with a name based on the directory path and filename without
mustache.tpl
*/
$basePath = Yii::app()->basePath;
$templates = array();
$options= array('fileTypes'=>array('tpl'));
$templateFiles = CFileHelper::findFiles(realpath(Yii::app()->basePath),$options);
foreach($templateFiles as $file){
// stupid additional check due to the findFiles function cannot handle . seperated filenames
if (strpos($file,'mustache') !== false) {
$templateId = str_replace(array($basePath,DIRECTORY_SEPARATOR,'mustache.tpl','.'),array('','_','',''),$file);
array_push($templates, array(
'name' => $templateId,
'template' => $this->stripEndLine($this->readTemplate($file)))
);
}
}
$templatesJs = "$.each(".json_encode($templates).", function (index, template) {ich.addTemplate(template.name, template.template);});";
$this->writeTemplateFile(Yii::getPathOfAlias( 'clientsideviews.assets.javascripts' ), $templatesJs);
}
private function writeTemplateFile($path,$fileContents)
{
$my_file = $path. DIRECTORY_SEPARATOR . 'mustache.tpl.js';
$handle = fopen($my_file, 'w') or die('Cannot open file: '.$my_file);
fwrite($handle, $fileContents);
fclose($handle);
}
private function readTemplate($file)
{
$handle = fopen($file, 'r');
$data = fread($handle,filesize($file));
fclose($handle);
return $data;
}
private function stripEndLine($template)
{
$output = str_replace(array("\r\n", "\r"), "\n", $template);
$lines = explode("\n", $output);
$new_lines = array();
foreach ($lines as $i => $line) {
if(!empty($line))
$new_lines[] = trim($line);
}
return implode($new_lines);
}
}</pre>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">By initilizing the <a href="http://icanhazjs.com/">ICanHaz</a> javascript Mustache template wrapper in the mustache.tpl.js file then the templates are ready for use in the browser without manually having to register the templates.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;"></span><br />
<pre class="brush: javascript">$.each(
[
{"name":"_views_meeting_meetingList",
"template":"<table class='responsive'><th>Meeting ID<\/th><th>Meeting Name<\/th><th>Create Time<\/th><th>Running<\/th>{{#meetings}}<tr><td>{{meetingID}}<\/td><td>{{meetingName}}<\/td><td>{{createTime}}<\/td><td>{{running}}<\/td><\/tr>{{\/meetings}}<\/table>"}
],
function (index, template) {ich.addTemplate(template.name, template.template);}
);</pre>
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<br />
<h3>
<span style="font-family: Arial,Helvetica,sans-serif;">3. Rendering templates should be clean.</span></h3>
<span style="font-family: Arial,Helvetica,sans-serif;">What now remains is the actual code that is rendering data from a controller call with the clientside template.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">The controller that is returning some data in json format</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"></span><br />
<pre class="brush:php">public function actionMeetingList()
{
...
//show all meetings
$meetings=$bbb->getMeetings();
echo json_encode($meetings) ;
}</pre>
<span style="font-family: Arial,Helvetica,sans-serif;">
</span>
<br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">And finally the clientside code responsible for the merging of data and the clientside templates.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;"></span><br />
<pre class="brush:javascript"><div id="meeting_list"></div>
<script type="text/javascript">
$(document).ready(function () {
/*
Getting data and rendering in template
*/
$.getJSON('/meeting/meetinglist', function (meetings) {
renderedTemplate = ich._views_meeting_meetingList({'meetings': meetings});
$('#user_list').append(renderedTemplate);
});
});
</script></pre>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">A final note, currently to avoid a full traversal of the filesystem on every request, I have added a check in the function init() in the ClientsideViews Yii component. This checks if the file exists on the file system. This is a tradeoff between my limited knowledge of Yii and how to better implement this in the framework, avoiding traversing the filesystem (very slow) for each request and manually being able to trigger a refresh by deleting the file located under /assets/javascripts/mustache.tpl.js. I am sure someone with more knowledge of the framework know the right way to hook this up to automate the refreshing when the files change. This however works for my purpose.
I have created a github repository that is available at <a href="https://github.com/kenneththorman/yiiclientsideviews">Yii ClientsideViews extension</a>. I have attempted to create a new extension on the YiiFramework website, but was not able to since apparently I am "too new" on the site.
</span>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com2tag:blogger.com,1999:blog-21633272.post-31584944617343226422012-07-04T07:52:00.000+02:002013-06-24T10:04:50.916+02:00Pentaho CE ReportViewer Localization is not localizing<div style="font-family: Arial,Helvetica,sans-serif;">
When creating reports and publishing them under Pentaho it would be nice if you could localize the web interface just like for dashboards etc...</div>
<div style="font-family: Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: Arial,Helvetica,sans-serif;">
This showed out not to be as simple as expected.</div>
<div style="font-family: Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: Arial,Helvetica,sans-serif;">
I spent some time searching Google and then I posted the following forum post at <a href="http://forums.pentaho.com/showthread.php?97199-How-to-translate-the-remaining-labels-on-the-ReportViewer">How to translate the remaining labels on the ReportViewer</a>. I was pretty busy for a few days and then I went back and searched Google some more and it seemed like I missed a pretty obvious link <a href="http://infocenter.pentaho.com/help/index.jsp?topic=%2Fcustomizing_pentaho_guide%2Ftask_reportviewer_localization.html">Report Viewer Localization</a>. After following this only just doing the steps on the pentaho-reporting-engine-classic-core-platform-plugin-4.1.0-stable.jar file some of the labels were translated.</div>
<div style="font-family: Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: Arial,Helvetica,sans-serif;">
All was however not well since there was quite a few labels and strings that could be localized still. Searching inside all the jar files for specific strings led me to belive that I needed to translate some messages.properties files in the pentaho-reporting-engine-classic-core-3.8.3-GA.jar file as well.</div>
<div style="font-family: Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: Arial,Helvetica,sans-serif;">
This task was a bit larger than the previous jar file so I was pretty satisfied when it was time to upload the file to the server and test it. My woes were not over yet though, and nothing I did seemed to work. I contacted some knowledgeable Pentaho developers at <a href="http://www.sigmainfo.net/">Sigma Infosolutions</a> and asked them if they could have a look at the problem and it showed out that a small code change was required in the Pentaho code to make the localization work.</div>
<div style="font-family: Arial,Helvetica,sans-serif;">
<br /></div>
<pre class="brush: diff">Index: Messages.java
===================================================================
--- Messages.java (revision 662)
+++ Messages.java (working copy)
@@ -1,36 +1,38 @@
-// Decompiled by DJ v3.12.12.96 Copyright 2011 Atanas Neshkov Date: 7/2/2012 5:25:55 PM
-// Home Page: http://members.fortunecity.com/neshkov/dj.html http://www.neshkov.com/dj.html - Check often for new version!
-// Decompiler options: packimports(3)
-// Source File Name: Messages.java
-
-package org.pentaho.reporting.engine.classic.core.parameters;
-
-import java.util.Locale;
-
-import org.pentaho.reporting.libraries.base.util.ObjectUtilities;
-import org.pentaho.reporting.libraries.base.util.ResourceBundleSupport;
-
-public class Messages extends ResourceBundleSupport
-{
- private static Messages instance;
-
- public static Messages getInstance()
- {
- // its ok that this one is not synchronized. I dont care whether we have multiple instances of this
- // beast sitting around, as this is a singleton for convinience reasons.
- if (instance == null)
- {
- instance = new Messages();
- }
- return instance;
- }
-
- /**
- * Creates a new instance.
- */
- private Messages()
- {
- super(Locale.getDefault(), "org.pentaho.reporting.engine.classic.core.parameters.messages",
- ObjectUtilities.getClassLoader(Messages.class));
- }
-}
+// Decompiled by DJ v3.12.12.96 Copyright 2011 Atanas Neshkov Date: 7/2/2012 5:25:55 PM
+// Home Page: http://members.fortunecity.com/neshkov/dj.html http://www.neshkov.com/dj.html - Check often for new version!
+// Decompiler options: packimports(3)
+// Source File Name: Messages.java
+
+package org.pentaho.reporting.engine.classic.core.parameters;
+
+import org.pentaho.platform.util.messages.LocaleHelper;
+
+import org.pentaho.reporting.libraries.base.util.ObjectUtilities;
+import org.pentaho.reporting.libraries.base.util.ResourceBundleSupport;
+
+public class Messages extends ResourceBundleSupport
+{
+ private static Messages instance;
+
+ public static Messages getInstance()
+ {
+ // its ok that this one is not synchronized. I dont care whether we have multiple instances of this
+ // beast sitting around, as this is a singleton for convinience reasons.
+ /*Changed the singleton Instance to the non-singleton Instance by removing the if(instance=null) condition */
+ //if (instance == null)
+ //{
+ instance = new Messages();
+ //}
+ return instance;
+ }
+
+ /**
+ * Creates a new instance.
+ */
+ private Messages()
+ {
+ /*Changed the Locale.getDefault() to LocaleHelper.getLocale() in order to localize the parameters according to the selected locale*/
+ super(LocaleHelper.getLocale(), "org.pentaho.reporting.engine.classic.core.parameters.messages",
+ ObjectUtilities.getClassLoader(Messages.class));
+ }
+}</pre>
<div style="font-family: Arial,Helvetica,sans-serif;">
<br /></div>
Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-77487800735408065382012-06-14T17:25:00.000+02:002013-06-24T10:05:05.765+02:00Upgrading Redmine from 1.2.2 to 2.0.2<div style="font-family: Arial,Helvetica,sans-serif;">
I needed to upgrade a Redmine installation that was pretty old to the newest release 2.0.2. I was following this document on how to upgrade Redmine <a href="http://www.redmine.org/projects/redmine/wiki/RedmineUpgrade">http://www.redmine.org/projects/redmine/wiki/RedmineUpgrade</a>. When I got to <a href="http://www.redmine.org/projects/redmine/wiki/RedmineUpgrade#Step-4-Update-the-database">Step 4 - Update the database</a></div>
<div style="font-family: Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: Arial,Helvetica,sans-serif;">
I received the following error</div>
<br />
<pre class="brush: bash">root@xxx #rake db:migrate RAILS_ENV=production --trace
** Invoke db:migrate (first_time)
** Invoke environment (first_time)
** Execute environment
rake aborted!
no such file to load -- dispatcher
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/activesupport-3.2.5/lib/active_support/dependencies.rb:251:in `require'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/activesupport-3.2.5/lib/active_support/dependencies.rb:251:in `block in require'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/activesupport-3.2.5/lib/active_support/dependencies.rb:236:in `load_dependency'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/activesupport-3.2.5/lib/active_support/dependencies.rb:251:in `require'
/vol/www/sites/redmine2.appinux.com/plugins/redmine_scm/init.rb:2:in `<top (required)>'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/activesupport-3.2.5/lib/active_support/dependencies.rb:251:in `require'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/activesupport-3.2.5/lib/active_support/dependencies.rb:251:in `block in require'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/activesupport-3.2.5/lib/active_support/dependencies.rb:236:in `load_dependency'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/activesupport-3.2.5/lib/active_support/dependencies.rb:251:in `require'
/vol/www/sites/redmine2.appinux.com/lib/redmine/plugin.rb:129:in `block in load'
/vol/www/sites/redmine2.appinux.com/lib/redmine/plugin.rb:120:in `each'
/vol/www/sites/redmine2.appinux.com/lib/redmine/plugin.rb:120:in `load'
/vol/www/sites/redmine2.appinux.com/config/initializers/30-redmine.rb:13:in `<top (required)>'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/activesupport-3.2.5/lib/active_support/dependencies.rb:245:in `load'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/activesupport-3.2.5/lib/active_support/dependencies.rb:245:in `block in load'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/activesupport-3.2.5/lib/active_support/dependencies.rb:236:in `load_dependency'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/activesupport-3.2.5/lib/active_support/dependencies.rb:245:in `load'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/railties-3.2.5/lib/rails/engine.rb:588:in `block (2 levels) in <class:Engine>'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/railties-3.2.5/lib/rails/engine.rb:587:in `each'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/railties-3.2.5/lib/rails/engine.rb:587:in `block in <class:Engine>'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/railties-3.2.5/lib/rails/initializable.rb:30:in `instance_exec'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/railties-3.2.5/lib/rails/initializable.rb:30:in `run'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/railties-3.2.5/lib/rails/initializable.rb:55:in `block in run_initializers'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/railties-3.2.5/lib/rails/initializable.rb:54:in `each'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/railties-3.2.5/lib/rails/initializable.rb:54:in `run_initializers'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/railties-3.2.5/lib/rails/application.rb:136:in `initialize!'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/railties-3.2.5/lib/rails/railtie/configurable.rb:30:in `method_missing'
/vol/www/sites/redmine2.appinux.com/config/environment.rb:14:in `<top (required)>'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/railties-3.2.5/lib/rails/application.rb:103:in `require'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/railties-3.2.5/lib/rails/application.rb:103:in `require_environment!'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/railties-3.2.5/lib/rails/application.rb:292:in `block (2 levels) in initialize_tasks'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/task.rb:205:in `call'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/task.rb:205:in `block in execute'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/task.rb:200:in `each'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/task.rb:200:in `execute'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/task.rb:158:in `block in invoke_with_call_chain'
/usr/local/rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/task.rb:151:in `invoke_with_call_chain'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/task.rb:176:in `block in invoke_prerequisites'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/task.rb:174:in `each'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/task.rb:174:in `invoke_prerequisites'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/task.rb:157:in `block in invoke_with_call_chain'
/usr/local/rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/task.rb:151:in `invoke_with_call_chain'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/task.rb:144:in `invoke'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/application.rb:116:in `invoke_task'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `block (2 levels) in top_level'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `each'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `block in top_level'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/application.rb:133:in `standard_exception_handling'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/application.rb:88:in `top_level'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/application.rb:66:in `block in run'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/application.rb:133:in `standard_exception_handling'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/lib/rake/application.rb:63:in `run'
/usr/local/rvm/gems/ruby-1.9.2-p180/gems/rake-0.9.2.2/bin/rake:33:in `<top (required)>'
/usr/local/rvm/gems/ruby-1.9.2-p180/bin/rake:19:in `load'
/usr/local/rvm/gems/ruby-1.9.2-p180/bin/rake:19:in `<main>'
Tasks: TOP => db:migrate => environment </pre>
<span style="font-family: Arial,Helvetica,sans-serif;">This was solved by removing the redmine_scm plugin that I copied to the plugin folder in <a href="http://www.redmine.org/projects/redmine/wiki/RedmineUpgrade#Option-1-Downloaded-release-targz-or-zip-file">Step 3 - Perform the upgrade - Option 1 - Downloaded release (tar.gz or zip file)</a></span>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com0tag:blogger.com,1999:blog-21633272.post-17207622851668758942012-06-04T23:22:00.000+02:002012-06-04T23:24:03.602+02:00SugarCrm 6.3.1: Enhancing Importer - allow importing of related module data<span style="font-family: Arial,Helvetica,sans-serif;">Often when importing data into <a href="http://www.sugarcrm.com/">SugarCrm </a>the built-in importer at a module level is working great. However if you have a need to import data that is actually modeled in SugarCrm as being 2 modules (tables) with a <a href="http://en.wikipedia.org/wiki/Many-to-many_%28data_model%29">many-many</a> relationship in between then the built-in importer comes up short.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">The full patch can be found at the bottom of this posting. This is a non upgradesafe change. Apply at your own risk. </span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">Let us assume that we have the following <a href="http://en.wikipedia.org/wiki/Entity-Relationship_Model">entity relational model</a> in SugarCrm that has been configured in SugarCrm Studio or in SugarCrm Module Builder.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCLTBfck-ccp3DmqZdli517ma0ws81wELJlexyxwOsNRscHiXjI07JyZcrqE6a0pGE4IKdpT_7mIERFWanjfULnVaG0_jwIPbleVH_vqlUlk6I5uYMnzEyQ4j9ONXCbtHSSdqJOQ/s1600/0_dataModel.png" imageanchor="1" style="margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCLTBfck-ccp3DmqZdli517ma0ws81wELJlexyxwOsNRscHiXjI07JyZcrqE6a0pGE4IKdpT_7mIERFWanjfULnVaG0_jwIPbleVH_vqlUlk6I5uYMnzEyQ4j9ONXCbtHSSdqJOQ/s1600/0_dataModel.png" /></a></div>
<br />
<blockquote class="tr_bq" style="font-family: Arial,Helvetica,sans-serif;">
<h2>
Module Builder</h2>
<div class="grid_7 alpha omega">
The <a href="http://www.sugarcrm.com/feature/platform#modulebuilder">Sugar Module Builder</a> enables users to build custom modules from
scratch or combine existing or custom objects into a brand new CRM
module. Developers can leverage existing Sugar Objects such as Contacts,
Accounts, Documents, Cases, and Opportunities to build a new module or
create their own custom objects from scratch to form a new module. Users
can build an unlimited number of custom modules, which interoperate
seamlessly with Reporting, Workflow, and Sugar Studio tools. Upgrades
for custom modules are fully supported. Building new modules allows
developers to extend Sugar beyond the typical CRM functions and optimize
Sugar for any xRM (any Relationship Management) function.<br />
<h3>
Positive Impacts</h3>
<ul>
<li>Create custom modules to track information critical to your business</li>
<li>Use pre-defined objects or create custom objects for the new module</li>
<li>Share or charge for custom objects on SugarForge and Sugar Exchange.</li>
</ul>
</div>
</blockquote>
<br />
<blockquote class="tr_bq">
<h2 style="font-family: Arial,Helvetica,sans-serif;">
Sugar Studio</h2>
<div class="grid_7 alpha omega" style="font-family: Arial,Helvetica,sans-serif;">
<a href="http://www.sugarcrm.com/feature/platform#sugarstudio">Sugar Studio</a> is the starting place for an administrator to
configure the way information is presented in Sugar. Administrators can
use Sugar Studio to create and add custom fields, hide fields that are
not relevant, and use the extensive customization capabilities of Sugar
Logic to create calculated, dependent, and related fields. Sugar Studio
is a very simple but powerful WYSIWYG interface that administrators and
developers use to configure Sugar to complement a company’s existing
business processes.<br />
<h3>
Positive Impacts</h3>
<ul>
<li> Rearrange the order of fields according to your company’s requirements</li>
<li> Hide fields that are not relevant to your business process</li>
<li> Create and add new custom fields</li>
<li> Calculate variable values based on the value of other fields like opportunity amount or expected close date</li>
<li> Present fields only when necessary using dependent fields</li>
</ul>
</div>
</blockquote>
<br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">If <a href="http://support.sugarcrm.com/04_Find_Answers/02KB/01Getting_Started/Importing_Made_Easy">importing</a> has been enabled for the Skill and the Language module when these modules were created then we can import Skill or Languages as standalone data. The Contact module is a built-in SugarCrm module and it has importing enabled but can also only import contact standalone data. </span><br />
<br />
<div style="font-family: Arial,Helvetica,sans-serif;">
<br />
<br />
<br />
There are a few challenges importing related data into SugarCrm:</div>
<ol style="font-family: Arial,Helvetica,sans-serif;">
<li>We need to handle relationships and splitting data into the correct modules.</li>
<li>We need to handle the fact that the id column might not contain the actual id of the data but that another column might do instead. </li>
</ol>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">Without the 2 challenges solved we will not be able to import the below data the way I would prefer.</span><br />
<br />
<div class="separator" style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiN2LyaqHDjesScG4a9YUbTkgCRvIbU39Ao6YOJaEVQdaML3Ske0qNnUdGAgumopk-wM_ooDo6G2xyue3qE5ryp0I9VJkBoVkbo-ssHvLi0aAhVDUI2dXnrWHCansaWYGpQNGFFkA/s1600/3_dataNoId.png" style="margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiN2LyaqHDjesScG4a9YUbTkgCRvIbU39Ao6YOJaEVQdaML3Ske0qNnUdGAgumopk-wM_ooDo6G2xyue3qE5ryp0I9VJkBoVkbo-ssHvLi0aAhVDUI2dXnrWHCansaWYGpQNGFFkA/s1600/3_dataNoId.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtpQwdOjJFBwqkvcgDYS9lgDb6BwzQdyVp5ct5tSWbXjFTIbGNxA4Vg49IJL47qK2ymtSJcbD6SZrJ5QmefmxQryoO0joZc8xyqSpt_inxaabGSSFdkfXPd1Sv7vnZf0U-YBtwew/s1600/1_csvData.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"></a></div>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">What we really want in the database after importing the above data is. </span><br />
<ul>
<li><span style="font-family: Arial,Helvetica,sans-serif;">2 contacts: John and Jane</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">3 skills: English literature, European history and PHP</span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">4 languages: English, German, Danish and Swedish </span></li>
<li><span style="font-family: Arial,Helvetica,sans-serif;">as well as the many-many relations needed so we can recreate the data pictured above</span></li>
</ul>
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">The 2 above mentioned challenges have been solved the following ways:</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">#1: In the importer step 3 we have added all the "main module's" (Contacts in this case) related modules' fields in the module field dropdown. This way you can map any data you want into a related module's field. </span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">#2: A primary key field selector has been added on Step 2 in the import if you have selected "Create new records and update existing records" on Step 1 in the importer. </span><br />
<br />
<br />
<br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">Lets look at the proposed solution.</span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">Step 1: the importer has not been modified visually and looks as below.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"></span><br />
<div class="separator" style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYmVgpXpXwEegSfUxAanzAGIEG6RhZSMHct8xGUAkPwnkirxaI51fct0pENSxxtqiIZ9wFlO58ZmrXdjImXJcFnmz-JFTCgM7U5VrqTNBrpPJVU9UMPmPDAF7XKcoJf7SyH-rlqQ/s1600/2_importStep1.png" imageanchor="1" style="margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYmVgpXpXwEegSfUxAanzAGIEG6RhZSMHct8xGUAkPwnkirxaI51fct0pENSxxtqiIZ9wFlO58ZmrXdjImXJcFnmz-JFTCgM7U5VrqTNBrpPJVU9UMPmPDAF7XKcoJf7SyH-rlqQ/s1600/2_importStep1.png" /></a></div>
<span style="font-family: Arial,Helvetica,sans-serif;"> (Please note the support for
saving an import configuration is working with the new importer functionality).</span><br />
<br />
<br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">Step 2: We have added a new field that allows you to select the field that is to be used as "id". It defaults to the standard importer behavior which is using the id column.</span><br />
<div class="separator" style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnOAmVOGeJdsZQTS23olC-eqmoQhXAXb0M-draPsAdRsGpo-snjinqHbzm1PYNbaSPkVJDtn9nJIRWWyseZSKZR52aBYKGROVqWJCnyyoj_S92q1tuGm1ZRiXHXDfYGq8EFazspQ/s1600/4_importStep2.png" imageanchor="1" style="margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnOAmVOGeJdsZQTS23olC-eqmoQhXAXb0M-draPsAdRsGpo-snjinqHbzm1PYNbaSPkVJDtn9nJIRWWyseZSKZR52aBYKGROVqWJCnyyoj_S92q1tuGm1ZRiXHXDfYGq8EFazspQ/s1600/4_importStep2.png" /></a></div>
<br />
<br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">Step 3: We have added a marker for the field used as primary id ( purple box below ). Please note the text in the dropdown marked with red boxes. The field selected in the dropdown is prefixex with a module name (<modulename>.<fieldname>) this indicates that it is a field from a related module you want to map to.</span><br />
<div class="separator" style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqFfYifiviHAStm5mS9kq73Q2KlS-Hx0DVPE8u70uSDORbXF9V64HVYVQdi8ImYiuSWbbSB6zgG1lakjBF5tRKJ9GLsRq-Z5YKjvS827p5rAAf7G8NYnF9chFXZJj8AdT2ZRUPJQ/s1600/5_importStep3.png" imageanchor="1" style="margin-bottom: 1em; margin-right: 1em;"><img border="0" height="281" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqFfYifiviHAStm5mS9kq73Q2KlS-Hx0DVPE8u70uSDORbXF9V64HVYVQdi8ImYiuSWbbSB6zgG1lakjBF5tRKJ9GLsRq-Z5YKjvS827p5rAAf7G8NYnF9chFXZJj8AdT2ZRUPJQ/s640/5_importStep3.png" width="640" /></a></div>
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">As you can see from the screenshot below, the module field dropdown now contains all fields from the main module (Contacts) (main module fields are not prefixed with the module name), as well as the related modules fields (skill_Skill, and lang_Language, these fields are prefixed with the module name).</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<br />
<div class="separator" style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjp-GuKFksU0owA__lxPmukkUUV0y4dnyLw-B2HnDgWuabWV7KDBnkblNZutZJ5xhRZ22Fnm7p7Evynq3aDnENLppKoZRLS-fyBRVzFIFgsj_C7BVzHy0W6fAE7Az5mlUWTsdiK3A/s1600/6_dropdownValues.png" imageanchor="1" style="margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjp-GuKFksU0owA__lxPmukkUUV0y4dnyLw-B2HnDgWuabWV7KDBnkblNZutZJ5xhRZ22Fnm7p7Evynq3aDnENLppKoZRLS-fyBRVzFIFgsj_C7BVzHy0W6fAE7Az5mlUWTsdiK3A/s1600/6_dropdownValues.png" /></a></div>
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span>
<span style="font-family: Arial,Helvetica,sans-serif;">Note: Do not map any value to a related module's "ID" field. This is handled behind the scenes.</span><br />
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span><br />
<span style="font-family: Arial,Helvetica,sans-serif;">Step 4: No visual changes has happened here</span><br />
<div class="separator" style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvAeD3MrElMZxdIOUdOCn6z_VfeHSZdboXqkslkdNjBG2zRNS4aETjAuwO8ihwjIfm9aSTwAtqUIGnkWIyM9jhdKSr4HeptrQXYs6XEts4JXN6NZtTp6WbX91RJlQOPPESkKozpw/s1600/7_step4.png" imageanchor="1" style="margin-bottom: 1em; margin-right: 1em;"><img border="0" height="435" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvAeD3MrElMZxdIOUdOCn6z_VfeHSZdboXqkslkdNjBG2zRNS4aETjAuwO8ihwjIfm9aSTwAtqUIGnkWIyM9jhdKSr4HeptrQXYs6XEts4JXN6NZtTp6WbX91RJlQOPPESkKozpw/s640/7_step4.png" width="640" /></a></div>
<span style="font-family: Arial,Helvetica,sans-serif;"><br /></span><br />
<br />
<span style="font-family: Arial,Helvetica,sans-serif;">There are quite a few code changes to add this new functionality into the importer and you can see the Subversion patch file below.</span><br />
<pre class="brush: diff">Index: include/database/DBHelper.php
===================================================================
--- include/database/DBHelper.php (revision 73)
+++ include/database/DBHelper.php (working copy)
@@ -1185,8 +1185,8 @@
$before_value=(float)$bean->fetched_row[$field];
$after_value=(float)$bean->$field;
} else {
- $before_value=$bean->fetched_row[$field];
- $after_value=$bean->$field;
+ $before_value=$bean->fetched_row[$field];
+ $after_value=@$bean->$field;
}
//Because of bug #25078(sqlserver haven't 'date' type, trim extra "00:00:00" when insert into *_cstm table). so when we read the audit datetime field from sqlserver, we have to replace the extra "00:00:00" again.
Index: modules/Import/Importer.php
===================================================================
--- modules/Import/Importer.php (revision 73)
+++ modules/Import/Importer.php (working copy)
@@ -59,6 +59,7 @@
*/
private $importColumns;
+ private $importRelatedColumns;
/**
* @var importSource
*/
@@ -104,9 +105,11 @@
//Get the default user currency
$this->defaultUserCurrency = new Currency();
$this->defaultUserCurrency->retrieve('-99');
-
+
//Get our import column definitions
$this->importColumns = $this->getImportColumns();
+ $this->importRelatedColumns = $this->getRelatedImportColumns();
+
$this->isUpdateOnly = ( isset($_REQUEST['import_type']) && $_REQUEST['import_type'] == 'update' );
}
@@ -132,7 +135,7 @@
protected function importRow($row)
{
global $sugar_config, $mod_strings;
-
+ global $beanList,$beanFiles;
$focus = clone $this->bean;
$focus->unPopulateDefaultValues();
$focus->save_from_post = false;
@@ -140,6 +143,11 @@
$this->ifs->createdBeans = array();
$this->importSource->resetRowErrorCounter();
$do_save = true;
+
+ $primary_key = (isset($_REQUEST['primary_key']) && !empty($_REQUEST['primary_key'])) ? $_REQUEST['primary_key'] : false;
+ if(!$primary_key){
+ $primary_key = 'id';
+ }
for ( $fieldNum = 0; $fieldNum < $_REQUEST['columncount']; $fieldNum++ )
{
@@ -322,46 +330,58 @@
return;
}
}
-
+
+ $focus_array = array();
+ $newRecord_array = array();
+
// if the id was specified
$newRecord = true;
- if ( !empty($focus->id) )
+ if ( !empty($focus->$primary_key) )
{
- $focus->id = $this->_convertId($focus->id);
-
+ if($primary_key=='id')
+ $focus->id = $this->_convertId($focus->$primary_key);
// check if it already exists
- $query = "SELECT * FROM {$focus->table_name} WHERE id='".$focus->db->quote($focus->id)."'";
+ $join = '';
+ $_def = $focus->getFieldDefinition($primary_key);
+ if($_def['source'] == 'custom_fields') $join = "JOIN {$focus->table_name}_cstm ON id_c = id";
+ $query = "SELECT * FROM {$focus->table_name} $join WHERE $primary_key='".$focus->db->quote($focus->$primary_key)."'";
$result = $focus->db->query($query)
or sugar_die("Error selecting sugarbean: ");
- $dbrow = $focus->db->fetchByAssoc($result);
+ while($dbrow = $focus->db->fetchByAssoc($result)){
- if (isset ($dbrow['id']) && $dbrow['id'] != -1)
+ if (isset ($dbrow[$primary_key]) && $dbrow[$primary_key] != -1)
{
- // if it exists but was deleted, just remove it
- if (isset ($dbrow['deleted']) && $dbrow['deleted'] == 1 && $this->isUpdateOnly ==false)
+ $focus->id = $dbrow['id'];
+ // if it exists but was deleted, just remove it //skiping deleted rows
+ if (isset ($dbrow['deleted']) && $dbrow['deleted'] == 1 && $this->isUpdateOnly == false /*&& $primary_key=='id'*/)
{
$this->removeDeletedBean($focus);
$focus->new_with_id = true;
}
else
{
+ //skiping deleted rows
+ if(isset ($dbrow['deleted']) && $dbrow['deleted'] == 1 ) continue;
if( ! $this->isUpdateOnly )
{
- $this->importSource->writeError($mod_strings['LBL_ID_EXISTS_ALREADY'],'ID',$focus->id);
+ $this->importSource->writeError($mod_strings['LBL_ID_EXISTS_ALREADY'],$primary_key,$focus->$primary_key);
$this->_undoCreatedBeans($this->ifs->createdBeans);
return;
}
-
+
$clonedBean = $this->cloneExistingBean($focus);
if($clonedBean === FALSE)
{
- $this->importSource->writeError($mod_strings['LBL_RECORD_CANNOT_BE_UPDATED'],'ID',$focus->id);
+ $this->importSource->writeError($mod_strings['LBL_RECORD_CANNOT_BE_UPDATED'],$primary_key,$focus->$primary_key);
$this->_undoCreatedBeans($this->ifs->createdBeans);
return;
}
else
{
+ $focus_array[] = clone $clonedBean;
+ $newRecord_array[] = FALSE;
+
$focus = $clonedBean;
$newRecord = FALSE;
}
@@ -369,19 +389,100 @@
}
else
{
+ if($focus->id){
$focus->new_with_id = true;
+ }else{
+ $focus->new_with_id = false;
+ $focus_array[] = $this->cloneExistingBean($focus);
+ $newRecord_array[] = true;
}
}
-
+ }
+ }
+ if(count($focus_array)==0){
+ $focus_array[] = $focus;
+ $newRecord_array[] = $newRecord;
+ }
+ for($focus_i=0,$focus_count=count($focus_array);$focus_i<$focus_count;$focus_i++){
+ $focus = $focus_array[$focus_i];
+ $newRecord = $newRecord_array[$focus_i];
if ($do_save)
{
$this->saveImportBean($focus, $newRecord);
+ $mainModuleId = $focus->id;
+ //Save records for related module
+ foreach($this->importRelatedColumns as $k=>$v)
+ {
+ $exploedFields = explode("$",$v);
+ $relatedModule[] = $exploedFields[0];
+ $relatedModuleField[$exploedFields[0]][] = $exploedFields[1];
+ }
+ if(isset($relatedModule) && is_array($relatedModule))
+ $relatedModules = array_unique($relatedModule);
+ if(isset($relatedModule) && is_array($relatedModule))
+ foreach($relatedModuleField as $modName => $fldsArr)
+ {
+ $haveValue = 0;
+ $relatedModuleFile = $beanList[$modName];
+ include_once($beanFiles[$relatedModuleFile]);
+ $relatedModuleObj = new $relatedModuleFile();
+ $whereArr = array();
+ $is_custom = false;
+ foreach($fldsArr as $k=>$fldName)
+ {
+ $fld = $modName."$".$fldName;
+ $fieldNo = array_search($fld,$this->importRelatedColumns);
+ if($row[$fieldNo]!="")
+ {
+ $haveValue++;
+ $relatedModuleObj->$fldName = $row[$fieldNo];
+ $whereArr[] = $fldName ."='". $row[$fieldNo]."'";
+ if(@$relatedModuleObj->field_defs[$fldName]['source'] == 'custom_fields') $is_custom = true;
+ }
+ }
+ if($haveValue)
+ {
+ $where = "";
+ $where = implode(" AND ",$whereArr);
+ $sql = "select id from ".$relatedModuleObj->table_name;
+ if($is_custom) $sql .= ' inner join '.$relatedModuleObj->table_name.'_cstm on id=id_c';
+ $sql .= " where ".$where." and deleted=0";
+ $result2 = $relatedModuleObj->db->query($sql);
+ $rowQuery = $relatedModuleObj->db->fetchByAssoc($result2);
+ if(count($rowQuery)==0 || empty($rowQuery))
+ {
+ $this->saveImportBean($relatedModuleObj, $newRecord);
+ $relatedId = $relatedModuleObj->id;
+ }
+ else
+ {
+ $relatedId = $rowQuery['id'];
+ }
+ $rel_table = strtolower($relatedModuleObj->table_name);
+ $foc_table = strtolower($focus->table_name);
+ if(strlen($rel_table)>16) $rel_table = substr($rel_table,0,11).substr($rel_table,-5);
+ if(strlen($foc_table)>16) $foc_table = substr($foc_table,0,11).substr($foc_table,-5);
+ $rel_field = $foc_table.'_'.$rel_table;
+ if(!$focus->load_relationship($rel_field)){
+ $rel_field = $rel_table.'_'.$foc_table;
+ if(!$focus->load_relationship($rel_field)){
+ $GLOBALS['log']->error("SugarBean.load_relationships, relationship not found.");
+ $rel_field = false;
+ }
+ }
+ if($rel_field){
+ $focus->$rel_field->add($relatedId,array());
+ }
+ $focus->save();
+ }
+ }
+
// Update the created/updated counter
$this->importSource->markRowAsImported($newRecord);
}
else
$this->_undoCreatedBeans($this->ifs->createdBeans);
-
+ }
unset($defaultRowValue);
}
@@ -544,11 +645,15 @@
$firstrow = unserialize(base64_decode($_REQUEST['firstrow']));
$mappingValsArr = $this->importColumns;
- $mapping_file = new ImportMap();
+ $mappingRelatedArr = $this->importRelatedColumns;
+ $mappArr = $mappingRelatedArr+$mappingValsArr;
+ $mapping_file = new ImportMap();
+
+
if ( isset($_REQUEST['has_header']) && $_REQUEST['has_header'] == 'on')
{
$header_to_field = array ();
- foreach ($this->importColumns as $pos => $field_name)
+ foreach ($mappArr as $pos => $field_name)
{
if (isset($firstrow[$pos]) && isset($field_name))
{
@@ -557,6 +662,8 @@
}
$mappingValsArr = $header_to_field;
+ } else{
+ $mappingValsArr = $mappArr;
}
//get array of values to save for duplicate and locale settings
$advMapping = $this->retrieveAdvancedMapping();
@@ -679,7 +786,26 @@
return $importColumns;
}
+ protected function getRelatedImportColumns()
+ {
+ $importRelatedColumns = array();
+ foreach ($_REQUEST as $name => $value)
+ {
+ // only look for var names that start with "fieldNum"
+ if (strncasecmp($name, "colnum_", 7) != 0)
+ continue;
+ // pull out the column position for this field name
+ $pos = substr($name, 7);
+
+ if ( strpos($value,"$") )
+ {
+ // now mark that we've seen this field
+ $importRelatedColumns[$pos] = $value;
+ }
+ }
+ return $importRelatedColumns;
+ }
protected function getFieldSanitizer()
{
$ifs = new ImportFieldSanitize();
Index: modules/Import/tpls/confirm.tpl
===================================================================
--- modules/Import/tpls/confirm.tpl (revision 73)
+++ modules/Import/tpls/confirm.tpl (working copy)
@@ -170,8 +170,30 @@
</tr>
<tr>
<td colspan="2"><div class="hr" style="margin-top: 0px;"></div></td>
+ </tr>
+ {if $TYPE != 'import'}
+ <tr>
+ <td colspan="2"><h3>Select primaty field&nbsp;{sugar_help text="It will update the row with the field equal to the value of the csv row"}</h3></td>
</tr>
<tr>
+ <td colspan="2">
+ <select name="primary_key">
+ {foreach from=$SAMPLE_ROWS item=row name=row}
+ {foreach from=$row item=value}
+ {if $smarty.foreach.row.first}
+ {if $HAS_HEADER}
+ <option value="{$value}"{if $value|lower == 'id'} selected='selected' {/if}>{$value}</option>
+ {/if}
+ {/if}
+ {/foreach}
+ {/foreach}
+ </select>
+ </td>
+ </tr>
+ {else}
+ <input name="primary_key" value="" type="hidden" />
+ {/if}
+ <tr>
<td colspan="2"><h3>{$MOD.LBL_THIRD_PARTY_CSV_SOURCES}&nbsp;{sugar_help text=$MOD.LBL_THIRD_PARTY_CSV_SOURCES_HELP}</h3></td>
</tr>
<tr>
Index: modules/Import/tpls/dupcheck.tpl
===================================================================
--- modules/Import/tpls/dupcheck.tpl (revision 73)
+++ modules/Import/tpls/dupcheck.tpl (working copy)
@@ -84,6 +84,7 @@
<input type="hidden" id="enabled_dupes" name="enabled_dupes" value="">
<input type="hidden" id="disabled_dupes" name="disabled_dupes" value="">
<input type="hidden" id="current_step" name="current_step" value="{$CURRENT_STEP}">
+<input type="hidden" name="primary_key" value="{$primary_field}">
<div class="hr"></div>
<div>
Index: modules/Import/tpls/step3.tpl
===================================================================
--- modules/Import/tpls/step3.tpl (revision 73)
+++ modules/Import/tpls/step3.tpl (working copy)
@@ -118,10 +118,34 @@
{/if}
<tr>
{if $HAS_HEADER == 'on'}
- <td id="row_{$smarty.foreach.rows.index}_header">{$item.cell1}</td>
+ <td id="row_{$smarty.foreach.rows.index}_header">
+ {if $primary_field==$item.cell1}
+ <b>{$item.cell1}
+ <small>*Primary Field</small>
+ </b>
+ <input type="hidden" name="primary_key" value="{$primary_field}" id="primary_key" />
+ <script>
+ {literal}
+ (function(obj, evType, fn){
+ if (obj.addEventListener){
+ obj.addEventListener(evType, fn, false);
+ return true;
+ } else if (obj.attachEvent){
+ var r = obj.attachEvent("on"+evType, fn);
+ return r;
+ } else {
+ return false;
+ }
+ })(window,'load',function(){document.getElementById('primary_key').value=document.getElementById('primary_key_list').value})
+ {/literal}
+ </script>
+ {else}
+ {$item.cell1}
{/if}
+ </td>
+ {/if}
<td valign="top" align="left" id="row_{$smarty.foreach.rows.index}_col_0">
- <select class='fixedwidth' name="colnum_{$smarty.foreach.rows.index}">
+ <select class='fixedwidth' name="colnum_{$smarty.foreach.rows.index}"{if $primary_field==$item.cell1} id="primary_key_list" onblur="document.getElementById('primary_key').value = this.value" {/if}>
<option value="-1">{$MOD.LBL_DONT_MAP}</option>
{$item.field_choices}
</select>
Index: modules/Import/views/view.confirm.php
===================================================================
--- modules/Import/views/view.confirm.php (revision 73)
+++ modules/Import/views/view.confirm.php (working copy)
@@ -63,7 +63,7 @@
global $sugar_config, $locale;
$this->ss->assign("IMPORT_MODULE", $_REQUEST['import_module']);
- $this->ss->assign("TYPE",( !empty($_REQUEST['type']) ? $_REQUEST['type'] : "import" ));
+ $this->ss->assign("TYPE",( !empty($_REQUEST['type']) ? $_REQUEST['type'] : ( !empty($_REQUEST['import_type'])?$_REQUEST['import_type'] : "import") ));
$this->ss->assign("SOURCE_ID", $_REQUEST['source_id']);
$this->instruction = 'LBL_SELECT_PROPERTY_INSTRUCTION';
Index: modules/Import/views/view.dupcheck.php
===================================================================
--- modules/Import/views/view.dupcheck.php (revision 73)
+++ modules/Import/views/view.dupcheck.php (working copy)
@@ -73,6 +73,7 @@
$this->ss->assign("IMPORT_MODULE", $_REQUEST['import_module']);
$this->ss->assign("CURRENT_STEP", $this->currentStep);
$this->ss->assign("JAVASCRIPT", $this->_getJS());
+ $this->ss->assign("primary_field", @$_REQUEST['primary_key'] ? $_REQUEST['primary_key'] : 'id' );
$content = $this->ss->fetch('modules/Import/tpls/dupcheck.tpl');
$this->ss->assign("CONTENT", $content);
Index: modules/Import/views/view.last.php
===================================================================
--- modules/Import/views/view.last.php (revision 73)
+++ modules/Import/views/view.last.php (working copy)
@@ -109,6 +109,8 @@
$this->ss->assign("errorrecordsFile",ImportCacheFiles::getErrorRecordsWithoutErrorFileName());
$this->ss->assign("dupeFile",ImportCacheFiles::getDuplicateFileName());
+ $this->ss->assign("primary_field", @$_REQUEST['primary_key'] ? $_REQUEST['primary_key'] : 'id' );
+
if ( $this->bean->object_name == "Prospect" )
{
$this->ss->assign("PROSPECTLISTBUTTON", $this->_addToProspectListButton());
Index: modules/Import/views/view.step3.php
===================================================================
--- modules/Import/views/view.step3.php (revision 73)
+++ modules/Import/views/view.step3.php (working copy)
@@ -63,7 +63,7 @@
public function display()
{
global $mod_strings, $app_strings, $current_user, $sugar_config, $app_list_strings, $locale;
-
+ global $beanList,$beanFiles;
$this->ss->assign("IMPORT_MODULE", $_REQUEST['import_module']);
$has_header = ( isset( $_REQUEST['has_header']) ? 1 : 0 );
$sugar_config['import_max_records_per_file'] = ( empty($sugar_config['import_max_records_per_file']) ? 1000 : $sugar_config['import_max_records_per_file'] );
@@ -200,7 +200,91 @@
if (!empty($importColumns)) {
$column_sel_from_req = true;
}
-
+ foreach($this->bean->field_defs as $field)
+ {
+ if(0 == strcmp($field['type'],'link') && (!empty($field['module']) || !empty($field['relationship'])))
+ {
+ if(!empty($field['module']))
+ {
+ $related_module = $field['module'];
+ }
+ elseif(!empty($field['relationship']))
+ {
+ require_once("data/Relationships/RelationshipFactory.php");
+ $test = SugarRelationshipFactory::getInstance()->getRelationship($field['relationship']);
+ if($test->def['relationships'][$field['relationship']]['lhs_module'] == $this->bean->module_dir)
+ {
+ $relatedMods[] = $test->def['relationships'][$field['relationship']]['rhs_module'];
+ }
+ else
+ {
+ $relatedMods[] = $test->def['relationships'][$field['relationship']]['lhs_module'];
+ }
+ }
+ }
+ }
+ //add extra related fields
+ foreach($relatedMods as $key=>$relatedModule)
+ {
+ if(array_key_exists($relatedModule,$beanList))
+ {
+ $relatedModuleFile = $beanList[$relatedModule];
+ include_once($beanFiles[$relatedModuleFile]);
+ $relatedModuleObj = new $relatedModuleFile();
+ $fieldsRel = $relatedModuleObj->get_importable_fields();
+ $relModuleStrings = return_module_language($current_language, $relatedModuleObj->module_dir);
+ foreach ( $fieldsRel as $fieldname => $properties ) {
+ if($properties['relationship'])
+ continue;
+ // get field name
+ if (!empty($relModuleStrings['LBL_EXPORT_'.strtoupper($fieldname)]) )
+ {
+ $displayname = str_replace(":","", $relModuleStrings['LBL_EXPORT_'.strtoupper($fieldname)] );
+ }
+ else if (!empty ($properties['vname']))
+ {
+ $displayname = str_replace(":","",translate($properties['vname'] ,$relatedModuleObj->module_dir));
+ }
+ else
+ $displayname = str_replace(":","",translate($properties['name'] ,$relatedModuleObj->module_dir));
+
+ $selected = '';
+ if ($column_sel_from_req && isset($importColumns[$field_count])) {
+ if ($fieldname == $importColumns[$field_count]) {
+ //$selected = ' selected="selected" ';
+ $defaultField = $fieldname;
+ $mappedFields[] = $fieldname;
+ }
+ } else {
+ if ( !empty($defaultValue) && !in_array($fieldname,$mappedFields)
+ && !in_array($fieldname,$ignored_fields) )
+ {
+ if ( strtolower($fieldname) == strtolower($defaultValue)
+ || strtolower($fieldname) == str_replace(" ","_",strtolower($defaultValue))
+ || strtolower($displayname) == strtolower($defaultValue)
+ || strtolower($displayname) == str_replace(" ","_",strtolower($defaultValue)) )
+ {
+ //$selected = ' selected="selected" ';
+ $defaultField = $fieldname;
+ $mappedFields[] = $fieldname;
+ }
+ }
+ }
+ // get field type information
+ $fieldtype = '';
+ if ( isset($properties['type'])
+ && isset($mod_strings['LBL_IMPORT_FIELDDEF_' . strtoupper($properties['type'])]) )
+ $fieldtype = ' [' . $mod_strings['LBL_IMPORT_FIELDDEF_' . strtoupper($properties['type'])] . '] ';
+ if ( isset($properties['comment']) )
+ $fieldtype .= ' - ' . $properties['comment'];
+
+ $optionsRelated[$relatedModuleObj->module_dir.'$'.$fieldname] = '<option value="'.$relatedModuleObj->module_dir.'$'.$fieldname.'" title="'. $displayname . htmlentities($fieldtype) . '"'
+ . $selected . $req_class . '>'. $relatedModuleObj->module_dir.".". $displayname . '</option>\n';
+ }
+
+ }
+ }
+
for($field_count = 0; $field_count < $ret_field_count; $field_count++) {
// See if we have any field map matches
$defaultValue = "";
@@ -225,7 +309,17 @@
$defaultField = '';
global $current_language;
$moduleStrings = return_module_language($current_language, $this->bean->module_dir);
+
+ $related_options_clone = $optionsRelated;
+ if(
+ isset($firstrow_name) &&
+ isset($field_map[$firstrow_name]) &&
+ isset($related_options_clone[$field_map[$firstrow_name]])
+ ){
+ $related_options_clone[$field_map[$firstrow_name]] = str_replace('<option ','<option selected="selected" ',$related_options_clone[$field_map[$firstrow_name]]);
+ }
+
foreach ( $fields as $fieldname => $properties ) {
// get field name
if (!empty($moduleStrings['LBL_EXPORT_'.strtoupper($fieldname)]) )
@@ -278,7 +372,6 @@
$options[$displayname.$fieldname] = '<option value="'.$fieldname.'" title="'. $displayname . htmlentities($fieldtype) . '"'
. $selected . $req_class . '>' . $displayname . $req_mark . '</option>\n';
}
-
// get default field value
$defaultFieldHTML = '';
if ( !empty($defaultField) ) {
@@ -306,6 +399,11 @@
$cellOneData = isset($rows[0][$field_count]) ? $rows[0][$field_count] : '';
$cellTwoData = isset($rows[1][$field_count]) ? $rows[1][$field_count] : '';
$cellThreeData = isset($rows[2][$field_count]) ? $rows[2][$field_count] : '';
+
+ if(count($optionsRelated)>0 && @$_REQUEST['primary_key'] != $firstrow_name)
+ {
+ $options = array_merge($options, $related_options_clone);
+ }
$columns[] = array(
'field_choices' => implode('',$options),
'default_field' => $defaultFieldHTML,
@@ -382,6 +480,7 @@
$this->ss->assign("COLUMNCOUNT",$ret_field_count);
$this->ss->assign("rows",$columns);
+ $this->ss->assign("primary_field", @$_REQUEST['primary_key'] ? $_REQUEST['primary_key'] : 'id' );
$this->ss->assign('datetimeformat', $GLOBALS['timedate']->get_cal_date_time_format());
@@ -768,4 +867,4 @@
EOJAVASCRIPT;
}
-}
+}
\ No newline at end of file</pre>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com10tag:blogger.com,1999:blog-21633272.post-56708633566912729962012-01-24T14:02:00.000+01:002012-01-24T15:34:25.326+01:00Backing up SugarCrm and MySql<div style="font-family: Arial,Helvetica,sans-serif;">
When running <a href="http://www.sugarcrm.com/">SugarCrm</a> in a production environment, you need to do backup. There are many opinions about how to do backup:
</div>
<blockquote style="font-family: Arial,Helvetica,sans-serif;">
Only wimps use tape backup: real men just upload their important stuff on ftp, and let the rest of the world mirror it ;)
<br />
<br />
<i>(Torvalds, Linus (1996-07-20). Post. linux.dev.kernel newsgroup. Google Groups. Retrieved on 2006-08-28.)
</i></blockquote>
<div style="font-family: Arial,Helvetica,sans-serif;">
However the above might not be what your boss want, so we need to schedule backup.<br />
<br />
Backing up SugarCrm is basically 2 separate operations.<br />
<ol>
<li>zipping up your working site directory structure</li>
<li>dumping the associated SugarCrm database</li>
</ol>
On a Linux system #1 can be done with the <a href="http://en.wikipedia.org/wiki/Tar_%28file_format%29">tar</a> command (or any other file packer of preference).</div>
<pre class="brush: bash">tar -czf SITE_DIRECTORY.tar.gz SITE_DIRECTORY</pre>
<div style="font-family: Arial,Helvetica,sans-serif;">
<br />
For #2 it depends on the database that you are using. In the remaining part of this posting I will be using <a href="http://en.wikipedia.org/wiki/MySQL">MySQL</a> and <a href="http://en.wikipedia.org/wiki/InnoDB">InnoDB</a> as <a href="http://en.wikipedia.org/wiki/Database_engine">storage engine</a>.<br />
<br />
There are numerous scopes for backing up a database: <br />
<ul>
<li><a href="http://en.wikipedia.org/wiki/Backup">full backup</a></li>
<li><a href="http://en.wikipedia.org/wiki/Incremental_backup">incremental backup</a></li>
<li>block level backup</li>
</ul>
<br />
Normally I prefer the full backup option combined with a block level backup plan for incremental backup. <br />
<br />
Simple full backup can be controlled using <a href="http://sourceforge.net/projects/automysqlbackup/">AutoMySQLBackup</a>. There are however additional parameters that I like to add to the mysqldump command used inside the script. If you are using any kinds of functions/stored procedures in the database, then you want to add <a href="http://dev.mysql.com/doc/refman/5.0/en/mysqldump.html#option_mysqldump_triggers">--triggers</a> <a href="http://dev.mysql.com/doc/refman/5.0/en/mysqldump.html#option_mysqldump_routines">--routines</a> to the <a href="http://dev.mysql.com/doc/refman/5.0/en/mysqldump.html">mysqldump </a>command. I also like to use the <a href="http://dev.mysql.com/doc/refman/5.0/en/mysqldump.html#option_mysqldump_single-transaction">--single-transaction</a> option on InnoDB storage engine (this is also mentioned <a href="http://sourceforge.net/tracker/index.php?func=detail&aid=3252077&group_id=101066&atid=628967">here</a> as an upcoming feature for AutoMySQLBackup).<br />
</div>
<div style="font-family: Arial,Helvetica,sans-serif;">
</div>
<div style="font-family: Arial,Helvetica,sans-serif;">
The above mentioned options/parameters will change the lines in the script from </div>
<div style="font-family: Arial,Helvetica,sans-serif;">
</div>
<div style="font-family: Arial,Helvetica,sans-serif;">
<br />
mysqldump --user=XXX -- password=YYY ... </div>
<div style="font-family: Arial,Helvetica,sans-serif;">
</div>
<div style="font-family: Arial,Helvetica,sans-serif;">
<br />
to </div>
<div style="font-family: Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: Arial,Helvetica,sans-serif;">
mysqldump --user=XXX -- password=YYY --triggers --routines --single-transaction ...</div>
<div style="font-family: Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: Arial,Helvetica,sans-serif;">
For incremental backup please see the <a href="http://dev.mysql.com/doc/mysql-enterprise-backup/3.5/en/backup.incremental.html">ibbackup</a> command or the <a href="http://dev.mysql.com/doc/mysql-enterprise-backup/3.5/en/mysqlbackup.incremental.html">mysqlbackup</a> command.</div>
<div style="font-family: Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: Arial,Helvetica,sans-serif;">
The majority of our servers are running on <a href="http://aws.amazon.com/ec2/">Amazon EC2</a>. We use a combination of running full backups nightly as well as <a href="http://aws.amazon.com/ebs/">Amazon Elastic Block Store (EBS)</a> (block level backup) snapshots through the day. This requires a slightly special setup with regards to your filesystem of choice. You can read more at <a href="http://alestic.com/2009/09/ec2-consistent-snapshot">Creating Consistent EBS Snapshots with MySQL and XFS on EC2</a> or if you prefer CentOS then you can have a look at my blog posting <a href="http://kenneththorman.blogspot.com/2011/09/setting-up-centos-56-with-php-53-on.html">Setting up Centos 5.6 with PHP 5.3 on Amazon EC2 with ec2-consistent-snapshot</a>.</div>Kenneth Thormanhttp://www.blogger.com/profile/17647032459684296822noreply@blogger.com1