Showing posts with label Mono. Show all posts
Showing posts with label Mono. Show all posts
Mono on Android: State of the Union
Having mostly finished up my surprisingly successful ROM Manager and ClockworkMod Recovery, I finally turned my attentions back to androidmono. Though Miguel recently announced the MonoDroid project, there's still room for a open source/hobbyist version. :) (And I need something to work on when I am bored)
When I put the project down a few months ago, I had very rough bindings to the Java API. I recently dumped the approach I took (albeit very cool), in favor of using jni4net. jni4net is a similar project that allows embedding a JVM into Microsoft's CLR, or vice versa. It only took a little bit of tweaking to make it work on Dalvik with Mono (I also hacked off a lot of unnecessary stuff that jni4net provides). So the following file:
Gives you this activity:
(Note: the code above is being displayed from my github repo and may change and leave the image out of date.)
I hacked up a Java/C# project to reflect on the Android SDK jar and create a C# android project that binds to the Java APIs via JNI. So, how it sort of works:
There's still a lot of work left to do: MonoDevelop template and plugin, a tool to create Java dopplegangers for each C# object via reflection, soft debugger support, translating Java's "OnSomeEvent" interfaces into true C# events/delegates, convenence extension methods, support for constructors (a Java constructor can not be "native"), and more. I wouldn't even say this is in Alpha stages yet, but it *works*.
Android.dll
Picture above represents the results of 3 sleepless nights. No, I didn’t rewrite the Android framework. I generated a bunch of proxy classes to access them from Mono though. Rough idea of how it works:
- Java application reflects through Android framework SDK jar and outputs an XML file that represents the object model.
- C# application generates thousands of code files of said object model. (This is similar to jni4net, but that doesn’t work on anything but Windows).
- Tag all the methods with the [MethodImpl(MethodImplOptions.InternalCall)] attribute.
- Implement the method calls at runtime with a combination of dynamically created expression tree delegates and mono_add_internal_call.
The class contents actually have no code, so the total dll size is pretty small (800k). Here’s what the LinearLayout “implementation” looks like:
namespace android.widget
{
public class LinearLayout : android.view.ViewGroup
{
public class LayoutParams : android.view.ViewGroup.MarginLayoutParams
{
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public virtual extern java.lang.String debug(java.lang.String arg0);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public extern LayoutParams(android.content.Context arg0, android.util.AttributeSet arg1);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public extern LayoutParams(int arg0, int arg1);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public extern LayoutParams(int arg0, int arg1, float arg2);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public extern LayoutParams(android.view.ViewGroup.LayoutParams arg0);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public extern LayoutParams(android.view.ViewGroup.MarginLayoutParams arg0);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
internal extern LayoutParams();
}
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public virtual extern void setGravity(int arg0);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
protected override extern void onLayout(bool arg0, int arg1, int arg2, int arg3, int arg4);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public override extern int getBaseline();
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
protected override extern void onMeasure(int arg0, int arg1);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
protected override extern bool checkLayoutParams(android.view.ViewGroup.LayoutParams arg0);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public virtual extern new android.widget.LinearLayout.LayoutParams generateLayoutParams(android.util.AttributeSet arg0);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
protected virtual extern new android.widget.LinearLayout.LayoutParams generateLayoutParams(android.view.ViewGroup.LayoutParams arg0);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
protected virtual extern new android.widget.LinearLayout.LayoutParams generateDefaultLayoutParams();
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public virtual extern bool isBaselineAligned();
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public virtual extern void setBaselineAligned(bool arg0);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public virtual extern int getBaselineAlignedChildIndex();
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public virtual extern void setBaselineAlignedChildIndex(int arg0);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public virtual extern float getWeightSum();
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public virtual extern void setWeightSum(float arg0);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public virtual extern void setOrientation(int arg0);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public virtual extern int getOrientation();
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public virtual extern void setHorizontalGravity(int arg0);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public virtual extern void setVerticalGravity(int arg0);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public extern LinearLayout(android.content.Context arg0, android.util.AttributeSet arg1);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
public extern LinearLayout(android.content.Context arg0);
[global::System.Runtime.CompilerServices.MethodImpl(global::System.Runtime.CompilerServices.MethodImplOptions.InternalCall)]
internal extern LinearLayout();
}
}
A year of on and off hacking coming to fruition soon…
Building Mono using the Android NDK
I finally got around to looking at the Android NDK yesterday. It is basically a subset of the Android Source build environment, but it has all the bits necessary to compile Mono. It only took a bit of finagling with the Android.mk to make everything work right. Building for Android is pretty easy now; the longest part is actually downloading and building the mcs assemblies. There’s a script to handle that though.
The androidmono repository has updated instructions.
Mono 2.5 for Android Released
On and off over the last few weeks, I’ve been working on getting the Mono 2.6 branch, and its super awesome soft debugger, working on Android. I’ve also spent some time documenting and cleaning up the currently horrendous build process involved (I was really the only one that knew how to do it). Anyways, the Android Market now has the current 2.6 branch of Mono available for download (2.6 is not complete yet, hence the reason I am referring to it at 2.5). This release supports Cupcake, Donut, and Eclair!
The project is hosted on Github in the androidmono repository.
For those interested in building/hacking this themselves, here are the instructions:
Developing and Debugging .NET applications on Android with Mono(Develop)
Lots of cool stuff has been happening in Mono recently! First MonoTouch was released, which allows the development of iPhone applications in C#/MonoDevelop. The catch was that there was no debugger support. To address this glaring problem, Mono released created and released a Soft-Mode Debugger, which is basically an in-process debugger that lives in the runtime. After peeking at the code, I realized that porting this to make it work on Android would be a cinch! The result: you can now create a console application in MonoDevelop, deploy it to an Android emulator or device, and debug:
I blogged a while ago that I was working on an interop layer for Mono to allow it to utilize the Android SDK. This is mostly complete. Once I write up a tool to generate convenient wrapper classes, writing full blown applications for Android will become a reality!
Android’s Linker makes Baby Jesus Cry
release-1.0 branch:
static int open_library(const char *name)
{
int fd;
char buf[512];
const char **path;
TRACE("[ %5d opening %s ]\n", pid, name);
if(strlen(name) > 256) return -1;
if(name == 0) return -1;
fd = open(name, O_RDONLY);
if(fd != -1) return fd;
for(path = sopaths; *path; path++){
sprintf(buf,"%s/%s", *path, name);
fd = open(buf, O_RDONLY);
if(fd != -1) return fd;
}
return -1;
}
cupcake branch:
/* TODO: Need to add support for initializing the so search path with
* LD_LIBRARY_PATH env variable for non-setuid programs. */
static int open_library(const char *name)
{
int fd;
char buf[512];
const char **path;
TRACE("[ %5d opening %s ]\n", pid, name);
if(name == 0) return -1;
if(strlen(name) > 256) return -1;
if ((name[0] == '/') && ((fd = _open_lib(name)) >= 0))
return fd;
for (path = sopaths; *path; path++) {
snprintf(buf, sizeof(buf), "%s/%s", *path, name);
if ((fd = _open_lib(buf)) >= 0)
return fd;
}
return -1;
}
Summary: Android’s linker used to look in the current working directory to resolve library references. Now it doesn’t. (And it never used LD_LIBRARY_PATH at all) This is really only annoying for 3rd party command line applications which have references to libraries that aren’t part of the standard Android build (and contained in /system/lib). I need to figure out how to make gcc, et al, create binaries that store the full path to the libraries they reference, rather than just the base file name...
Edit: I tried patching it to use LD_LIBRARY_PATH, but calling getenv from within the linker would always return NULL. Not sure why. So I did the next best thing and added “.” to the list of search paths. I think that is default behavior anyways on other Linux based systems (will verify later). Submitted the change, hope it gets accepted.
Microsoft's DLR and Mono bring Python and Ruby to Android
With my recent work on porting Mono to Android, I began needing a way to create Java classes and invoke their various methods from C#. I had read briefly about the Dynamic Language Runtime and upcoming C# support for dynamic types; so that naturally led to me considering implementing a JNI-P/Invoke interop layer that allows access to Java/Dalvik code through C# and vice versa. However, I wasn't sure if the DLR would even run on Mono, but I was pleasantly surprised. Support for the DLR was added to Mono around a year and a half ago!
After copying the necessary DLLs over to the phone, I was able to run the simple ipy.exe console application that comes with the DLR source code. The neat thing I discovered while perusing through the documentation is that IronRuby and IronPython are actually compiled into CIL byte code and thus eventually native code! Anyhow, as you can see, IronPython is working dandily on the phone. [1]
The Dynamic Language Runtime is probably the coolest new toy in the .NET arsenal since lambda expressions. Couple the DLR with the new dynamic keyword in C# 4.0, and you can invoke dynamic languages from C# as easily as you can access .NET classes from IronPython or IronRuby![0]
Now to get back to implementing Java interop...
[0] The dynamic keyword basically provides compile time parsing of an expression tree, but the expression isn't actually validated until runtime.
[1] Though Mono + DLR is clocking in at a hefty 13mb in files. But, applications will soon be installable onto the SD card so the storage constraints will be a moot point.
Compiling Mono under the Android Build Evironment
I mentioned previously that I got Mono compiling under the Android build environment. I have also submitted all the changes I made to Mono back to their team. In addition, all external changes are hosted on Google code. The external changes were filling holes in Bionic, setting up a config.h that works on Android, and converting the Makefile to an Android.mk. Click here to go to the AndroidMono repository.
The build instructions are straightforward, and assumes you have a Release 1.0 repository (not Master or Cupcake) already set up:
- In a shell, go to your mydroid/external directory and type:
- svn co http://androidmono.googlecode.com/svn/trunk/ androidmono
- cd androidmono
- svn co svn://anonsvn.mono-project.com/source/trunk/mono mono
- Apply the androidmono.patch file to the mono directory make the necessary changes to the mono project. (This won't be necessary once they accept my patch)
- Run make in your mydroid directory.
Now you should have a mono and libmono.so! The build process does not create the managed assemblies. That needs to still be done the old fashioned configure/make way. The resultant mono binary will look for the managed assembles in /data/data/com.koushikdutta.mono/assets/lib/mono/2.0/(mscorlib.dll).
Mono and Bionic Together At Last
I finally got Mono building in the Android build environment (I.e., a project located at mydroid\external\mono). The great part about this is that I am now doing cross compiles to build the project, as opposed to an ARM native build on the emulator; those were painfully slow. Converting Makefiles to Android.mk files was pretty tedious, but that didn't compare to the significant differences between glibc and Bionic (Android's libc implementation). When Google said Bionic was "lightweight", they weren't joking. For example, Bionic does not implement div. Seriously? (However, the cupcake branch seems to have it now)
Regardless, I have Hello World working linked to Bionic now, which is quite exciting. I also tried out some PInvokes on libGLES_CM.so (OpenGL ES), and the initialization function eglGetDisplay was working just fine. So things are looking good, and this project should be able to really start taking off now.
Edit:
Mono on Bionic is around 50% faster... check my original Mono vs Dalvik post for updated performance comparisons. I am guessing the speed increase is due to Bionic being compiled for the Thumb instruction set as opposed to ARM. Or maybe because Google is using some CPU/architecture specific compiler optimizations in their build process.
Android's Linker
"Next task: create a JNI lib that can access Mono... should be fairly easy." Famous last words. I guess I didn't think through the problem of Android using a non-standard linker. Not only is it named and located differently (/system/bin/linker as opposed to ld-linux.so.3), the format of the dynamic libraries it loads is also different as well. So, Android's linker can not load Linux .so files, and vice versa.
This basically means that Mono with its current build configuration can not interact with the Android system at all, other than through RPC. And while it certainly is possible[0] to do that (and Android uses RPC extensively), I feel that would be a clumsy solution, as my goal is to get Mono running side by side with Java/Dalvik, and allow seamless interaction with runtime through usage of Dynamic Language Runtime (I'm not sure if Mono supports the "dynamic" keyword/type yet, but when it does... this will be quite amazing).
There are two possible solutions to this:
- Recompile Mono and the glibc dependencies using the linker script for Android.
- Compile Mono within the Android build environment so it uses Bionic instead of libc.
Using Crosstool, I tried to build a new toolchain tailored for Android's linker. I quickly found that whoever set up the toolchain/glibc build process must be a sadist. I wasn't too hesitant to give up on this approach anyway: this would result in two copies of libc being loaded (if that is even possible).
Although getting Mono fully integrated to work with Android's libraries will take considerably longer, I can rest knowing it is the proper solution.
[0] Set up an RPC server for OpenGL ES or skia? Ugh...
Mono for Android - First Update
I mentioned in my previous post that Mono was displaying assertions and exceptions on run when not being under the Debian environment. I have resolved these issues by tweaking a few variables on configure, and one by patching mono/metadata/security.c.
I am also now building Mono from the trunk (instead of 2.0.1) and using eglib instead of glib. Here's what I run to build Mono (still done on the emulator, under Debian):
export 'CFLAGS=-DARM_FPU_NONE -DPLATFORM_ANDROID -Wl,-rpath,.,-rpath,/data/data/com.koushikdutta.mono/assets/lib,--dynamic-linker,/data/data/com.koushikdutta.mono/assets/lib/ld-linux.so.3'
export CPPFLAGS=$CFLAGS
export 'LDFLAGS=-Wl,-rpath,.,-rpath,/data/data/com.koushikdutta.mono/assets/lib,--dynamic-linker,/data/data/com.koushikdutta.mono/assets/lib/ld-linux.so.3'
./configure --disable-mcs-build --with-tls=pthread --disable-shared-handles --with-static_mono=no --with-glib=embedded --prefix=/data/data/com.koushikdutta.mono/assets --exec-prefix=/data/data/com.koushikdutta.mono/assets
make
However, "make" fails while building docs. That is ignorable though. I actually comment out docs in my Makefile, as I haven't looked into how to disable it during configure.
With this setup, the base mono installation directory is /data/data/com.koushikdutta.mono/assets (instead of /usr/local). The binaries, libraries, and assemblies should be placed in their usual relative locations: ie, mono binary should be in bin, the libraries in lib, etc.
Now, if you were to run "./mono test.exe" from a terminal, you will see a lengthy assertion. This is because Mono is failing to find the user's home directory, because Android does not set that environment variable. And thus Mono doesn't have a valid location for the ~/.wapi/ directory. The ".wapi" directory is used by Mono for some transient files and communication (from what I understand). For example, if you were to run Mono under Terminal Emulator, HOME needs to be a location the Terminal emulator process and uid has full read/write permissions: /data/data/com.android.term:
export HOME=/data/data/com.android.term
./mono test.exe
I have updated the APK on the Market with my latest changes. I also have the APK available for direct download.
Next task: create a JNI lib that can access Mono... should be fairly easy. (Calling into the lib will automatically set the HOME directory for you as well)
Mono for Android - Now Available on the Market
After another day of messing around compiling Mono, I finally got it working on Android out of the box. The final hurdle was to tweak a few environment variables at build time to pass the "--dynamic-linker /data/data/com.koushikdutta.mono/ld-linux.so" option into the linker. This way the mono binary will actually find the dynamic linker at a valid location in the Android file system rather than fail with a "not found" error.
The application on the market contains the Mono runtime, which will be unpackaged on first run. It will allow you to run simple command line applications from a tool like Terminal Emulator. Here's some more Mono on Android action, sans Debian. The video shows the simple installation followed by a demonstration of the "Hello World" program that is included with the package.
Note that the application is downloaded, installed, and executed, all without root permissions. There are still some outstanding issues though. If you watched carefully, you will see that I redirected stderr to /dev/null. This is because Mono not running under Debian reports a couple of assertions and errors; once upon starting up and again upon exiting. But they don't really seem to be affect anything from as far as I can tell. It is something I'll need to look into. But first... calling into Mono from Java/Dalvik.
Building Mono for Android
<Edit>I have figured out how to get Mono building in the Android build environment fairly easily. Click here for the updated instructions.</Edit>
As I mentioned in previous posts, I managed to get Mono compiled and running under Debian running side by side with Android. I built Mono "natively" by setting up the Debian environment in the emulator. For the sake of having all the instructions in one place on how to set up a native build environment under Debian, I will be consolidating and tweaking the directions from Saurik's post on how to set up Debian on the G1.
Update 1: If you want an installable Mono APK, which is trimmed down to bare minimum dependencies (3MB) and works on non rooted phones, click here or get it off Market. I have updated the instructions below to reflect the configuration and steps I took to build Mono to create this APK.
Update 0: If you just want a premade SD card image for your phone, skip to the very end of this post about deploying Mono to a real phone and follow the directions from there.
Setting up a Native Build Environment under Debian
If you follow the instructions to build Mono for ARM, you will fail miserably. Doing native builds on a G1 is not an option: the hardware is slow, runs out of memory, and just becomes unstable and unresponsive. Ultimately the watchdog timer kicks in and restarts the phone. However, the emulator does not have this issue. It runs faster than a G1, and you can tweak the qemu options to allocate extra RAM. So, the emulator makes a perfect "native" build environment!
As I mentioned, Mono will run under Debian. The Debian installation will be run off the SD card, contained in an ext2 image file. We will set up the ext2 image in the emulator, build Mono, and then transfer the image to a real phone.
Go to a directory somewhere and create a workspace for the installation:
mkdir monoimage
cd monoimage
Set up the SD card image to be used by the emulator by using the mksdcard command that comes with the Android SDK:
# use whatever size you wish
mksdcard 800MB sdcard.img
Build the Debian image as per the instructions in Saurik's post:
sudo apt-get install debootstrap
# use a size smaller than your SD card, must be less than 4GB as that is the max file size for FAT
dd if=/dev/zero of=debian.img seek=749999999 bs=1 count=1
mkefs -F debian.img
sudo mount -o loop debian.img debian
sudo debootstrap --verbose --arch armel --foreign lenny debian http://ftp.de.debian.org/debian
sudo umount debian
Now create the Debian kit on the SD card:
mkdir sdcard
sudo mount -o loop sdcard.img sdcard
sudo mkdir sdcard/kit
sudo cp debian.img sdcard/kit
rm debian.img # don't need this anymore!
# this script sets up the debian mount and gets you a bash prompt
wget http://koushikdutta.blurryfox.com/MonoWalkthrough/bash.sh
sudo cp bash.sh sdcard/kit
# this unmounts the debian image
wget http://koushikdutta.blurryfox.com/MonoWalkthrough/umount.sh
sudo cp umount.sh sdcard/kit
# busybox. handy tools...
wget http://koushikdutta.blurryfox.com/MonoWalkthrough/busybox
sudo cp busybox sdcard/kit
sudo umount sdcard
Time to start the emulator with SD card image we created. However, before we can can do that, we need to make a quick change. The emulator does not support ext2 out of the box, but that can be fixed by recompiling the kernel. I have a prebuilt emulator kernel available for download. (For further do it yourself instructions on how to compile the Android kernel manually, check out Motz's instructions. After following his instructions, you can use "make menuconfig" to tweak the kernel build options and then use "make" to build it.)
# get the custom emulator kernel
wget http://koushikdutta.blurryfox.com/MonoWalkthrough/zImageExt2
Ok, we're set now! Start her up!
# the last 2 arguments tell qemu to provide 512MB of virtual RAM to the emulator
emulator -sdcard sdcard.img -kernel zImageExt2 -qemu -m 512
Now, let's get into a shell on the emulator to finalize the Debian installation:
# This will get you to a "sh" instance on the emulator. Let's get a real bash prompt next.
adb shell
# Ignore any errors you may have with this command!
#You should see "I have no name!@localhost:/#" as your bash prompt after executing this.
. /sdcard/kit/bash.sh
/debootstrap/debootstrap --second-stage
echo 'deb http://ftp.de.debian.org/debian lenny main' > /etc/apt/sources.list
echo 'nameserver 4.2.2.2' > /etc/resolv.conf
# Let's restart the prompt to get rid of that "I have no name!" problem.
exit
# Ignore any errors again. You should get a "localhost:/#" bash prompt now.
. /sdcard/kit/bash.sh
Building Mono
Whew! You now have a full Debian distribution on the emulator. Now that we're set up, let's get the ARM portion of Mono built. But first, let's grab all the updates, dependencies, and tools we will need to build it.
apt-get update
apt-get install bison libglib2.0-0 libglib2.0-dev bzip2 gcc make
cd ~
# The following URL is the trunk snapshot at the time of writing this entry.
# Go to http://mono.ximian.com/monobuild/snapshot/sources-trunk/ to get the latest snapshot.
wget http://mono.ximian.com/monobuild/snapshot/snapshot_sources/mono/mono-122990.tar.bz2
tar xvf mono-12290.tar.bz2
cd ~/mono-122990
wget http://anonsvn.mono-project.com/viewvc/trunk/mono/eglib.tar.gz?view=tar
# wget mangles the file name...
tar xvf eglib.tar.gz?view=tar
And now we can finally begin the native build process. First let's set up some environment variables.
# CPPFLAGS=-DARM_FPU_VFP is used because the G1 supports VFP.
# Also, tell the linker where the runtime linker is found, and the path to the libraries.
export 'CFLAGS=-DARM_FPU_NONE -DPLATFORM_ANDROID -Wl,-rpath,.,-rpath,/data/data/com.koushikdutta.mono/assets/lib,--dynamic-linker,/data/data/com.koushikdutta.mono/assets/lib/ld-linux.so.3'
export CPPFLAGS=$CFLAGS
# We need to set the linker flags specifically as well...
export 'LDFLAGS=-Wl,-rpath,.,-rpath,/data/data/com.koushikdutta.mono/assets/lib,--dynamic-linker,/data/data/com.koushikdutta.mono/assets/lib/ld-linux.so.3'
# --disable-mcs-build configures it not to build the managed assemblies.
# Managed assemblies will be built on the host.
# --with-tls-=pthread changes how thread local storage is implemented.
# The default __thread does not work on Android.
# --disable-shared-handles is used because the G1 kernel does not have SYSVIPC enabled.
# --with-static-mono=no forces mono to dynamically link to libmono to save space.
# --with-glib=embedded embeds a trimmer version of glib into Mono.
# --prefix and --exec-prefix inform Mono of where it's binaries, libraries, and assemblies are found.
./configure --disable-mcs-build --with-tls=pthread --disable-shared-handles --with-static_mono=no --with-glib=embedded --prefix=/data/data/com.koushikdutta.mono/assets --exec-prefix=/data/data/com.koushikdutta.mono/assets
# This will take a couple hours... make a pot of coffee or 6.
make
Note that we have only built the Mono runtime at this point. We need to build the managed code portion now. First let's exit the emulator cleanly and unmount the ext2 image cleanly:
# Exit bash
exit
# Unmount the ext2 image
. /sdcard/kit/umount.sh
# Exit adb shell
exit
You can now close the emulator and remount the SD card on your host PC. You should be in the monoimage directory you created earlier.
sudo mount -o loop sdcard.img sdcard
sudo mount -o loop sdcard/kit/debian.img debian
Get the Mono sources again and build them on the host:
wget http://ftp.novell.com/pub/mono/sources/mono/mono-2.0.1.tar.bz2
tar xvf mono-2.0.1.tar.bz2
cd mono-2.0.1
./configure --with-glib=embedded --prefix=/data/data/com.koushikdutta.mono/assets --exec-prefix=/data/data/com.koushikdutta.mono/assets
make
So, both portions of Mono have been built. Let's install the managed portion directly to the ext2 image.
sudo make install DESTDIR=/full/path/to/monoimage/data/data/com.koushikdutta.mono/assets
Unmount the images:
sudo umount debian
sudo umount sdcard
Restart the emulator to finalize the Mono installation (this will overwrite the PC runtime that was installed):
emulator -sdcard sdcard.img -kernel zImageExt2 -qemu -m 512
. /sdcard/kit/bash.sh
cd ~/mono-2.0.1
make install
Mono should now be installed and functional! Let's test it:
cd ~
wget http://koushikdutta.blurryfox.com/MonoWalkthrough/hello.cs
# gmcs IS a managed executable; and obviously quite a complex one at that.
# If it succeeds. Then Mono is working... quite well.
gmcs hello.cs
# Hello World!
mono hello.exe
Before we transfer the Debian image to a real phone, let's shut down gracefully again:
# Exit bash
exit
# Unmount the ext2 image
. /sdcard/kit/umount.sh
# Exit adb shell
exit
Putting Android onto a Real Phone
If you just want a a premade SD card image that contains a working Debian install and Mono runtime, click here. Unzip the sdcard.zip and follow the instructions below to transfer the debian.img (which is contained in the sdcard.img) to your phone.
Your phone must be rooted and have the latest version of modified RC30 (1.3+) . Connect your phone and copy the kit directory from the SD card image to your real SD card. You should preferably do this by mounting your phone's SD card.
# Mount the SD card image first so we can access the kit.
sudo mount -o loop sdcard.img sdcard
# It is actually much faster to mount your phone's SD card as a removable disk on your
# computer and copy the kit directory over from the sdcard.img to the real SD card.
# It is also less likely to be corrupted as well. Don't use adb push unless you really want to.
adb push sdcard/kit /sdcard/kit
Now, go to a bash prompt on your phone and you will be able to run Mono!
adb shell
. /sdcard/kit/bash.sh
cd ~
mono hello.exe
What Now?
Well, now Mono is running on Android, side by side with Debian. However, attempting to invoke it from outside of a chroot /bin/bash will result in a "not found" error. This is because Mono is not configured to use Android's linker (/system/bin/linker), and looks for it at "/lib/linux-ld.so.3". You can fix that by doing the unionfs trick that Saurik posted about in his blog.
However, this process still requires the phone to have root, and have a Debian installation. For this to really be viable, Mono would need to be compiled statically, with all its dependencies. And I have no idea how to do that, yet. If someone does, please let me know!
Dalvik vs Mono
<Edit 2>I managed to get Mono natively on top of Android's libc, Bionic! This resulted in another 50% or so speed improvement; I am guessing this is because Bionic is compiled for the Thumb instruction set, which is supposedly faster. The post has been updated once again with the new results</Edit>
<Edit 1>I've updated the performance scores with Mono as pulled from the trunk (I was using 2.0.1) and using eglib. It's gotten faster and uses even less memory.</Edit>
<Edit 0>The original article was only about Dalvik vs Mono, and begged the question, why didn't Google leverage Mono's existing open source technology? However, I found that Sun has an ARM version of their Java Runtime environment available as a 90 day trial. So I also ran the tests against that as well. Sun's ARM JRE is around as fast Mono, but at a greater memory cost. However, the Sun Java Runtime Environment is not open source, unlike Mono. So, it is not a viable runtime for the Open Handset Alliance's platform.</Edit>
Google has had a little egg on its face recently. They wrote up a 40 page comic touting the awesomeness Chrome V8's performance, only to be thoroughly trounced by TraceMonkey and Squirrelfish Extreme in comparative benchmarks (It's ok guys, I still prefer Chrome as my browser though).
So after that embarrassing showing, I was naturally a little skeptical about the supposed benefits that Dalvik provided for mobile devices. To better understand Dalvik's goals and inner workings, I watched an hour long presentation starring its creator, Dan Bornstein. The two line summary is that Dalvik is designed to minimize memory usage by maximizing shared memory. The memory that Dalvik is sharing are the common framework dex files and application dex files (dex is the byte code the Dalvik interpreter runs).
The first thing that bugged me about this design, is that sharing the code segments of dex files would be completely unnecessary if the applications were purely native. In Linux, the code segments of libraries are shared by all processes anyways. So, realistically, there is no benefit in doing this. In fact, Mono's managed assemblies also reap these same benefits of multiple processes sharing the same code segment in memory.
The second thing that bugged me about this presentation was Dan starts out talking about how battery life is not scaling with Moore's law, which is certainly true. But if the battery is the primary constraint on the device, why is Dalvik so concerned with minimizing memory usage? I am by no means a VM design guru, as I'm sure he is, but I can say the following with certainty:
- Total memory usage has absolutely no impact on battery life. The chips are being powered regardless of how much of their memory is being used. Increasing the total memory available on a device will also only cause marginal increase in battery drain. Memory is not something that taxes the battery compared to other components of the system.
- Battery life is primarily affected by how much you tax the processor and the other hardware components of the device: especially the use of 3G/EDGE and WiFi radios.
- Interpreting byte code will tax the processor and thus the battery much more than native/JIT code.
- Modern (Dream/iPhone comparable) hardware running Windows Mobile is rarely memory constrained, and they don't have a fancy memory sharing runtime. Memory constraints (in my mobile experience) become an issue on Windows Mobile when several applications are running at the same time. And this problem can be solved at the application framework level; such as how the Android Application life cycle is implemented. If all applications can suspend and restore at the system's whim, then memory consumption is trivialized. However, the application framework is not tied to the Dalvik runtime. (I.e., it can be ported to work with native code, Mono/.NET, JVM, whatever)
- Generally in applications, the code's memory footprint is trivial compared to the application data memory footprint (images, text, video, etc). Dalvik is overly concerned with optimizing the memory size of dex files and sharing memory. Dan's presentation did a comparison between the Browser's Java .class files versus the Dalvik .dex files (the .dex file is around 250k, around half the size of the .class files). My reaction to that is whoopity-shit. What happens when you start up the Browser? You head to your favorite webpage, it loads up a half dozen images which decompress to a raw R5G6B5 format, which then clocks in at several megabytes. That really trivializes the few hundred kilobytes that Dalvik is trying to save.
This leads me to believe that Google committed a classic performance optimization mistake: they are optimizing an aspect of the system that is trivial in the grand scheme. To poke a little nerdy fun at a portion of Dan's presentation, it is akin to tweaking your for loops to iterate downwards for better performance. And all the while the loops are being used perform an inefficient selection sort.
Regardless, all speculations and theories aside, let's let real world scenarios speak for itself. The T-Mobile G1, aka HTC Dream, has terrible battery life when compared to its siblings of the Windows Mobile variety. (I own or have owned a Dream, Touch, Touch Cruise, and Touch Diamond in the past year)
Runtime Memory Usage
My first test was to create a simple hello world program for both runtimes. Hello World would be printed to the screen, and then the thread would sleep for 30 seconds, allowing me to peek at the process' memory usage.
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 14500 43.0 3.8 14788 3816 pts/0 Rl+ 01:35 0:01 ./mono MonoTests.exe ...
root 1605 49.0 4.8 34940 4812 pts/2 Sl+ 20:10 0:00 /system/bin/dalvikvm ...
root 4464 64.0 6.7 180388 6740 pts/1 Sl+ 18:21 0:02 java -jar JavaTests ...
Ok, so this surprised me a bit. Mono needs around half the memory to start up? Using pmap on the dalvikvm process shows that it is referencing a lot more "base" system libraries than Mono. I suppose at the end of the day, it doesn't matter, because on Linux, libraries are loaded and shared between processes. I also took a pmap snapshot of Mono and Java for those interested (Sun's ARM JRE is quite bloated...).
Performance Comparisons
I'll be the first to admit, these comparisons aren't fair at all. No interpreter will ever run as fast as native code. But, I'll test it anyway. These tests purposely steer clear of the calling into underlying libraries. The goal is to benchmark the memory usage and performance of the runtimes themselves by way of very simple applications. Click here to view the code for the Java and the C# tests.
Selection Sort Test
This test creates a reverse sorted array of integers between 0 and 1000 and sorts them into increasing order (and does it 10 times, excluding the results of the first). Lower numbers are better. Results:
| Time (ms) | |
| Dalvik | 4668 |
| Mono | 411 |
| Java SE for Embedded | 895 |
Class (and Structure) method call Test
This test creates instantiates an array of 10000 FibContainer instances. FibContainer is a simple class:
| C# | Java |
| class FibContainer public void Compute(FibContainer previous, FibContainer beforePrevious)
struct FibContainerStruct public void Compute(FibContainerStruct previous, FibContainerStruct beforePrevious) | class FibContainer public int getValue() public void setValue(int value) public void Compute(FibContainer previous, FibContainer beforePrevious) |
It then iterates over the array and calculates and stores the Fibonacci series. The test notes 3 things: total memory in use by the runtime after allocating the array, the time to allocate the array, and the time to calculate the Fibonacci series (the method calls are intentional). Note that I also performed this same test on Mono with a feature not available in the Java language: I used a struct instead of a class. Smaller numbers are better in all cases. This test was run 50 times (the first excluded):
| Memory (bytes) | Allocation Time (ms) | Calculation Time (ms) | |
| Dalvik Class | 9817240 | 5388 | 3013 |
| Mono Class | 7376896 | 933 | 167 |
| Mono Struct | 2007040 | 12 | 107 |
| Java Class | 10438176 | 319 | 211 |
| C++ Class | 1960000 | N/A | N/A |
Equivalent C++ code would allocate the amount of memory shown above. So, as you can see, Mono has around 33% less overhead when allocating classes. It is also around 8% faster at doing those allocations, and the calculation time completely blows Dalvik out of the water. And by way of intelligent usage of structs in Mono, you can leverage near bare metal memory usage. (Not to mention that arrays of structs containing blittable types are themselves blittable. This is very friendly for processor/memory caches. It also provides for easier interaction with native calls, such sending an array of vertices to OpenGL. But I digress.)
Long story short: from my initial, limited, and naive testing, Mono is faster and uses less memory than Dalvik. And it is not even designed to run on mobile devices. So it begs the question, why didn't Google just convert the .class files to CIL and use the Mono runtime? That way they wouldn't have alienated Java developers, would have access to open source Java libraries they so covet, wooed .NET developers, and wouldn't have needed to invent their own sub-par runtime!
I also want to test out the performance of the two Garbage Collectors as well as native function invocation (P/Invoke and JNI), but I'm hungry and will do that in a later post. Until next time friends!
Mono on Android - With Gratuitous Shaky Cam Action
I thought I had Mono working; however, that was just on the emulator. The phone was failing due some triple combo exception (StackOverflow, OutofMemory, NullReference) that was ultimately related to my temporary hack/kludge. Getting it working on a phone took a couple more tweaks and fixes provided by Miguel de Icaza and Mark Chricton. The following video shows me natively compiling a simple hello world application using gmcs and then running it. Performance comparisons coming soon (for real this time)! (Note: The initial load times on mono and gmcs are abysmal since the Debian install is running off the SD card. That shouldn't affect execution time once everything is loaded into memory.)
Setting up a Native Build Environment for Android
My initial attempts at cross compiling Mono were on Scratchbox. But Scratchbox was having technical difficulties building Mono from the trunk (not sure why), did not have a working apt-get, the Mono binary would not actually run when it did build, etc. What I really wanted was to do real native builds. However, the G1/Dream has limited RAM (96MB total I think). As a result, compilation took forever, and often the watchdog daemon would kick in and restart the phone due to unresponsiveness. So that is not an ideal build environment. So my solution was to get Debian running on the Android emulator.
I figured it would be straightforward enough, but I ran into a snag mounting the ext2 image containing the Debian boot strap. That was solved by recompiling the Android Kernel with ext2 support added:
- Follow these directions to prepare to build the kernel.
- Before building the kernel, run make menuconfig, and add ext2.
- Now make.
That will give you a an image with ext2 compiled into the kernel. For the lazy, click here for the compiled kernel image.
When you start the emulator from the command line, use the following flags:
emulator -kernel zImage -sdcard sdcard.img -qemu -m 512 # The '-qemu -m 512' options tells qemu to start with 512MB of virtual RAM!
Now install Debian with the same instructions Saurik's post. Note that you will not need to insmod the ext2.ko module. Always remember to unmount the ext2 partition before closing the emulator. Otherwise you may end up getting some nondescript NFS error; probably a result of file system corruption. However, this is fixable by mounting the ext2 file in Linux and running the fsck tool.
Mono on Android - Success At Last!
Last week, I blogged about my attempts to get Mono running on Android. I was running into a nasty Illegal Instruction issue, which I didn't look into further until today. And after implementing a quick hack/fix I got it working!
After running strace on Mono, I was able to determine that the Illegal Instruction was occuring in mini-arm.c in the mono_arch_flush_icache function. I wasn't able to decipher whether the inline assembly was correct or not; but I got the jist of what it was trying to do: it was calling cacheflush, but failing with the Illegal Instruction in there. I looked at the mini-x86.c and saw that the equivalent mono_arch_flush_icache was empty; the comment indicating it was "not needed". I don't know if this is some vestigial code or what, but I figured it would be relatively benign to not flush the instruction cache and just see what happens. And the grand results:
root@localhost:~/Desktop# mono test.exe
Hello World
This would not be possible without the help of my co-worker, Kevin Eustice, and my abhorrence towards Java.
Expect to see some performance comparisons between Mono vs Dalvik shortly! Though I don't expect it to even be a fair competition, because Dalvik does not JIT. Lame.
Mono on Android
It's been a little quiet on my blog lately... too quiet. It's cause I've been bashing my head against a wall trying to get Mono on Android (well, Debian running side by side with Android really). So far, I've gotten the Mono runtime cross compiled for Android, but attempting to run anything results in the following:
localhost:~/Desktop/junk# mono
Usage is: mono [options] program [program-options]
Development:
--aot Compiles the assembly to native code
--debug[=<options>] Enable debugging support, use --help-debug for details
--profile[=profiler] Runs in profiling mode with the specified profiler module
--trace[=EXPR] Enable tracing, use --help-trace for details
--help-devel Shows more options available to developers
Runtime:
--config FILE Loads FILE as the Mono config
--verbose, -v Increases the verbosity level
--help, -h Show usage information
--version, -V Show version information
--runtime=VERSION Use the VERSION runtime, instead of autodetecting
--optimize=OPT Turns on or off a specific optimization
Use --list-opt to get a list of optimizations
--security[=mode] Turns on the unsupported security manager (off by default)
mode is one of cas, core-clr, verifiable or validil
localhost:~/Desktop/junk# mono test.exe
Illegal instruction
Note that mono works fine, but attempting to run a program fails. So yeah, I'm at a loss. I'm guessing there is a bug in the ARM JIT somewhere. Or maybe my make process went horribly wrong due to user error somewhere. I'll try getting Mono from the trunk and building and installing. Apparently there are some more ARM related fixes in there.
I managed to get this far by:
Cross Compile Mono and its dependency chain using Scratchbox (there are a lot, which I'll detail in a later post). I also did some of the compilation on the G1 itself rather than Scratchbox.
Subscribe to:
Posts (Atom)