Ken's Digital Den

Category: java

Strategies for Multiple Android Products with Eclipse and Git

Problem: What’s the cleanest, easist way of sharing code between multiple “products” within a given Android app?

Background: An Android application may have free and paid versions.  Both have some common code and other files, but also have specific code and some files must differ between the two versions, such as the Android Manifest.  When bugs or features are added in one version of the product, it would be nice for those changes to be applied to others if appropriate.  This is similar to the “one product multiple versions” workflow, but different in than specific well-known files will always differ, and the requirement of merging between the concurrent branches (at least for me) is far more frequent.

Requirements:

  • Simple to migrate code from one product to another.
  • Easy to isolate code which should be shared and code that belongs to a specific product.
  • Relies on SCM.
  • Unlikely to let unwanted changes to slip between versions.
The Simplest Thing that Could Work:
Create two isolated projects, one for the free, one for the paid.  When code sharing opportunities present themselves, just copy the files or sections of files across.  Do not rely on SCM or build conventions.  This works but as projects grow, it becomes more difficult to keep things straight and not overwrite changes unintentionally.

My Current Approach:

I’m working on my first free/paid application and I’ve decided to represent the two products (free, paid) as branches in a git repository.  In this case, I’m using master for free.  Since the Android market requires unique product IDs for each product, I use a Java package namespace that corresponds to the product ID for code that only belongs to one product, and a general Java package name for common code.  To migrate code from one version to another, I make careful commits of only the shared java namespace and use git-cherry-pick.  For now I’m using the command-line rather than egit.  I find it’s best to first get familiar with a particular git feature via the command-line tools.  Then any troubles encountered when using the GUI provided by egit can be better understood.

This so far is working out acceptably, but it’s not perfect.  It can be hard to remember to specify commits for specific in-common code, as required by cherry-pick.  Also, depending on how modular the code is, there can be a lot of refactoring to keep the code bases in-sync.

Another approach I may take in the future is to move all common code into separate Android library projects, and only have product-specific code live in actual project associated with the product.  This will reduce the amount of cherry-picking required.  On the downside it seems that the ADT Eclipse plugin has a proprietary  way of managing code in Library projects and changes to library code does not automatically propagate to downstream projects, until the entire workspace is rebuilt.  I’m hoping there is something I can do to fix this, because using the ADT Eclipse library feature is not very useful with this restriction.

Google searches didn’t turn up much for me, but I’d love to know what techniques others are using to solve this problem…

Java 7 (OpenJDK/IcedTea) for the BUG 2.0

NOTE: this post was originally at http://community.buglabs.net/kgilmer/posts/251-Java-7-OpenJDK-IcedTea-for-the-BUG-2-, however the webserver seems only intermittently available so I’m moving the post here.

For anyone interested in Java 7, I have built OpenJDK 7 (via IcedTea) for BUG 2.0 (Angstrom, armv7, Zero VM).  The binaries are available here.  The binaries are not provided as Angstrom opkgs, so the installation process is manual. This is because it was built natively on a BUG, rather than using the OpenEmbedded cross compiling build system.  You’ll also need to install libcups2 via opkg if you want to replace the Java 6 JRE to run BUG applications.  Here is my installation steps, assuming the j2re-image tarball is decompressed in /home/root:


$ mkdir -p /usr/lib/jvm/java-7-openjdk/jre
$ mv ~/j2re-image/* /usr/lib/jvm/java-7-openjdk/jre
$ rm /usr/bin/java
$ ln -s /usr/lib/jvm/java-7-openjdk/jre/bin/java /usr/bin/java
$ opkg install libcups2

Now if you restart Felix (via Knapsack if you are running a 3.0 release), you’ll notice that the JVM running is java.runtime.version = 1.7.0_147-icedtea-b147 via http://bug/support.html.  This is provided on an experimental basis and I would not be surprised if problems pop up!

An interesting benefit that IcedTea 7 provides (other than ARM compatibility via Zero) is alternative VM implementations CACAO and JamVM.  The binary build I have provided unfortunately does not include these but I’m going to try and build them.  Hopefully I can get Zero, Shark, CACAO, and JamVM all happily living inside the BUG!  :)

Special thanks to xranby@irc.oftc.net#openjdk for his invaluable assistance!

Fresh from the oven: Knapsack for Apache Felix

Both for my various weekend projects and on the BUG, I’ve been unhappy with the bundled launcher that comes with Felix.  Equinox is no better.  Typically people use the Eclipse tooling to create and configure framework instances, but when setting up for production or working with embedded targets this is often not possible. So I set out to make something that would allow me to easily get a framework up and running from the terminal.  Luckily, Felix has excellent embedding support and documentation so it was pretty easy to get started.

What is it?  Essentially an alternative launcher for the Felix OSGi framework (3.2.2) that uses the OS’s native shell for framework management, and creates all of the configuration files automatically, letting the user just download, run, and start adding their application bundles, with interactive feedback.

I’ve much more detail on the design details and usage on the project page and in the readme.

However, here is the summary of how to go from Zero to OSGi in 30 seconds*:

$ mkdir foo && cd foo
$ wget https://leafcutter.ci.cloudbees.com/job/knapsack/lastSuccessfulBuild/artifact/knapsack.jar
$ java -jar knapsack.jar &

* Depending, of course, on your copy-paste skillz.

Release 0.1 for Sprinkles

After several months of actively developing with Sprinkles I think it’s time to make a tag and call it a version.  I’ve still got far to go in terms of documentation but with Cloudbees free Jenkins instance I have a build running with JUnit and Javadocs available.  I’ve also created a recipe for OpenEmbedded so embedded devices can also taste the functional fruits in Java.  Oh, and Sprinkles exports its package just like a good OSGi bundle should!

Sprinkles: The functional fruits must be tasted!

Some problems are a natural fit for functional style programming.  By this I mean applying functions, often recursively, to lists of data.  Some languages support this out of the box, in a compact elegant fashion, such as Python.  Frustrated with the OOP approach to a problem involving files, I began to write some static helper methods to try to restructure my program in functional way.  The result, after two weeks of thoroughly enjoyable exploration of recursion and self-reference, is Sprinkles.  Sprinkles consists of a “function” interface and a set of static methods that allow the functions, such as map() and fold(), to be applied to elements and collections.  The base function interface is simply:


public interface Function {
    public Object apply(Object element);
}

I know I know, how could anything useful come from such a simple interface…?  Calling the Fn.map(Function, Object) static method will cause every element contained in Object (assuming what’s passed implements Iterable) to be passed to Function, with the result being put in a new Collection as the return value.  When this pattern is applied multiple times, I find that certain problems that would require a lot of code in a traditional OOP approach are quite clean and concise.

To get a sense of the value of Sprinkles, let’s look at some examples.  In these examples I use annonymous inner classes for brevity.  But in real code, I typically define my functions explicitly, as I’ve found I want to reuse them.

Let’s say I want to “walk” the filesystem for all the files from a given directory:

Collection results =
    Fn.map(ReturnFilesFunction.GET_FILES_FN, new File("/tmp"));

See how easy that was? results now contains every file that is a child of /tmp. Ok, I admit this is cheating, but bear with me. I wrote some static functions for common activites like this.
Well, now let’s say I want to figure how the total size of all those files.  Well, the results of my first function is a list of all files.  What if apply another function that returns the sizes of those files:

Collection results = Fn.map(
    new Function(){
          public Object apply(Object element) {
                  return ((File) element).length();
          }
    }, Fn.map(ReturnFilesFunction.GET_FILES_FN, new File("/tmp")));

Now, results will contain a list of the sizes of all the files.  Thankfully, Java autoboxes the primitive types for me since I’m using Java 6.

To get my final result, I simply need to “collapse” the list of sizes into a sum using the fold() function:

</span>

<span style="font-family: Consolas, Monaco, 'Courier New', Courier, monospace; font-size: 12px; line-height: 18px; white-space: pre;">Object result = Fn.fold(new FoldFunction() {</span>
<pre>    public Object apply(Object element, Object result) {
        Long ival = (Long) element;
        Long rval = (Long) result;
        if (rval == null)
               return ival;
        return ival + rval;
    }
    }, Fn.map(new Function() {
            public Object apply(Object element) {
                    return ((File) element).length();
            }
    }, Fn.map(ReturnFilesFunction.GET_FILES_FN, new File("/tmp"))));
System.out.println("Total size: " + result);

You may be thinking, well that’s really hard to read, and I admit it’s true.  There are a few things to make it better.  First, as I mentioned, do not use the anonymous inner classes:

Object result = Fn.fold(
    new AddLongElements(), Fn.map(
        new FileToSize(), Fn.map(
                ReturnFilesFunction.GET_FILES_FN, new File("/tmp"))));

The other trick is using the version of Map() that takes in a Collection<Function>. This let’s the function chain be composed in a linear, non-nested way:

Collection<Fn.Function> functions = new ArrayList<Fn.Function>();
functions.add(ReturnFilesFunction.GET_FILES_FN);
functions.add(new FileToSize());
Object result = Fn.fold(new AddLongElements(),
              Fn.map(functions, new File("/tmp")));

Albiet, in our file example this approach doesn’t help much, since we are only applying a few functions.  But as the chain grows, this code style quickly ends up being easier to read.

One of the interesting twists of Sprinkles is that a Function can return a Collection (list), and that list is then acted upon as such.  When a Function does not want to return a value, it simply returns null and it will not be appended to the list.  However, where ever Collections are used in the API, empty Collections are passed/returned instead of null, making the code much cleaner, since there is no need for null checks everywhere.  And finally, anything can be passed into map().  If what’s passed in is iterable, then the function is applied as a list, otherwise the function is applied as if it’s a list of single element.  This has great impact when composing sets of recursive functions that can expand and contract based on the results of a function.

Drawbacks?  Well, in general this approach works well for certain kinds of problems.  Also, the Object-in-Object-out approach essentially boils down to dynamic typing, since only at runtime is it know what types will be passed and returned.  However since types can be constrained from the entry point of a function, in practice I don’t have much problem with this.  Function reuse is somewhat limited in that a function has to be explicitly written to handle all static types that may be passed to it.  And finally, there are no generics in the API.  I simply don’t understand generics as implemented in the Collections framework well enough yet.

There are more techniques to cover later, such as using find() and depth-first function traversal.  With Sprinkles, I can finally taste the forbidden fruits of functional programming in Java.  It’s amazing to find how easy functions can be composed and reused!

Thoughts on OSGiFS

I’ve spent some time working with the prototype implementation for a filesystem-based management interface to OSGi.  Some points that have become apparent:

  • It should probably be called something like “filesystem admin” or “native admin” to be more inline with the OSGi naming style.
  • I’m unable to find a way around the ‘one thread per file’ problem.  I was hoping there was some magical NIO thing, but if there is I can’t find it.  Due to this, I’m considering either using FUSE (which would limit it to Linux only) or consolidating what files are available in the interface to as few as possible.  The former has the possibility of a much richer user experience, but requires JNI integration with FUSE.  I have had little luck with with existing libraries for this on Linux.  The latter is less work but could end up making the whole thing more of a pain than it’s worth.
  • The logic of having directories for run levels, and symlinks in them that point to an install directory is a mixed bag.  It’s nice to have one directory where all the bundles go, and the runlevel convention is pretty simple, but having the management agent (osgifs) delete the symlinks when a bundle fails to start is not a good approach.  In general, osgifs should never delete files, it just causes user confusion.  So, there needs to be another way (via the filesystem metaphor) of defining runlevels for bundles, providing a clear view of state and handling cases of run level changes and bundle start failure.
  • The felix auto install feature is quite handy.  It’s nice to be able to just scp the jar to a remote, machine, run it, and have an OSGi framework ready to go.
  • The shell integration becomes quite natural.  And when combined with screen, tee, etc. it is a very nice way of running and managing remote OSGi framework instances.  Also, running a framework in the background of a running shell great for debugging, in that any log or stdout output from the framework is printed in the shell.  This gives you immediate feedback if something happens.
  • I should look into integrating with Karaf, but it seems like the first order of business would be dealing with the Maven dependencies, which is something I’ve never quite had the time to learn.

So, in summary I’d say the prototype is a “success” to the degree that I’m going to continue to use it, but it needs a lot of work before I’d consider it relevant and valuable in a larger scope.

Some notes on XMPP, pubsub, Smackx, and ejabberd.

  • Smack and smackx employ a custom classloading scheme to load extensions at runtime.  This mechanism looks for a configuration file in a specific place in a jar.  If you are running from source (in an IDE) or do not have the file (from a custom build), the classloading scheme will fail.  Rather than an error like “Can’t load extensions”, you’ll simply find intermittent ClassCastExceptions being thrown.  Most if not all of the test suites will fail as well.  Running from a binary jar or from a build utilizing the included build scripts is recommended.
  • For PubSub to work with ejabberd, when creating the PubSubManager, you must specify the toAddress in the PubSubManager() constructor.  If you do not do this subscriptions will fail.  The toAddress that worked for me was “pubsub.” + connection.getHost().
  • Psi is a great XMPP chat client for Linux that lets you see the raw XML.  Smack also has a debug mode triggered by a system property that will create a Swing XMPP message log view.
  • Do not expect to get blown away by the detail and verbosity of error messages from ejabberd.
Follow

Get every new post delivered to your Inbox.