Maori.Geek

CBSE 2010 Report

2010-07-11T19:32:00.000-07:00

This is from a report for BuildIT
This year’s Component Based Software Engineering (CBSE) conference was held in Prague, at Charles University, in conjunction with the other CompArch conferences, Quality of Software Architectures (QoSA), International Symposium on Architecting Critical Systems (ISARCS) and the Workshop on Component-Oriented Programming (WOCOP).
I arrived in Prague early so registered to attend the WOCOP headed by Ralf Reussner and Ivica Crnkovic. At this workshop I was able to see many presentations about continuing work from other Ph.D. students and discuss it with them in the question sessions and breaks.
After the presentations we broke into groups to discuss topics of interest, our topic was discussing Services vs. Components. Our group, consisting of people from various fields (including embedded, performance prediction, security and software engineering) all saw these two concepts very differently. Therefore instead of presenting conclusions from our discussion, we enumerated the fundamental questions that we encountered when discussing these topics. I was elected to present our questions to the group which elicited much discussion, culminating in a great quote from Ralf, paraphrased as “If you ask a professor what a component is, they can answer. However each professor gives a different answer.”
The next few days CBSE, QoSA and ISARCS were held, and I attended relevant talks on my Ph.D. project, these included “Reliable Dynamic Reconfigurations in a Reflexive Component Model” by Leger et al. ,“Component Composition Using Feature Models” presented by Ralf Mitschke and “Automated Creation and Assessment of Component Adapters with Test Cases” presented by Oliver Hummel.
Opportunities where taken to discuss work and research outside the conference setting. I talked to the presenters Ralf Mitschke, Claas Wilke and Kerstin Falkowski at length about their work and how it relates to mine, and progress and problems they have encountered, as well as many other presenters.
I presented second to last on the last day. I spoke for about 18 minutes and had 12 minutes of questions. Many questions came from Kung-Kiu Lau, who presented on “(Behavioural) Design Patterns as Composition Operators”, and Kiev Gama who presented “A self-healing Component Sandbox for Untrustworthy Third Party Code execution”, both of whom I had discussed their work with earlier in the conference. The presentation went well and the feedback and questions where thoughtful and useful for future research.

Submitted first Patch To Felix

2009-12-03T19:06:00.000-08:00

First submitted issue and patch to Felix, it was just a bug I found in the framework package while I was using it in an embedded manner.

In OSGi Require-Bundle manifest header, never mix up "version" and "bundle-version".
I discovered some bundles do this, and Felix does not like it, here is what happens...

Require-Bundle: org.w3c.dom.smil;version="[1.0.0,1.1.0)";
This means 2 things, A) that the version range wont work, and B) that the bundle must have a property version which equals "[1.0.0,1.1.0)".

This means that bundles will not resolve with really weird errors, and nothing will work.

The better idea of course is never to use Require-Bundle, which is of course a static dependency and one that breaks all that modularity that OSGi gives you.

Software Engineering Lessons (1)

2009-10-21T14:48:00.001-07:00

A lacking team member is not only a hindrance to progress, but is also a detriment. Not only are they unproductive, but what they produce must be more carefully analysed by better programmers consuming time and resources which could be better used.
--Joel on Software

Azureus Logging Replaced

2009-09-13T15:36:00.000-07:00

Azureus was using a FEW custom logging frameworks mixed in with some Log4J, which was really annoying when trying to separate the project into bundles. So what did I do, I moved it over to the JDK logging package. Took me all weekend but I am finished.

To make it OSGi-y I also wrote a package that implements a service that inserts itself as a handler into the JDK logging service and then passes those LogRecords onto the OSGi logging service, so that any bundle wanting to listen to the Torrent bundles logs can listen to either the JDK logger, or the OSGi Logger service.

Next is removing the custom plugin framework, and all the ad-hoc stuff that is implemented around it to make it modular.

All the internal stuff that goes on inside Azureus can be greatly simplified with the use of OSGi. But the process of moving is difficult, and the tools from going from legacy to OSGi code is arduous. Refactoring tools need to be developed to help this process.

OSGi + Azureus Torrent Bundle

2009-09-10T17:13:00.000-07:00

TLDR) I have been pulling apart the code of the Azureus Torrent Client to create a Torrent Bundle and Service in OSGi. It is more difficult than I first though it would be.
If you would like to see such a bundle, please let me know, this would encourage my efforts.

I have been working on it for a while now, and this is the first post that is describing my process, and the problems I faced.

I chose Azureus because it was my old client, it has alot of functionality that basic torrent clients dont, like selecting files within the torrent to download and others not to. And because I assumed moving it from the Azureus plugin model to OSGi would be trivial.

Here is what I learnt

First 1) No matter how modular you think your code is, forcing it into OSGi shows you all the dependencies errors caused by hacks.

This was obvious for Azureus when pulling out the SWT packages. I wanted a core package that depended on no external UI bundles, like SWT, or Apcahe CLI.
The problem is that some of Azureus's Table classes, that are abstract UI classes organizing information and handling the internationalization, depended on SWT listeners by using instance of a UI util to check weather it is SWT then adding the correct listener. This then caused a core dependency on SWT interface, SE 101 bad.

Second 2) Before OSGi people solved the problems it addresses in a custom manner, making it difficult to learn and build extensions for those models.

Azureus uses its own plugin model to create and deploy plugins, for this it then creates its own class loader, its own life cycle handlers, update managers, version comparitors, a way of creating and inserting plugins, and a whole lot of code that replicates functionality.

This model does not easily break down into OSGi modules which I thought it would.
It is not an easy process taking legacy code and moving it to OSGi bundles.
Dependencies on the plugin model is tight.

A side project that I would like to see is a set of refactoring tools to break legacy java code into OSGi bundles. If they exist please contact me, anything that would help. One such project I tried was Barrio plugin for Eclipse, this is beta and did not work, for unknown reasons.

EMF EObject dynamic toString() method

2009-08-12T21:15:00.000-07:00


 public static String toString(EObject eobject)
 {
  StringBuffer sb = new StringBuffer();
  EClass eClass = eobject.eClass();

  for (int i = 0, size = eClass.getFeatureCount(); i < size; ++i)
  {
   // Ignore derived features.
   //
   EStructuralFeature feature = eClass.getEStructuralFeature(i);
   if (!feature.isDerived())
   {
    sb.append(feature.getName() + ": ");
    Object obj = eobject.eGet(feature);
    if(obj == null) {
     sb.append("null"); 
    }
    else if(obj instanceof EObject)
    {
     sb.append(toString((EObject)obj));
    }
    else if(obj instanceof List)
    {
     List list = (List)obj;
     for(Object object : list)
     {
      if(object instanceof EObject)
      {
       sb.append(toString((EObject)object));
      }
      else
      {
       sb.append(object.toString());
      }
      sb.append(", ");
     }
    }
    else
    {
     sb.append(obj.toString());
    }
    sb.append("; ");
   }
  }
  return sb.toString();
 }

Laptop LCD + PS3 Eye Cam = Multitouch

2009-07-28T14:40:00.001-07:00

Problem: Want a multitouch for fun!!

Background: I started to build one a while back trying to use a wiimote for the camera. This seemed like a good idea as the wiimote does all its computation on the device, meaning that the computer would have more free CPU as it has to do no image processing. However the wiimote is far to insensitive for FTIR, it responds best to direct light from a IR source.
This setback made me rethink the problem. For a few months.
Some PhD students, from FlipMu and studying at the music school in Victoria University (Wellington, New Zealand), came and gave a talk on their uses of multitouch surfaces creating musical instruments. These guys inspired me to finish the project, and I had a little money that was doing nothing so I started the project back up.

Solution: Build it!

Step 1) The first step is to get the necessary parts together.
Camera: I am going to use the PS3 Eye Cam because it can do 120 frames per second, and that is impressive. I am using a 950nm filter that has been taxed from another project.
Display: I am using an old laptop that I ripped apart and turned (which I had a lot of help with) the LCD display into a flat unit.
Touch: 1cm thick Plexiglas that I bought as an off cut very cheap. This is backed by 2 layers of 3mm Plexiglas to hold the LCD in place.
Light: 8 950nm IR 1.5V at 100mA LED's.
Method: FTIR seems to be the only way for an LCD multitouch screen to work.
Unit: Initially I am building a testing rig, this way I can test different methods and find the optimum method and strategy before putting all the work into a proper case.

Step 2) Unit: Build experimental unit, cut grind and make Plexiglas layers. This step took significantly longer than it took you to read that sentence.

Step 3) Electronics: Solder up LED's, extract laptop LCD and parts, remove IR filter from PS3 eye (the best instruction videos are here). This is another step that took longer than it seems.

Step 4) Put Together: So put the unit together and experiment.

Experimentation) Increase LED Count: Initial Experiments show that I need more LEDs. At high frame rates in the Eye Cam the exposure is not enough to see the blobs. So the first step is to increase the LED amount, I am thinking of increasing it to 30.
LED Efficacy: Maximising the effect of the LEDs by placeing them inside the surface, correct angle, liberal use of reflective tape, all to hopefully greatly increasing the efficacy of the LEDs.
Complaint Surface: I have no surface over the Plexiglas for prtection or increasing the FTIR effect. I am going to experiment with different layers of silicone. I have read it is difficult to keep the surface see though with a coat of silicone.

Future) Programming: What I really want is a nice interface between a multitouch screen and the OS. I do not want a custom multitouch OS, because then the technology will always stay niche. I want an interface that can replace the keyboard and mouse while staying fully functional.
The end goal of which is a full media center, probably XBMC, fully interactive with a multitouch display, for selecting movies and music.

Links: Jeff Han: for Multitouch renasonce
Jonny Chung Lee: for Wiimote experiments

EMF + Map = EMap

2009-07-16T00:00:00.000-07:00

Problem: Using and EMF Model to model data, I need a way of retrieving objects quickly, via a key.

Solution: Use an EMF Map.

Step 1) Create a Element Class, "Instance Type Name" set to "java.util.Map$Entry"

Step 2) In this class you need a Attributes or References called "key" and "value". Don't forget containment.

Step 3) Set this type as the reference, with UpperBound set to "-1".

Step 4) The keys "hashcode" and "equals" methods should be implemented. Neither the Key nor the Element class can be dynamic (annoying I know).

More (but not much) at http://wiki.eclipse.org EMF/FAQ#How_do_I_create_a_Map_in_EMF.3F

Heap Dump, With OverFlow

2009-06-30T16:48:00.000-07:00

Problem: A Heap Stack Overflow caused by something unknown in the code.

Solution: Dump the stack when it crashes and then analyze using a Memory Analyzer.

Step 1: Dump Stack with crash: Add "-XX:+HeapDumpOnOutOfMemoryError" this to the VM arguments in the Run Configuration.

(Note: the size of the stack can be adjusted with this "-Xmx300m" argument. Lowering the heap size to reduce the dump size can speed up analysis but may remove the actual cause of the problem.)

Step 2: Run program get dump in the projects' directory (you will have to refresh to see it in Eclipse).

Step 3: Install Analyzer plugin site http://www.eclipse.org/mat/

Step 4: Double click heap dump, should analyze and present information.

Setting up a JUnit Benchmark

2009-06-22T16:50:00.000-07:00

Problem: I want a way of testing multiple algorithms, with the same repeatable tests, and generate a report from the outcomes, to then generate conclusions.

Solution: JUnit 4 Benchmark with Parametrized Tests, and a Report Writer.

Precondition: Must have the same interface to each algorithm, treating each test class as the same may require as large adapter class.

Step 1: Write the base classes:
Step 1.1: First we must create our own Parameterized Test Runner, as the default one from JUnit defines the name of the test as a the paramter number, which does not mean much. It is far more readable if the test name where specified by the users. So copy the contents of the org.junit.runners.Parameterized class into a CustomParameterizedTestRunner (Note: you can use the JUnit inner parametrized annotation class that to define your own). Then change the getName method in the inner TestClassRunnerForParameters class to look like:


 @Override
 protected String getName() {
  return fParameterList.get(fParameterSetNumber)[0].getClass().getSimpleName();
 }

Step 1.2: Write the base Test Class: This class should be extended by all your tests, it defines what classes to run, it inserts the class through the constructer.
First create a JUnit Test, @RunWith annotation should be towards your customized test runner just created. Add a public static method that returns a Collection (a 2D array) and annotate with @Parameters, Then create a constructer that takes the parameters defined in. The class should look like:


@RunWith(CustomParameterizedTestRunner.class)
public abstract class AbstractAlgorithmTest{

@Parameters
public static Collection data()
{
 Object[][] data = new Object[][] {{ new Algorithm1()}, { new Algorithm2() },{new Algorithm3()}};
 return Arrays.asList(data);
}

public AbstractAlgorithmTest(Algorithm a)
{
 this.algorithm = a;
}
...

Step 2: Write the tests that extend the base class. Include static attributes to be accessed by the reporter for results analysis.

Step 3: Write the reporter class, it should have a static main method. It should create a JUnitCore instance, add a RunListener and then call the run method on the classes that should be tested.


 public static void main(String[] args)
{
 JUnitCore juc = new JUnitCore();
 juc.addListener(new RunListener(){
  public void testFinished(Description description) throws Exception {
   System.out.println(AbstractAlgorithmTest.attribute1);
   System.out.println(AbstractAlgorithmTest.information);
  
  }
 });


 juc.run(new Class[] {
   AlgorithmTest1.class,
   AlgorithmTest2.class,
  });
}

Scala/Java Mixed Bundle

2009-06-14T16:01:00.000-07:00

Problem: Creating an OSGi Bundle with both Scala and Java code inside the Eclipse Framework.

Result: Currently too difficult to bother.

Issue: Classpath and Builder : After editing the .project file like said in both this and this articles, the main issue is that the building of the code into a single bin folder, seems to have some issues. If you have a package nz.maori.geek, like me, and you have the Scala class in nz.maori.geek, references in Scala to nz.maori.geek say that classes in this package are not found.

Solution: More time mucking round, or research into the problem, which I dont have as this is a spike.

Workaround: in the two articles discussed above give a good guide into creating a Scala bundle, this is not mixed, but at least there is decent support for it, I have not come across any major problems yet. From the Scala bundle just expose some code that implements a Java interface (because the Java editor still doesnt like Scala methods), as a exported package and use.

Building and Running Eclipse P2

2009-06-03T18:58:00.000-07:00

Problem: Building and Running the Head of Eclipse P2, with SAT4J.

PreConditions: Have Subclipse and Maven plugins installed and setup as described in this blog entry.

Solution :
Step 1) Setup Network: Using pserver on some networks (for some reason) does not require going through the proxy. If this is so add the P2 CVS Server to the "Windows-->Preferences-->General-->Network Connections-->No Proxy" list.

Step 2) Get Eclipse: Get the most recent stable build of Eclipse and use this as the Development Environment.

Step 3) CVS Checkout: Add CVS repository ":pserver:anonymous@dev.eclipse.org:/cvsroot/rt"

Step 3.1) Checkout "HEAD --> org.eclipse.equinox --> p2 --> org.eclipse.equinox.p2.releng"
Step 3.2) Select org.eclipse.equinox.p2.releng project and "Right-Click -->Import Project Set"

(Note: Select the quoted text, open the CVS Repositories view, and select Paste Connection or hit Ctrl+V to add the connection)

Step 4) Get SAT4J: Add the SAT4J and Apache SVN repositories.
Check out the "SAT4J-->maven-->trunk" project as a maven project.
Check out the Apache "commons-->proper-->beanutils && cli --> trunk"
(Note: Dont forget to Update Project Configuration)

Step 5) Validate Tests: Run some tests over P2 with "Run-As Junit Plugin Test". Go to the test "Run Configuration-->Main-->Run an Application-->[No Application] - Headless Mode", if this is not set some tests will fail or freeze.
Also to speed things up restrict the test environment by deselecting all the unessesary plugins in "Run Configuration-->Plugins".

Useful Links (where I plagiarized most of this from):
http://wiki.eclipse.org/Equinox_p2_Getting_Started_for_Developers
http://wiki.eclipse.org/Equinox_p2_tests

Building and Running Apache Felix in Eclipse

2009-06-02T23:22:00.000-07:00

Problem: Getting Apache Felix into Eclipse for Development and Testing.

Solution :
Step 1) Intaling the Plugins into Eclipse:
The Plugins we will need for this are (Links are to the plugin site)

Subclipse
Maven (note make sure to install SCM for SVN)

Step 2) Setup Maven:
First you need to change Eclipse default run time to the JDK. This is done in "Windows-->Preferences-->Installed JRE's". Add the file path to your JDK and untick the default JRE, this will force all things to run in the JDK.

Also is may be necessary (it has not been determined to be, but is reccommended) to run in Eclipse itself in the JDK, so alter the eclipse.ini file found in the Eclipse.exe folder to include the arguments with a link to the jdk virtual machine i.e.
"-vm
C:\Program Files\Java\jdk1.6.0_13\bin\javaw.exe"
(Note: The end line character is not a typo, it is a must)

Next you need to setup the Proxy for Maven to download through this is done by editing (or creating) the ~/.m2/settings.xml file to include the proxy information





myproxy
true
http
tur-cache.massey.ac.nz
8080
User
1111

Step 3) Setup Subclipse:
Setting up the Subversion Proxy in ~/Application Data/Subversion/servers file. It is pretty straight forward, just do all the editing under the "[global]" tag.

Step 4) Get Felix:
Adding the Felix SVN repository (<-----) to your subclipse. Right click Felix-->trunk, and select checkout as Maven Project.

(Note: This will build all the projects (a lot of them) and add them to your workspace. Go get a coffee, this is going to take a while!)

Step 4.1) Once the projects have been downloaded to setup the sources folders "Right-Click-->Maven-->Update Project Configuration"

Step 5) Check it is Working: To check that you have a working copy of Felix, open up a few of the Maven Projects created and right click --> Run As --> Maven :: Test. This will run their test suits and hopefully you will get the all green passes from JUnit that I like to see.

Google Gadget Customization Exercise

2009-06-02T22:26:00.001-07:00

Problem: Gadget had a border I didn't like and I wanted to remove it to use on my blog.

Solution:
Step 1) Find the code: The source code, a basic XML file, is located in the html source of the "Configure Gadget" in a hidden input tag called "gadgetUrl". Go to this link, and copy the source.

Step 2) Replicate Functionality: Go to the Google Gadgets Editor and paste the copied source, save, and at the top left there is a link to the XML file you just created.

Note) The Google Gadgets Editor allows the editing and upload an publication of an XML files. This would be a great place to upload loads of small RDF descriptions for publication.

Step 3) Edit and Test: open up "Add a Gadget" in blogger layout, and select "Add your own", in the text field add the URL to the recently created document. As you change it click update, and it will update the gadget for you!

Proxy Problem with Eclipse 3.5 RC2

2009-06-02T20:00:00.000-07:00

Problem: Eclipse 3.5 RC2 was intermittently throwing errors at different times when trying to access the outside world (SVN, CVS and update sites).

Tried (Unsuccessfully): Setting the proxy in different places, "netstat 100 -b" shows network connections listing the programs that created them. This showed that Eclipse was using the proxy but it was not working.

Solution: The Socks Proxy takes authority over the HTTP proxy, so that if it is set it uses this one. Unsetting the Socks Proxy solved all of the problems except CVS connection.

Problem 2: CVS not connecting when everything else does.

Solution: The CVS protocol pserver which I was using to access Eclipse P2 Source, is not needed to be proxied on my network. This means that adding it to the proxy ignored sites let me access it over CVS.

Why did noone tell me about HashSets Before

2009-05-28T01:47:00.000-07:00

So I have some massive calculations to do while looking into Component Repositories. I am trying tor work out the total amount of possible solutions that a OSGi Bundle can have in a repository, currently testing OSGi Bundle Repository. The maximum number of solutions a query could have is the numbeof depended on bundles (D(b)) (2^D(b))-1. This is pretty big, if you have 10 you could have 1023 combinations installed in OSGI, and this is a small number, when it gets into the hundred I worry.

So I have to query over a large space. I was using linkedLists and ArrayLists till I discovered the HashSet. Unbelievable the LinkedList's removeAll function is O(n^2) why is this, because for every item, it must check contains, and it checks contains by going through each element, this is O(n) so O(n) * n ... HashSet is O(1), it is a lockup.

It changed my running time on small sets from 30 seconds to 3. I love HashSets

OSGi Package Classpath Problem

2009-05-27T17:29:00.000-07:00

So after looking into KnopflerFish OSGi packages on the OSGi Bundle Repository I realized something strange. Bundles import a package they export! I.E.

Bundle-SymbolicName: org.knopflerfish.bundle.console
Import-Package: org.knopflerfish.service.console,org.osgi.framework,org.osgi.service.log
Export-Package: org.knopflerfish.service.console;uses:=org.osgi.framework;specification version=1.0.0

I wondered if this was superfluous, but apparently it is not. Stupid me! I thought that the first place on the classpath would be packages in the bundle, but it is the resolved imported packages.

To test this out I created two simple bundles, A and B, each exported a package "nz" and imported it as well, this package had a class A that in its constructor sysouted different lines. Both had activators that simply created A();

Each time it was run both activators wrote the same message on the console, until you explicitly stated versions and version ranges to exclude each others package. Without the version range the package selected is always the greater version.

What this means is that I need to read more of the OSGi Sevice Platform Spec to get a better understanding of resolving issues.

Reasons for Going back from Ubuntu

2009-05-26T14:42:00.001-07:00

I installed Ubuntu 8 months ago, since this I have loved and hated it.
Here is a list of reasons why I am going back to Vista, in an attempt to let Ubuntu developers in on my perspective.

I was running Ubuntu on my Laptop, it has an Intel Graphics chipset, this means that I cannot run compiz when running an external monitor as well, which I always do. I cannot use any programs that depend on compiz to run, so it makes my desktop ugly, annoying and slow.

Unstable wifi connection, dont know why but when using vista it doesnt constinaly drop my connection to my university network. I attpempted to fix the problem and got no where.

A large part of my day is dedicated to procrastonation, and a Slow Firefox is incredibly painful, each time I get on a windows machine and use firefox, it pisses me off that it seems so much faster!

What I liked are basically all the nice programming stuff, like bash, python libs, Graphviz, apt-get, not having to restart for updates, and loads of other really nice things.

So instead of losing these, I am Virtualizing my Ubuntu using VMWare under windows.

I will writeup later about how this goes.

EMF Contains Problems

2009-05-21T20:34:00.000-07:00

I had this problem where I would add an object to a EObject, and then later in the code it would not be there. This became a large puzzle to me, I am not an expert in EMF, but after much trial and error, I have found the problem.

Containment!! This is a greatly annoying problem. EMF is smart, it sets an owner for an object. If you add it to a EList where .isContainment() returns true, it will remove its self from its previous container and put it in the new one. This is also how the serialization works out if an object is not contained anywhere.

My solution to this, was pretty hacky, though turning out to be extremely useful. I set up a tree of clone() methods where it would clone an object down to its primary information. This is a little expencive, but it is sometimes necessary. For instance I have an object that is a container, I want to generate another container from that with a slightly different state, this is where the clone method is necessary.

The implications from this are that the standard containment list, EObjectContainmentEList, doenst use .equals() when checking for containment, so I had to override this, it is easy, but a bit of a hack. It requires some duplicate code to a super super class, because of some optimazation code in the super class.