Developer Bits on Sustainable Computing

Monday, August 9, 2010

The world is trying to “green up” our cars, light bulbs, clothing, and even our laptops. Apple is pushing the greenest laptops of all time. Google goes lead free and commits to 100% recycling practices. Massive data centres recycle water and implement radical new energy efficient cooling technologies. But, as consultants, we primarily write code – what role do we play in sustainable computing? Some will argue that our role here is best served by the principle of, “Every little bit.”.

A quick search of the Internet reveals thousands upon thousands of books, articles, and studies focused on becoming more efficient. Blogs on the agile methodology preach bringing speed to development and TDD strives to make the QA cycle a blip on the project timeline. Grails and Spring Roo bring projects and prototypes online in minutes rather than days. It cannot be argued that more efficient development reduces cost and makes good business sense – but are there other benefits as well?

Doing the same work in less time
As developers our days mandate carbon output – what we get in return for our expenditure is variable. As we develop we need to be conscious about making the most of our machines each minute that we are in front of them as well as time we spend away from them. An few moments of consideration can save thousands of watts of power each day. Do you stare at your console while compiling or do you execute other tasks during that time? Do you practice TDD to remove much of the overhead of manual testing? Do you hibernate or fully shut down your machine when leaving for meetings or lunch? These are all small things by themselves, but few million persons doing them everyday makes a very large difference.

Conserving bandwidth
Each bit we push and pull across the wire has a cost associated with it. As such, it is in everyone’s best interest to conserve bandwidth (unless you are in the business of selling bandwidth) at every turn. Servers will do less work, switches and firewalls breathe more easily, pages load faster, and all while using less energy and costing your client less. Here again, the work we do can have a large impact over time.

  • Have we minimized and optimized CSS and JavaScript?
  • Are we serving only compressed images?
  • Have we done all we can to leverage client side caching?
  • Is Ajax being used to reduce full page loads?

All of these things result in more speed for the user, less cost for the client, and less carbon out of the pipe.

Optimising code
Servers are like athletes – the harder they work the more energy they need to consume to keep going. World class athletes have figured out that at a certain level of performance even the smallest gains in efficiency determine victory or loss. Cyclists shave fractions of grams of weight off each component to save weight, then test in wind tunnels to reduce drag. Swimmers now wear full body suits of technologically advanced material modeled after shark skin to reduce friction in the water. They do this all to finish fractions of a second faster than they could before. Can we learn from the world’s elite athletes and reduce total clock cycles necessary to compete a task?

Here again we have a situation where everyone wins. When we fully optimize our code we will use fewer resources to complete the same set of tasks. Our code will be faster, the user experience better, client costs reduced, and less energy will be wasted. While techniques for code optimization is beyond the scope of this writing it is something that we as developers need to be conscious of and work to master.

Summary
While the methods and mechanisms that can be used to make software an integral part of sustainable computing are seemingly infinite, we can all work to implement a small, finite set of options every day and together make a huge difference. Whether we simply add break statements to for loops or implement reduced carbon data centres, we can all do something everyday to reduce the number of bits of carbon we produce.

How Do Annotations Work?

Friday, August 6, 2010

Recently, I was giving an overview of Java to a bunch of C/C++ developers to help them bridge the gap. Mostly I ended up assuring them that they knew what they were doing and filled in very few gaps. I did enlighten them on some of the fun things like Collections and some confusing things like Date. We covered lots of ground in those few hours of presentation and banter, but one question I couldn’t answer was about annotations.

I had put a few code samples in my presentation that I’d pulled from a recent project. Stuff I knew worked, so I wouldn’t be surprised with any hastily thrown together code with style, format, or syntax errors. One of the classes was an annotated Spring class, with some code not too much unlike this snippet:

@Autowire private SomeBean someBean;

We went down a path of queries related to how to create new ones, which is a simple @interface declaration like the following.

public @interface Foo{ }

They queried how to put them in the code, and I pointed back to the example. I also showed them some other examples related to using them on class definitions, methods, and on parameter values, as defined by the @Target annotation and ElementType enum. We looked at examples of commonly encountered annotations such as @Deprecated and @SuppressWarning among others. We even looked at the different uses of @Retention and discussed the RetentionPolicy enum; how SOURCE is used to provide hints for the compilers (and IDEs) but aren’t retained in the compiled code, how CLASS is retained in the code but not necessarily available at runtime, and how RUNTIME is certain to be available at runtime.

Then they wanted to know how to use them. Not how to implement them, but how to access the @Retention(RetentionPolicy.RUNTIME) declared annotations.

I hated to admit that I didn’t really know. I knew there weren’t any frequently used utility classes for grabbing annotations, but that there were annotation-related methods on the reflection classes. I knew in practice that they were used, but not directly how, so I set out to learn so I could completely satisfy their questions.

To be sure, I use annotations all day long. I probably @Deprecate a method every day, sometimes permanently, sometimes just to quickly find uses as Eclipse is faster at adding them to problems than it is at searching. Any class that extends or implements another is surely fraught with @Override annotations. I probably write twice as many @Test annotated methods than anything else. I have @Autowire in nearly every function-full class of a Spring application, and every Hibernate project is filled with the @Column and all the other JPA annotations.

What I hadn’t had to do, though, was write any code to find and use annotations. I can’t even recall passing an annotation to a method to try to identify them. After much digging, I ascertained that indeed there aren’t any utility classes that aid in finding or using annotations. There’s got to be some heavy work behind the classpath scanning in frameworks like Spring and Hibernate to find annotated classes and methods. I’m not quite prepared to dive into that, but let’s look at some simple cases of using annotations.

First, a simple set of annotations should be built for our various uses. Let’s take one for each kind of element: class, method, and property.

@Target(ElementType.TYPE) @Retention(RetentionPolicy.RUNTIME)
public @interface TypeAnnotation { }

@Target(ElementType.FIELD) @Retention(RetentionPolicy.RUNTIME)
public @interface FieldAnnotation { }

@Target(ElementType.METHOD) @Retention(RetentionPolicy.RUNTIME)
public @interface MethodAnnotation { }

@Target(ElementType.PARAMETER) @Retention(RetentionPolicy.RUNTIME)
public @interface ParameterAnnotation { }

There are actually eight different ElementTypes that can be used, but these are surely the most common, and are certainly enough to provide a thorough example. I’ll expand on the simple examples here, but I wanted to first discuss the annotations on our annotations.

The @Target annotation lets the compiler know where the annotation may be used. Compilers and IDEs will warn if the annotation is used on the wrong type of element. If the @Target is left off, the annotation may be applied anywhere without warning. The ElementType must be defined, and allows multiple, but not duplicate entries. Multiple ElementType allow the annotation to be used in any of those instances.

The @Retention annotation lets the compiler know how long to maintain the annotation. As mentioned, there are three RetentionPolicy values that are allowed, SOURCE, CLASS, and RUNTIME. If the @Retention annotation is used, a RetentionPolicy must be declared. If the @Retention annotation is not used, the default is CLASS, which will keep the annotation in the compiled class, but it may not be available to the runtime.

The @interface is used to define the annotation. The name of the interface is then used when putting the annotation in source later. Other than that modification to the naming, the annotation is declared much like any other interface, in that you can declare member variables and methods. It is important to note these aren’t intended for use as regular interfaces, so there are some other restrictions. Member variables must be public, static, or final, but may be of any valid type. Only public and abstract modifiers are allowed on methods. And only a small set of return types are allowed for methods; strictly, primitives, Strings, enumerations, other annotations are allowed, or 1-dimensional arrays of those types.

One more bit, if you want to make an annotation that takes a value without a name, you must declare a member variable named “value.” That is, like @SuppressWarnings(“unchecked”). It can be a single value or array, depending on your needs. If you define it as an array you can provide several values by wrapping them in squiggly-braces like @SuppressWarnings({“unchecked”,”unused”}) would do. If you do not wish to use “value,” the variable must be explicitly declared (and, truly, value is optional) such as @SuppressWarnings(value=”unchecked”).

Other differences will become apparent as the article continues, and as you play with annotations of your own. In trying to keep the article short, and not delve too much into every possible use case, I’ll show some simple uses, point out some of these other nuances of @interface definitions, and leave it to the reader to expand on that.

We’ve got some trivial annotations set up, so let’s apply them to a simple class.

@TypeAnnotation class Foo {
  @FieldAnnotation public Object object;

  @MethodAnnotation public Object setObject(@ParameterAnnotation final Object object){
    this.object = object;
  }
}

If you’ve used annotations in your code, you’ve probably used some or all of these kinds of annotations. Certainly, the annotations used before have actually done something. Before we get into doing something, let’s try to find the annotations at runtime.

I mentioned previously that Spring and Hibernate and other frameworks have functionality to scan classes in the classpath for annotations. If you’ve taken a look at how they do it (they’re both open-source projects, so dig in…) or hit a search engine looking for the answer, you probably have seen or correctly surmised that interrogating a classpath is not as easy as it seems. As such, I’m not going to try to lay that out here (sorry to whet your appetite), but instead will use the annotations in a more explicit manner.

First, a simple method to see if the annotation exists, just as an example of accessing them.

public boolean isAliased(final Class type) {
    final TypeAnnotation typeAnnotation = type.getAnnotation(TypeAnnotation.class);
    return (typeAnnotation != null);
}

The trivial example interrogates the provided class using the Class.getAnnotation() method. There’s a similar Class.getAnnotations() that will return an array (zero-length if empty) of the annotations if you want to check for more than one. This would be used very simply by something like the following snippets, the first returning true, and the other false.

isAliased(Foo.class);
isAliased("yes, a literal".getClass());

It isn’t a terribly useful example, so let’s make a method that does a little more. First, let’s expand one of our annotations to give us some runtime differences from source.

@Target(ElementType.FIELD) @Retention(RetentionPolicy.RUNTIME)
public @interface FieldAnnotation { String value(); }

Here we’ve changed our FieldAnnotation to require we provide some additional information in the form of a String property. It’s been named “value” for convenience, so we can use it without forcing the use of the name. Note that it looks like a method declaration, but we’ll be treating it as both a variable and a function.

If we applied the same annotation to the same class as before, we’d run into an error as we’re missing a required property. Let’s update our class a little and give this value some functionality.

@TypeAnnotation class Foo {
    @FieldAnnotation("here") public Object object;

    @FieldAnnotation("there") public Object other;

    @MethodAnnotation public Object setObject(@ParameterAnnotation final Object object){
      this.object = object;
    }
}

Again, the class isn’t terribly useful, but we now have two annotated member variables with different names. Let’s make a quick method to set those values by their annotation name, ignoring their object name.

void setPropertyByAnnotationName(final Object object, final String name, final Object value) {
    final Field[] fields = object.getClass().getDeclaredFields();
    for (final Field field : fields) {
        final FieldAnnotation fieldAnnotation = field.getAnnotation(FieldAnnotation.class);
        if(fieldAnnotation!=null && name.equals(fieldAnnotation.value()){
            field.set(object, value);
        }
    }
}

Lots of spots for error there, as there’s no null checking, it assumes the field is accessible, and the object is of the right type…but it matches our simple Foo class, so we’ll let it go for now. As with the Class, there is a Field.getAnnotations() in case you’d like to investigate more than one annotation. What we have is a method that interrogates our object (first one) to look for fields (member variables) annotated with our FieldAnnotation. When it finds one, it checks to see if the name matches the value (note it looks like a function here) in the annotation. When there’s a match, it sets the value as provided by our value parameter. This would be used simply as thus:

Foo foo = new Foo();
setPropertyByAnnotationName(foo, "here", "this is set!");
setPropertyByAnnotationName(foo, "there", Calendar.getInstance());

With this, we should end up with a Foo.object that contains the String “this is set!” and Foo.other that contains a Calendar with the current time in it.

One might desire to ask what good is that? We surely have access to Foo.object and Foo.other directly in this class, but it will certainly be the case that we might not have such access in other classes. Additionally, with the annotation, we don’t even really need to care that it’s a Foo object. We could very easily declare another class with similar annotated members and have their fields accessed with the same method we just created. This is roughly how Spring’s @Autowire works. They, of course, do a bit more in terms of recognizing names and types and whatnot without explicit definition, but this is the crux of it.

Accessing methods is done pretty much the same way. Let’s use the @MethodAnnotation to use the setter we’ve got. Note that calling methods is a lot trickier because the annotation itself has no way to ensure that the parameter list is correct. It is up to the method using reflection to determine if the prototype of the method is something it can work with. For brevity, I’ll leave this using the original annotation, which means every annotated method will be called and its value set to the same thing; the same modifications on the @FieldAnnotation can be done here for similar functionality.

void callMethodByAnnotation(final Object object, final Object value) {
    final Method[] methods = object.getClass().getDeclaredMethods();
    for (final Method method : methods) {
        final MethodAnnotation methodAnnotation = method.getAnnotation(MethodAnnotation.class);
        if(fieldAnnotation!=null){
            method.invoke(object, value);
        }
    }
}

Again, a slew of opportunities for errors with missing null-checks, accessibility, and the mentioned parameter list validation. Also, there is a similar Method.getAnnotations() method that will return an array of annotations in case you want to check for more than one. This method simply checks the class for all methods with the right annotation, and if found, tries to invoke the method with the provided object as the parameter. Again, with this functionality you can find methods in classes without caring about their real names. This, again with more guts, is how Spring handles its @RequestMapping methods in @Controller classes. As before, this can be implemented with this simple

Foo foo = new Foo();
callMethodByAnnotation(foo, ""this is set!");

In the end, we end up with Foo.other set to a String saying “this is set!” using the annotation instead of calling the method directly.

Trickier is handling the ElementTypePARAMETER annotations. This is because there’s no direct reflection access to the parameter list. There are a few arrays that can be gathered to help identify the parameters, but the parameters are actually only realized when you use the Method.invoke(), so to use these, you need to handle all of the parameters by annotation, name, or type. A typical use would be some form of IOC or other runtime association of parameters to other values available to the application.

Why this is tricky is that there is no opportunity, except perhaps inside an invoked method, to access the parameter annotations when calling the methods directly. With that understanding, we can see that these are best paired with annotated classes or methods, where the functions are going to be called indirectly, and therefore there is time to interrogate the Method for annotations before calling Method.invoke. That said, let’s modify our other method-calling method to take a map of values to tie to our parameters.

Before we do, though, this brings about another sticky bit. Unlike classes, member variables, and methods, parameters don’t always keep their names. When classes are compiled, the parameters are reduced essentially to a type and order. The variable name associated with it is not kept with the method unless the class is compiled with debugging symbols. As such, like we did for the fields, we’ll add a required value to represent the name of the parameter. This way, the annotation will maintain the value that we’ll use to pull values from our map, and it won’t matter if debug is enabled or not; our annotation will continue to work.

@Target(ElementType.PARAMETER) @Retention(RetentionPolicy.RUNTIME)
public @interface ParameterAnnotation { String value(); }

And since it’s required with no default, we’ll have to change our class appropriately.

@TypeAnnotation class Foo {
    @FieldAnnotation("here") public Object object;

    @FieldAnnotation("there") public Object other;

    @MethodAnnotation public Object setObject(@ParameterAnnotation("inside") final Object object){
        this.object = object;
    }
}

And we’ll modify the method to take advantage of that annotation.

public void callMethodByAnnotation(final Object object, final Map map) {
    final Method[] methods = object.getClass().getDeclaredMethods();
    for (final Method method : methods) {
        final MethodAnnotation methodAnnotation = method.getAnnotation(MethodAnnotation.class);
        if (methodAnnotation != null) {

            final Annotation[][] parameterAnnotations = method.getParameterAnnotations();
            final Object[] parameters = new Object[parameterAnnotations.length];
            for (int i = 0; i < parameterAnnotations.length; i++) {
                parameters[i] = null;

                final Annotation[] annotations = parameterAnnotations[i];
                for (final Annotation annotation : annotations) {
                    if (annotation instanceof ParameterAnnotation) {
                        parameters[i] = map.get(((ParameterAnnotation) annotation).value());
                    }
                }
            }
            method.invoke(object, parameters);
        }
    }
}

Note the busy work necessary to grab the annotation. The annotations come from an array of arrays. The first dimension is one for each parameter, and the second dimension is the array of annotations for the parameter at that first dimension. If there are no parameters, the first dimension array is zero-length. For each parameter, if there are no annotations, that second dimension array is zero length. There is a related Method.getParameterTypes() that could be used to be certain that the type in the map was compatible with the type in the parameter, but ours is an Object, so we can be a little sloppy.

The new method can be used very much like the other, except we'll pass the parameters in a Map. We could, of course, use any mechanism for keeping track of the things that could be put in the parameter list. If you've used Spring to annotate a @Controller with @RequestMethod methods, you'll notice that you can just toss in a whole slew of parameters such as ModelMap or HttpServletRequest and it will figure them out by type.

Map map = new Map();
map.put("inside", "this is set!");
Foo foo = new Foo();
callMethodByAnnotation(foo,map);

Same caveats as before regarding shortcuts taken for brevity... Now when we call our method, it will check our map for the value. If it doesn't find an element in the map that matches our annotation name, it puts null in the parameter list. As before, our Foo.other will have a String with the value "this is set!" when we're done.

That seems like a lot of busy work to make sense of annotations. Really it's a lot of fluff code around it to make it understandable. Really, the magic lies in the getAnnotations() or getAnnotation() methods on the Class and the reflection classes Field and Method, and the Method.getParameterAnnotations() method. And, of course, our trivial examples don't carry much insight into any kind of usefulness, so let's expand on this and make a one-class annotated XML parser. A very simple one, mind you.

Parsing XML is a pain that way too many of us go through. There are plenty of tools to help us do this, so this isn't intended as a replacement or any kind of competition for them. It's just familiar territory, and one I think I can squeak in a small pair of classes for a solid example of annotation use.

Consider this fairly trivial XML, where we have a Foo element with a single attribute, id, and a nested element also named Foo.

<Foo id="alpha">
    <Foo/>
</Foo>

The astute among us can quickly envision a likewise trivial POJO that would do the same. I'm making public members for brevity; for real, we'd use getters and setters, right?

class Foo {
    public String id;
    public Foo foo;
}

With this class in mind, we can see we'll need a couple annotations to handle identifying the class and our members.

@Target({ElementType.TYPE, ElementType.FIELD}) @Retention(RetentionPolicy.RUNTIME)
@interface XMLElement { String value() default ""; }

@Target(ElementType.FIELD) @Retention(RetentionPolicy.RUNTIME)
@interface XMLAttibute { String value() default ""; }

A quick word about the two noticeable differences from the previous examples. First, we'll note that the XMLElement has been annotated with two ElementType targets. This allows the same attribute to be used to identify types and members, as XML elements are often neatly identified with POJOs. Also, I've added a default to each. This is both to show what it looks like and allow us to optionally rename the item in question. A small annoyance with the defaults is that there is no null value. When the value is an array, you can declare an empty list as the default, but that's not really null either.

@interface ArrayExampleAnnotation { String[] value() default {}; }

This allows us to alter our class to add the annotations as follows:

@XMLElement class Foo {
    @XMLAttribute public String id;
    @XMLElement public Foo foo;
}

We could be more explicit and annotate it thusly:

@XMLElement("Foo") class Foo {
    @XMLAttribute("id") public String id;
    @XMLElement("Foo") public Foo foo;
}

Or even make the POJO and XML a little less tightly coupled by changing the POJO names from the expected XML names (this is where the value makes more sense!):

@XMLElement("Foo") class FiddleStix {
    @XMLAttribute("id") public String identifyingAttribute;
    @XMLElement("Foo") public FiddleStix next;
}

Now we just need a function to digest our XML string and populate our POJO. To be short and sweet, and not emphasize the XML processing too much, I'm just going to use the typical W3C DOM classes that come with the Java runtime. Easy, short-ish.

public static  POJO turnXMLStringIntoPOJO(final String string, final Class type) {
    final POJO pojo = type.newInstance();
    final XMLElement topXmlElement = type.getAnnotation(XMLElement.class);
    final Document document = DocumentBuilderFactory.newInstance().newDocumentBuilder()
            .parse(new ByteArrayInputStream(string.getBytes()));
    if (document.getNodeName().equals(topXmlElement.value())) {
        final NamedNodeMap namedNodeMap = document.getAttributes();
        for (int i = 0; i < namedNodeMap.getLength(); i++) {
            final Node node = namedNodeMap.item(i);
            String name = node.getNodeName();
            final Field[] fields = type.getDeclaredFields();
            for (final Field field : fields) {
                final XMLAttribute xmlAttribute = field.getAnnotation(XMLAttribute.class);
                if (xmlAttribute != null) {
                    if (("".equals(xmlAttribute.value()) && field.getName().equals(name))
                            || xmlAttribute.value().equals(name)) {
                        field.set(pojo, node.getNodeValue());
                    }
                }
            }
        }
        final NodeList nodeList = document.getChildNodes();
        for (int i = 0; i < nodeList.getLength(); i++) {
            final Node node = nodeList.item(i);
            final String name = node.getNodeName();
            final Field[] fields = type.getDeclaredFields();
            for (final Field field : fields) {
                final XMLElement xmlElement = field.getAnnotation(XMLElement.class);
                if (xmlElement != null) {
                    if (("".equals(xmlElement.value()) && field.getName().equals(name))
                            || xmlElement.value().equals(name)) {
                        final Object object = field.getType().newInstance();
                        field.set(pojo, object);
                    }
                }
            }
        }
    }

    return pojo;
}

With a little recursion or other care, this simple method could be used to handle nesting our XML so that Foo could have Foo wtih Foo containing Foo... But for our purposes, this meets our needs. Again, much with the caveats of missing care exchanged for brevity. Notice it's a generic class, giving us flexibility to just send it any old class. With our caveats, it makes assumptions that the class is annotated (or null pointer exceptions will occur). The string is quickly (since it's small) into a DOM tree, and we make two passes through it, once for the attributes and once for the elements. We compare the names of the nodes to the names (value) of our annotations, if they're defined, or to the name of the field, if the value is empty. If a match is found for the annotation, we set the value to whatever the XML contained. If a match is contained for the element, we simply instantiate the right type and add it to our object. We could explore the DOM deeper and set its values, but that's a little larger example.

This class could take all three of our example classes (careful, as two of the classes will conflict) and parse the same XML and get the expected results.

String xmlString = "<Foo id=\"alpha\"><Foo /></Foo>";
Foo foo = turnXMLStringIntoPOJO(xmlString, Foo.class);
FiddleStix fiddleStix = turnXMLStringIntoPOJO(xmlString, FiddleStix.class);

What should result is that our objects would have an id String of "alpha" and their same-type member populated with an empty, but instantiated item. Both classes, from the same XML just because of the annotations.

And that, my friends, is how you use annotations. Well, one way...

Packaging your Java application for the Mac

Tuesday, August 3, 2010

The Mac has some nice tools to turn your jar file into a first class Mac application. This is most useful if the application has a GUI interface; Swing, SWT, etc. There are three parts to this process:

  1. Package your application as a jar file
  2. Create an icon for your application
  3. Create an application bundle

Some applications referenced below may not be installed on your Mac by default. If not, you can either install them from your distribution or download them.

I assume you already know how to do step one so we’ll move on to step two.

Find an icon you like for your application. A little Googling will produce a variety of free icons. The png file format will work for you. Be sure to follow any licensing restrictions that apply to your icon and do the following:

  1. Download the png file.
  2. Open the Icon Composer application (its in /Developer/Applications/Utilities).
  3. Drag the png file into the largest box in the icon composer.
  4. From the largest box drag it to the next largest then the next and so on to create various size icons. Be sure to go from larger to smaller for the best results.
  5. Choose File->Save in the Icon Composer and save your new icns file.

Finally, build your application bundle:

  1. Open the Jar Bundler (its in /Developer/Applications/Utilities).
  2. Select the Classpath and Files tab and add your jar file to the classpath.
  3. Select the Build Information tab.
  4. Click Choose next to the Main Class drop down, select your application’s jar file again.
  5. Select the entry point for you application from the Main Class drop down.
  6. Select your JVM Version from the drop down
  7. Click Choose Icon and select the icns file you created above.
  8. Click Create Application, enter a name for your bundle, and click Create

Now you have an application bundle file you can just drop in your /Applications directory to install.

Sun Java 1.6.0_21 Not Detected By Eclipse

Friday, July 30, 2010

A “new” bug has been swirling around places like Slashdot, making it sound like Eclipse is broken. The bug is related to a change in the latest release of Java, v1.6.0_21, not Eclipse. It’s getting some attention because a lot of people are encountering it after accepting an automatic update to Java (particularly on Windows). It is a bug manifested in Eclipse by a change in the new Java runtime. It really goes back to an old bug (squashed in 3.3) where Eclipse couldn’t detect the Sun JVM; same bug now, because they’re changing the VM since Oracle’s sticking their name in everything. A note has been recently added to the permgen size FAQ page at Eclipse.org giving a little more detail than this.

It can be circumvented (or rather, postponed to the point of probably not happening) by increasing the garbage collection memory size by adding (or updating) the following two lines to your eclipse.ini file (on Windows and Linux, look right in the root of the Eclipse folder, for Mac, it’s deeper in the Library folders). If the settings aren’t there to be updated, add them just before the -vmargs, making sure not to split any other parameters.

--launcher.XXMaxPermSize
512m

You can choose a size larger than that if your machine supports it, or the delay until it runs out of memory isn’t enough. Always enter values in MB.

Alternatively (or additionally), you can specify the JVM to use by adding it’s path to the eclipse.ini file (again, before the -vmargs) to avoid the bug in the 1.6.0_21 runtime. This example uses the Java installed with the JDK instead of just the Java SE runtime that the path suggests.

-vm
C:/Program Files/Java/jdk1.6.0_20/bin/javaw.exe

Of course, your path may vary.

The Complete Team

Tuesday, July 27, 2010

I think it’s safe to assume that any software development project has developers participating. In a team environment, the better a developer is at collaborating and working with others, the better the project will turn out. This includes not only interpersonal interaction, but also manging things like overlapping work and code conflicts. It sometimes works to segregate developers or groups of developers on slices of projects, and shuttle the work between the groups via APIs and interfaces, much like the software ultimately will.

It also works well to have collaborative code ownership, where all developers get their keyboards into all of the source and apply their special talents to each part of the project. Fostering this kind of collaboration will ultimately bring the strengths of the entire team of developers to all of the parts of the project. When projects are sliced up and developers are brick-walled from each other, opportunities to have brilliance throughout are lost.

Good Developers All

All software developers are good at it. At least good enough to get jobs. It’s not an easy thing to bluff for very long. It’s important to try to keep them that way.

In many domains, including game shows, the phrase “weakest link” is used to describe a likely point of failure of a system. In software consulting, there are occasionally “developers that shouldn’t be.” We’ve all run into them. They’ve taken the classes, and even done some work, but there’s something off. Perhaps it’s the pace they can’t maintain. They might have a comfort zone, a favorite type of software or tools or product, that once they leave that realm they become…less good. Maybe they have a problem grasping things out of context or without so much understanding that everyone else could have already written the software already!

It’s important to recognize these trouble spots early, and wed them out. I’m not suggesting anyone should be lose their job over weaknesses, but everyone should know if there’s a place they’re good and where they’re not good. Having developers work on things they’re not familiar with or comfortable with is a good way to get them to learn, but it’s also a likely spot for schedule slips, performance dips, and bugs. When someone is stepping into a new technology or programming paradigm, give them some training, and not always on their own, or pair them with someone who’s done it or something like it.

It is often the case that the weakest link may not be an issue of skill, but one of utilization. If a project is segmented, it can very well happen that one developer on one project is overwhelmed with work queuing up, while others are idle. This happens for a variety of reasons. It sometimes waxes and wanes in cycles that are impractical to try to predict.

When someone can’t get the hang of it or when someone is underutilized, try finding a place on the project they can succeed and add value. In a divided software project, there may be developers who aren’t contributing fully to their vertical bit for whatever reason. It may very well be the case that a gap exists elsewhere on the project that this developer could provide much value. In a collaborative environment, the developer who’s lost or idle can find tasks that suit their skills and time and the project can be better for it.

Ideally, everyone working on a project should be able to add their bit to every bit.

Stickler for Standards and Patterns

It’s easy to get carried away, but it’s important to have some members of a complete team that encourage and enforce following standards and recognizable patterns.

It’s easy to get into a flow and let the little things get away. Really, when software executes, the names of the packages and classes and methods and members and variables are largely irrelevant. Case-in-point: a great deal of JavaScript, C, and other software is run through obfuscation before being released in an effort to reduce having someone swipe it and claim it as their own (or whatever nefarious purpose they may have). Anyone who’s seen obfuscated code recognizes that the time spent ensuring that variable names follow some coding guidelines is wasted effort.

However, anyone who’s had to adapt or maintain someone else’s code recognizes the time saved when the not-obfuscated code is properly adorned with a standard or guideline-compliant set of names. Having someone (or a few) on a team who enforces even simple things like naming standards can add incredible value and time savings during development and later maintenance of any project. Done right, their influence and encouragement will bring everyone on the team to embrace and follow whatever guidelines have been agreed to.

The same thing goes for following common software patterns. By and large, I’m all for patterns as a way to communicate and provide direction, but I’ve run into a few extremists who insist on wrapping every little thing in some kind of pattern label. Sometimes it can’t be done, so relax a little. At the same time, having everyone familiar with the frequently used and sometimes necessary patterns is essential. Everyone should be able to use Singelton, Fly-weight, Factory, and Visitor patterns; and probably has done so without even knowing it. Anti-patterns, too, should be known and shared, and avoided…surely we’ve all been on projects that have been one big “ball of mud” or faced “interface bloat.”

Ideally, what will happen is that the Sticklers for Standards and Patterns will be cruising through the code working on their bits, and either see and correct a non-compliant bit of code, or point it out to the individuals or group at large. When near-misses occur and are corrected just through collaborative code ownership, all of the software gets better for the effort.

Also, the more standard following and pattern recognition the team can do is better for development overall. Each developer improved with standards and patterns is a little bit better for it.

Relentless Testers

In a large project, it’s important to test the software as it is developed. This includes sitting in front of a running version of the software as well as writing unit test software on the software base itself.

Some developers don’t agree in the value of writing tests on software. Some developers dig writing the tests on software as much as writing the software itself. A balance needs to be met to ensure that all of the time isn’t spent writing tests, or that software isn’t getting written just because it satisfies a test. Balance needs to include the usefulness and necessity of having the software.

On a large project, especially one where every developer is free and comfortable to change any of the software, unit tests are measurably necessary. Some disciplines call for “Test Driven Development,” but I contend that it’s really often more about “Test Defended Development.” In a proper unit test, the implementation inside a method isn’t necessarily what’s being tested. Sometimes that’ll happen in order to deal with that blurry line between unit test and integration test. In the end, though, a test should prep for a method call, call a method, and evaluate the results, ensuring that the desired and expected changes or events happen. Once a method is so protected by a test, changing the method appreciably should affect and break a test; this will encourage the developer making the change to ensure that the change is worthwhile and necessary. The dread of breaking tests should keep nitpicking to a minimum. Conversely, correcting broken tests when necessary changes are made will further strengthen the software.

The relentless testers will ensure that the tests add value to the project, weeding out the useless tests added just for “coverage numbers.” I loathe the test wrapped in a try/catch block that allows the test to pass regardless of the actual outcome of the method, neither preparing or evaluating results, and often not really testing anything; just getting the count up on the number of tested methods.

Ideally, some of the developers on the team will be the sort that does unit testing right. As they do their parts on collaborative projects, they may encounter weakly covered software or tests that aren’t efficient or valuable. They’ll take their moment to shine and correct the tests, or even help the other developers improve their testing skills. As these developers relentlessly add to the coverage and improve the defense and definition of the software through testing, the project as a whole gets stronger and defects are greatly reduced.

Master of Performance and Brevity

Similar to the Stickers for Standards and Patterns, the Masters of Performance and Brevity will be those developers with eyes and knacks for identifying software that can be improved or made ore concise or both. We affectionately call them “tweakers,” but they add value in all kinds of ways.

Sure, we’ve all written hasty loops and methods that inefficiently step through collections and whatnot. A second pass is often used to clean up that “get it done” initial effort. The tweakers will take it a step extra and optimize the code, for both performance and brevity. Extra temporary declarations and casts will be removed. Inefficient and clumsy code will be streamlined and refined to be elegant. Elegant and clean code is easier to maintain.

Sometimes the tweakers need to be reigned in as they’ll spend large and quantifiable time working to improve processes that will result in shaving machine cycles from software that can result in immeasurably small gains such that the input will never equal the output.

Ideally, all of the developers on a project will write efficient code, at least by the time it gets committed to the team repository. The tweakers will then further clean the code when they encounter it, improving the project as a whole. When these developers are on segmented projects, only some of the parts get optimized. Also, as the not-quite-tweaker developers return to those parts of the code, they’ll get exposed to the tighter and cleaner code, and hopefully begin to incorporate that.

Wizard of Tools and Environments

Unless you’re a total die-hard, editing code with the most bare of text editors, compiling, linking, and otherwise packaging from command-line interfaces, you’ll be using an IDE. Sometimes it doesn’t matter which IDE is used; the code should be compliant to the language in which it’s written, not the editor used to write it. Some bits of the environment leak in, however, and influence the project structure, if nothing else. Sometimes the combination of tools used (e.g., Eclipse, Subversion, and Maven) have myriad options and combinations of configuration. It helps to have someone familiar with the ideal development environment.

Likewise, the targeted testing or deployment environments can have an impact. If the team uses a continuous integration tool (as they should) to do an out-of-the-IDE and off-the-desktop build and run of the tests, this will add complexity as there needs to be an additional build tool (like Ant or Maven or make) that might do things differently than the IDE does. And whether deploying a stand-alone application or some kind of container-deployed application, there are additional considerations to have regarding the end of the development stream, when the deployable is made. If a web-application, for example, is being developed, separate packages may need to be built and tested for the different application servers; a simple WAR file gets a little complicated when deploying to Jetty or Tomcat or JBoss or Glassfish as each may include different bits in their default classpath or may work differently with the classes and archives provided in the WAR as they implement their classloaders differently. And let’s not forget the differences between deploying on a Windows and non-Windows environment, or even the occasional nuances and inconveniences of the differences between the non-Windows (and heck different varieties of Windows) environments.

Ideally, the Wizards on the team will help ensure that the tools used combine to make a harmonious, not conflicting environment. Sometimes this can include moving files around to correct a multiple-platform build consideration, or to correct a version or dependency conflict. A developer may add a Maven dependency, for example, that is already resolved by another dependency, so the Wizards will remove the unnecessary one, or correct the version of the other, to be sure the tree is neat and clean, and easier to maintain.

Regulator of Plans and Tasks

Many of us dread the endless meetings to hash and rehash project specifics. Some of us live for those meetings. In moderation, there’s value in collaborating together, if for nothing else than disseminating information quickly.

As a project cycles along, things will need to be tracked and changed. Velocity is hard to maintain otherwise. It is imperative that a list of some sort be maintained that allows developers to identify, and possibly define, work to be done, and for their managers and peers to know what work has been done. Efficiently tracking tasks, more importantly than time (sorry, bosses), ensures that dependencies are fulfilled and identified as the project stretches through the development cycles. Following a Getting Things Done habit of just doing what can be done quickly and noting what must be postponed ensures that nothing is missed.

Additionally, if a developer in the middle of a task notices something that needs to be done, either to meet some missing or improved requirement or process, then another developer who may be between things to do, or even working on the same or similar tasks, can do that and perhaps be done before the developer who noticed it has finished their initial task.

Ideally, the regulators will help keep things flowing smoothly. They’ll recognize duplicate or similar tasks and eliminate or relate them, keeping task lists from getting overwhelming. If everyone participates in the management of plans and tasks, at least insofar as they take tasks off the queue that they can do, and add tasks to the queue that they cannot do (for time or skill reasons), it works out that projects can actually kind of self-manage.

Subject-Matter Experts

Of course, it helps to have someone who knows about the software being written. It’s quite possible to have developers who don’t know all of the specifics of the project. Certainly to some degree they should all know something about the project, but that can still be at a pretty high level.

Arguably, most business software is some kind of interface-to-database-and-back-again. That interface can be a stand-alone application or a series of web pages, or maybe even just some web service or RSS feed kind of thing. The database could be a bunch of in-memory data, files, or an actual database. It could be complex and send information to other applications in a cloud, or simple and just run on its own.

Knowing that to some degree, some parts of all software are frankly just software, it’s possible to have developers writing software without knowing about the stuff they’re writing.

Someone should, though.

Ideally, there would be some developers who are experts to some degree on the subjects that make up the project. This could mean all about the finances of the banking application, or just about the security technologies used in it. In a segmented environment, these experts get blocked from sharing their expertise with the other parts of a bigger project, but in a collaborative environment, they get to help everyone fill in knowledge gaps.

Jacks of All Trades

Having people that have more than one of these skills is a bonus. Further improving a complete team is having multiple people with multiple skills that aren’t all the same sets, overlapping to provide multiple coverage in all of the collaborative tools and disciplines.

Certainly, some shining stars can do some or a lot of all of these things. It happens sometimes that when one focuses too hard on one area they suffer in others. Conversely when they focus on all areas, it happens that all areas suffer. A balance must be struck, but breaking out of the “one-developer, one-task” will strengthen everyone working on the team, and strengthen the project overall.

Dojo Datagrid

Friday, July 23, 2010

Recently at a client site, I was assigned a task to find a good data table widget, that would handle column sorting, highlighting, and the like.  But it had to work with the current technology stack.  Since Dojo was already part of that stack, I started there.

To my joy and suffering, I found a little hidden gem called the Dojo DataGrid. Joy, because it covered all the requirements of my client, but suffering, since the documentation for the latest version was completely non-existent.  When I say completely, I mean COMPLETELY.  The web page on  Dojo’s site had TODO under every section.  Doing a Google search found tons of information, but for older versions.

So I thought I would put my findings here, so you don’t have to hunt the info down.

To programmatically build the DataGrid you must first add in the Data grid class, the class of your data store and the source location of the data used for display.  Below I am using a csv file to load data into the grid.  Later, I will show you how to bring in a json object.

<script type="text/javascript">
  dojo.require("dojox.grid.DataGrid");
  dojo.require("dojox.data.CsvStore");
  dojo.addOnLoad(function() {

  // our test data store for this example:
  var store = new dojox.data.CsvStore({
  url: 'players.csv'
});

Then layout your table in your desired fashion.  Here I have 4 columns of data. “Field” represents the column heading from the csv file.  “Name” is the column heading title, which will be displayed in the table.  “Width” is the desired width of the column.  You can specify a specific pixel width or let the browser assign a width to the column by adding the key word ‘auto’ to the width parameter.

// set the layout structure:
var layout = [{
    field: 'Player',
    name: 'NHL All Time Point Leaders',
    width: '200px'
  },
  {
    field: 'Games',
    name: 'Games Played',
    width: '50px'
  },
  {
    field: 'Points',
    name: 'Points',
    width: 'auto'
  },
  {
    field: 'PPG',
    name: 'Points Per Game',
    width: 'auto'
  }];

Once you have the layout complete, create a instance of the DataGrid object.  Here you fill in your default query, link in you datastore, and tell the DataGrid you use your previously created layout.

// create a new grid:
var grid = new dojox.grid.DataGrid({
  query: {Player: '*' },
  store: store,
  clientSort: true,
  rowSelector: '20px',
  structure: layout
},
document.createElement('div'));

After that, all you need to to is tell the browser where the grid should be located once created

// append the new grid to the div "gridContainer":
dojo.byId("gridContainer").appendChild(grid.domNode);
// Call startup, in order to render the grid:
grid.startup();

<body>
  <div id="gridContainer" style="width: 100%; height: 100%;"></div>
</body>

To use a json object instead of a remove the csvStore class and replace it with a ItemFileReadStore object

dojo.require("dojo.data.ItemFileReadStore");

Then change the datastore creation like so,

dojo.addOnLoad(function() {
  // our test data store for this example:
  var store = new dojo.data.ItemFileReadStore({
  url: 'players.json'
});

The DataGrid can also be built declaratively, below creates the above json DataGrid table

<script type="text/javascript">
  dojo.require("dojo.data.ItemFileReadStore");
  dojo.require("dojox.grid.DataGrid");
  var jsonStore;
</script>

<div dojoType="dojo.data.ItemFileReadStore" url="pois.json" jsId="jsonStore"></div>

<table id="gridNode" jsId="grid" dojoType="dojox.grid.DataGrid" autowidth="true" autoheight="true"
query="{ player: '*' }"  clientSort="true" store="jsonStore" style="width:400px" >
<thead>
<tr>
<th field="Player" width="200px">NHL All Time Point Leaders</th>
<th field="Games" width="200px">Games Played</th>

<th field="Points" width="auto">Points</th>
<th field="PPG" width="80px">Points Per Game</th>
</tr>
</thead>
</table>

Once you get the gist of the DataGrid, it becomes easy to build.  Granted this is a simplistic example of it’s use, and there are many more bells and whistle you can add to the grid for functionality.  But, now you have a solid framework to start with.  The example code from the programmatic examples can be download here