Mark Watson's opinions on Java, Ruby, Lisp, and AI

Authors David Salter and Frank Jennings have written a very targeted book specifically for enterprise developers wanting to use NetBeans with the SOA plugins.

Service Oriented Architecture (SOA) can be loosely described as implementing business processes by combining web service calls to available services. Just as you would probably not want to perform database operations without transactions, when you combine web service calls for specific business processes you want to wrap them in transactions and using BPEL (Business Process Execution Language) modules makes it easier to express logic and error handling. The BPEL designer plugin lets you work with a graphical interface and avoid tedious hand editing of XML files.

Just like using full J2EE stack adds a lot of complexity, with attendant benefits, to large scale Java server side development, SOA also adds a lot of "baggage" to building systems using web services, but as systems become larger and more complicated that structure can really pay off.

There are no magic bullets in software design and development that remove the requirement for hard work in analyzing system requirements and iteratively work on design and staged prototypes: for SOA applications you need to understand your business processes, what events can occur, and what error conditions need to be handled. If you don't understand these basics, no framework is likely to save a project. If you do understand these basics, and you want to use Java SOA frameworks and tools, then this book is a good guide to getting started. I do have a few minor complaints about this book: figures are not numbered (perhaps not such a bad idea, I am just used to seeing figures identified) and there is sometimes too much detail given for some developers: the reader is carefully led through setup, using the IDE plugins specific to SOA (BPEL builder, WSDL and XML Schema editors, etc.) and sample applications with many screen shots. Even with these small complaints, this highly targeted book should be very useful to Java developers looking to develop SOA applications. It would be best if readers already had at least a little knowledge of SOA, BPEL, etc. since this book mostly covers using the tools in NetBeans to facilitate build SOA applications.

Labels: J2EE, Java, SOA

I earned my living between 1988 and 1997 developing and being a mentor using C++. I usually argue against using C++, a language that offers superb runtime performance (speed and memory utilization) with the penalty of greatly increased development costs.

For most applications, it is less expensive to optimize for developer productivity. In my decade of using C++, with hindsight, the only projects that needed the performance of C++ were a commercial product for Windows 1.03, some VR work for SAIC and Disney, work on two Nintendo video games, a real time expert system for PacBell, and a PC racing game. All other C++ projects could have been done with more economy and effectiveness using other languages.

I used to error on the side of staying current with too many programming languages, although in the last few years I have invested heavy time in only three: Ruby, Java, and Common Lisp. Even though I find C++ development to be less fun, slow, and sometimes even a little painful, I am considering replacing Common Lisp with C++ in my small set of 3 languages that I am willing to invest heavily in. This Benchmark game of C++ vs. SBCL Lisp is one reason for this (possible) decision. The other reason is that I find Ruby to also be very good for quick prototyping and fast agile development, and even though the runtime performance of Ruby is very poor, it seems like the combination of Ruby + Java + C++ covers a wider range of application development than the combination of Ruby + Java + Lisp. In other words: Ruby gives me a lot of what Lisp does and I feel like I need one agile scripting language in my programming language toolbox.

Sure, "the best tool for the job", but I like to also consider the costs of not totally mastering the tools that I use in my work. This is why I have given up (for serious work) some great programming languages like Python, Smalltalk, C#, and Prolog.

Labels: C++, Java, Lisp, Ruby

Although I have been paid to work on both the Enhydra Enterprise Java application server and on the JBoss based Jaffa framework, I must admit that Tomcat has always been my favorite platform: lean, and add just what I need.

As a result of my 'build up just what I need' preferences, I just got around to experimenting with Glassfish. The Update Center is great, and as more instant install frameworks and applications become available, this will be a time saver. The web admin tool is refined - no complaints there.

I do have one complaint about the 200 MB resident memory footprint. A lot of what I do involves deploying to low cost servers, often semi-managed VPS systems. Smaller memory use is cheaper, but for most large server deployments, an extra 100 MB makes no difference.

I thought that it was very cool that one of the available instant install components is JRuby with the most excellent Goldspike. I have written before in this blog about the ease of running Rails web apps with Goldspike and JBoss. With Glassfish and the Update Center, it is even easier. Cool stuff.

Labels: J2EE, Java, JRuby

In the 1970s, I very much enjoyed working through Jean Sammet's classic book "PROGRAMMING LANGUAGES: History and Fundamentals". For one thing, it was my first real exposure to Lisp, and it was fun learning many new languages at once. At one point I had over a dozen programming languages on my resume (due to needing to use some one-off languages for military hardware, and several different assembler languages).

Generalization is good because the more experience with different languages, libraries, frameworks, and development styles that you have, then the easier it is to choose a good technology to solve new problems. I would argue that broad experience is at least a little better than narrow but deep experience.

Unfortunately there is another side to this issue: whenever I see really great design and code, it almost always seems to be written by someone who deeply specializes in one, or perhaps two, programming languages.

In the last few years, due to customer requirements, I have had to work in Java, Common Lisp, Ruby, and Python (and a little work with Prolog, and even less fortunately in C++). I am now in my mid-50s, and semi-retired (I try to limit myself to working no more than 25 hours per week). I would like to specialize in just one or two languages and not have the overhead of staying current with a half dozen programming languages.

The language that I enjoy the most is Ruby, and with the maturation of JRuby, it is great to be able to use the same language for scripting and general software development (using Matz's C based Ruby) and when I need to use existing Java libraries and frameworks (JRuby). While I really enjoy Lisp and Smalltalk (never professionally), my hope is that as JRuby continues to get faster and integration with the Java platform improves that the combination of (C or J)Ruby and Java will cover everything that I need for my work. BTW, the comment about future work on a "compiler that allows compiling a given Ruby class 1:1 to a Java class, producing a class you can construct, with specific signatures you can call from Java code" posted by Charles Nutter a few days ago to my JRuby blog article was very good news indeed.

Labels: Java, Lisp, Ruby

Pardon me for playing with the title of Edsger Dijkstra's famous paper, but as a Java developer I am starting to believe that arrays should be seldom used, given Java generics and the revised collection classes. I started thinking about this while reading someone else's code and noticing some artificial looking conversions between arrays and collections. Then for fun, I did some measurement and noticed over 15% of processing time was spent in toArray() methods.

The big cost in IT is usually in development and maintenance, not in deployment costs. Most projects are small, not Amazon size deployments so I believe that unless you are sure that you are building a very large scale system, it is best to optimize development to reduce costs for small and medium size systems. Smaller code size means smaller costs. Java programs should be as short and concise as possible and generics and use of collections helps a lot. Sometimes when I read the source for Java programs it looks like the authors were paid by line of code :-)

Labels: Java

Based on experience with consulting jobs developing web applications using several Java frameworks, Ruby on Rails, and Portable AllegroServe with WebActions (open source Common Lisp frameworks), I believe that choice of framework is less important than:

• Programming language: choose a language that both fits the application domain and has good library support for your application
• Data modeling: while I believe in interactive bottom up development, spending time up front getting object models 'right' makes development easier
• Object persistence: there are lots of good choices (prevalence, object relational mapping for relational databases, distributed memory only, etc.) but choose a scheme that makes sense both for development and deployment

I think that these 3 issues are all more important than choosing a web UI framework.

I have been investing a fair amount of time learning Erlang this year and the ErlyWeb framework (that uses the high performance Yaws Erlang web server) looks very good for both interactive development and distributed deployment. For web applications that map well to Erlang, ErlyWeb allows Erlang to be the development language of choice, but again the important choice is programming language selection rather than web framework.

Labels: Erlang, Java, Lisp, Ruby, web applications

Although most of my work is in AI development and general web services, I greatly enjoy writing user interfaces - this interest started in 1982 when SAIC bought me a Xerox 1108 Lisp Machine - the windowing system and general development tools were so good that they make most recent software development environments look weak.

Google Web Toolkit (GWT) and Squeak Smalltalk based Seaside are pleasant exceptions. Just like using Rails, GWT and Seaside provide that pleasant "everything just works" experience and great interactive development environments.

Last night I used GWT and IntelliJ to prototype some web UI ideas. If you have not tried working with the GWT Development Shell, give it a try: changes to client side resources (style sheets and HTML) and to Java client side source code (automatically compiled to Javascript during development) are immediately reflected in the test web app. I don't have any current tasks dealing with Java web applications right now, but the next time I do, GWT looks like a very good framework choice.

Seaside also supports the same fluid interactive development style: changes made in a Squeak Smalltalk source browser are immediately reflected in running web application. Seaside uses continuations to provide a more linear programming model. Blocks of code are used instead of URLs; inside a code block, you can call another code block - after a "return", you continue where you left off in the first code block. There is a performance penalty for using continuations, but it looks straight forward to run multiple Squeak processes behind Apache so it look like Seaside scales the same way that Rails scales. DabbleDB is written using Seaside - a great proof of concept for scaling.

Labels: Java, Smalltalk

Here is the download link. These tools come in a few 'flavors': Java, Ruby, C++, and C#. I expect to add two larger NLP projects in the next month.

BTW, I consider the LGPL to be "business friendly". You are allowed to mix my LGPL software with your own commercial products without open sourcing your products. You may also mix my LGPL software with open source with projects with Apache, BSD, MIT, or Mozilla style licenses. If you have any questions, ask me. If the LGPL license prevents you from using this material, please let me know about it.

Labels: AI, C#, C++, Java, Ruby

I dislike re-writing code that works well without a very good reason. Sometimes there is a temptation to rewrite code in order to move to a new language and platform. The Quercus PHP run time WAR file from Caucho looks like a good way to mix and match existing server side Java and PHP projects. Several years ago, a customer asked me to loosely integrate (just share a common login) SugarCRM and the JSP+POJO+Prevayler based web app that I had written for them to manage their research papers and devliverables. Not difficult to do, but my integration solution was a hack. Compiling PHP to byte code for the Java platform and easier (non-web service) calls between PHP and Java is an obvious win for developers. When JRuby's performance gets better (and it will :-) I look forward to the same type of integration with both Ruby and Rails.

Labels: Java, Ruby

As a developer, I have spent about equal time the last few years doing paid-for work in three programming languages: Common Lisp, Ruby, and Java. As you would expect, I try to apply the same tools and ideas for all three languages. This helps prevent the small "mental overhead" when switching languages. After reading the Guice User's Guide while having lunch today, I believe that I am going to use Guice for one or two of my "personal time" projects in order to try it out and add it to my toolkit.

The question that interests me is: how to use Guice patterns in dynamic languages like Common Lisp and Ruby? For Common Lisp, I recently started studying the project Closer - a wrapper for smaller projects for AOP and uniform interfaces (and enhancements) for MOP and context oriented programming. Good stuff!

For Ruby, there are a few projects for injection of control (e.g., Needle).

I am not sure how well spreading this pattern over three programming languages will work for me, but it makes sense to put some effort into consistency.

Labels: Java, Lisp, Ruby

I spent 20 minutes this morning going through the struts 2 examples and browsing the demo source code. Good stuff, but I am going to sick with my minimalist JSP + POJO + CSS + AJAX supporting Javascript tool kits for most applications. If I need to do anything complex, I will choose between 2 great frameworks: OpenLazlo or GWT. I have also taken a careful look at Flex 2 - another great toolkit but I like OpenLazlo's support for both Flash and DHTML.

Another picture (one block from our house at our local hiking trail head - taken a month ago):

Labels: Java

Ergonomics is the study of human capabilities in relationship to work demands. There are a few aspects of the Java language that tend to make programmers less effective than developers who use concise scripting languages like Ruby or languages like Lisp that enable the development of new domain specific languages on top of the Lisp language. I would like to address two techniques that I use to make Java development more ergonomic, given the limitations of Java.

Fixable problem #1 for me is that Java is not a concise language: I believe that the fewer lines of code in a project, then the easier to read, understand and maintain. I try to crunch down the size of my code by writing getters/setters on one line and initializing multiple variables per line. Also, since most laptop and desktop displays are now a wide screen format, I tend to use more "horizontal real estate" when editing - try to maximize the code that I can see on one screen by putting some small code blocks on one line, etc. - but be careful to keep code readable.

Fixable problem #2: application size and complexity: I always prefer to chop projects up into sub projects. Modules can be tested in isolation and then generate JAR files for use in larger aggregate projects. Including source code in your JAR files makes them more Java IDE friendly for reading library code.

The non-dynamic nature of Java does have one large advantage: IDEs like IntelliJ and Eclipse can offer better auto completion and refactoring capabilities. By carefully structuring Java projects, I believe that while Java will never have the high degree of programmer productivity of Ruby, Lisp, Smalltalk, etc., it is still a great language to develop in - important because of the obvious benefits of server side Java.

Labels: Java

When I do any database work in Java, I like to support both PostgreSQL and MySQL. The MySQL drivers since 2002 have a GPL license and are not suitable at all (for license reasons) for use in non-GPL projects when you need to distribute your application (but is fine for internal projects). Thus, for non-GPL projects, I use the older LGPL MySQL driver org.gjt.mm.mysql.Driver to stay clean, license-wise.

I do notice that just about all online tutorials that use MySQL with Java use the newer GPL licensed drivers, so I would imagine that many people end up using the GPLed driver in non-GPL projects. Being an open source enthusiast also means respecting license choices of copyright owners.

The old LGPL driver does not support some things like bit values, but since the old driver is open source, I could always tweak the source code if need be. I am a little surprised that the Java community does not fork the old LGPL driver and maintain it.

Labels: Java, open source

I have been investing a fair amount of time learning the Dojo Javascript library but I had an application where I only needed AJAX to handle HTML form input - I wanted a server side function to return HTML to be inserted below the form. Dojo is a huge library so I pulled the prototype.js library out of one of my old Ruby Rails projects (prototype.js is only about 70KB). I put the following in my form JSP page:

<form id="myForm2">
 <input type="text"  id="input_test_form_text" />
 <input type="button" value="Search" onclick="getS()" />
</form>
<div id="searchresults"></div>

<script type="text/javascript" src="prototype.js"></script>
<script>
 function getS() {
  var params = 'query=' + $F('input_test_form_text');
  var x = new Ajax.Updater('searchresults', '/ajax_search.jsp',
                           {method: 'get',parameters: params});
 }
</script>

The follow is a trivial server side JSP ajax_search.jsp that does nothing but returns the original query as an HTML snippet:

<%@ page contentType="text/html;charset=UTF-8" language="java" %>

<%
    String req = request.getParameter("query");
    out.println("<p>request for 'query' is:" + req + "</p>");
%>

Labels: AJAX, Java, Javascript, JSP

I have used Nutch for two contracting jobs and Lucene for many jobs. Until today, I have viewed Nutch simply as:

• Quick to configure for target websites to spider and to administer spidering
• Trivial to run search web application
• Web service provider (OpenSearch API)

Today however I started looking more closely at the underlying Hadoop architecture (like the distributed Google file system and their map reduce client library) and at both the available plugins and the plugin architecture. New opinion: Nutch is a platform for building more complex web applications and knowledge management applications.

Labels: Java, search

JRuby 0.9.2 was released a few weeks ago but I just got around to trying it out this morning. My favorite feature is the "JRuby IRB Console". An easy way to experiment with JRuby is to simply to download the console JAR file that conveniently contains the JRuby run time system, has a "tab auto complete", and has readline support. If you have not given JRuby a try yet, download the IRB Console, and follow through the tutorials. Good stuff!

JRuby performance problems:

I love programming in Ruby but find myself working a lot in Common Lisp instead for vastly better runtime performance. JRuby performance is very poor at this time (much slower than the standard Ruby system, which is itself slow), but with support from Sun and some development time, performance and Java platform integration will improve.

Labels: Java, Lisp, Ruby

I have been reading some negative comments on adding web services, embedded database, etc. to the core Java libraries but I think that this is in general a good thing. If you don't like the now "standard" web services libraries, then just use the stack you prefer. The good thing is that it is just about trivial to do two things:

• Export methods from one of your POJO classes as a web service
• Build a client from a WSDL resource (e.g., "wsimport -d generated http://test.com/stock?wsdl")

Java DB (Apache Derby) is a good embedded database and having it in the JDK (but not the JRE) by default makes it easier to deliver small applications to customers and feel more confident that they will have an easy installation experience. I could care less if the JDK download is a few megabytes larger.

Labels: Java

I feel like I have come full circle (almost) in AJAX development: I started out a few years ago adding some simple AJAX enabled forms to a JSP based application. When first starting out, it took hours to get something working. Then a year ago, I discovered how simple it is to use AJAX in Rails: fine, except that Ruby does not have high enough performance for some applications (unless large parts are written in C - Ferret, for example).

I have spent many evenings playing around with various release versions of the Google Web Toolkit (GWT) and it is very compelling, especially if you already are used to developing GUI applications in Java - the only new wrinkle worth mentioning is getting used to handling events asynchronously. The problem with GWT is that it really does tie you to the Java platform. I spend most of my time developing AI applications, but that said, who does not want basic knowledge and competence at building web applications?

I use Common Lisp, Java, and Ruby for development, so for the occasional AJAX tasks that I have, I have settled with the well respected dojo Javascript toolkit because it plays very well with both Lisp and Java JSP based web applications. Dojo is also easy to use with Rails, if you want an alternative to Rail's great AJAX support.

By using Dojo, I basically have to deal with just one learning curve no matter what platform I am developing on. Here is a simple example for a JSP page (assuming that this would be more interesting to most people than a Lisp example):

Add this to your <head> section on a top level JSP page:

    <script type="text/javascript">
     var djConfig = { isDebug: true };
    </script>
    <script type="text/javascript" src="dojo.js"></script>
    <script type="text/javascript">
            dojo.require("dojo.io.*");
            dojo.require("dojo.event.*");
    </script>

Then try putting this somewhere on your JSP page (note: this assumes that you have another JSP page ajax.jsp that gets the form values and returns content to be placed into the DIV element with ID=ajxplydiv. Anthing that ajax.jsp writes using out.print() gets inserted asynchronously into the "ajxplaydiv" DIV element):

    <h3>test AJAX Form:</h3>
    <form id="myForm">
        <input type="text" name="input_test_form_text" />
        <input type="button" name="button1"
               value="Try AJAX form" id="ajaxButton" />
    </form>

    <div id="ajxplaydiv">
       initial context for AJAX play div
    </div>

    <script type="text/javascript">

    var buttonObj = document.getElementById("ajaxButton");
    dojo.event.connect(buttonObj, "onclick",
                       this, "my_onclick_button");

    function my_onclick_button() {
      var ajaxargs = {
        url: "ajax.jsp",
        load: function(type, data, evt){
           dojo.byId('ajxplaydiv').innerHTML = data;
        },
        error: function(type, data, evt){
         alert("Error occurred!");
        },
        mimetype: "text/plain",
        formNode: document.getElementById("myForm")
     };
     dojo.io.bind(ajaxargs);
   }
</script>

The call to dojo.io.bind sends a request containing the form data to the ajax.jsp page, and whenever the results are returned then the Javascript functions "load" or "error" defined in the data block in my_onclick_button() is called. This is just one example of processing form data and adding HTML below the form without refreshing the page - but is a common use case for using AJAX. This example assumes that I have both the dojo "src" directory and dojo.js in my top level web resources directory. The great thing about dojo is that it encapsulates all asynchronous event handling, offers a good supply of visual components (that map to HTML elements) and is simple to use no matter what programming language you are using for a web application.

Labels: AJAX, Java, Javascript, Ruby, search

This is a good thing, to be sure! However, I have to wonder how this might affect Sun's revenue for Java in embedded devices, etc. Sun has a huge investment in Java and as a corporation they really do need to generate revenue from Java.

One thing that I would like to see happen is tight JRE integration with Linux systems: a mechanism for running a shared JVM instance for small Java utilities and porting Apple's contributions for sharing more memory between JVM instances running on the same system.

Labels: Java, open source

I plan on doing a major overhaul to our "knowledge intensive" cooking and recipes web portal including AJAX support and a better artificial intelligence recipe wizard. The portal is written in Java using JSPs and custom tag libraries.

I have actually considered re-writing the whole thing in Ruby Rails, partially for the AJAX support and partially because Rails is easier to develop with. However, after spending a few hours experimenting with the IntelliJ 6 Google Web Toolkit (GWT) plugin and wizard support, a lot of the advantage of Rail's excellent AJAX support is minimized. Both free developers from most Javascript hassles of using AJAX.

The is a short flash demo that is worthwhile watching if you are considering using GWT.

Labels: AJAX, Java

I am about 4 months into a project using a very expensive object persistence mechanism (Franz Lisp AllegroCache) that almost automatically wraps Common Lisp CLOS classes for automatic persistence and B-Tree indexing of as many slots as you want to tag as indexed. As is sometimes is the case, here commercial software tools save time and effort. There is a very interesting open source project for Common Lisp (Elephant) that does something similar by using BerkleyDB or a relational database as the backend - it looks good, but it is not yet an out of the box solution.

For Java work, although I was once a fan of Hibernate, I try to use Prevayler instead: if you add new instance variables to the end of class definitions you can add to your object model without breaking your persistent storage - a neat trick, to be sure.

When it comes to the best programmer productivity using object relational mappers, I personally think that Ruby Rail's ActiveRecord wins hands down - with the slight cost that you design database schema instead of class models.

Labels: Java, Lisp, object mapping, Ruby