Mark Watson's opinions on Java, Ruby, Lisp, AI, and the Semantic Web

When I first started using Ruby, Matz's C-Ruby was mostly the only game in town.

I am also an enthusiastic user Ruby 1.9.x, JRuby, and MacRuby. Seeing the Ruby spec being developed in Japan under government funding, and that it is for Ruby 1.8.7, I get a feeling of déjà vu as a long time Lisp user.

The balkanisation of Lisp has been more than a small nuisance for me in the last 25 years. I would prefer that all Ruby implementations eventually implement a common specification, but I would rather it be for something that looks like 1.9.x.

Labels: Ruby

I use Rubymine for most of my Ruby/Rails/Sinatra development on Ubuntu, and use it in conjunction with TextMate on OS X. I find it convenient enough to alternate between TextMate when I don't need IDE features, and Rubymine when I do.

One of the biggest improvements is that indexing now occurs in the background and auto-complete and other features become available that depend on knowledge of an application and the gems that it uses.

This is subjective, but once Rubymine 2.0 loads up and is done with any background indexing then the CPU use is minimal, and I think improved from earlier versions (nice to not have the fan kick in on my laptop when the CPU cores heat up). For the Rails application that I am coding on right now, Rubymine is using about 360MB of resident memory - this is OK with me.

Labels: IDEs, Ruby

I have been spoiled by great support for indexing and search in relational databases (e.g., Sphinx, native search in PostgreSQL and MySQL, etc.)

I was pleased to discover, after a little bit of hacking this morning, how easy it is to do indexing and search using the MongoDB document-centered database. I have two common use cases for search, and MongoDB seems to handle both of them fairly well:

• Search for words inside of text fields
• Efficient word prefix search to support AJAX "suggest" style lists

My approach does require combining search results for multiple search terms in application code, but that is OK. Assuming the use of MongoRecord, here is a code snippet:

class Recipe < MongoRecord::Base
  collection_name :recipes
  fields :name, :directions, :words
  def to_s
    "recipe: #{name} directions: #{directions[0..20]}..."
  end
  def Recipe.make collection, name, directions
    collection.insert({:_id => Mongo::ObjectID.new, :name => name,
                       :directions => directions,
                       :words => (name + ' ' + directions).split.uniq})
  end
end
 
host = 'localhost'
port = Mongo::Connection::DEFAULT_PORT
MongoRecord::Base.connection = Mongo::Connection.new(host,port).db('mongorecord-test')

db = MongoRecord::Base.connection

coll = db.collection('recipes')
coll.remove({})

coll.create_index(:words, Mongo::ASCENDING)

Recipe.make coll, 'Rice Soup', 'Cook the rice, then add extra water to thin it out.'
Recipe.make coll, 'Cheese and Rice Crackers', 'Slice the cheese and layer on top of crackers.'

puts "\nSimple find"
puts Recipe.find_by_name(:name => 'Rice Soup').to_s

puts "\nFind recipe by regular expression (ignoring case) in array of words /water/i"
Recipe.find(:all, :conditions => {:words => /^water/i}).each { |row| puts row.to_s }

According to the MongoDB documentation, a regular expression match like /^water/i will use an index just as a relational database match in the form like 'water%' does.

I am still in a learning mode with MongoDB, so I would appreciate any comments on improving this aproach.

Labels: MongoDB, Ruby, search

I spend a lot of time experimenting with infrastructure software, sometimes for customer jobs and sometimes just because it is fun to learn new things. For non-SQL data stores, I have spent a lot of time in the last year experimenting with and using CouchDB, AppEngine datastore, Tokyo Cabinet, MongoDB, Cassandra, and SimpleDB. Tokyo Cabinet and SimpleDB store hash values as strings, and don't have the great client APIs that the others have because limitations in string-only hash values. That said, for an Amazon hosted application SimpleDB can be a good choice and Tokyo Cabinet is light weight and easy to install and use. Casandra looks great, and as I have written about here before, Cassandra is easy to use from ruby and has great features.

MongoDB has great performance and similar capabilities as Casandra. Chris Kampmeier has a great writeup that covers installing MongoDB on OS X, including setting it up as a system service. I followed Chris's directions. A pleasant surprise is that MongoDB has a light footprint, and leaving it running as a service like I do with PostgreSQL and MySQL is reasonable.

Along with the mongo and mongo_record gems, MongoDB is an awesome tool, and always keeping it running makes it easy to experiment with. BTW, I also keep Sesame and CouchDB running on my local network for much the same reasons.

Labels: MongoDB, no-SQL, Ruby

I noticed that Microsoft allows free use of their search and spelling correction APIs. I just played with the APIs for a few minutes. Here is a Ruby code snippet that I just wrote:

API_KEY = ENV['BING_API_KEY']

require 'rubygems' # needed for Ruby 1.8.x
require 'simple_http'
require 'json'

def search query
  uri = "http://api.search.live.net/json.aspx?AppId=#{API_KEY}&Market=en-US&Query=#{CGI.escape(query)}&Sources=web+spell&Web.Count=4"
  JSON.parse(SimpleHttp.get(uri))["SearchResponse"]["Web"]["Results"]
end

def correct_spelling text
  uri = "http://api.search.live.net/json.aspx?AppId=#{API_KEY}&Market=en-US&Query=#{CGI.escape(text)}&Sources=web+spell&Web.Count=1"
  JSON.parse(SimpleHttp.get(uri))["SearchResponse"]["Spell"]["Results"][0]["Value"]
end

You need a free Bing API key - notice that I set the key value in my environment. If you get a key, then try:

search "semantic web java ruby lisp"
correct_spelling "semaantic web jaava ruby lisp"

The first method does spelling correction before search.

Labels: Ruby, search

I have everything running under Ruby 1.9.1 on my development system that I need. I may upgrade/migrate my production servers to version 1.9.1 in a week or two if everything continues to go OK.

The last problem that I worked around was a problem in Rails/webrick in dealing with headers+bodies if the bodies contained binary data. I locally patched webrick (my Google groups post).

I can't get mongrel to build with Ruby 1.9.1 so that is an issue since I usually deploy using nginx + mongrel(s) + memcached.

Labels: Ruby, Ruby Rails

I have been converting (on my development system, not in production yet) my Rails applications to work with Ruby 1.9.1. I converted my non-Rails stuff (that I care about) to work with 1.9.x a while ago.

I still use ferret for single process (or single thread) web applications. I have wanted to switch to Sphinx and UltraSphinx for a while and since ferret is not compatible with Ruby 1.9.x because of changes for unicode (RString struct, specifically) I did that conversion late last night. I was pleasantly surprised how simple it was to switch to Sphinx and UltraSphinx: lots of little edits, but quick and easy.

A little more frustrating: the auto_complete Rails plugin does not seem to be Ruby 1.9.x compatible in one of my Rails web apps. I am going through the plugin code, adding debug output, and looking for the problem. The problem may just be specific to my application.

Because of the performance improvements of 1.9.x and because of the long term maintenance overhead of having (eventually) both ruby 1.8.x and ruby 1.9.x applications in deployment, I am willing to spend a lot of effort on the conversion. Another issue is that I expect to have a Ruby book published early summer of 2009 and I would like all the examples to be Ruby 1.9 compatible (and some JRuby specific code).

Labels: Ruby, Ruby Rails

I use bash aliases to quickly change my environment to use Ruby 1.8.x, JRuby, or Ruby 1.9.x. I am changing the default to use Ruby 1.9.1 - an important psychological hurdle :-)

Labels: Ruby

I have some mixed feelings about the integration of Merb and Rails because Merb is a small "micro kernel" type framework that seems just about perfect for projects like light weight web services and web portals.

On the good side of this merger: if Rails becomes more modular and the resulting Rails 3.0 can also be tailored down to 'just what is needed', then this merge should end up being good for developers.

I have just signed a publishing contract for a new Ruby book that has one part on publishing information for consumption by both humans and software agents. After waiting a few months, I will grab early Rails 3.0 builds and write to Rails 3.0. A bit funny because before my holiday break I was deciding whether to use Merb or Rails for this book example.

Labels: Merb, Ruby, Ruby Rails

IntelliJ 8.0 was released yesterday and after installing the Ruby + Rails plugin, IntelliJ is very competitive with the NetBeans for Rails development.

One feature that I particularly like is the jump links in the editor that let you jump from a controller method to the corresponding view template. There are also links from a method to the super class method that is being overridden (if any). There is currently a small bug in the plugin: multiple identical jump links are shown; all work the same.

In some ways it is nice to have Java and Ruby support in one IDE, but there are "Java only" menus shown while working on Rails projects - that is one advantage of the new RubyMine IDE: basically IntelliJ with all Java support removed. At the current time, Rails support for IntelliJ 8.0 seems to be more stable than the prerelease version of RubyMine but it will be interesting to compare the two next year when RubyMine is released as a product.

Labels: IDEs, Ruby, Ruby Rails

I am re-writing a quick one or two evening hack from 3 years ago (RubyPlanet.net) in Ruby + Merb. I originally wrote this web site in Java - humorous since it is a Ruby blog feed aggregator :-)

I am also using this as the example program for a DevX article on Merb that I am working on. I just did a very quick first cut at the example program for the article (i.e., what soon will be the new RubyPlanet.net) and without doing any caching and hitting a database a lot, I was surprised to see that

ab -n 5 http://localhost:4000/

showed 44 page requests per second. My goal is to get 100 to 200 page requests handled per second (depending on the number of blog entries on the main page) using a single process on my MacBook, and that looks like it may be easy. Deployed on a very low cost VPS with a bunch of other applications, this web app may not be all that fast "out in the field" but I want that to be a limitation of the server, not the code.

Labels: Merb, Ruby

I am back home after attending MerbCamp in San Diego last weekend. Merb is sort of like a micro-kernel architecture version of Rails: a small core with many plugins (and also complete "slice" mini-apps) that (hopefully) do not depend on each other. The idea is that you only add in what you need. Right now is probably not the time to try Merb for the first time: the developers are working right now to release version 1.0 RC1 (with version 1.0 to follow as quickly as possible). I am currently using Merb 0.9.9 and I am not going to update until I can do a "gem update merb" to move up to the 1.0 APIs. As announced at MerbCamp, the developers want to stabilize the APIs for version 1.0 and then continually work with minor 1.x releases for about one year, then release 2.0 that is likely to not be very backwards compatible with 1.0. Also, 1.x releases will be backwards compatible with 1.0 but not necessarily other 1.y releases. I think that this is a good plan, and matches the way I have used Rails for the last three years: I tend to freeze individual projects against a specific version of Rails, and only do security updates. I plan on using Merb for a semantic web project, and I will probably just stick with 1.0 and not follow the API changes during the 1.x path towards 2.0.

Labels: Merb, Ruby

I am at MerbCamp - so far an enjoyable conference. Merb is a more modular and more efficient version of Rails. As you might expect, Merb does not yet have the easy 'out of the box' experience that Rails provides. That said, I will probably start to use Merb instead of Rails on some new projects because of performance (run time and much less memory required) reasons.

Labels: Merb, Ruby

Although there are several Semantic Web libraries or frameworks that I like to use, I had to choose just one for a DevX article that I am finishing up. I chose to use Sesame. After covering what I think are some "big wins" of using RDF/RDFs/OWL (for some applications) I present some example programs that I hope that readers have lots of fun with. The "wrapper" library that I wrote for Sesame works fine for both Java (which Sesame is written in) and JRuby. I must say that for experimenting with Sesame, JRuby is a lot nicer because the example programs are much shorter and with Ruby duck typing it is easier to write callback handlers, etc. for my wrapper library. Being able to work interactively in a JRuby jirb shell is also a big win for experimenting with code, different SPARQL queries, etc.

Labels: Java, RDF, Ruby, semantic web

First, 2.1 looks like a great update: I have not seen any compatibility problems with 2.0 that could not be instantly fixed. The named_scope (has_finder) changes look good for organizing database queries, and I especially like the way they nest. I have not tried using the new gems dependency functionality yet but this looks very useful when deploying applications to fresh servers.

The news about the large performance boost using MagLev looks interesting, but I will reserve my enthusiasm until the project is further along and I can try running it myself. I find myself reverting from coding in Ruby back to Common Lisp or Java to get around performance issues, so a much faster Ruby runtime environment sounds good. Ruby is such a slow language, that there is plenty of room for improvement.

Labels: Ruby, Ruby Rails

JSON is very much lighter weight than XML and is meeting a need for easily calling some Scheme code from a Ruby program. The Scheme code I am using is old, I wrote it years ago to extract entities from plain text. Since the Scheme program starts very quickly, I am able to simply start a Scheme interpreter as a separate process using back quotes to capture any output to stdout in a Ruby string variable:

require 'json'
s = `gsi extraction.scm -e '(get-proper-names "President Bush moved to South America.")'`
# parse JSON text...

Here I am using Gambit-C Scheme interpretively. It is very easy to write any structured data in JSON format in the Scheme (or Lisp) code and parse it on the Ruby side. It is also easy to speed things up and compile my program:

gsc -c extraction.scm
gsc -link extraction.c
gcc -o extraction extraction.c extraction_.c -lgambc -I/Library/Gambit-C/current/include/ -L/Library/Gambit-C/current/lib/

which both makes the program faster and reduces the startup time down to a few milliseconds. Since the extraction program starts very quickly, this solution is good enough. However, I have been looking at an alternative idea in case I ever need to use a Lisp or Scheme program that has a long startup time. I have not done this yet, but here are my ideas for Gambit-C Scheme:

• Write a C wrapper for the Scheme (a good reference) to create a Ruby callable class whose initialization method performs the long running initialization for the Scheme library
• Wrap the Scheme side library according to this reference

This looks easy enough, but getting memory allocation and deallocation correct might be difficult. For now, I am calling my old Scheme code from Ruby the easy way :-)

Labels: Lisp, Ruby, Scheme

I earned my living between 1988 and 1997 developing and being a mentor using C++. I usually argue against using C++, a language that offers superb runtime performance (speed and memory utilization) with the penalty of greatly increased development costs.

For most applications, it is less expensive to optimize for developer productivity. In my decade of using C++, with hindsight, the only projects that needed the performance of C++ were a commercial product for Windows 1.03, some VR work for SAIC and Disney, work on two Nintendo video games, a real time expert system for PacBell, and a PC racing game. All other C++ projects could have been done with more economy and effectiveness using other languages.

I used to error on the side of staying current with too many programming languages, although in the last few years I have invested heavy time in only three: Ruby, Java, and Common Lisp. Even though I find C++ development to be less fun, slow, and sometimes even a little painful, I am considering replacing Common Lisp with C++ in my small set of 3 languages that I am willing to invest heavily in. This Benchmark game of C++ vs. SBCL Lisp is one reason for this (possible) decision. The other reason is that I find Ruby to also be very good for quick prototyping and fast agile development, and even though the runtime performance of Ruby is very poor, it seems like the combination of Ruby + Java + C++ covers a wider range of application development than the combination of Ruby + Java + Lisp. In other words: Ruby gives me a lot of what Lisp does and I feel like I need one agile scripting language in my programming language toolbox.

Sure, "the best tool for the job", but I like to also consider the costs of not totally mastering the tools that I use in my work. This is why I have given up (for serious work) some great programming languages like Python, Smalltalk, C#, and Prolog.

Labels: C++, Java, Lisp, Ruby

Good news: Apple is supporting Ruby 1.9 using the Objective C runtime. This mainly to support building native Mac applications but this will eventually be good for other more general purpose Ruby development on Macs. Very cool since both Objective C and Ruby have their roots in Smalltalk.

Labels: Ruby, Smalltalk

My most recent DevX article has just been published. This was fun material to write about because after some experimentation I feel like I have my Ruby on Rails development environment and server deployment strategy just right, at least for my needs. I should mention that although I have been professionally writing Ruby on Rails applications for a few years, I have not yet written an application that will not run nicely on a single server using nginx, memcache, and a few mongrels. I set my development.rb environment for my MacBook and my production.rb environment for the Linux server I am deploying to, and svn is the glue that holds everything together. If you are interested in deploying very large scale applications, my article will not be very useful to you.

Labels: NetBeans, Ruby, Ruby Rails

I have been putting this off - I have been busy. Ruby 1.9 uses the YARV and seems to be about twice as fast as Ruby 1.8 - a nice speed improvement which is likely to get much better before a Ruby 2.0 release. I tried the new version on several of my POR (plain old Ruby) programs, and everything worked fine for me. Rails would not work for me, but I did not expect it to. I like the way that gem and rake are now bundled with Ruby 1.9 - that makes good sense.

With the good work also being done on JRuby and Rubinius, it is great to know that Ruby is under active development.

Labels: Ruby

One of my customers uses Common Lisp for a lot of development and we go with Franz, largely because of AllegroCache object persistence and great support.

Until today, I had never bothered getting Elephant installed and set up with a back end data store (I chose CL-SQL with native PostgreSQL). Since I own my own licensed copy of Franz Lisp (and my customer provides me with another license for their work), I was always happy to just use AllegroCache.

Anyway, after reading Vetle Roeim's blog post Implementing a blog in Common Lisp I decided to take a little time setting up Elephant and Edi Weitz's Hunchentoot web server and his HTML-TEMPLATE Common Lisp templating system. I used a fairly recent release of SBCL Common Lisp on a MacBook. This is all free open source software. I found the Elephant API to be easy to use.

After having some fun with this, I do have a few comments on Common Lisp vs. Ruby development:

• Common Lisp compiled code runs much faster than Ruby: typical benchmark results are about 30 times faster - but Ruby's slowness is mitigated if a lot of processing is performed in native libraries like Ferret.
• Development with Common Lisp takes longer than Ruby (for me, at least). It would have taken me 10 minutes to get set up with Ruby for this tutorial while it took a full 90 minutes to get 20 different Common Lisp packages and install them in the project directory for Vetle's tutorial. It is true that I will keep this project around, and if I ever need to use Elephant, HTML-TEMPLATE, Hunchentoot, etc., then I will just clone and modify this project (if I don't care about using the most up to date libraries). Common Lisp ASDF is a good package manager but in my opinion is not as good as Ruby Gems.
• I find that I work a little faster in Ruby than Common Lisp, mostly because the language is terser: simply less code to write and read. That said, Common Lisp is also a great language to work in.

Labels: Lisp, Ruby

In the 1970s, I very much enjoyed working through Jean Sammet's classic book "PROGRAMMING LANGUAGES: History and Fundamentals". For one thing, it was my first real exposure to Lisp, and it was fun learning many new languages at once. At one point I had over a dozen programming languages on my resume (due to needing to use some one-off languages for military hardware, and several different assembler languages).

Generalization is good because the more experience with different languages, libraries, frameworks, and development styles that you have, then the easier it is to choose a good technology to solve new problems. I would argue that broad experience is at least a little better than narrow but deep experience.

Unfortunately there is another side to this issue: whenever I see really great design and code, it almost always seems to be written by someone who deeply specializes in one, or perhaps two, programming languages.

In the last few years, due to customer requirements, I have had to work in Java, Common Lisp, Ruby, and Python (and a little work with Prolog, and even less fortunately in C++). I am now in my mid-50s, and semi-retired (I try to limit myself to working no more than 25 hours per week). I would like to specialize in just one or two languages and not have the overhead of staying current with a half dozen programming languages.

The language that I enjoy the most is Ruby, and with the maturation of JRuby, it is great to be able to use the same language for scripting and general software development (using Matz's C based Ruby) and when I need to use existing Java libraries and frameworks (JRuby). While I really enjoy Lisp and Smalltalk (never professionally), my hope is that as JRuby continues to get faster and integration with the Java platform improves that the combination of (C or J)Ruby and Java will cover everything that I need for my work. BTW, the comment about future work on a "compiler that allows compiling a given Ruby class 1:1 to a Java class, producing a class you can construct, with specific signatures you can call from Java code" posted by Charles Nutter a few days ago to my JRuby blog article was very good news indeed.

Labels: Java, Lisp, Ruby

It takes a few lines of Ruby code to process OpenOffice.org document files:

require 'rubygems'
require 'rexml/document'
require 'rexml/streamlistener'
require 'zip/zipfilesystem' # install with gem
include REXML

class OOXmlHandler
  include StreamListener 
  attr_reader :plain_text
  def initialize; @plain_text = ""; @last_tag_name =""; end     
  def tag_start name, attrs; @last_tag_name = name; end
  def text s
    @plain_text << s << "\n" if @last_tag_name.index('text')
  end
end

class ReadOpenOffice
  attr_reader :text
  def initialize file_path
    Zip::ZipFile.open(file_path) { |zipFile|
      xml_handler = OOXmlHandler.new
      Document.parse_stream((zipFile.read('content.xml')), xml_handler)
      @text = xml_handler.plain_text
    }
  end
end

puts ReadOpenOffice.new('KBrecipes.odt').text

I have spent too much of my time over the last 10 years dealing "programatically" with Microsoft document formats. I am tired of wasting my time when open document formats are so much easier and less expensive to use.

Labels: ODF, Ruby

Check it out on DevX.

In this article I write about why a Java developer might want to also use Ruby and I cover a few cool Ruby features.

Labels: Ruby

Based on experience with consulting jobs developing web applications using several Java frameworks, Ruby on Rails, and Portable AllegroServe with WebActions (open source Common Lisp frameworks), I believe that choice of framework is less important than:

• Programming language: choose a language that both fits the application domain and has good library support for your application
• Data modeling: while I believe in interactive bottom up development, spending time up front getting object models 'right' makes development easier
• Object persistence: there are lots of good choices (prevalence, object relational mapping for relational databases, distributed memory only, etc.) but choose a scheme that makes sense both for development and deployment

I think that these 3 issues are all more important than choosing a web UI framework.

I have been investing a fair amount of time learning Erlang this year and the ErlyWeb framework (that uses the high performance Yaws Erlang web server) looks very good for both interactive development and distributed deployment. For web applications that map well to Erlang, ErlyWeb allows Erlang to be the development language of choice, but again the important choice is programming language selection rather than web framework.

Labels: Erlang, Java, Lisp, Ruby, web applications

Compact languages and libraries have the advantage that one person can understand most of an implementation. Two of the most elegant programming languages, Squeak Smalltalk and Ruby, have sufficiently compact implementations to understand, given some effort. And this effort is worthwhile - a lesson I learned in the 1970s when I worked as a systems programmer: I kept listings of key parts of the PrimeOS (Multics based) operating system and for the Lisp and FORTRAN implementations handy at home and in my office. Time spent reading through the code paid huge dividends whenever I had to do any systems level programming. Reading the code helps create a mental map of whatever software that you are using.

The implementation of Ruby and the standard Ruby libraries is compact, basically divided into the C implementation of Ruby itself and the Ruby code for the standard libraries. If you are going to be working much with Ruby then consider creating at least a project (using Eclipse, IntelliJ, TextMate, etc.) with the library source code. Good for library reference, looking up APIs, and for just reading code.

I would make the same suggestion to developers who use Squeak Smalltalk (or other implementations): you have the source code to the standard libraries: read it!

I also enjoy and am very productive using the Java and Common Lisp frameworks, but these languages with their standard libraries are so huge that I would personally never attempt to dive in and understand their implementations. Not having a mental road map of the implementations of Common Lisp and Java reduces the effectiveness of these platforms.

Labels: Ruby, Smalltalk

Here is the download link. These tools come in a few 'flavors': Java, Ruby, C++, and C#. I expect to add two larger NLP projects in the next month.

BTW, I consider the LGPL to be "business friendly". You are allowed to mix my LGPL software with your own commercial products without open sourcing your products. You may also mix my LGPL software with open source with projects with Apache, BSD, MIT, or Mozilla style licenses. If you have any questions, ask me. If the LGPL license prevents you from using this material, please let me know about it.

Labels: AI, C#, C++, Java, Ruby

I dislike re-writing code that works well without a very good reason. Sometimes there is a temptation to rewrite code in order to move to a new language and platform. The Quercus PHP run time WAR file from Caucho looks like a good way to mix and match existing server side Java and PHP projects. Several years ago, a customer asked me to loosely integrate (just share a common login) SugarCRM and the JSP+POJO+Prevayler based web app that I had written for them to manage their research papers and devliverables. Not difficult to do, but my integration solution was a hack. Compiling PHP to byte code for the Java platform and easier (non-web service) calls between PHP and Java is an obvious win for developers. When JRuby's performance gets better (and it will :-) I look forward to the same type of integration with both Ruby and Rails.

Labels: Java, Ruby

As a developer, I have spent about equal time the last few years doing paid-for work in three programming languages: Common Lisp, Ruby, and Java. As you would expect, I try to apply the same tools and ideas for all three languages. This helps prevent the small "mental overhead" when switching languages. After reading the Guice User's Guide while having lunch today, I believe that I am going to use Guice for one or two of my "personal time" projects in order to try it out and add it to my toolkit.

The question that interests me is: how to use Guice patterns in dynamic languages like Common Lisp and Ruby? For Common Lisp, I recently started studying the project Closer - a wrapper for smaller projects for AOP and uniform interfaces (and enhancements) for MOP and context oriented programming. Good stuff!

For Ruby, there are a few projects for injection of control (e.g., Needle).

I am not sure how well spreading this pattern over three programming languages will work for me, but it makes sense to put some effort into consistency.

Labels: Java, Lisp, Ruby

It pays off in the long run to write some custom code and tweak your own working environment. How many programmers spend time writing code to improve other people's productivity but don't invest the time to automate their own work flow? Probably too many.

I have been working fairly hard in my free time on two "free web books" for my main we site. In the past I have simply created PDF files and made them available for my site. Although I write my "free web books" using a Creative Commons license, I decided that I wanted to make at least a small revenue stream so that I can spend more time on these projects. My plan has been to have all of my web books readable for free online with a small Google Adsense advertisement at the beginning of each chapter and offer through lulu.com the ability to buy PDFs with no advertising for a few dollars or get a printed book for about $12.

I would prefer spending my time writing rather than preparing HTML, PDF, and printer ready PDF. I think that I now have this process automated about as fully as I can, using custom Ruby code, shell scripting, and a "do everything" Makefile. I had experimented with using OpenOffice.org and writing some utilities to modify the generated HTML. I also experimented with the very cool SiSU system.

In the end, I went back to automating my writing setup using Latex, htlatex, pdflatex, custom Ruby code and shell scripts, and the OS X Latex editor TexShop. It feels good to get things working "just right".

Labels: Latex, Ruby, technical writing

I built Yarv on my MacBook following these directions. The Yarv VM is a lot faster than C Ruby 1.8.X and it looks like Rails will work with Yarv before too long. I tried a lot of my code with Yarv without problems - mostly computationally expensive text handling.

Labels: Ruby

JRuby 0.9.2 was released a few weeks ago but I just got around to trying it out this morning. My favorite feature is the "JRuby IRB Console". An easy way to experiment with JRuby is to simply to download the console JAR file that conveniently contains the JRuby run time system, has a "tab auto complete", and has readline support. If you have not given JRuby a try yet, download the IRB Console, and follow through the tutorials. Good stuff!

JRuby performance problems:

I love programming in Ruby but find myself working a lot in Common Lisp instead for vastly better runtime performance. JRuby performance is very poor at this time (much slower than the standard Ruby system, which is itself slow), but with support from Sun and some development time, performance and Java platform integration will improve.

Labels: Java, Lisp, Ruby

I spent an evening or two a year ago setting up RubyPlanet.net for my own use: I wanted a one stop Ruby news page. I spent an hour yesterday putting RSS/Atom spidering in a background work thread to avoid occasional delays in processing requests.

Labels: Ruby

I feel like I have come full circle (almost) in AJAX development: I started out a few years ago adding some simple AJAX enabled forms to a JSP based application. When first starting out, it took hours to get something working. Then a year ago, I discovered how simple it is to use AJAX in Rails: fine, except that Ruby does not have high enough performance for some applications (unless large parts are written in C - Ferret, for example).

I have spent many evenings playing around with various release versions of the Google Web Toolkit (GWT) and it is very compelling, especially if you already are used to developing GUI applications in Java - the only new wrinkle worth mentioning is getting used to handling events asynchronously. The problem with GWT is that it really does tie you to the Java platform. I spend most of my time developing AI applications, but that said, who does not want basic knowledge and competence at building web applications?

I use Common Lisp, Java, and Ruby for development, so for the occasional AJAX tasks that I have, I have settled with the well respected dojo Javascript toolkit because it plays very well with both Lisp and Java JSP based web applications. Dojo is also easy to use with Rails, if you want an alternative to Rail's great AJAX support.

By using Dojo, I basically have to deal with just one learning curve no matter what platform I am developing on. Here is a simple example for a JSP page (assuming that this would be more interesting to most people than a Lisp example):

Add this to your <head> section on a top level JSP page:

    <script type="text/javascript">
     var djConfig = { isDebug: true };
    </script>
    <script type="text/javascript" src="dojo.js"></script>
    <script type="text/javascript">
            dojo.require("dojo.io.*");
            dojo.require("dojo.event.*");
    </script>

Then try putting this somewhere on your JSP page (note: this assumes that you have another JSP page ajax.jsp that gets the form values and returns content to be placed into the DIV element with ID=ajxplydiv. Anthing that ajax.jsp writes using out.print() gets inserted asynchronously into the "ajxplaydiv" DIV element):

    <h3>test AJAX Form:</h3>
    <form id="myForm">
        <input type="text" name="input_test_form_text" />
        <input type="button" name="button1"
               value="Try AJAX form" id="ajaxButton" />
    </form>

    <div id="ajxplaydiv">
       initial context for AJAX play div
    </div>

    <script type="text/javascript">

    var buttonObj = document.getElementById("ajaxButton");
    dojo.event.connect(buttonObj, "onclick",
                       this, "my_onclick_button");

    function my_onclick_button() {
      var ajaxargs = {
        url: "ajax.jsp",
        load: function(type, data, evt){
           dojo.byId('ajxplaydiv').innerHTML = data;
        },
        error: function(type, data, evt){
         alert("Error occurred!");
        },
        mimetype: "text/plain",
        formNode: document.getElementById("myForm")
     };
     dojo.io.bind(ajaxargs);
   }
</script>

The call to dojo.io.bind sends a request containing the form data to the ajax.jsp page, and whenever the results are returned then the Javascript functions "load" or "error" defined in the data block in my_onclick_button() is called. This is just one example of processing form data and adding HTML below the form without refreshing the page - but is a common use case for using AJAX. This example assumes that I have both the dojo "src" directory and dojo.js in my top level web resources directory. The great thing about dojo is that it encapsulates all asynchronous event handling, offers a good supply of visual components (that map to HTML elements) and is simple to use no matter what programming language you are using for a web application.

Labels: AJAX, Java, Javascript, Ruby, search

I use Ruby a lot for small database tasks (love ActiveRecord!) and for using Rails for some web applications (not with scaffolding, mostly generating controllers and writing my own code).

That said, for a lot of what I do, Ruby is much more than an order of magnitude slower that compiled Common Lisp. I find myself still using Ruby and Java when appropriate, but for most tasks, I am going back to using Common Lisp. Better language, and the learning curve is not so bad (I have been using Lisp for over 25 years, and I am already up to a 'moderate' level of competence :-)

Labels: Lisp, Ruby

When writing large software systems, rapid access to data is often important: what can be kept in memory or more slowly: processes on the same local network and on disk. In software development, we see the same effect: maximum speed and efficiently if a single person can understand the architecture and comprehend the entire system. Moving from a single developer to a very small team adds a little overhead: design notes and pencil and paper drawings turn into casual but more explicit short documents and conversation. The optimisation is minimising cost between two people talking and sharing information vs. maintaining documentation and reading time. Talking is almost always better because communication is a two way street, but if you have N developers, O(N^2) "talking overhead" is too expensive with a large N, so back to the one way street of documentation.

I like to view programming languages in terms of how they allow me to deal with complexity, keeping as much stuff in my head at once:

• Lisp: great for building up the language from the bottom and extending towards an application domain. The new application "programming language" is higher level and the remaining part of a system is more concise code and easier to keep track of.
• Ruby: concise, so programs are much shorter and easier to understand.
• Java: the language does little for me as far as reducing complexity, but great IDEs like IntelliJ at least allow rapid code browsing, "who calls this" queries, etc.

Labels: AI, Lisp, Ruby

I am about 4 months into a project using a very expensive object persistence mechanism (Franz Lisp AllegroCache) that almost automatically wraps Common Lisp CLOS classes for automatic persistence and B-Tree indexing of as many slots as you want to tag as indexed. As is sometimes is the case, here commercial software tools save time and effort. There is a very interesting open source project for Common Lisp (Elephant) that does something similar by using BerkleyDB or a relational database as the backend - it looks good, but it is not yet an out of the box solution.

For Java work, although I was once a fan of Hibernate, I try to use Prevayler instead: if you add new instance variables to the end of class definitions you can add to your object model without breaking your persistent storage - a neat trick, to be sure.

When it comes to the best programmer productivity using object relational mappers, I personally think that Ruby Rail's ActiveRecord wins hands down - with the slight cost that you design database schema instead of class models.

Labels: Java, Lisp, object mapping, Ruby

There has been a lot of buzz over Sun hiring Charles Oliver Nutter and Thomas Enebo (two core JRuby developers). Certainly a fine thing for people who use both Java and Ruby. My hope is that they do a lot of work on Tomcat/WEBrick/Rails integration for hybrid JSP/Rails web apps.

I have been using both Ruby and Rails a lot in the last year, and have not missed Java too much. That said, an efficient Rails platform on top of a Hotspot JVM sounds good. My quick experiments with JRuby have not been totally without problems, but with Sun's obvious motivation to get a first rate Ruby environment running on the JVM, I expect things to get better. The smooth integration of Java classes in an interactive Ruby IRB shell environment is fun.

Labels: Ruby, Ruby Rails

For the last year most of my work has been using dynamic programming languages (Ruby and Common Lisp). I have written about this before but given that programmers are properly trained/educated, dynamic languages simply are more efficient when measured against programmer time. I think that the reason is fairly simple: dynamic programming, meta object protocol, Ruby's missing method, etc. all make it possible to solve some difficult problems with relatively few lines of code.

I have made a good living using the Java platform (starting with Java 1.0 beta) but I think that it is time to move on. Sure, I am still likely to take on interesting work even if it has to be done in Java, but I think that both Java and the .Net platform languages missed the boat on programmer productivity.

I have a few comments on Ruby vs. Common Lisp also: the beauty of Ruby is that it is a simple language to learn and to master. It is difficult to find programmers who are willing to make the effort to master Common Lisp and CLOS. Too bad, but that is the way it is. Personally, I still like Common Lisp a lot because I have already invested decades (part time) learning Lisp and Common Lisp blows Ruby away performance wise. BTW, my positive comments on Ruby and Common Lisp largely also apply to Smalltalk.

Labels: Lisp, Ruby, Smalltalk

Ruby, with great libraries like ActiveRecord and Ferret let you solve problems with just a few lines of code. I had 4 database tables that I wanted to index all fields of each row, and enable a plain text search that would quickly point me back to the table name and row index. The indexing takes about 20 lines of code and took less than 10 minutes to get working:

class CreateIndex
  def initialize
    @index = Ferret::Index::Index.new(:path => './search_index')
    ActiveRecord::Base.establish_connection(   
      :adapter => 'postgresql',
      :host => 'localhost',
      :username => 'postgres',
      :database => 'test_database')
    process_class(Item)
    process_class(Plan)
    process_class(Procedure)
    process_class(Visit)
    @index.close
  end
  def process_class the_class
    the_class.find_all.each {|obj|
      doc1 = Document.new
      doc1 << Field.new("text",  obj.attributes.values.join(' '),  Field::Store::YES, Field::Index::TOKENIZED)
      doc1 << Field.new("class", the_class.to_s,  Field::Store::YES, Field::Index::NO) 
      doc1 << Field.new("id", obj.id.to_s,  Field::Store::YES, Field::Index::NO) 
      @index << doc1
    }
  end
end

This would take a lot of code in Java, Common Lisp, etc. Unless my customers specifically ask me to use another programming language, I just about always use Ruby now.

Labels: object mapping, Ruby

I am doing a lot of coding using both Lisp and Ruby for my current project. I am finding interop between the languages to be especially easy because of the dynamic nature of both languages. When I use REST style calls I can dynamically create classes and instances just from the XML so changing classes in one language does not break code in the other.

Labels: Lisp, object mapping, Ruby

One thing that I like about Common Lisp and Ruby is the ability to examine class definitions, etc. at runtime. I have several model classes in a Common Lisp application and I want to implement a web services API for my application that in some cases returns model objects to the requester.

I wrote a very short utility function today that takes any class instance looks up all of the slot names, and then gets the slot values. I output to an XML stream.

I don't have to manually change any code to marshall any of my classes to XML if I change the model class definitions.

Labels: Lisp, object mapping, Ruby

I am working on a project that has many disparate data sources, and before I can do the interesting AI part everything needs to be normalized into our object models. Usually, this would be a mountain of work, but I am finding that using Ruby scripts with the ActiveRecord library is saving lots of time. I am basically taking advantage of both the dynamic nature of Ruby and the dynamic ORM functionality of ActiveRecord to cut the amount of required code (compared to say Java) by a large amount.

Ruby on Rails has a deserved reputation for saving time on web application development, but you should really consider ActiveRecord a separate part of Rails that can be used standalone.

Labels: object mapping, Ruby

I like this! From Matz:

Ruby is a language designed in the following steps:

  * take a simple lisp language (like one prior to CL).
  * remove macros, s-expression.
  * add simple object system (much simpler than CLOS).
  * add blocks, inspired by higher order functions.
  * add methods found in Smalltalk.
  * add functionality found in Perl (in OO way).

So, Ruby was a Lisp originally, in theory.
Let's call it MatzLisp from now on. ;-)

Since the late 1970s, I have always been happy writing software in Lisp, but except for the development of three commercial products and quite a bit of R&D, Lisp has been a hard sell. With the mass enthusiasm for Ruby on Rails, Ruby is starting to be an easy sell. My only problem with Ruby is that you don't get the white-hot high performance of compiled Common Lisp - but I can live with that.

Labels: Lisp, Ruby

I just had another reminder why I like late binding and flexible typing so much: I had written a handy little utility in Ruby for collecting some data, writing to an open file as data was collected and processed. Initially, I thought: why buffer everything in memory? Then I had the need to use the same functionality to dynamically generate a download from a web app, but I did not have to change any code: I need to collect text and send it out in a Rails controller (instead of using a view) back to a client web browser - I needed in-memory data, not a local file.

Well, since both String and File classes support << (which is what I used in the first version of the code for writing to an open file), I did not have to change any code since I could just pass in an empty string instead of an open file, and return what used to the a file descriptor (instead of closing it) as the value of the accumulated text. Try that in Java or C++ :-)

Anyway, not a big deal, but it did save me refactoring some old code for a new use, saving a few minutes.

Labels: Ruby