Tuesday, May 06, 2008

I am back from 14 days of travelling in Northern Mexico

It was a good trip: visits to private homes, meals with locals, etc. I have a few pictures online

Saturday, April 19, 2008

Using JSON for communicating between Ruby and Lisp or Scheme

JSON is very much lighter weight than XML and is meeting a need for easily calling some Scheme code from a Ruby program. The Scheme code I am using is old, I wrote it years ago to extract entities from plain text. Since the Scheme program starts very quickly, I am able to simply start a Scheme interpreter as a separate process using back quotes to capture any output to stdout in a Ruby string variable:
require 'json'
s = `gsi extraction.scm -e '(get-proper-names "President Bush moved to South America.")'`
# parse JSON text...
Here I am using Gambit-C Scheme interpretively. It is very easy to write any structured data in JSON format in the Scheme (or Lisp) code and parse it on the Ruby side. It is also easy to speed things up and compile my program:
gsc -c extraction.scm
gsc -link extraction.c
gcc -o extraction extraction.c extraction_.c -lgambc -I/Library/Gambit-C/current/include/ -L/Library/Gambit-C/current/lib/
which both makes the program faster and reduces the startup time down to a few milliseconds. Since the extraction program starts very quickly, this solution is good enough. However, I have been looking at an alternative idea in case I ever need to use a Lisp or Scheme program that has a long startup time. I have not done this yet, but here are my ideas for Gambit-C Scheme:This looks easy enough, but getting memory allocation and deallocation correct might be difficult. For now, I am calling my old Scheme code from Ruby the easy way :-)

Labels: , ,


Tuesday, April 15, 2008

Looks good: JStreet.org: pro-peace and pro-Israel political action group

In my opinion, for far too long the far right wing group (neoconservatives) American Israel Public Affairs Committee (AIPAC) has been pushing a militaristic foreign policy in the Middle East. This is a group that everyone in Congress lives in fear of because they are so well funded and organized. If you want to research the connections between the neoconservatives and AIPAC then Google is there to help you.

We live in a free country and AIPAC members are certainly free to do what they want but personally I hope that JStreet.org gets support from who I think are the vast majority: people who look for a more rational approach to problems in the Middle East, including the mess in Iraq and who disagree with the radical right wing crowd that now seems to want war with Iran. Most of us do not want unending war.

Labels:


Tuesday, April 08, 2008

New Google Web App Engine

This looks very good. You get access to Google's infrastructure: GFS, big table, email, etc. The IDE is cool: it only requires Python 2.5 to run.

I was too late to get one of the first 10,000 developer invites, but I am hopeful that before too long my turn will come :-)

The Google App Engine supports many Python web development frameworks like Django, CherryPy, Pylons, and web.py. I have only used CherryPy to any great extent, but Django has a good reputation, and I once worked through it's tutorial and it looked good (also, very good documentation). Google also supplies their own Python web application framework webapp.

Sunday, April 06, 2008

National Public Radio: "Our Confusing Economy, Explained" - excellent!

www.npr.org
Perplexed by the U.S. economy? You're not alone. Law professor Michael Greenberger joins Fresh Air to explain the sub-prime mortgage crisis, credit defaults, the shaky future of other types of loans and what we can expect from the U.S. financial markets.

Greenberger is a professor at the University of Maryland School of Law and the director of the University's Center for Health and Homeland Security.
This is a long listen (39 minutes), but is the sort of thing that if every voting American listened to, and if every politician in Washington knew everyone listened to, that a lot of the future corruption in Washington might be avoided. Professor Greenberger also gives very solid advice on what we must do as a country to avoid having Asian countries "eat our lunch": emphasize science and technology and build companies that produce real products - and don't idolize people and companies who make money speculating instead of producing. I liked that he seemed to think our problems are fixable, but only with positive action. Listening to this interview, and then emailing your 2 senators and your representative in congress regarding this material would be a good start.

Labels: ,


Friday, April 04, 2008

Blu-ray and Java vs. DVDs

As a consumer I have not bought a Blu-ray player yet - I think that it will still be some time before the technology settles. That said, I am intrigued by the BD-J Java programming environment for Blu-ray and something that I would like to look at. About 8 years ago I wrote a set top box emulation in Java for Disney, perhaps similar to what people may do in BD-J with custom directory viewers, ways to annotate material, etc. Who knows what smart developers will do with the platform? Not me.

I am turned off by the extremely high cost of writable Blu-ray media - yuck! The prices will come down, but by how much?

Massive personal storage does not (yet) excite me. We have hidef satellite TV with a hidef DVR, but we don't need to store too many hours of video, and there is nothing so far that we want to keep around for more than a week. My brother owns 1500+ Blu-ray, HDVD, and DVDs and he takes great pleasure in owning entertainment media. My wife and I prefer renting (NetFlicks) and time delayed DVR viewing. It is easy enough to re-order a NetFlix movie in the future if I want to watch something again.

I still find the storage on a DVD-R to be great for most of my uses, especially since I often compress backup data before writing. If you deal with a lot of text, like I do, it is amazing what fits onto a DVD-R.

Thursday, March 27, 2008

Book review: "Building SOA-Based Composite Applications Using NetBeans IDE 6"

Authors David Salter and Frank Jennings have written a very targeted book specifically for enterprise developers wanting to use NetBeans with the SOA plugins.

Service Oriented Architecture (SOA) can be loosely described as implementing business processes by combining web service calls to available services. Just as you would probably not want to perform database operations without transactions, when you combine web service calls for specific business processes you want to wrap them in transactions and using BPEL (Business Process Execution Language) modules makes it easier to express logic and error handling. The BPEL designer plugin lets you work with a graphical interface and avoid tedious hand editing of XML files.

Just like using full J2EE stack adds a lot of complexity, with attendant benefits, to large scale Java server side development, SOA also adds a lot of "baggage" to building systems using web services, but as systems become larger and more complicated that structure can really pay off.

There are no magic bullets in software design and development that remove the requirement for hard work in analyzing system requirements and iteratively work on design and staged prototypes: for SOA applications you need to understand your business processes, what events can occur, and what error conditions need to be handled. If you don't understand these basics, no framework is likely to save a project. If you do understand these basics, and you want to use Java SOA frameworks and tools, then this book is a good guide to getting started. I do have a few minor complaints about this book: figures are not numbered (perhaps not such a bad idea, I am just used to seeing figures identified) and there is sometimes too much detail given for some developers: the reader is carefully led through setup, using the IDE plugins specific to SOA (BPEL builder, WSDL and XML Schema editors, etc.) and sample applications with many screen shots. Even with these small complaints, this highly targeted book should be very useful to Java developers looking to develop SOA applications. It would be best if readers already had at least a little knowledge of SOA, BPEL, etc. since this book mostly covers using the tools in NetBeans to facilitate build SOA applications.

Labels: , ,


Wednesday, March 26, 2008

Minimum number of languages that a developer should master?

A funny moment: recently I heard a senior computer scientist blast Java on very poor performance (server side Java performance is very good - the language has other problems in my opinion) while extolling the virtues of Common Lisp as, if I understood him correctly, the one true 'do everything' language. I enjoy Lisp (I wrote 2 Springer-Verlag Lisp books, many years ago), but it is a marginal language if you count developers, available frameworks, and relatively few large deployed systems (a few notable exceptions to this). Lisp is a language that I can recommend to some people as a second or third language, but I would have a difficult time recommending it as any developer's primary language for writing production code. Lisp is great for research.

In any case, whenever I hear someone blasting Java, I know that they likely don't have real experience developing large scale Java server side applications and don't understand the benefits of the server JVM and a huge collection of good tools and frameworks.

Whenever I hear someone knocking scripting languages like Ruby, Python, or Perl I always think that what a shame it is that they don't save themselves a lot of development time using an agile scripting language - when appropriate.

Then there are times when it is "just right" that development be slow and painful, and C++ is used for runtime speed and memory efficiency :-)

Also if you need a carpenter to do some maintenance on your home: don't hire a carpenter who only uses a saw.

Advantages of open source: quickly working around problems

Yesterday was fun. We use SBCL (Common Lisp) on a project I work on, and my customer hit a limit of SBCL not using all the memory on one of their new servers. A quick email to a SBCL developer, and we were patched and running very quickly (thanks Nikodemus!)

A decision to use any open source project is usually based on how "vibrant" and active the community is surrounding an open source project. This is easy to determine developer activity by release history and user base by newsgroups + wikis.

I definitely have my favorite commercial developers (plugs to Franz Lisp, Jetbrains/IntelliJ, OmniGroup/OmniGraffle, and TextMate) and my decision to purchase and use commercial products is based strongly on support, patches, and new versions.

Sunday, March 16, 2008

C++, taking a second look

I earned my living between 1988 and 1997 developing and being a mentor using C++. I usually argue against using C++, a language that offers superb runtime performance (speed and memory utilization) with the penalty of greatly increased development costs.

For most applications, it is less expensive to optimize for developer productivity. In my decade of using C++, with hindsight, the only projects that needed the performance of C++ were a commercial product for Windows 1.03, some VR work for SAIC and Disney, work on two Nintendo video games, a real time expert system for PacBell, and a PC racing game. All other C++ projects could have been done with more economy and effectiveness using other languages.

I used to error on the side of staying current with too many programming languages, although in the last few years I have invested heavy time in only three: Ruby, Java, and Common Lisp. Even though I find C++ development to be less fun, slow, and sometimes even a little painful, I am considering replacing Common Lisp with C++ in my small set of 3 languages that I am willing to invest heavily in. This Benchmark game of C++ vs. SBCL Lisp is one reason for this (possible) decision. The other reason is that I find Ruby to also be very good for quick prototyping and fast agile development, and even though the runtime performance of Ruby is very poor, it seems like the combination of Ruby + Java + C++ covers a wider range of application development than the combination of Ruby + Java + Lisp. In other words: Ruby gives me a lot of what Lisp does and I feel like I need one agile scripting language in my programming language toolbox.

Sure, "the best tool for the job", but I like to also consider the costs of not totally mastering the tools that I use in my work. This is why I have given up (for serious work) some great programming languages like Python, Smalltalk, C#, and Prolog.

Labels: , , ,


Saturday, March 15, 2008

Excellent video: Dan Ingalls demos Lively Kernel at Google

I wrote about Sun's Lively Kernel web browser based programming environment last month and watched this excellent Lively Kernel talk by Dan Ingalls this morning. I especially liked one point that he made: with the right architectural decisions systems can be fairly simple and still be general.

The demonstration on the video is both impressive and serves as documentation if you want to experiment with the Lively Kernel yourself. As it is, the Lively Kernel is a good prototype but if the performance is improved a bit and the programming environment and IDE (which is like Squeak Smalltalk, only for Javascript) continues to improve, then Lively Kernel is on my (personal) radar for something that I will use in the future.

I remember getting a Xerox 1108 Lisp Machine in 1982 and I also wanted the optional Smalltalk environment which was not in the budget. However, a sales engineer at Xerox did let me use Smalltalk for a month on a trial basis. It is amusing to be able to get a smalltalk like programming environment like Lively Kernel for free that is hosted in a web browser. Something about Moore's Law and also about free software projects...

Off course, by far the best free Smalltalk experience is Squeak but Lively Kernel is still very cool, if only for its possible future use of quickly writing web interfaces using existing web services, etc.

Labels: ,


Wednesday, March 12, 2008

My Spanish4.us web portal

My Spanish4.us web portal is a simple 1 page fits all study center for a Spanish class that my wife and I are taking.

This is something that I love about Ruby on Rails: I had a simple idea and in less than 2 hours I had prototyped the application and deployed it to one of my leased servers. I will add more phrases to the popup translation tool and more general information over the next few weeks.

Labels:


Friday, March 07, 2008

Ruby becoming a first class language on Mac OS X

Good news: Apple is supporting Ruby 1.9 using the Objective C runtime. This mainly to support building native Mac applications but this will eventually be good for other more general purpose Ruby development on Macs. Very cool since both Objective C and Ruby have their roots in Smalltalk.

Labels: ,


Friday, February 29, 2008

Importance of understanding business and sales issues; dynamc languages

I tend to view work from a consultant's point of view, but this also probably applies to you if you work for a company: while staying on top of a few technologies is obviously important (I find that the combination of Ruby, Ruby on Rails, Java, and Common Lisp covers most of what I need to get just about any job done using tools that are at least reasonably appropriate), success in any information processing career requires more than just technical savvy:

What is much more difficult, but in some ways more fun and rewarding, is the effort to learn as much as possible about the business and sales processes relevant to each project. Ultimately, most software is written and maintained to meet business and sales goals, so it pays to understand non-IT related issues as well as technical IT issues.

One reason that I like to use Ruby is that the language is so concise and terse, that I can spend more time thinking about the larger issues of problems that I am trying to solve - the technical aspects of writing code are diminished.

Saturday, February 23, 2008

Ruby client code for accessing OpenCalais and Metaweb/Freebase web services

I wrote a Ruby API for accessing OpenCalais this morning. OpenCalais processes text and extracts semantic information.

On a slightly related subject: I have enjoyed experimenting with the Python Metaweb APIs in the last year, and I just wrote about in my AI blog about Christopher Eppstein's new ActiveRecord like API for accessing structured data in Freebase.

Thursday, February 21, 2008

Heavy weight Javascript client applications vs. lighter weight AJAX

I experimented with Mjt last year: Mjt is a client side template system: Javascript is used to merge data from JSON web service calls with templates to generate HTML - all in the browser (except for data fetched from a server). Mjt looks solid and has been fairy widely used; an alternative client side framework is Sun's experimental (not for production use!) Lively Kernel project. If you have not played with Lively Kernal, give it at least a one minute try - it uses the Morphic GUI framework, so if you have used Squeak, it will seem familiar.

The big problem, as I see it, of client side Javascript frameworks is issues of maintainability. I have worked with Javascript heavy web applications that other people originally wrote and they are definitely much more difficult to jump into, understand, and modify compared for example to AJAX heavy Rails applications or GWT web applications.

That said, there is something tidy about the idea of writing web applications in two intertwined but separate tasks:I like to recognize technologies as early as possible that I might use in the future. Although I don't (yet) feel really comfortable working with frameworks like Mjt my gut feeling is that this is the future because it makes it easier to work with multiple languages and platforms for implementing web services and makes it easier to mix up data from multiple sources.

Labels:


Microsoft live.com, Yahoo attempted buyout

I have been following the attempted Yahoo buyout with great interest because I buy into the idea of universal access to online information using many types of devices: PCs, Macs, iPhones, Nokia N800s, secret decoder rings, etc.

In the future that I predict and look forward to, following and exploiting standards will be absolutely required for success. As part of my own research (and fun), I just about continuously try and evaluate every type of online information service (Amazon's web services, Google gdata, freebase.com, dabbledb.com, etc.)

Microsoft's live.com seems to be getting better as far as supporting Mac, Linux, Firefox, etc. The question to me is: how open is Microsoft willing to become?

If I were to sit down and enjoy a beer with Bill Gates and Steve Balmer (unlikely unless they are vacationing in Sedona, Arizona) I would have some good advice for them: do a sea change and embrace open standards, stop selling new versions of Windows and instead sell yearly subscriptions to Windows and Office (slow improvements, no more big "XP", "Vista", etc. releases), and use their resources to make their software and infrastructure flexible, standard, and valuable to users.

If Microsoft does buy Yahoo, it will be interesting to see if they try to force changing to Microsoft infrastructure: they certainly had problems after buying Hotmail and doing a major conversion to Microsoft server side infrastructure. Yahoo is doing some great things with Open Source (Hadoop, Javascript libraries, etc.) and it will be interesting to see if Microsoft will permit using competing infrastructure software for internal systems.

Labels: ,


Wednesday, February 20, 2008

Hillary Clinton: corporate shill

I had intended to not write any more political blogs, but I need to express myself here:

Hillary Clinton served on the board of directors for Walmart while that company was cutting worker benefits. I do not like her and Bill's long standing enabling of Monsanto's push of genetically modified food and seeds that produce plants that generate infertile seeds - forcing many farmers in third world countries into bankruptcy. When she became a US senator, she and Fritz Hollings accepted very large soft money payments from Disney and promptly promoted the awful pro-Disney SSSCA legislation.

I just read yesterday that she is getting more defense industry PAC money than John McCain. Read between the lines on that one.

The problem that my country has is huge: being a military super power is just not a viable economic strategy. Spending more on our military industrial complex than every other country combined just does not pay for itself, but is making many politicians and their corporate masters a lot of money. So a fundamental flaw in Hillary Clinton's campaign for president is that not only is she herself a bad pick because I believe that she will keep our country on the non-competitive current "military super power" strategy, but even worse, she is unlikely to win a general election against John McCain - on the other hand, every poll that I have seen seems to indicate that Barack Obama could fairly easily beat McCain in a general election. Hillary Clinton is acting as a spoiler, an agent to get John McCain elected, and she is being enabled and supported by many democrats who are not clearly analyzing how to achieve their political goals. BTW, I like John McCain, but his current views on foreign policy make him, in my opinion, a poor candidate for president.

While I am not as excited about Barack Obama's run for president as my more liberal friends, he seems to be the logical choice for president.

Labels:


Friday, February 15, 2008

My DevX article "Real-Life Rails: Develop with NetBeans, Deploy on Linux"

My most recent DevX article has just been published. This was fun material to write about because after some experimentation I feel like I have my Ruby on Rails development environment and server deployment strategy just right, at least for my needs. I should mention that although I have been professionally writing Ruby on Rails applications for a few years, I have not yet written an application that will not run nicely on a single server using nginx, memcache, and a few mongrels. I set my development.rb environment for my MacBook and my production.rb environment for the Linux server I am deploying to, and svn is the glue that holds everything together. If you are interested in deploying very large scale applications, my article will not be very useful to you.

Labels: , ,


IBM's Project Zero

IBM has an interesting idea with Project Zero, which borrows a lot from ideas behind frameworks like Ruby on Rails: use of a dynamic scripting language (Groovy or PHP), use of a "script aware" HTML template language, and built in support for REST and AJAX.

I worked through the tutorial that uses Groovy (instead of the other supported scripting language PHP), and my first impression is that the Eclipse plugin support is well done (although color and syntax support for editing templates would be good) and the framework meets its goals: support building interactive web applications with little required knowledge of the underlying technologies.

I would be more enthusiastic about Project Zero if I were a Groovy enthusiast. For Groovy loving developers, Project Zero looks to be very useful.

Friday, February 08, 2008

NetBeans 6.1 development build: almost there for my work

I just tried the daily dev build (NetBeans 6.1 Dev 200802080008) for OS X. It is almost there for my daily work - my current Java development project (a commercial version of my old NLBean open source project with a new AI NLP module), Scala coding experiments, and new Rails projects all work great. The one problem: I get errors when using existing Rails NetBeans projects (actually, I get the same errors when trying to modify project properties in new Rails projects but new projects can be created with the desired properties). Close, but not quite there. BTW, the Scala NetBeans plugins, which are very new, are looking very good.

Labels:


Tuesday, February 05, 2008

PostgreSQL 8.3 on OS X: I like the full text indexing/search features

I built the latest version from source, with one problem: I was only able to install readline from source using "--disable-shared" so I ended up also building PostgreSQL statically linked - oh well so much for being in hurry, I have 2 gigs of RAM on my MacBook, so what is a little memory between friends :-)

I have been waiting for version 8.3 because of the full text indexing/search features. Here is the Text Search documentation - enjoy! Here is a little sample of the SQL extensions to support indexing and search:
test=# create table test (id integer, name varchar(30), email varchar(30));
CREATE TABLE

test=# create index test_name_idx on test using gin(to_tsvector('english', name));
CREATE INDEX
test=# insert into test values (1, 'Mark Watson', 'mark@mark.com');
INSERT 0 1
test=# insert into test values (2, 'Carol Watson', 'carol@mark.com');
INSERT 0 1
test=# select * from test where to_tsvector(name) @@ to_tsquery('mark');
id | name | email
----+-------------+---------------
1 | Mark Watson | mark@mark.com
(1 row)

test=# select * from test where to_tsvector(name) @@ to_tsquery('watsons');
id | name | email
----+--------------+----------------
1 | Mark Watson | mark@mark.com
2 | Carol Watson | carol@mark.com
(2 rows)

test=# test=# select * from test where to_tsvector(name) @@ to_tsquery('mark & watson');
id | name | email
----+-------------+---------------
1 | Mark Watson | mark@mark.com
(1 row)

test=# select * from test where to_tsvector(name) @@ to_tsquery('mark | watson');
id | name | email
----+--------------+----------------
1 | Mark Watson | mark@mark.com
2 | Carol Watson | carol@mark.com
(2 rows)

test=#
Obviously, if you were creating a new table with many rows, add the index after the data is added to the table. "gin" refers to a complete inverted word index. Specifying 'english' ensures that a word stemmer if used that understands English language conventions. Note that a search for 'watsons' matches because the search terms are stemmed before search.

The search syntax looks odd, but I expect to get used to it quickly. For Rails: I use "acts_like_ferret" a lot; I'll wait a month to see if any handy plugin is written for PostgreSQL specific search - I would rather that someone else write it. I need to check out acts_as_tsearch, but I don't think that it is updated yet to work with the final 8.3 release.

Monday, February 04, 2008

Snowing in Sedona Arizona

My wife and I took a drive this morning after it stopped snowing. Some nice pictures taken near our home in Sedona:
near Boynton Canyon
Dry Creek Road - looking west

XMPP (Jabber)

I had experimenting with XMPP on my long term list of things to do. I took a 90 minute break from work this afternoon to set up a playground: OpenFire XMPP server and the Ruby XMPP4r client library. Setting up the OpenFire service on one of my leased servers was easy - a very good administration web application and in general an easy install.

I had more problems with XMPP4r but setting Jabber::debug = true helped. I installed the easier to use wrapper library xmpp4r-simple but decided that its API was probably too limited (long term), so I might as well get used to XMPP4r.

I also grabbed the Common Lisp XMPP client cl-xmpp but experimenting with Ruby clients is probably easier. The OpenFire developers also supply a Java client library (Smack) that is on my list of things to try.

I think that XMPP may be a good "push" technology for distributed knowledge sharing systems (an interest of mine). XMPP has a lot going for it: a good security model, straight forward bi-directional communication between any two connected clients, and a publish/subscribe capability like the Java Message System (JMS). The Comet architecture (uses HTTP and JSON, instead of socket connections and XML) looks interesting but XMPP seems to have a head start and I don't think that I need to learn both technologies (yet).

Getting Things Done: a perspective from a work at home programmer

While I like to automate repetitive tasks (server deployments, builds, tests, etc.), I also enjoy "tuning up" my personal work habits, tweaking them to get things just right. Hopefully, you will find something useful here (and please add comments on how you "tune up" your own work flow):

This page is powered by Blogger. Isn't yours?

Subscribe to Posts [Atom]