Artificial Intelligence Blog

Protégé OWL Ontology Editor

2008-03-29T08:07:00.000-07:00

I installed Protégé version 4 alpha last night and it has been solid for me so far. It has been over a year since I upgraded my local Protégé installation, and I like these (new ?) features a lot:

Saved XML for OWL ontologies is very readable, with good automatically generated comments and a nice layout
Use of the Java OWL API
Both Fact++ (using JNI) and Pellet 1.5 are smoothly integrated
The Owlviz Plug-in seems to display graphs faster
Drag and drop can be used rearrange class hierarchies

I have started working again on an old KnowledgeBooks project: an OWL ontology for news stories and associated "Semantic Scrappers" to populate the ontology from plain texts of news stories. I am still in the experimentation stage for the semantic scrapper: I am trying to decide between pure Ruby code, Java using the available OWL APIs, or a combination of JRuby and the Java OWL APIs. I am currently playing with these three options - for now, I am in no hurry to choose a single option. Using a dynamic language like Ruby has a lot of advantages as far as generating "OWL classes" automatically from an ontology, etc. That said, the Java language has the best semantic web tools and libraries.

Long term, I would like a semi-automatic tool for populating ontologies via custom scrapper libraries. I say "semi-automatic" because it would be useful to integrate with Protégé for manual editing and browsing, while supporting external applications accessing data read-only (?) via the Java OWL APIs.

Ruby API for accessing Freebase/Metaweb structured data

2008-02-23T12:35:00.000-08:00

I had a good talk with some of the Metaweb developers last year and started playing with their Python APIs for accessing structured data. I wanted to be able to use this structured data source in a planned Ruby project and was very pleased to see Christopher Eppstein's new project that provides an ActiveRecord style API on top of Freebase. Here is the web page for Christopher's Freebase API project. Assuming that you do a "gem install freebase", using this API is easy; some examples:

require 'rubygems'
require "freebase"
require 'pp'

an_asteroid = Freebase::Types::Astronomy::Asteroid.find(:first)
#pp "an_asteroid:", an_asteroid
puts "name of asteroid=#{an_asteroid.name}"
puts "spectral type=#{an_asteroid.spectral_type[0].name}"

#all_asteroids = Freebase::Types::Astronomy::Asteroid.find(:all)
#pp "all_asteroids:", all_asteroids

a_company = Freebase::Types::Business::Company.find(:first)
#pp "a_company:", a_company
puts "name=#{a_company.name}"
puts "parent company name=#{a_company.parent_company[0].name}"

You will want to use this API interactively: use the Freebase web site to find type hierarchies that you are interested in, fetch the first object matching a type hierarchy (e.g., Types -> Astronomy -> Asteroid) and pretty print the fetched object to see what data fields are available.

My OpenCalais Ruby client library

2008-02-23T10:49:00.000-08:00

Reuters has a great attitude about openly sharing data and technology. About 8 years ago, I obtained a free license for their 1.2 gigabytes of semantically tagged news corpus text - very useful for automated training of my KBtextmaster system as well as other work.

Reuters has done it again, releasing free access to OpenCalias semantic text processing web services. If you sign up for a free access key (good for 20,000 uses a day of their web services), then you can use my Ruby client library:

# Copyright Mark Watson 2008. All rights reserved.
# Can be used under either the Apache 2 or the LGPL licenses.

require 'simple_http'

require "rexml/document"
include REXML

require 'pp'

MY_KEY = ENV["OPEN_CALAIS_KEY"]
raise(StandardError,"Set Open Calais login key in ENV: 'OPEN_CALAIS_KEY'") if !MY_KEY

PARAMS = "¶msXML=" + CGI.escape('<c:params xmlns:c="http://s.opencalais.com/1/pred/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><c:processingDirectives c:contentType="text/txt" c:outputFormat="xml/rdf"></c:processingDirectives><c:userDirectives c:allowDistribution="true" c:allowSearch="true" c:externalID="17cabs901" c:submitter="ABC"></c:userDirectives><c:externalMetadata></c:externalMetadata></c:params>')

class OpenCalaisTaggedText
  def initialize text=""
    data = "licenseID=#{MY_KEY}&content=" + CGI.escape(text)
    http = SimpleHttp.new "http://api.opencalais.com/enlighten/calais.asmx/Enlighten"
    @response = CGI.unescapeHTML(http.post(data+PARAMS))
  end
  def get_tags
    h = {}
    index1 = @response.index('terms of service.-->')
    index1 = @response.index('<!--', index1)
    index2 = @response.index('-->', index1)
    txt = @response[index1+4..index2-1]
    lines = txt.split("\n")
    lines.each {|line|
      index = line.index(":")
      h[line[0...index]] = line[index+1..-1].split(',').collect {|x| x.strip} if index
    }
    h
  end 
  def get_semantic_XML
    @response
  end
  def pp_semantic_XML
    Document.new(@response).write($stdout, 0)
  end
end

Notice that this code expects an environment variable to be set with your OpenCalais access key - you can just hardwire your key in this code if you want. Here is some sample use:

tt = OpenCalaisTaggedText.new("President George Bush and Tony Blair spoke to Congress")

pp "tags:", tt.get_tags
pp "Semantic XML:", tt.get_semantic_XML
puts "Semantic XML pretty printed:"
tt.pp_semantic_XML

The tags print as:

"tags:"
{"Organization"=>["Congress"],
 "Person"=>["George Bush", "Tony Blair"],
 "Relations"=>["PersonPolitical"]}

OpenCalais looks like a great service. I am planning on using their service for a technology demo, merging in some of my own semantic text processing tools. I might also use their service for training other machine learning based systems. Reuters will also offer a commercial version with guaranteed service, etc.

NLTK: The Natural Language Toolkit

2008-02-04T08:55:00.000-08:00

I have a 22 year history of working with natural language processing, but for the most part this was a low level of effort (perhaps averaging 3 to 5 weeks a year). For learning (and perhaps for some production work if you extract the parts that you need) I can very much recommend NLTK.

NLTK developers (or aggregators since NLTK is an aggregate of smaller projects, with a lot of new work added) Steven Bird, Ewan Klein, and Edward Loper are writing a complete book on NLP using NLTK that looks good. I wish that I had a good resource like this 22 years ago!

texai.org

2008-01-22T06:56:00.000-08:00

Stephen Reed is doing some interesting research at texai.org. This project is interesting to me because there is a lot of overlap with my own KBSportal.com project (which I am currently re-writing in Ruby and Ruby on Rails: the first two versions were in Common Lisp and Java): RDF, Sesame, knowledge representation, and NLP.

For me, working on KBSportal.com has been a learning process, and I think that texai also serves that purpose for Stephen.

RapidMiner machine learning, data mining, and visualization tool

2008-01-18T11:29:00.000-08:00

When analyzing data for customers I usually start with a tool like Weka or occasionally write custom analysis code. The GUI tool RapidMiner (used to be called YALE) provides a nice environment for setting up experiments by selecting operations and models to apply to your data sets. RapidMiner uses Weka, libsvm, JAMA, and several graphics visualization libraries. RapidMiner has data loaders for relational databases, spreadsheets, comma delimited files, etc.

It is well worth watching the demo video for a quick introduction before working through the tutorial.

Machines Like Us

2007-12-14T06:54:00.000-08:00

Norm Nason emailed me with a link to his site Machines Like Us that contains an eclectic sampling of philosophy of AI, some interesting interviews (e.g., with my friend and old boss Ben Goertzel), biographies, etc.

Using the PowerLoom reasoning system with JRuby

2007-10-02T10:46:00.000-07:00

I am writing a free web book on AI programming with Ruby (a work in progress).

I wanted to use the PowerLoom reasoning system for the book chapter on reasoning. I had two choices: using the C++ PowerLoom libraries with Ruby bindings or writing a Java wrapper class and then using JRuby. I decided to use JRuby. After some more work on converting to native Ruby types for query results, the material in this blog will eventually make its way into my Ruby AI book (along with a larger example). Here is a simple PowerLoom source file "test.plm" that I will use (part of an example by Robert M. NacGregor in the PowerLoom distribution):

(defmodule "BUSINESS"
  :documentation "Module for the Business demo example used in the PowerLoom Manual."
  :includes ("PL-USER"))

(in-module "BUSINESS")

;; clear any info from previous runs:
(clear-module "BUSINESS")
(reset-features)

;;; Concepts

(defconcept company)
(defconcept corporation (?c company))

;;; Relation

(defrelation company-name ((?c company) (?name STRING)))

and here is a JRuby example program that uses the wrapper (seen later):

require "java"
require "powerloom.jar"
require "pl.jar"
require 'pp'

include_class "PowerLoomWrapper"

pl = PowerLoomWrapper.new

pl.load("test.plm")
pl.changeModule("BUSINESS")
pl.assertProposition("(and (company c3) (company-name c3 \"Mom's Grocery\"))")

answers = pl.doQuery("all ?x (company ?x)")
answers.each {|answer| puts answer}

answers = pl.doQuery("all (?x ?name) (and (company ?x) (company-name ?x ?name))")
answers.each {|answer| puts answer}

Assuming that you get the powerloom.jar and stella.jar files from the PowerLoom web site (grab the latest download), this Java wrapper should work for you:

import edu.isi.powerloom.*;
import edu.isi.powerloom.logic.*;
import edu.isi.stella.Cons;
import java.io.File;

public class PowerLoomWrapper {

    public PowerLoomWrapper() {
        System.out.print("Initializing...");
        PLI.initialize();
        System.out.println("    done.");
    }
    public int load(String fpath) {
        try {
            PLI.load(fpath, null);
            return 1;
        } catch (Exception ex) {
            System.out.println("Error loading " + fpath + " is: " + ex);
            return 0;
        }   
    }
    public int changeModule(String workingModule) {
        try {
            PLI.sChangeModule(workingModule, null);
            this.workingModule = workingModule;
            return 1;
        } catch (Exception ex) {
            System.out.println("Error changing module " + workingModule + " is: " + ex);
            return 0;
        }   
    }
    public int assertProposition(String proposition) {
        try {
            PLI.sAssertProposition(proposition, workingModule, null);
            return 1;
        } catch (Exception ex) {
            System.out.println("Error asserting proposition '" + proposition + "' is: " + ex);
            return 0;
        }   
    }
    public int createRelation(String relation, int arity) {
        try {
            PLI.sCreateRelation(relation, arity, workingModule, null);
            return 1;
        } catch (Exception ex) {
            System.out.println("Error creating relation '" + relation + "' with arity=" + arity + " is: " + ex);
            return 0;
        }   
    }

  // following method donated by Thomas Russ:
  public java.util.ArrayList doQuery(String query) {
    java.util.ArrayList ret = null;
      try {
       PlIterator rawAnswer = PLI.sRetrieve(query, workingModule, null);
       ret = new java.util.ArrayList(rawAnswer.length());
       java.util.Iterator answer =
         new edu.isi.stella.javalib.StellaIterator(rawAnswer);
       while (answer.hasNext()) {
         Object obj = answer.next();
         int nItems = PLI.getColumnCount((edu.isi.stella.Stella_Object)obj);
         java.util.ArrayList r2 = new java.util.ArrayList(nItems);
         for (int i = 0; i < nItems; i++) {
           r2.add(fix(PLI.getNthValue((edu.isi.stella.Stella_Object)obj, i, null, null)));
         }
         ret.add(r2);
       }
      } catch (Exception ex) {
       System.out.println("Error with query '" + query + "': " + ex);
       ret = new java.util.ArrayList(0);
    }
      return ret;
  }


    private String workingModule = "PL-USER";
}

Here is the Makefile that I use for building a second jar file pl.jar" and running the sample program:

all:
 javac -classpath .:powerloom.jar:stella.jar PowerLoomWrapper.java
 jar cvf pl.jar PowerLoomWrapper.class PowerLoomWrapper.java
 /Users/markw/bin/jruby-1.0/bin/jruby jruby_powerloom.rb

Good video: Knowledge Representation and the Semantic Web

2007-06-23T07:06:00.000-07:00

Peter Patel-Schneider gave this talk at Google January, 2006.

Peter Patel-Schneider has an interesting perspective: while he has been very involved with Semantic Web technologies like OWL and descriptive logic reasoners (author of the Classic system), he also appears to have a reasonable amount of skepticism. His talk is good because he starts off clearly explaining RDF, RDFS, and the improved formalism and expressiveness of OWL. I especially enjoyed his summarization of different types of logic, what their computational limitations and capabilities are.

I have mixed feeling about trying to implement the Semantic Web using a formal semantically rich language like OWL: I am concerned that encoding information in OWL simply takes too much effort. At the other end of the complexity spectrum, tagging is a very light weight expression of the Semantic Web but tagging is inadequate (but easy enough that many people make the effort).

I have a short list of Semantic Web resources that I find useful.

N-GRAM analysis using Ruby

2007-06-12T17:16:00.000-07:00

I dusted off some old code today to look at common word pairs in some customer data. NGRAM analysis finds the most common bi-grams (2 word combinations), tri-grams (3 word combinations), etc. The code is simple, and I share it here in case you ever need to do the same thing:

require 'zip/zipfilesystem'

def words text
  text.downcase.scan(/[a-z]+/)
end

Zip::ZipFile.open('../text.txt.zip') { |zipFile| # training data
  $words = words(zipFile.read('text.txt'))       # is in a ZIP file
}

bi_grams = Hash.new(0)
tri_grams = Hash.new(0)

num = $words.length - 2
num.times {|i|
  bi = $words[i] + ' ' + $words[i+1]
  tri = bi + ' ' + $words[i+2]
  bi_grams[bi] += 1
  tri_grams[tri] += 1
}

puts "bi-grams:"
bb = bi_grams.sort{|a,b| b[1] <=> a[1]}
(num / 10).times {|i|  puts "#{bb[i][0]} : #{bb[i][1]}"}
puts "tri-grams:"
tt = tri_grams.sort{|a,b| b[1] <=> a[1]}
(num / 10).times {|i|  puts "#{tt[i][0]} : #{tt[i][1]}"}

Output might look like this:

bi-grams:
in the : 561
in Java : 213
...
tri-grams:
in the code : 119
Java source code : 78
...

Cool stuff. Ruby is my favorite language for tool building.

I have a new page on Knowledge Management

2007-06-10T16:03:00.000-07:00

I have a new page on Knowledge Management where I share some philosophy on why I believe that open source plays such an important role in building effective KM systems. I believe that as much as possible it is best to use available open source infrastructure software and use available resources for "value add" user modeling and AI components. I list on the right side of this web page some of the better free resources for doing KM work.

The URL for this blog has changed

2007-06-03T09:31:00.000-07:00

Please use the new URL http://markwatson.com/aiblog/ The old URL will redirect.

The new RSS feed is: http://artificial-intelligence-theory.blogspot.com/atom.xml

My experiences writing AI software for vehicle control in games and virtual reality systems

2007-04-30T15:11:00.000-07:00

I have in the past offered advice in the comp.ai.games news group re: AI control of vehicles. Here are a few short notes based on some experience that I have had. I have had the good fortune of getting to spend a fair amount of time (both on my own, and while being paid) writing code to control vehicles in games, etc.:

* Writing example code for my book AI Agents in Virtual Reality Worlds
* Working on the Indy 500 car simulation system at SAIC. I actually worked on the motion platform control, network programming, and sound system software. For this project, you always raced against other people, also in simulators.
* At Angel Studios working on 2 Nintendo games and a PC-based hover craft racing game.

Anyway, I do have some advice on how to write AI driving opponents:

The first thing that you want to do is to define data structures for navigational "way points" that AI vehicles follow and simply have the vehicles move between these "way points". If possible, AI controlled vehicles should use the same physics simulation code as the player's vehicle (at least when the AI vehicle is close and in the player's field of view).

After your AI controlled vehicles can move along using "way points", then, based on the game theme, add logic for detecting the proximity of the player's vehicle (e.g., AI vehicle might swerve into the player's vehicle under specified circumstances), etc. Do not bother with complex behavior if a vehicle is not in the rendered scene for a player.

It should be fairly easy to get the code running to follow way points, but that is definitely your first step. Combining driving logic control with "lerping" between weigh points will also make the vehicles look a lot more realistic while more or less keeping them where you want them.

When possible, your movement logic should depend heavily on where the AI vehicle is in the player's field of view. For efficiency, it might be expedient to use simple finite state machines for the control logic. However, I have experimented (again, see my book) with using expert systems, neural networks, and genetic algorithms. For Java programmers, my book "Intelligent Java Applications" contains a (not practical, but fun!) example of using genetic algorithms to train simple control programs in an "asteroids" style arcade game.

Good luck, and have fun!

Something like Google page rank for semantic web URIs

2007-04-17T08:34:00.000-07:00

A key idea of the semantic web is reusing other people's URIs - this reuse forms interconnections between different RDF data stores. The problem, as I see it, is that while URIs are unique, we will end up with a situation where (as an example) there might be thousands of URIs for NewsItem. The Swoogle semantic web search engine is useful for finding already existing URIs - a good first step toward reuse.

Swoogle will return results using their type of 'page ranking'.

So, we need two things to accelerate the adoption and utility of the semantic web: web sites must start to include RDF and this included RDF must reuse common URIs for concepts and instances.

Great Overview of Knowledge Representation

2007-04-09T12:55:00.000-07:00

Bill Clementson mentioned Conrad Barski's new open content article on Knowledge Representation. Nice! Conrad provides not only a great overview, but just as important, the motivation for treating Knowledge Representation as a science.

Defining AI and Knowledge Engineering

2007-04-01T07:32:00.000-07:00

Our own intelligence is defined by our abilities to predict and generalize. As Jeff Hawkins points out, we live our lives constantly predicting what will happen to us in the next few seconds. (See Numenta.com - Hawkin's company - for the source code to NTA hierarchical temporal memory system.)

We also generalize by learning to recognize patterns and ignore noise.

AI systems need to implement prediction and generalization, and do this in a way that scales so that we can move past small toy problems. Scalability is most important in prediction because of the size of data required to model the environment that an AI will live in and the real time requirements (prediction does us little good if the calculation takes too long).

Knowledge Engineering is not AI, it is the engineering discipline for the understanding and re-implementation in software of human level expertise in narrow problem domains.

Java FastTag part of speech tagger is now released under the LGPL

2007-03-03T11:14:00.000-08:00

FastTag is now released under the LGPL. I expect to release version 3 of my KBtextmaster NLP toolkit under the same licensing in the next few weeks. A separate version of FastTag uses the MEDPOST medical term lexicon.

I also released taggers written in C++, C#, and Ruby using the LGPL.

Semantic Web: through the back door with HTML and CSS

2007-02-09T14:59:00.000-08:00

I have spent a lot of time over the last 10 years working on technology for harvesting semantic information from a variety of existing sources. I was an early enthusiast for Semantic web standards like RDF and later OWL. The problem is that too few web site creators invest the effort to add Semantic Web meta data to their sites.

I believe that as web site creators start using CSS and drop HTML formatting tags like <font ...>, etc. (HTML should be used for structure, not formatting!), writing software that is able to understand the structure of web pages will get simpler. Furthermore, the use of meaningful id and class property values in <div> tags will act as a crude but probably effective source of meta data; for example: a <div> tag with an id or class property value that contains the string "menu" is likely to be navigational information and can be ignored or be of value depending on the requirements of your knowledge gathering application.

Just as extracting semantic information from natural language text is very difficult, analyzing the structure and HTML/CSS markup to augment web data scraping information software is difficult. That said, HTML + CSS is likely to be much simpler to process in software than plain HTML with formatting tags. BTW, I am in the process of converting all of my own sites to using only CSS for formatting - I have been writing HTML with embedded formatting since my first web site in 1993 - time for an update in methodology.

New version of my NLP toolkit

2006-10-25T20:08:00.000-07:00

I have done a fair amount of work in the last year on my KBtextmaster product (although not in the last 5 months due to a consulting contract). I hope to release version 4 next spring. For previous versions I did my R&D in Common Lisp, then converted to Java (bigger market!). While I may eventually do a Java port, I decided that I would rather stick with Common Lisp and go for maximum features and performance for the next release.

I did just formed a VAR relationship with Franz to use their Allegro Common Lisp for development and deployment. Allegro has support for compiling to a library that is accessible from Java applications, so that may be be OK for Java customers. The high runtime performance of Allegro is amazing.

Software environments for working on AI projects

2006-10-04T14:36:00.000-07:00

In the new global economy of driving production and service costs towards zero, it makes a lot of sense for computer scientists to learn specialized skills to differentiate themselves in the marketplace. Since you are reading this blog I assume that you are interested in learning more about AI so I thought that I would list the AI development environments that I have found to be particularly useful - and a lot of them are free.

Classic AI Languages
Although not strictly required for work in AI, a few AI oriented languages have proven especially useful in the past: Lisp, Scheme, and Prolog. Scheme is a great language but suffers from an "embarrassment of riches": there are almost too many fine implementations available to choose from. That said, I would recommend the excellent and free DrScheme and MzScheme as a very good place to start because it is supported by a repository of useful libraries that are very easy to install. If you want to mix logic programming with Scheme then the following book (with examples that work with DrScheme) is recommended: The Reasoned Schemer

If you want to use Common Lisp (which is what I use for most of my AI development consulting) there are two commercial products that are very good and have free (non-commercial only!) versions: Franz Lisp and LispWorks. There is no need however to stick just with commercial offerings: SBCL (MIT license) and CLisp (GPL license) are two good choices among many.

If you want to use Prolog, the open source (LGPL) SWI-prolog and the commercial Amzi Prolog are both excellent choices and have lots of third party libraries.

Scripting Languages
I have found two scripting scripting languages to be particularly useful for AI projects: Ruby and Python. Python has more third party libraries and projects for AI but I personally enjoy developing in Ruby.

Pick an environment and stick with it
Believe it or not, I tend to follow this advice myself: I tend to use one language for a year or so, and then switch (usually because of customer preference or the availability of a great library written in one specific language). It pays to take the time to master one language and environment, then use that environment a lot.

So my advice is to spend just a few hours each with a few of my suggestions in order to pick one to learn really well. Once you pick a language stick with it until you master it.

Using Amazon's cloud service for computationally expensive calculations

2006-08-24T16:43:00.000-07:00

I did not get too excited about Amazon's S3 online storage web services, but the Amazon cloud web services look very compelling: $0.10 per instance-hour consumed (or part of an hour consumed) where an instance is roughly equivalent to 1.7Ghz Xeon CPU, 1.75GB of RAM, 160GB of local disk, and 250Mb/s of network bandwidth.

I sometimes do large machine learning runs, and the two computers on my local LAN that I usually use (dual CPU Mac tower with 1.5 gigs RAM, Dell dual Pentium with 1 gig of RAM) are fairly fast, but there is often that pesky overnight wait to get a run in.

I need to spend more time checking out Amazon's code samples and documentation, but the idea of spending perhaps $10 and getting a long running machine learning run done quickly is very compelling.

PS. I just tried to sign up for the service, but their beta is full :-(

PSS. I just got an account :-)

Classic reasoning systems like Loom and PowerLoom vs. more modern systems based on probalistic networks

2006-08-17T16:57:00.000-07:00

My current project (for a large health care company) involves three AI components (a semantic network, and reasoning system based on PowerLoom, and a module using probalistic networks).

We are in a hedge our bets mode, basically using three promising approaches - I can not talk too much about the application domain, but it will be useful to see which approaches end up being most useful.

BTW, I have written about this on my regular web blog, but the release of a new version of PowerLoom under liberal open source licensing (that is, commercial use OK) is a great addition to anyone's 'AI toolkit'.