Wednesday, August 29, 2007
Good book: "Programming Collective Intelligence"
This book
is a great introduction to the techniques that I use almost daily in my own personal research and work for customers, and I can recommend it without reservation. The choice of Python for the examples is not optimum for me, but OK, especially because the techniques in the book for machine learning, categorization, clustering, filtering, optimization, support vector machines, etc. are mostly short and can be used as is or converted to whatever programming language that you need to use. The data used to present the book material is mostly from collaborative web sites. The book relies heavily on existing Python libraries and I like this approach since it mirrors rational software development practice: build custom code on top of existing libraries and software tools. Good book!
is a great introduction to the techniques that I use almost daily in my own personal research and work for customers, and I can recommend it without reservation. The choice of Python for the examples is not optimum for me, but OK, especially because the techniques in the book for machine learning, categorization, clustering, filtering, optimization, support vector machines, etc. are mostly short and can be used as is or converted to whatever programming language that you need to use. The data used to present the book material is mostly from collaborative web sites. The book relies heavily on existing Python libraries and I like this approach since it mirrors rational software development practice: build custom code on top of existing libraries and software tools. Good book!Labels: AI, clustering, machine learning, Python, support vector machines
Tuesday, April 17, 2007
The Semantic Web, Parrots, and AI
Two different subjects today: I just added a blog entry on the semantic web on my AI blog and our pet parrot. One (possible) route to understanding how to do AI is to appreciate problem solving abilities in the natural world. Our young Meyers parrot is a good problem solver but it takes him a while. Earlier this morning, I was reading in bed and had fetched our parrot so he could run around like crazy on and under our bedspread - good for burning off energy. Our parrot wanted to get at some of my stuff on my night stand, but his way was blocked, except for a space between two water bottles which, try as he might he could not squeeze through and he could not move the water bottles. He spent about 2 minutes walking back and forth thinking about the sad situation he was confronted with when he suddenly lowered one wing, raised the other, moving his shoulders close together and then simply walked right through the "water bottle gap" :-)
Our small parrot must have some abstract world model of objects and his own body. Why and how he thought of raising one shoulder while lowering the other to compress the width of his shoulders is a mystery to me, but I believe that this was possibly an example of abstract thinking.
Our small parrot must have some abstract world model of objects and his own body. Why and how he thought of raising one shoulder while lowering the other to compress the width of his shoulders is a mystery to me, but I believe that this was possibly an example of abstract thinking.
Labels: AI, semantic web
Friday, March 16, 2007
metaweb.com and freebase.com
I am always on the lookout for freely available sources of data in useful formats. Metaweb was founded by Danny Hillis and their first public system is at www.freebase.com.
"Freebase is a vast, free, open online database of structured knowledge" - from their web site.
One interesting thing, besides the interesting technology for storing and querying structured data where both the user can define her own categories and use system wide categories, is that the content that hosted is freely licensed under Creative Commons, GNU documentation license, or in the public domain.
You need to request an invitation, and then the documentation provides information on accessing Freebase. I experimented during lunch time with their Python client APIs - cool stuff.
"Freebase is a vast, free, open online database of structured knowledge" - from their web site.
One interesting thing, besides the interesting technology for storing and querying structured data where both the user can define her own categories and use system wide categories, is that the content that hosted is freely licensed under Creative Commons, GNU documentation license, or in the public domain.
You need to request an invitation, and then the documentation provides information on accessing Freebase. I experimented during lunch time with their Python client APIs - cool stuff.
Labels: AI, data mining, Python
Wednesday, March 14, 2007
I have released some NLP (natural language processing) tools with a LGPL license
Here is the download link. These tools come in a few 'flavors': Java, Ruby, C++, and C#. I expect to add two larger NLP projects in the next month.
BTW, I consider the LGPL to be "business friendly". You are allowed to mix my LGPL software with your own commercial products without open sourcing your products. You may also mix my LGPL software with open source with projects with Apache, BSD, MIT, or Mozilla style licenses. If you have any questions, ask me. If the LGPL license prevents you from using this material, please let me know about it.
BTW, I consider the LGPL to be "business friendly". You are allowed to mix my LGPL software with your own commercial products without open sourcing your products. You may also mix my LGPL software with open source with projects with Apache, BSD, MIT, or Mozilla style licenses. If you have any questions, ask me. If the LGPL license prevents you from using this material, please let me know about it.
Labels: AI, C#, C++, Java, Ruby
Saturday, March 03, 2007
Source code for FastTag released - free for non-commercial use
I have started to release my commercial products with a free for non-commercial license, with source code. I repackaged the FastTag part of speech tagger this morning and it is now available. A separate version of FastTag uses the MEDPOST medical term lexicon.
Labels: AI
Tuesday, October 10, 2006
Human minds, programming, and the "caching problem"
When writing large software systems, rapid access to data is often important: what can be kept in memory or more slowly: processes on the same local network and on disk. In software development, we see the same effect: maximum speed and efficiently if a single person can understand the architecture and comprehend the entire system. Moving from a single developer to a very small team adds a little overhead: design notes and pencil and paper drawings turn into casual but more explicit short documents and conversation. The optimisation is minimising cost between two people talking and sharing information vs. maintaining documentation and reading time. Talking is almost always better because communication is a two way street, but if you have N developers, O(N^2) "talking overhead" is too expensive with a large N, so back to the one way street of documentation.
I like to view programming languages in terms of how they allow me to deal with complexity, keeping as much stuff in my head at once:
I like to view programming languages in terms of how they allow me to deal with complexity, keeping as much stuff in my head at once:
- Lisp: great for building up the language from the bottom and extending towards an application domain. The new application "programming language" is higher level and the remaining part of a system is more concise code and easier to keep track of.
- Ruby: concise, so programs are much shorter and easier to understand.
- Java: the language does little for me as far as reducing complexity, but great IDEs like IntelliJ at least allow rapid code browsing, "who calls this" queries, etc.
Friday, August 18, 2006
I started a new blog just on AI theory
Artificial Intelligence Theory will probably be a very low volume blog. I am planning on using it more in an essay or white-paper writing mode. One thing that I will probably write about, in addition to more practical topics like probabilistic networks and reasoning systems is a long time interest that started in 1976 when I bought Bertram Raphael's great book The Thinking Computer: Mind Inside Matter: Computer Go Programs. I spent a lot of free time in the late 1970s writing what I am quite sure was the world's first commercial Go playing program Honnibo Warrior. I am still very interested in trying to develop some cross between NGRAM style hashes for local board positions and efficient storage mechanisms like AllegroCache to solve some tasks that if not strictly required by a Go program, would at least be more like the way human experts play Go: in other words, figure out how to implement the temporal and spatial memory in the human neocortex, but in software, and efficiently.
Labels: AI
Monday, August 14, 2006
Indie Game Development, AI in games
Slashdot has a discussion on Microsoft's "free" PC and XBOX 360 game development kit. There are also other good low cost alternatives for Indie development like Torque.
I spent a few years doing AI game development at Angel Studios (2 Nintendo games, prototype networked PC hovercraft game, and a VR system for Disney) and although I have been working more on 'practical' AI applications since moving to Sedona 7 years ago, I still have a keen interest in gaming and AI for games. A few years ago I thought of setting up a cooperative game development community for fun and maybe some profit, but my consulting business keeps me too busy, at least for now. Another thing that keeps me from making a large investment in an independent game making co-op is thinking how much money was spent writing commercial games at Angel Studios: teams with dozens of professional artists, programmers, a few musicians, etc. are expensive. That said, game AI programming is great fun and surprisingly difficult.
I spent a few years doing AI game development at Angel Studios (2 Nintendo games, prototype networked PC hovercraft game, and a VR system for Disney) and although I have been working more on 'practical' AI applications since moving to Sedona 7 years ago, I still have a keen interest in gaming and AI for games. A few years ago I thought of setting up a cooperative game development community for fun and maybe some profit, but my consulting business keeps me too busy, at least for now. Another thing that keeps me from making a large investment in an independent game making co-op is thinking how much money was spent writing commercial games at Angel Studios: teams with dozens of professional artists, programmers, a few musicians, etc. are expensive. That said, game AI programming is great fun and surprisingly difficult.
Thursday, August 10, 2006
OpenCyc 1.0, AI in general
I noticed on Slashdot that OpenCyc 1.0 has been released. I spent a short while reading comments and realized how different my own views on AI are from many Slashdot commentators.
To me, AI is all about writing software that makes decisions given uncertain and sometimes contradictory information. AI is about modeling problem domains and working both within that model and changing the model as new information becomes available. AI is about using problem domain models to provide human users with useful, interesting, and unexpected results by matching a model of a user's inquiry. AI is about solving the game of Go: the branching complexity of the game is so great that having perfect information is not enough.
So, a tool like OpenCyc is not really a match to my personal view of what AI development is: Cyc and OpenCyc try to define ontolological knowledge of real world common sense knowledge. I appreciate decades of hard work, and I have myself spent many hours experimenting with earlier versions of OpenCyc - so kudos for the 1.0 delivery.
Still, I tend to view "AI problems" as being problems restricted to narrow domains but still made very difficult or impossible by uncertainty, missing information, and time or memory constraints on algorithms.
To me, AI is all about writing software that makes decisions given uncertain and sometimes contradictory information. AI is about modeling problem domains and working both within that model and changing the model as new information becomes available. AI is about using problem domain models to provide human users with useful, interesting, and unexpected results by matching a model of a user's inquiry. AI is about solving the game of Go: the branching complexity of the game is so great that having perfect information is not enough.
So, a tool like OpenCyc is not really a match to my personal view of what AI development is: Cyc and OpenCyc try to define ontolological knowledge of real world common sense knowledge. I appreciate decades of hard work, and I have myself spent many hours experimenting with earlier versions of OpenCyc - so kudos for the 1.0 delivery.
Still, I tend to view "AI problems" as being problems restricted to narrow domains but still made very difficult or impossible by uncertainty, missing information, and time or memory constraints on algorithms.
Labels: AI, knowledge management
Saturday, August 05, 2006
Yes languages affect our thoughts, even in programming
The Sapir-Whorf hypothesis poses that our native language affects how we think. Computer programming languages also strongly affect how we think about, design, implement, and maintain code.
I was working on some tricky code this morning that builds on some Common Lisp CLOS class libraries. The new code is really orthogonal to the existing functionality and it seemed like a poor idea to merge the new in with the old, especially since the old codebase will probably be used as-is for a while. I decided to start a new module (as defined by physical file organization) that added the new functionality to the existing classes as generic methods. The new module stays small, and anyone needing to use the original codebase is not confused with extra code for functionality that they do not need.
In Ruby, I like to do the same sort of thing: have different modules (as defined by physical file organization) where new orthogonal functionality is added by defining new methods to existing classes in new file modules.
In Java (and other languages), you can always use Aspect-oriented programming (AOP) to add new orthogonal behavior to class libraries, but, to be honest, I dislike AOP - this is not the fault of AOP per se, but because I have only used AOP with Java.
I was working on some tricky code this morning that builds on some Common Lisp CLOS class libraries. The new code is really orthogonal to the existing functionality and it seemed like a poor idea to merge the new in with the old, especially since the old codebase will probably be used as-is for a while. I decided to start a new module (as defined by physical file organization) that added the new functionality to the existing classes as generic methods. The new module stays small, and anyone needing to use the original codebase is not confused with extra code for functionality that they do not need.
In Ruby, I like to do the same sort of thing: have different modules (as defined by physical file organization) where new orthogonal functionality is added by defining new methods to existing classes in new file modules.
In Java (and other languages), you can always use Aspect-oriented programming (AOP) to add new orthogonal behavior to class libraries, but, to be honest, I dislike AOP - this is not the fault of AOP per se, but because I have only used AOP with Java.
Thursday, May 25, 2006
New PowerLoom site
Thanks to reader Vinodh Das for pointing this out to me: the PowerLoom web site has been updated and as one of the developers told me, PowerLoom is now released under an open source license. PowerLoom is a great system - if you are interested in AI, logic, reasoning systems, etc., then check it out.
Labels: AI, knowledge management
Tuesday, May 09, 2006
Integrating a semantic network with a reasoning system
For a long term AI project, I am using Common Lisp and CLOS to model customer application specific nodes in a semantic network. This morning I worked out the non-obvious (to me!) bits for integrating my stuff with the Loom reasoning system by deriving Loom concepts from my CLOS classes. Cool stuff!
Labels: AI, knowledge management, Lisp
Friday, February 24, 2006
Wonderful mix of functional and logic programming
I received the book The Reasoned Schemer last week. The authors use the same socratic teaching style (ask questions of the reader) that they used in the Little Schemer to introduce the implementation of logic programming using a functional programming style. I can just feel my brain twisting a little from new ways of thinking about old problems.
I mostly use object oriented programming (Java, Ruby, Smalltalk, and Common Lisp's CLOS) so it is healthy to switch to a functional programming style if only for research and learning projects. Anyway, this is an awesome book.
last week. The authors use the same socratic teaching style (ask questions of the reader) that they used in the Little Schemer to introduce the implementation of logic programming using a functional programming style. I can just feel my brain twisting a little from new ways of thinking about old problems.I mostly use object oriented programming (Java, Ruby, Smalltalk, and Common Lisp's CLOS) so it is healthy to switch to a functional programming style if only for research and learning projects. Anyway, this is an awesome book.
Subscribe to Posts [Atom]
