Archive for the ‘ruby’ Category

oai/sru and ruby

Thursday, April 20th, 2006
biblio:~/Projects/ruby-oai ed$ ruby test.rb
Loaded suite test
Started
..........
Finished in 171.247595 seconds.

10 tests, 280 assertions, 0 failures, 0 errors

So after about 4 hours of hacking I’ve got ruby-oai which is a OAI-PMH client library for ruby. Included is a test suite that puts all 6 oai-pmh verbs through their paces using OAI-PMH servers at the loc.gov, lanl.gov, pubmedcentral.gov.

Just a few days before I did something with sruby which is a SRU client library for Ruby. Now these are just the initial versions, and I’m sure there are ways that they can be improved. But after doing a few years of solid Java coding it’s just another reminder of how dynamic languages such as Ruby and Python can really help catapault productivity.

I have to admit, I do miss javac sitting on my shoulder reminding me of things I’m doing wrong at compile time compared with discovering bugs/errors at run time in python and ruby. But I’ve really come around to Bruce Eckel’s point about Strong Typing vs Strong Testing. Building sruby and ruby-oai using test driven development really makes me more confident that my code is working properly…and what’s more it makes me much happier with the look and feel of the API. This look n’ feel aspect is something that the mechanical javac can’t really help with. Doing test driven development in Java with Eclipse approaches this level–but somehow isn’t as fun–but I imagine it scales better to larger teams. However, I don’t work on any of these uber large teams anyhow. Hopefully I’ll find a moment to talk about the new work I’m doing in the coming weeks.

xbib in subversion

Wednesday, February 22nd, 2006

Bruce D’Arcus has started putting his XBib project into subversion at sourceforge. The code is going to include ruby, python and styling libraries. I didn’t realize that sourceforge was offering up svn now…which is a welcome change. I want to set aside some time to get familiar with Bruce’s model and code.

testing, testing, is this thing on?

Thursday, January 26th, 2006

Marcel Molina on testing in Rails

Begins: Mon, 06 Feb 2006 at 6:30 PM

Ends: Mon, 06 Feb 2006 at 10:00 PM

Location:

Thoughtworks

651 W. Washington Blvd

Chicago, IL 60661

USA

Link: meetup page

I’m looking forward to attending this talk by Marcel Molina of 37Signals on testing in Rails. One of the things that has impressed me the most about Rails so far is how test stubs are automatically written out for you when you generate classes. I also really like how fixtures can load test data for each test…and the custom assertions rock. Anyhow, hopefully I will find the time to attend this the week before I head out to Oregon.

good fences and the frankenweb

Tuesday, January 3rd, 2006

Ian Bicking has some interesting notes about competing web development technologies–mainly in response to some posts from Ivan Krystic. The discussion is definitely recommended, especially if you find yourself looking at web application frameworks for Python and Ruby. I found the pivot point of the discussion to be around a new term (for me) — the “frankenweb”.

My understanding is that like Frankenstein (a being created by stitching together random body parts from dead humans) the frankenweb is an unholy mixture of MVC components pulled from different projects, when put together result in an ugly partially functional whole. I think this characterization of Ian’s work is really unpleasant, but strangely compelling. I think that this is mainly because of Ian’s response:

The “Frankenweb” is a feature, and it describes the web we have, the software we have, and the future that is inevitable. The world was never all J2EE, or ASP(.NET), or PHP, and it won’t be all Rails either.

I think Ian is right on about this: “frankenweb” does describe the web we have, and hopefully the web we will continue to have–and the degree to which we can all interoperate is the degree to which the web will succeed. Perhaps I’m seeing the frankenweb through Weinberger-Colored-Glasses, having just finished Small Pieces Loosely Joined (which I thoroughly enjoyed and plan to write about later if there is time). Weinberger does an excellent job of distilling the essence of the web, and how its architecture enabled it to pull itself up from it’s own bootstraps, grow and adapt:

In the real world, I can’t just put in a door from my apartment to my neighbor’s so that anyone can go through. But that’s exactly how the web was built. Tim Berners-Lee orginally created the web so that scientists could link to the work of other scientists without having to ask their permission. If I put a page into the public Web, you can link to it without having to ask to do anything special, without asking me if it’s alright with me, and without even letting me know that you’ve done it…The web couldn’t have been built if everyone had to ask permission first.

Of course I’m conflating links between pages, and API links between software components…but what Ian says about embracing the frankenweb seems to resonate with this somehow.

It’s also quite disorienting to hear Ivan and others lauding tight coupling:

You don’t see the Ruby on Rails guys modularizing Rails to the point of pain. You see them delivering a single, high-polish, tightly coupled product that does its job well.

Given the various pluggable modules that make up Rails I think “tightly coupled” is largely an overstatement. Granted they are available in the same code base, and I haven’t tried to use one of them in isolation–but I imagine it could be done if someone wanted to say, use a activerecord model in a script or something. The Pragmatic Programmer has a really nice chapter on decoupling, and the authors are actually heavily involved in the Ruby/Rails community. The chapter starts out with a nice quote from Robert Frost’s poem The Mending Wall:

Good fences make good neighbors.

It seems to me that Ian is doing the hard work of patching some of these fences, and building a few and deserves a lot of credit for the effort and cat herding.

selenium/ruby sprint

Wednesday, December 7th, 2005

In case you missed it, or aren’t subscribed the chicago-ruby group is getting together for a Selenium demo/sprint session on Dec 13th. I’ve seen Selenium demo’d before at a chipy meeting, and look forward to a more in depth look since several of my friends really like this testing tool. I think Jason is particularly interested in getting some Ruby driver support.

snakes and rubies

Sunday, December 4th, 2005


I managed to attend Snakes and Rubies yesterday where Adrian Holovaty and David Heinemeier Hansson talked about their respective web frameworks: Django and Rails.

The event started at 2PM and went to 6:30PM or so, and was attended by over 100 people! I watched this little event take shape out of the mists of the local python and ruby mailing lists and was just amazed to see how vibrant the Chicago software development scene is, or has become in the 3 years I’ve lived in the area.

What’s more Adrian and David did a great job promoting both of their projects, while remaining amiable and respectful of the other camp. It’s hard to imagine a similar event between two commercial frameworks. Both were given about 45 minutes or so to talk about their software in any way they wanted. They both had extremely different yet effective presentation styles, and their projects had one important thing in common: disillusionment with PHP.

Rather than talking technical details Adrian spent most of his time focused on how Django came to be down in Kansas at lawrence.com. lawrence.com began it’s life as a PHP application which served as a community site for all sorts of goings on in Lawrence, Kansas. The site interleaved all sorts of local entertainment content: music, dining, art, movies…and it encouraged user participation. For example you can listen to mp3s from local musicians, but here’s the twist, you listen to them as they are playing in town…so if you like a song you can jump over to the venue later that week to see them live. Another example was a full on sports site for the local little leagues which posted details of games, scores, weather conditions, etc. All of this was detailed to show how deeply intertwined all the data was.

The really interesting stuff for me was when Adrian described how journalism informed his software development practices…and how Django fed into this process. In the same way that journalists work against the clock to get news stories out quickly and accurately Adrian and his team worked to get software projects done often on similar deadlines (sometimes like 4 hours). They quickly found that their PHP infrastructure wasn’t allowing them to respond quickly enough without introducing side effects, and decided that they needed new tools…which is how Django was born. In fact the little league application mentioned above was the first Django application.

Adrian has since moved on to the Washington Post, where he is their resident web technology mad scientist. Apparently they are using Django in some form at the Post, or are planning to since he mentioned Django’s caching can scale to the 9 million odd requests the Post gets in a single day.

Unfortunately my lead pencil ran out of lead just a bit of the way into David’s talk, so I don’t have as much written down from the Rails presentation. David dropped some wonderful one liners that I wish I could have written down. Much unlike Adrian, David let actual code do most of the talking for him.

Early on he had a screen with a quote from Richard Feynman on the importance of finding beautiful solutions to problems (if you remember the quote please let me know). This quote kind of guided the rest of the talk where David showed off beautiful Model, View and Controller code from RubyOnRails…and it really was beautiful stuff. David’s thesis was that beautiful things make you happy, and happiness makes you more productive…so beautiful code will make for happy, productive programmers. Much of this comes back to the essential philosophy of Ruby–to give joy to programmers. At any rate, the lights were dimmed and David gave us a tour of what RubyOnRails code looks like, while highlighting some of the strengths of the project and the Ruby language. On one of the pages there was some code to set a cookie expiration, and the date was created like so:

   20.years.from_now

How cool is that! I wasn’t sure if this was part of Ruby proper until I fired up my ruby interpreter to check:

biblio:~ ed$ irb
irb(main):001:0> 20.years.from_now
NoMethodError: undefined method `years' for 20:Fixnum
        from (irb):1

Whereas from the Rails console it works fine:

biblio:~/Projects/cheap ed$ script/console 
Loading development environment.
>> 20.years.from_now
=> Thu Dec 04 15:38:48 CST 2025

So Rails decorates the Fixnum class with the years method. Pretty awesome :-) Another thing David highlighted was that Ruby is used everywhere, from configuration, to writing XML, to writing JavaScript. I was even surprised to hear him argue for full on Ruby in view templates. His argument is that even when a framework offers only a limited set of tags, it’s still offering logic, and rather than creating some bastardized tag language why not just use tried and true Ruby.

The two presentations were followed by a few (not many) moderated questions, and some questions from the audience. The highlight for me was when Why the Lucky Stiff’s question was asked:

Looking a bit beyond web frameworks, how do you envision the world coming to an end?

David responded by “scoping” the world to mean the world of software development and said that this world would come to an end if the layers of Java “sedimentation” continue to accrue. He went on to predict that we’re at a crossroads in software development, and that a paradigm shift is underway…intentionally provocative, and pretty much right on as far as web development goes if you ask me. Adrian responded “Yoko Ono”.

So, as you can tell I’m still digesting the presentations and discussion. There was so much good stuff, and I was really struck by the collegiality between the two guys: open source software development at its finest. The two main things I took away were embracing the boundaries between software development and a particular industry like Journalism, or in my case Libraries; and always trying to strive for the beautiful in software, “boiling down” a thorny problem into its most simple and elegant expression.

BBC Catalogue’s Search

Saturday, November 26th, 2005

I did end up hearing back from Matt Biddulph about the search technology that he’s using with RubyOnRails to build the BBC Programme Catalogue.

The core of the search is nothing more than mysql 4.1’s fulltext indexer. I used to think very poorly of it until I discovered how to turn off its automatic stoplist and minimum indexable word length, and started using its boolean mode. Having the database manage the indexing without having to keep a separate index in sync is very valuable, and of course it’s portable to any client language.

The nice thing with a dataset the size and quality of the BBC’s is that you’re not solely dependent on the quality of the freetext indexer. I’ve done a little statistical analysis on the data to help with scoring the results. For example, programme contributors can be ranked according to how many shows they’ve contributed to, and commonly co-occurring contributors can be easily calculated with a bit of overnight batch processing. This kind of stuff contributes to a pretty good set of search results.

Given the visibility of the BBC Catalogue and that it has nearly a million records this says good things to me about the scalability of MySQL’s fulltext search. I’ll definitely consider it along with Ferret for Rails experiments that need search functionality.

ruby-chicago

Tuesday, November 8th, 2005

Last night I took the train into ThoughtWorks to check out the Chicago Area Ruby Group meeting. There wasn’t a planned talk, so I wasn’t sure what to expect (apart from getting a chance to chat with Jason and Chris). One thing I definitely didn’t expect was seeing close to 30 people there.

The room we met in was kind of an atrium type of space. Everyone arranged themselves into a circle, and at the center of the room there was a smaller table with 6 chairs. After everyone went round an introduced themselves the function of the central table was revealed by Joe O’Brien who got everyone to play this discussion game called fishbowl. Basically there are 6 seats, and any 5 people can sit in them at any time (always leaving one chair open). People start talking about stuff, and if at any point someone wants to join the discussion then they sit down in the empty seat, and someone who no longer wants to talk can get up and leave. At all times the 5 seats needed to stay filled.

This fishbowl actually worked out really well. The conversation ranged from ruby’s performance, to comparisons with java, rails, the community, the “Ruby Way”, joy, and practical examples of Ruby in the workplace. All in all it was a very pleasant meeting, and it was really interesting to see a good cross section of Chicago technology in an informal environment. Afterwards Jason, Chris and I went and had a few drinks at a nearby bar with Sam Stephenson and Marcel Molina of 37Signals. Sam and Marcel are both core developers for the Rails project, and recently moved to Chicago to join 37Signals. Sam is the developer behind prototype which I’ve been meaning to learn more about. Hopefully Sam can be coerced into doing a prototype talk at some point.

It’s been interesting watching local perl, python, ruby and java groups and how they regard each other. Chris mentioned that it’s unfortunate when discussion borders on digging at the other guy, and that the real thing that unites these groups is that everyone enjoys programming, and works on stuff on their own time. If the focus could brought to be that level I think there could very well be occasional cross language meetings. I mentioned to John Long that perhaps a meeting about javascript could bring folks from other languages together. He had a great idea of having someone like Sam talk about javascript, and then break off into smaller groups that talked about integration with various languages. Anyhow it was well worth the train ride in. Thanks for putting up with me being away for an evening Kesa :-)

ruby-zoom

Friday, October 7th, 2005

If you ever need to do z39.50 from ruby and have successfully built and installed ruby-zoom only to see:

   biblio:~ ed$ irb   irb(main):001:0> require 'zoom'   dyld: NSLinkModule() error   dyld: Symbol not found: _ZOOM_connection_search     Referenced from: /usr/lib/ruby/site_ruby/1.8/powerpc-darwin8.0/zoom.bundle     Expected in: flat namespace 

or a similar error about missing symbols…never fear! The YAZ toolkit doesn’t build a shared library by default. It’s confusing because the ruby-zoom package builds fine with header files. When building YAZ you’ll need to:

biblio:/usr/src/yaz-2.1.8 ed$ ./configure --enable-shared

Submitted here to help similar users who are flailing wildly in Google.