Michael Barton

Come to supercomputing. My whole talk is on this subject ;). Trying to figure out if I should start writing about it before or after - Deepak Singh

I understand the points in the comments about using flat files for better speed. However compared to when I used to use miscellaneous scripts and data files to do my research I find that using a 'Ruby on Rails' type of database-backed approach is much better for me because of the shorter development time and how much easier the code is to maintain. - Michael Barton

@Deepak I did consider mentioning Hadoop/NoSQL (see last paragraph of earlier draft http://bit.ly/ztjS9) as it's obvious to discuss these types of approaches when dealing with very large datasets. However I think these tools do still require a fair amount of work for maintain and use compared with a more standard kind or MySQL approach. I say that because I tried using map/reduce across the university cluster and had quite a few teething problems. - Michael Barton

More generally, "any DB + any ORM" is A Good Thing. I can see why people stick with (My)SQL. It's tried and tested. I find a lot of the newer developments interesting, exciting, fun - but often, "too agile" for real work. Libraries change too fast, documentation (if any) goes out of date, code moves to new repositories, in the space of 3 weeks. - Neil Saunders

@Neil I originally tried using DataMapper instead of ActiveRecord but so many Rails centric libraries assume ORM == ActiveRecord. This meant using DataMapper precluded the use of the factory_girl and shoulda libraries which I have come to find very useful. I think Rails needs to be is truly ORM agnositic and that the current changes in Rails 3.0 doesn't go far enough to address this. - Michael Barton

I agree. I really like DataMapper (and other ORMs - sequel, mongomapper), but using them with Rails components = ugly, not fully-functional hacks, as things stand. Be interesting to see how the new ActiveSupport looks. I'm even considering abandoning Rails for now and just plugging together components myself as required (e.g. ramaze/sinatra if web frontend required). - Neil Saunders