Deepak Singh
Moving on from what’s comfortable -
There is a flip side to this with things like Power Pivot that Excel may become both powerful enough to support different data forms natively as well as hiding that complexity behind a familiar interface - and that stuff from the Freebase guys is really impressive in that way as well. - Cameron Neylon
Definitely, and there are some projects that put an excel-like interface in front of Hadoop, but if you really want to do multi-dimensional analysis you need more robust approaches. - Deepak Singh
From an non-bioinformaticians perspective, there's another flipside. Excel is a low-threshold app for researchers who need to do programming-like tasks and do not know programming. Also, my experience is that our IT-professionals generally like it when we present them with an excel front-end that they can use for further development of data-handling. Taking away excel seems to me to be a great loss. Building on excel appears more productive. - Nils Reinton
Preserving the spreadsheet interface while extending the underlying capabilities would be a step in the right direction. For example BeanSheet includes a full Java interpreter, allowing far more robust programming than Excel's VBscript support. - Mike Chelen
Mike is heading down the right track. You can keep spreadsheet like interfaces. That's a presentation issue. I am talking about what you're doing. If you're using excel to do multi-table joins for a 2 TB database then you have problems - Deepak Singh
One specific situation that I am talking about here, 90% of time was spent just managing data in spreadsheets as opposed to having a robust data management system. Try storing technical replicates, biological replicates, treatment conditions, time points, associated metadata, etc all in a set of spreadsheets and try getting some biology out of that, and now do that for multiple terabytes of data. Non-starter - Deepak Singh
Good timing of the post. I think I convinced one of my colleagues today to move a project (that started a few months ago) from Excel to a relational database. Most users don't see the limitations of Excel. My arguments today: doesn't work for multiple users, difficult to connect multiple tables. - Martin Fenner
Deepak: Is Excel fundamentally less efficient at performing queries than any other database? If the desktop systems are underpowered, perhaps a client-server model would let dedicated boxes do the heavy lifting, used for example in Google spreadsheet. - Mike Chelen
Mike ... yes. It's not designed to be a query system that understands relationships - Deepak Singh from IM