On Business Intelligence and F#
It is high time that Business Intelligence get the benefits of the language "Cambrian Explosion" and agile revolution. Think about BI for a second. Most of the talk around BI is oriented around tools - a stack that ties together presentation, storage, and logic, all in the name of avoiding dealing with pesky programmers. To the point that "requires no writing code" becomes a feature point. How did we get here? And how do we get out?
In the olden days, if you wanted a report, leaving aside BI for the moment... you had to ask your IT department for the report. You were on their schedule, and more often than not, the backlog was very long. Leaving aside why this was the case (i.e.budget shortage, lack of IT/business alignment, etc.) - it was. This begat two primary developments, the shadow IT department, and the market for tools that empowered the business to, at least try, to generate their own reports independent of IT.
Now, in the intervening years, these two developments have not really stopped at all. There is still a ton of shadow IT, and a ton of tools that purport to help you generate business intelligence by using an integrated stack of tools that, in theory, allow BI to happen without programmers. The question is... is this a good thing?
I would say no. BI tools, more often than not, tie you to not only a platform, but frequently, a specific product. You can't take your BI developed in Microstrategy and run it in Cognos, at least not very easily. And it makes sense why - as each of these tools competes on the basis of capabilities, and there is therefore no motivation to port the capabilities of one BI product over to another. And because there is no obvious short term economic justification from the tool vendors point of view, it simply doesn't happen.
Of course, the medium to long term economic justification for tool vendors for this is very good indeed. By creating an ecosystem of BI that allows for greater innovation and better solutions, BI will receive much greater investment. The savvy players who take advantage of this will do really, really well, just like Microsoft prospered by having an open PC platform and Google prospered by having an open internet platform. What has to happen, however, is someone has to move first, and given the nature of the space - big corporate buyers - it has to be one of the big players to do it with any kind of credibility.
That said, it does not help that there has been little standards innovation in the world of SQL. Not to say that it doesn't happen, but lets put it this way - nobody is proposing SQL as a new .net language like they do for F#, Ruby, or even Boo. SQL is just now standardizing how objects work... and worst yet, the language continues to get balkanized - especially in BI land where extensions for doing cubes and other specialized functionality tend to differ from vendor to vendor.
So how do we untie this gordian knot and get to a place where BI is portable, testable, and exists in a manner that allows diversity in authoring tools, persistence mechanism, and presentation mechanism? I humbly submit that F# should be the language of BI.
Why F#? Well, functional programming in general is oriented towards the folding, summarization, reduction, and calculation of sets of information - that is - data. SQL is mostly a functional, declarative language anyway, so moving to F# as the lingua-franca of data should be a no-brainer. Imagine a world where BI is:
* Persistence Ignorant rather than Persistence Obsessed
* Portable from tool to tool - so long as it can parse F#
* BI authoring tools allow business users to use a GUI to write F# constructs rather than balkanized SQL constructs
* The benefits of a modern functional language (ASTs, automatic generalization, massive parallellization, etc.) are finally tools that are easily available to BI
* Allowed to have the benefits that the agile world has brought us (testability, etc.)
Imagine a world where you write a Domain Specific Language (DSL) in F#, and the BI tools manipulate the DSL. Imagine being being able to swap out different persistence mechanisms based on strict performance characteristics, rather than having to pay the port tax when you move from one persistence mechanism to another. Few people in the BI world have been exposed to the recent "Cambrian explosion" of new languages that have emerged in the last few years, and that's a shame, because some cross-pollenization would be very compelling for new kinds of solutions to emerge.
A recent Gartner CIO poll reported that CIOs must ‘Make the Difference’ by replacing generic IT with distinctive solutions that drive enterprise strategy. This means that true BI that differentiates will likely be invested in. It would be a shame if we continued to have all this BI live on vendor specific islands that were unable to leverage some of the state of the art work going on in computer science. On the other hand, BI that leverages these new capabilities that the computer scientists like Don Syme are giving us will have a great chance to "make the difference".
I conclude this with a call to action. If you are doing BI, ask why we are using the same basic language we were using 10 years ago. If you are a language geek or a software developer, ask why what you are doing, particularly if it generates information that is used in the strategic decision making process, isn't considered "BI". Whomever is the first tool vendor to get to this vision will probably get to have a great deal of control over how it gets done - and the field is very green at the moment for someone to fill this gap :)