A lot of discussion has been kicking-up lately about BI and SOA as combined forces in the IT enterprise. Much of the discussion has been around similar, or dissimilar skill sets and strategies of these architectures. Not to mention, is BI or is SOA a solution, architecture or strategy? To be honest, unless there is a driving and obvious business need to utilize SOA mechanisms in a BI solution, we cannot put forth a compelling argument to do so. Without that business case, we are just implementing architectures for technology's sake, not for the business' sake. And in the end, it will be the business that drives the decision.
So let's take a look at how data integration for your BI system can benefit from web services. Keeping in mind that this particular blog space is focused on industry specific vertical solutions in business intelligence, I will pick a particular industry solution example. Let's choose, for the sake of this article, a manufacturer of consumer goods meant for mass consumption, who is looking to implement customer segmentation so as to improve their product marketing and streamline their go-to-market process.
In this case, data sources may be from such varied heterogeneous sources as transactional databases, Excel files, mainframes or LOB systems (ERP, CRM). If you were to perform straight ETL by pulling defined fields based on database data types, you will fight the common ETL problems of tight coupling, high sensitivity to source and target changes, deep understanding of source schemas, etc. But a promise of SOA is to provide "contract" based integration and programming where you should be able to access line of business and legacy systems from a "semantic integration" perspective. In the case of a customer segmentation solution for manufacturing, you would, for example, integrate customer data from CRM to survey results from an external database along with financial figures for the business from Excel, for example.
Now, reaching a "nirvana" level of integration is a lofty goal, one that will greatly benefit those of us who hold responsibilities with business users to provide meaningful, actionable, aggregated knowledge. With an understanding that this should be a goal of the enterprise will help us to continue to work toward adding web services layers to legacy data access and a unified approach to SOA & BI in corporate IT.
Here are a few of my personal favorite links to continue reading about this topic:Microsoft BizTalk Adapter Pack is a set of adapters that we use with our customers for BI data integration through SOAFor a good look at ETL and SOA in the world of BI try this article by Arnon Rotem-Gal-Oz