|
|
-
Regularly, in this blog space, you read often about "vertical" business intelligence solutions, or "industry" BI solutions. Usually, I will frame this around our Microsoft Services and partner offering called Microsoft Enterprise Cube or MEC. That is probably appropriate terminology in the Microsoft space of platform software technology. In other words, at Microsoft, in the product groups, we make flexible tools and platforms for developing solutions such as business solutions based on business intelligence. For Microsoft, that means SQL Server, PerformancePoint Server, Excel and Sharepoint for the most part. And the development environment for the glue is .NET all residing on the Windows Server OS.
But in solutions group such as within Microsoft Services, we help our customers implement these products as high-impact business solutions that will solve immediate problems and provide ROI in the shortest payback period we can produce. In the world of business intelligence, there is really is no need to distinguish between an industry specific vertical BI solution or any other type of BI solution. In order produce a business solution, you must use industry knowledge, data and content based on the vertical area you are addressing. There is just no other way to do it. The distinction then is really boiled down to product vs. offering. So, PPS, SQL & MOSS are our products. Packaged solutions such as MEC are our offerings to solve your business problems.
Back back to the area of indsutry domain. If you look at a holistic approach to solving a business problem with BI, you need to start at the source. Or the data sources, to be more precise. If you are a healthcare provider or a manufacturer or a telecom company, your data sources will have some similarities in CRM, inventory, billing, etc. But there will be many differences in business processes, data models and operational data that will mean that scorecards with KPIs in one industry do not directly translate into another industry's business domain.
Once you work your way up the stack from data sources, you will again find similarities in areas such as collaboration and user access. But in manufacturing, for example, there may additional needs to remote access from portable devices to see what-if scenario output, forecasts and dashboards whereas an insurance company may require 90% PC access to their BI portals.
These differences should be discovered and recorded through requirements analysis using techniques such as E-R, ORM and UML diagrams that depict the processes, entities and relationships that will form the basis of our business intelligence solution. Performing these up-front tasks as essential to build the domain model that will produce high-impact business value with your Microsoft BI tools.
|
-
Clearly, there are a lot of reasons why business intelligence makes sense in the context of Customer Relationship Management (CRM). After all, both of these applications typically provide deep insights into customers. BI can give you the analysis and behavior patters or answers to the questions "why" and "how", while your CRM will tell you "who" and "what" among many other things.
For example, your CRM will tell you who your customers and what they have bought. There are a number of complex CRM systems that provide bundle software services to allow you to add in to that mix planning, scorecards, what-if scenarios and other BI domain functions. Things are much more complicated then that, but this is a blog and I don't want to bore you to death! So let's stick with simplicities here and focus on my little topic of the day, which is to look at the BI part of CRM and how these are independent.
At this point, once you are looking at a complex and independent BI infrastructure to provide, let's say, a BI portal (we'll assume Sharepoint) for a metals manufacturer. We'll want to have all points of the supply chain benefiting from both of these so that the financial planners, operations managers, sales team, marketers and strategic decision makers are looking at the same data points and making streamlined excellent decisions.
Great, now we have to worry about single source of truth and a single view of the customer. Is the BI infrastructure using the same customer database as the CRM system? Are your suppliers and B2B customers linked into your CRM? If not, we've immediate run into problems that can be resolved with CDI (customer data integration) and MDM (master data management).
MDM can provide the central hub to supply that single view of customer, product, employee, etc. These entities are common MDM modelling entities and is definitely a direction to begin thinking about with your company data. Any BI system that is supplying knowledge for decisions must use a unified, up-to-date, accurate vision of the business and the underlying process. Otherwise, you are doing more harm than good.
For a good dive into the Microsoft vision of MDM, I recommend starting here:
http://www.microsoft.com/sharepoint/mdm/default.mspx
Roger Wolter has some good writings on there and the CDI Institute is sort of rebranding as the MDM institute:
http://www.the-cdi-institute.com/
So thanks for hanging in there with me this posting! It is vitally important to get your data marts, data hubs, customer views, etc. correct before we can have fun with analysis and data mining. Which is what I will focus on next time. I am going to give you some samples of real-life data mining applications that created real business value for our MEC customers.
Best, Mark
|
-
Welcome back, lovers of business intelligence and solutions for business optimization!!
Today, I thought I would touch on the topic of driving business performance and optimizing processes and keeping your employees focused on your key business metrics.
In the world of BI, there are many terms that relate to measuring your business performance: KPIs, metrics, measures and so on. What I want to do here is to use our Microsoft Services business solution framework MEC to show a very interesting way that I have seen customers being successful at this.
The first thing you need to do is to decide which top 5 organization, strategic metrics do you need to have your employees have visibility to at all times. That can be tricky because you may have a large bucket of metrics and KPIs but they need to be categorized into strategic, operations and tactical. With a Microsoft BI projet such as MEC, we can provide you with a rich set of KPIs for your business that meet those categories and are appropriate for different parts of your organization.
So now you can decide which 5 need to be visible to groups such as marketing, sales, HR, operations, etc. At Microsoft, and in MEC in particular, we use Sharepoint as a centralized BI portal such as in this dashboard example:
Below those Virtual Earth and Silverlight controls live Microsoft PerformancePoint analyst dashboards and all of this is surfaced via MDX out of a SQL Server Analysis Services cube.
But if you take a look at the top part of the portal, you will see a scrolling Silverlight control that mimics a stock ticker which I have highlighted here in a red outlined circle:
What you achieve with this approach is that every time your employees log into their portals every day, they see the top business metrics that they need to be aware of to drive those business activites to their optimal level. You can see the current measure as well as what the trend looks like.
The objectives we are putting forth here is to give visibilty to all of your employees to coallesce around driving the overall business strategy through business performance management.
|
-
You hear me talk alot about business intelligence as a solution not a tool here in this blog. And in fact I am more than just an advocate of BI and demonstrating the value of BI to business, but I live and breathe it every day! To me, it seems that a successful business intelligence solution, implementation project or pilot, must, must, must utilize best practices, standards, common practices and leverage key learnings more so than most other IT ventures. Notice that I did not mention BI products or tools here! I am speaking of implementing your BI for return on your BI investment, to embed business intelligence into the fabric and rythm of your business so that your entire organization makes better decisions.
Ok, enough of that. I am biased, no doubt. But my past history in IT and software companies was not always BI. So I feel I am on pretty solid ground when I say that the sort of business intelligence projects that I am speaking of here will need a center of excellence (COE). And I am referring to my old concept of end-to-end BI (please read through my prior postings for more on that) which would include data integration, data quality, data management, reporting, KPIs, workflow and data visualization. That is a TON of data movement, data presentation and data governance all coming together in the end to empower your business to make excellent decisions.
A center of excellence (COE) is there to provide the guidance, standards, best practice and learnings to use while implementing these projects. In our group out of Microsoft Services where we work on vertical BI solutions for industries (which we call Microsoft Enterprise Cube or MEC). We have internal COE groups for BI and BPM and there are also good articles out there on MSDN, DM Review and BI Review which I recommend you search for BI COE. I am an absolute advocate of our internal groups at Microsoft, our partners and our customers. One of my favorite activites, in fact, is spending time to review, discuss and debate approaches to business intelligence implementations, scalability and lessons learned. An interesting approach that we are taking is to try and bake that knowledge base into MEC so that our customers can see the fruits of the labor somewhat tacitly in the form of very well designed and developed reports, KPIs, data models and ETL as well as explicitly in white papers, guide books and training.
I won't have the time here today to get into the different approaches to setting up an internal COE. Certainly those references I listed above are good places to begin your search. You'll have to remind me to speak about this more here or if you run into me at one of our events. As always, I would greatly enjoy speaking about the experiences or ideas around business intelligence center of excellence. Or you can reach out to me via email (makrom@microsoft.com) or speak with your Microsoft local rep about Microsoft BI best practice consulting and MEC. Being able to build up a go-to group with this sort of knowledge will save you enormously on these projects. In fact, measure your COE operational cost savings on your BI projects by reducing your internal project overhead or buffers by leveraging the COE.
|
-
I wanted to take just a few minutes out this month to post up some of my favorite, more interesting data visualization examples using Microsoft products (SQL, PPS, SSRS, Proclarity, MOSS & Silverlight) as a way to demonstrate making data analysis, scorecards and dashboards exciting and interesting. I will probably not have much of a chance to blog here for a couple of weeks while I am heads-down on getting our BI solution framework for industries launched and rolled out to our field and partners. Enjoy!!
Here is an example of a bubble chart surfacing data through SSRS ( thanks to Rob Wilson for this chart )
Here is an example of a PPS dashboard using Visio for a strategy map, a very nice way to utilize Visio to surface BI:
Here is an example of Sharepoint BI dashboard with Silverlight, Virtual Earth and Sharepoint subsites surfacing data from MDX queries against a SQL Server 2005 OLAP cube:
Perhaps not as cool or snappy is a favorite of mine. Excel is a very, very good reporting tool for data analytics and very popular for reporting. Excel now has support for Edward Tufte's Sparklines. Essentially, you are able to view a chart within a cell, which can help get across an important statistic in a small amount of space, which is a goal of Sparklines:
Finally, here is an example of allowing your users to see their scorecards remotely from their Windows Mobile device:
|
-
What is the truth? How do I find the truth? Just the truth, the real truth.
This is a common demand of police detectives, lawyers, relationships, or every day life. But let us now forget that those of us working in the BI community demand the the singular truth for our solutions to be a success. Our business users are most certainly demanding that. Without, we are providing business solutions used for critical decisions that would be based on possibly false knowledge or obfuscated facts.
Continuing on the most recent DBCF postings around data quality and data integration, I want to stretch this out to the concepts of SSOT (single source of truth) or SVOT (single version of the truth). These are essentially common database design objectives and are critical in BI solutions. A business scenario which includes BI as a component can be solved by simply putting in a cube and a spreadsheet. The sometimes hard & tedious work of wading through the sources of similar but different data and battling the organizational politics can be 80% of your efforts in a complete BI solution including data integration.
What does a fact table not contain facts? In the world of dimensional modeling for data warehouses, the analyst and the modeller must perform the exercises of modelling the star schema, but cannot ignore a deep analysis of data sources. Data profiling in this context is the primary activity where you will look for duplicates, similarities and differences and determine, along with the owners of the data, which customers, products, LOBs or sales are to be treated as the database of record. And how can the datawarehouse bring disparate sources together for a single version or vision of the truth. These are just a few examples. I encourage further research and have a look at this article, which is one that I enjoyed reading on this topic: http://www.computerworld.com/databasetopics/data/story/0,10801,88349,00.html.
When we work on BI solutions specific to industry business scenarios, we build data marts so that we can have the proper data sources integrated to generate reports and analytics needed to solve business problems. There are a number of Microsoft partners that have tools and practices to make this a success in your project. Searching Microsoft.com for data integration partners will provide a good start for you. For our MEC (Microsoft Enterprise Cube) framework customers, we provide data quality and integration tools and expertise to work with our customers on-site to uncover the SSOT and SVOT.
As always, drop us a line at bidis@microsoft.com for questions regarding these BI solution concepts and other related topics.
|
-
Happy new year everyone! As we enter into 2008 I thought I would start us off on a few musings that I have around data quality in relation to business intelligence and data warehouse projects. A few recent projects that I have worked on and witnessed had me thinking about the topic and some thoughts to relate to you all.
First, why is data quality important in BI projects? A very basic question to ask, indeed, but a valid starting point because I hear this question too often. Doesn't matter if you are using SQL Server with SSIS, custom ETL, any data integration whether that be with Oracle or specialized tools like Informatica ... data quality is crucial to you BI application project. In the end, not ensuring data quality will likely result in poor decisions being made by the business from faulty statistics or incorrect data. When your end users are looking at BI reports, they should be looking at actionable knowledge. The analytics must be the result of vetted quality data.
Second, profile your data. This is very important and can help you identify common sources of data quality problems such as duplicate data, missing data and suspicious data. I would also recommend having a look at the new Excel data mining add-in cpabilities to examine source data files so that you can identify outliers in your data sources. Missing data and duplicate data are common sources of downstream business intelligence flaws. I promise to follow up on this post with a few words about finding proper entity sources and other data integration concepts including MDM very shortly to address those concerns.
A nice source of data quality strategies utilizing SQL Server is available here: http://msdn2.microsoft.com/en-us/library/aa964137.aspx.
That's all for now, folks. Until next time ... BR, Mark
|
-
Is dimensional modeling too difficult? I do hear this question when debating sources for cube data in business intelligence solutions. At Microsoft, our OLAP engine and data warehouse tools are all wrapped up into SQL Server and its components: SQL Server database engine, SQL Server Analysis Services (SSAS) and SQL Server Integration Services. But if the requirements for your BI application are real time or near real time, there may be a need to build your cube directly off an OLTP or transactional database system.
Data warehouse and BI purists will generally tend to steer away from that. In fact, in SSAS, there are many capabilities allowing you to have your cube built and refreshed upon partitions and measure groups and sensing source data changes so that portions of the cube are refreshed. Many of these mechanisms give you essentially the near real time capability of what is sometimes refered to as "operational BI".
But most certainly a common good practice is to build your BI solution upon a cube in the OLAP engine that is based on a data warehouse of some sort. In my role at Microsoft, my group generally looks to build functional BI solutions built all Microsoft and partner products that give you a snap-in solution for a particular business problem. Therefore, we do not generally provide an enterprise data warehouse. Instead, we call them data marts with entities and attributes that fit just what is needed for a particular business problem.
Indeed, we utilize dimensional modeling based upon the TDWI methodolgy devised by Ralph Kimball. We have found that SQL Server is very well suited for building what we call a Unified Dimensional Models (UDM) for your data warehouse. But it can be challenging for database administrators or developers who are comfortable and well versed in relational database design. A very good source of information on dimensional models can be found from the Microsoft reference architecture of Project Real: http://www.microsoft.com/sql/solutions/bi/projectreal.mspx. I recommend looking at the sample data warehouse that ships with SQL Server to see examples of a star schema for storing your measures & dimensions. Taking the time to learn these techniques to build a data mart or data warehouse will provide the best foundation for a successful Microsoft BI implementation. And this way, your decisions can be based on application requirements such as realtime, near realtime, quarterly reporting, predictive reports, etc. as opposed to the technical challenges of building a star schema.
I think that's it from here at the DBCF BI Solutions desk for the year. Happy Holidays and I look forward to blogging in the new year! BTW, be sure to keep an eye out for our MEC BI solution launch events coming in 2008 ...
- Mark
|
-
A very common scenario that we have run across from our customers when speaking with them about BI projects, is "Where do I start"? Many IT professional and business professionals have an understanding of business intelligence systems and the power that they can bring to your organization and the value that they can bring an organization. Yet these are complex systems which can include data warehouses, analytics, reporting, workflow, business process management, portals, on and on. In fact, the Microsoft BI stack includes all of these through SQL Server, MOSS, Visual Studio, Proclarity and PerformancePoint Server.
So where to begin ... What are my data sources? Do they support the problem scenarios that I am trying to provide to the business? What should my UDM look like? Star schema or snowflake? How should I surface my heirarchies, KPIs, analytics ... ? How long is this going to take me?
For those that have been reading my blog over the past several months, you may have captured a few bits of information that I have released early before our official launch regarding Microsoft Enterprise Cube (MEC). This is a way in which Microsoft can provide for you a beginning baseline out of the box to begin the process of gaining insights into your customers, providing immediate ROI on your BI investments and lowering operating income.
These are all the lofty goals and intent of any BI project. Here Microsoft will provide data models to use for your staging and star schemas, preditictive analytics algorithms, KPIs, reports, etc. all based on industry standards from your particular industry.
Here is an example: With an MEC solution for Churn Management in a telecommunications customer, you would receive the Microsoft BI stack with these elements included:
1. Physical ODS & star schema
2. Cubes to support the scenario
3. KPIs & reports to measure the business
4. UI integrated with Virtual Earth and Silverlight
5. ETL processes to bring your existing data into an existing data mart
6. High speed source data adapters
I'm going to post a few notes this month, I promise, on some techniques that can be used to map data sources to an existing data mart and how to address such issues. Without clean data properly summarized in your data mart, your business intelligence solution will not bring you proper value to your business users.
|
-
If we are to look at the time it takes to bring a successful, complete Business Intelligence solution on the Microsoft platform to production in your environment, what components of building this solution would need to be accounted for in such a project?
Some of the most comprehensive examinations of the lifecycle of BI projects are available in the Kimball Group's Datawarehouse Toolkit books. But in a nutshell, what we mean in such cases is that what really makes a BI project complete and successful is to perform the necessary upfront analysis, interviews with the business stakeholders, build your data warehouse using proven dimensional modeling techniques and to build your schemas based on data models to support your intended BI application solution. Without the solid foundation to support your KPIs, measures, reports, etc. you will not be successful.
The Microsoft BI platform products fully support you to build these solutions: SQL Server (SSIS, SSRS, SSAS), PPS and Proclarity. But if you do not have the in-house staff, time or funds to invest in an end-to-end time & materials project to implement these solutions from scratch, or wish to cut down on your time to production, Microsoft offers pre-built BI solutions called Microsoft Enterprise Cube (MEC) solutions.
MEC solutions allow you to implement the Microsoft BI solution platform based on popular scenarios such as Churn Management, Customer Segmentation and Profitability Analysis with Microsoft Services or Microsoft BI partners. The intent is to provide you with 70% or so of what you need to get started around these scenarios including the database schemas, data models, KPIs, reports and workflow. In this way, you can bring your solution to production in your environment in 16-18 calendar weeks by utlizing the MEC data models and analytics.
This is not to say that your project lifecycle with this approach to BI solutions does not include analysis and modeling. Instead, we look at the efforts involved around extending an existing data model, star schema and cube structure. Since these are based on industry best practices and standards, standing up such a complete scenario takes much less time, effort and risk.
There are no good shortcuts in the BI project lifecycle. But there are methods to solve your business problems with BI scenarios in a much quicker method using a prebuilt solution like that we are speaking of here with MEC.
Happy Thanksgiving and I will report back in December with some further musings around Microsoft BI solutions ... BR, Mark K
|
-
Seems like there has always been a question that Microsoft has received from the data warehouse, business intelligence and business community: Does Microsoft really *get* BI? In other words, I think that means are we serious about the technology, the capabilities, and the benefits it brings to business. Also, can Microsoft speak the business talk in its products and solutions that address business pain points that can be solved with BI.
Those that have been working with BI systems for many years have seen large software vendors like Oracle, SAP and SAS perform very well in this area when you look at Gartner's magic quadrant for business intelligence and analytics systems. The BI-specific niche companies make a nice showing there as well, such as Business Objects and Cognos. Then there is a separate category of consulting services-driven solutions from CapGemini, HP, IBM, etc. that leverage their business consulting to add value to a customer solution.
With the very recent release of Microsoft Office PerformancePoint Server, this is all possible and being delivered by Microsoft today. Yes, Microsoft does indeed get it! We are putting together solutions based on these BI & BPM tools that are being driven by what OLAP Report lists as the #1 OLAP engine on the market today, SQL Server 2005 Analysis Services and our ETL engine, SSIS, and updating those in 2008 with brand new versions. By working with industry experts in the fields of telcommunications, health services and retail, Microsoft is taking those tools and creating data models and cubes to solve business problems for business users. And all of that analysis and knowledge is then surfaced through MOSS 2007 so that it is available to the masses: BI for everyone in your organization! Empower, enlighten and participate in excellent business decisions is what Microsoft BI solutions position for.
So that's it from Redmond for me for now. Have a very happy Fall & Halloween! Best regards, Mark
bidis@microsoft.com
|
-
For those that read the DBCF database blogs which I post here from time to time, you will notice my move into business intelligence solutions full time since the beginning of this year (2007) under the BIDIS Microsoft alias (bidis@microsoft.com). I wanted to write a brief entry here to sort of wrap-up a few comments about the pervasiveness of BI in business today that we see and that I have encountered in working with our customers, business decision makers and industry leaders.
Saying that BI is pervasive in today's business structure can seem very much like marketing hyperbole. Personally, I think it really does not do justice to the fact that I believe we are trying to put across here. The takeaway for me that I wish to highlight is that business intelligence as a solution to every day business problems such as common pain points including customer profitability, revenue management, customer churn analysis and customer segmentation, touches nearly every business in every sector. BI, when done right, is truly a solution to the issues that are raised by those problem areas.
I'll just spend 1 paragraph here focusing on a single area within the problem space I mention above. Let's take Customer Profitability, which Microsoft Services packages as a ready-to-deploy module within the Microsoft Enterprise Cube solution. A deep, analytical 360 degree vision of your customer is the only true way to decipher his and her effect on your business operating income, revenue streams, margins and profits. To be able to analyze your customer base at this micro level provides strategic advantages over competitors and enables you to understand your customers better than ever before. Turning vast amounts of data into knowledge is not only pervasive, but necessary to compete and win in your business.
The way that this is done is to extract the data points needed to form our pre-built Profitability Anlysis cubes through our ETL layer and into the analytics layer provided by Microsoft PerformancePoint Server. SQL Server 2005 in the engine that drives this functionality and provides the ODS for the data storage while the analytical charts, KPIs and scorecards are surfaced through MOSS 2007 and ProClarity.
For more information on Microsoft packaged BI solutions, please send an email to this alias: bidis@microsoft.com.
Thanks and take care! Mark
|
-
In the Microsoft Industry Solutions Group, we are developing pre-built BI solutions for vertical business needs. Each of these solutions solve specific problems within a vertical business such as Churn Management for Telcos, patient segmentation for healthcare, and customer profitability for retail businesses.
Prebuilt BI solutions in this context represent a package of the Microsoft BI products (SQL Server & OBA product line) with the specific required set of ETL packages, database connectors, data models, cubes, KPIs, scorecards, reports and dashboard. The idea here is to get your business running in a production environment with a specific pain point solution based on Microsoft BI quickly without the need for months of analysis, architecting and building the building-blocks needed to accomplish the feat of turning your mass amounts of data into knowledge for customer segmentation, churn analysis, profitability analysis, etc.
There would be no need to hire data modelers, DBAs, developers, business analysts and project managers for a 12-16 month project to figure out not only how to make this all happen in business intellegence tools. But also to figure out what exactly it is that you need in regards to data models, data extraction, cubes and analytics in order to track, measure and forecast your business performance.
The old adage of one size does not fit all certainly pertains to an off-the-shelf BI solution such as this. But if 70% of the lengthy, expensive and complex job of building and implementing a pure BI or BPM solution is already completed for you, then the BI solution has been a wise investment.
And in the case of these Microsoft BI solutions I spoke of here, they include analytics and workflows built in PerformancePoint and SQL Server that are based on industry standards and best practices from some of the top industry consultants and standards forums for each industry such as TMF and ARTS.
|
-
What are some of the most common problems encountered when integrating data from multiple sources to retrieve the data points necessary for business analytics? Very frequently asked questions that we hear when starting business intelligence projects include:
How to connect to the data sources? How do I manage different keys for similar data? How to handle large data feeds? Should I use a temporary staging table or load dimensions and facts immediately? Must I use a UDM for data model?
These are complex questions with answers that depend on many factors. But they are frequently asked and I am putting together a list of answers for these based on Microsoft SQL Server best practices for BI and DW using SSAS and SSIS and various solutions through our partners including those that offer solutions for high-speed adapters.
Those answers and responses will be posted here on my blog over the next several weeks, so please check back frequently ... hang in there!! Mark
|
-
As we wrap-up our first ever BI conference at Microsoft here at the Washington State Convention Center, I just wanted to take a few minutes to type in a few blog notes.
Our team was here representing the Microsoft Services Industry Solutions Group (ISG). Hopefully, if you were here in Seattle this week, that you had the time to stop by our booth in the Microsoft pavillion. Here we demonstrated the Enterprise Cube solution framework based on the Microsoft BI stack. This an enabling technology for you to adopt BI solutions in your enterprise in a quick manner (4-5 months) that fits a specific business need (Churn Management, Customer Profit Analysis, etc.) and is based on the industry-leading TCO toolset for BI solutions.
The sessions and keynotes during the conference hopefully portrayed the Microsoft vision accurately for you regarding our message of BI for Everyone, Pervasive BI in the enterprise and our solutions and tools for Business Performance Management, such as PerformancePoint Server and our group's framework, Enterprise Cube, for solving industry-specific business problems.
If you have interest in industry-specific pre-built solutions based on industry standards and common practices, please contact the product group here in Redmond, WA for further information about demos which we have ready and showed here this work, or to set-up a proof of concept for you. Our email address is bidis@microsoft.com.
Thanks and I hope to see you all back here next year!
|
|
|
|