Dynamics NAV Business Intelligence & Reporting : Business Analytics

What is new in Dynamics NAV 5.0 Business Analytics (BA)

clausba — Mon, 04 Jun 2007 22:58:00 GMT

A while a go we released Dynamics NAV 5.0 and with that a new version of Business Analytics Basic and Advanced. I want to spend a few lines of text to share what is new in this release and the reasons for these features making it into the product at this time.

First of all the Advanced client is now up to speed with version 2k5 of the Targit release with everything in that. This gives us support for presenting cubes created in SQL Server Analysis Services 2005.

Of other new things i can add gauges as a way of graphically representing KPIs in a trafficlight way. I also add improved integration with SQL Server Reporting Services.

In BA Basic list of new features looks as follows:

Ability to apply filters on fact tables
(Allows customers with large volumes of data to create cubes containing only subsets of data. Filters can be set by the members of any of the dimensions in a cube)
Create calculations between virtual cubes
(Allows for cross cube calculations to be handled by BA Basic. An example would be having a cube with actual figures and one with budget figures. It would be obvious to create a measure reflecting the deviation between the two. This is possible with this feature)
Multiple active configurations
(In previous version one could have only one active configuration. A configuration is made up of a set of cubes, dimensions etc that are implemented in databases with fixed names (BA for MBS Navision 400). This would mean that whenever a given configuration was implemented the database names etc was overwritten by the new content. In this version the database/configuration name is customizable meaning that you can create configurations that can be updated at differing schedules etc. without interfering with each other)
Support for named instances of SQL Server It is now possible to have the BA databases and or NAV run on something but the default instance in SQL)
Freehand MDX
(This is the ability to create formulas/calculated members that go beyond what the simplistic calculator will enable inside NAV. This is also something that potentially can break the implementation if syntax is wrong in the statements written. This unleashes the power of the language but with the freedom comes responsibility)

In coming posts I will create examples of the usage of each of these features so stay tuned. If you would like to see one over the other leave a comment and I will try to do this.

Creating a Fiscal Calendar in BA 4.00

clausba — Fri, 22 Jul 2005 12:07:00 GMT

By request I'll briefly run over generating new timedimension in BA. By default BA is not able to create a Fical Calendar that fits the content of the table AccountingPeriodes. Nor is it able to handle calendars from Service. These can however be created manually. It is really simple and requires only editing of the Analysis Services database. It is important to note that changes made this way will not show up in the BA configurator inside Navision and the changes will be overwritten if the configurator is run from inside Navision.

To start editing open Analysis Manager. In the database BA for MBS Navision 400 right click the Shared Dimensions node and choose New Dimension - Wizard as shown below.

Choose Start Schema as the dimension type and click next

In the Select dimension table box select the table wPostingDate that contains a list of dates.

Select Time dimension as the dimension type and make sure that the Date column is set to thedate column.

If you wish to create a Fiscal Calendar where the fiscal year begins in July this is set in the Create time dimension levels box shown below. Valid selections are any month and any given day within that month.

Clicking next will enter the Dimension Editor where advanced settings can be defined and the dimension can be setup to fit more specific needs. Since the dimension at the current point doesn't show a date as the lowest level by a day number this will have to be changed to obtain having a date shown as the lowest level of the dimension. This is done by marking the Day level which is shown in the upper left hand pane and afterwards editing the Member Name Column property in the lower left hand pane.

To have a date shown instead of a number choose DateName and click ok.

Clicking data to browse the dimension will render the result below. Save the dimension and process it.

The dimension can now be added to any cube in the database. Given the fact that the table wPostingDate already is added to some cubes in the Default setup no further editing should be necessary for those cubes when adding the new dimension. If the dimension is added to a cube where the table is not part of an existing dimension the join to the fact table should be specified when adding the cube. This should be done by pointing the column TheDate from wPostingDate to the appropriate datetime column of the fact table.

I is important to note that this works as long as no editing of the setup takes place from inside Navision as it overwrites any manual changes made since the last deployment when running.

The optimal solution to creating a Fiscal time dimension would be extracting the starting period end closing period from the AccountingPeriods table. If the interest is there I will do a post on this at a later point in time. Also please let me know if this was of any help and if you would have suggestions on other topics. My next post will be on incremental updates instead of the default full loads supported by BA. Especially I'm interested in the relevance on posts on Reporting Services in conjunction with Navision.

Running more than one active configuration in BA 4.00

clausba — Mon, 23 May 2005 09:35:00 GMT

As promised at my session last week I would post a detailed walk through of how to be able to run more than one active configuration with BA in Navision 4.00. First of all I would like to state that this is not a thing that I'd recommend doing unless you are used to working with both SQL Server, Analysis Services and Data Transformation Services. For more info on those technologies - see my previous post on the components used in BA 4.00.

First we will move the Relational database created by BA to new one. This is necessary because BA always will create a database with the same name. If we do not complete this step the database will be overwritten when creating a new configuration. Navigate to Query Analyzer located in the SQL Server program group in the Start menu. there connect to the database server and execute the script below:

sp_renamedb 'BA for MBS - Navision 400', 'BAVendorDB'
go
sp_detach_db BAVendorDB
go
xp_cmdshell 'rename "c:\Program Files\Microsoft SQL Server\MSSQL\Data\BA for MBS - Navision 400_dat.mdf" BAvendordb_dat.mdf'
go
xp_cmdshell 'rename "c:\Program Files\Microsoft SQL Server\MSSQL\Data\BA for MBS - Navision 400_log.ldf" BAvendordb_log.ldf'
go
EXEC sp_attach_db @dbname = N'bavendordb',
@filename1 = N'c:\Program Files\Microsoft SQL Server\MSSQL\Data\bavendordb_dat.mdf',
@filename2 = N'c:\Program Files\Microsoft SQL Server\MSSQL\Data\bavendordb_log.ldf'
go

Note that this is specific to your installation and rights on the server. You will have to be a member of the sysadmin group on the database server to be able to execute this. Also note that there should be no users running on the database when doing this. Normally I'd recommend scripting the database and then edit the script and run it appropriately or backing up the database and restoring it with a new name. For the sake of the demo I did it this way which is easy but requires a specific setup to run.

When this is done the cubes should be copied and edited to point to the new database. Start by navigating to the Analysis Services administration tool. Here browse to the server where the Navision Cubes are placed.

Copy the database by right-clicking the database and chose copy

Right-click on the Server name and chose paste.

Since there is already a database of that name you are asked to choose a new one. Write an appropriate name for the configuration at hand. In this example it will be Vendor Database which is the name I will use for the rest of this post.

You now have an exact copy of the Analysis Services database. Expand the Data Source folder and right-click the data-source that is in it. We need to edit this to make it point to the new database that we just created named BAVendorDB.

Point to the server (in this case (local)) and chose BAVendorDB from the drop-down.

After this we have a new Analysis Services database pointing to the relational database BAVendorDB in SQL Server.

The next task is to alter the DTS packages created by BA. Open Enterprise Manager and browse to the Data Transformation Services folder. In Local Packages there are 4 packages that we will need to alter.

First open the package BA for MBS-Navision 400 - initialize. This will open the package in DTS Designer which is where DTS Packages can be edited.

Double click BA for MBS-Navision and create new connection called BA Vendor DB. Point the connection to the new database called BAVendorDB in the drop-down.

After this the task Create Company needs to be altered to point to the new connection

The task Truncate tables also needs to be pointed to the new connection.

Click the Package menu – chose Save As and choose a new name for the package. In this case I will call the package BA Vendor DB Initialize

Now open the package BA for MBS-Navision 400 - Transfer [CompanyName]. In this package we also need to create a new connection called BA Vendor DB. this step equals the one above.

When clicking OK a pop-up called Task References will appear. DTS has detected that several tasks are using the connection we just changed and asks if it should delete the mappings. This is not needed and clicking OK without selecting any transformations will preserve the transformations. We can do this since the databases are exact matches.

To make sure that all transformations are pointing to BA Vendor DB all connections called BA for MBS-Navision 400 should be changed. For each of them repeat the step about preserving all mappings in the tasks. Screen shots are below for each connection in the package.

Click the Package menu – chose Save As and choose a new name for the package. In this case I will call the package BA Vendor DB Transfer [CompanyName]

Now open the package BA for MBS-Navision 400 - Post Process. In this package we also need to create a new connection called BA Vendor DB. this step equals the one above.

Double-click the task Populate Posting Date. Point the connection to the connection created above called BA Vendor DB.

Double-click the task Update Posting Date. Point the connection to the connection created above called BA Vendor DB.

To be able to edit the connection settings for the task Quality Check we need to do a Disconnected Edit. In disconnected edit all properties can be edited with no check on the validity of the value. This means that one can really mess up in this and I cannot stress enough that you should not do this unless you know what you are doing. Doing the wrong things here can ruin the package for good. To enter Disconnected Edit right-click the Quality Check task and chose Disconnected Edit.

In Disconnected Edit locate under Tasks the Quality Check task double-click the ConnectionID property.

In the Edit Property box set the value to 2 and click OK.

Exit Disconnected Edit and double-click the Quality Check task. The connection name should now be BA Vendor DB.

Double-click the Update Statistics tasks and set the connection to BA Vendor DB

Click the Package menu – chose Save As and choose a new name for the package. In this case I will call the package BA Vendor DB Post Process

Now open the package BA for MBS-Navision 400 - Process Cubes. Double-click the Process Shared Dimensions task and point it to the dimensions in the Vendor Database.

Double-click the Process Cubes task and point it to Cubes in the Vendor Database

Click the Package menu – chose Save As and choose a new name for the package. In this case I will call the package BA Vendor DB Process Cubes

The next task is to alter the Agent Job created by BA. The job will enable you to schedule the job in running the packages at a given time of day/night.

Browse to Management and SQL Server Agent. Under Jobs you will find a job called Update Business Analytícs Database.

Double-click the job. Here you will see four steps representing each of the four packages mentioned above.

Edit the first step and change the name of the command to reflect the new name of the initialize package. In this case BA Vendor DB - Initialize

Alter the following steps accordingly to reflect the new names of the packages. Also note that you will have to give the job a new name to be able to coexist with other parallel configurations.

To run the job either schedule it or right-click and chose run from the menu.

The components used by Business Analytics 4.00

clausba — Thu, 19 May 2005 03:40:00 GMT

In this post I will outline the components used in BA v.4. This will enable you to get an overview of the underlying technologies and ultimately give you an idea of what it takes to customize the setup delivered by BA.

The major components of Business Analytics are:

The Configurator which is the tool that consumes the XML containing the configuration chosen.
A relational Database in SQL Server 2000
A Set of DTS Packages
A Database in Analysis Services 2000

In this post I will describe in short what DTS is and how it can be used. I’ll also point to further reading. I will also give a short introduction to Analysis Services and here too point to further reading on the subject.

Data Transformation Services (DTS)

When creating a small configuration and running it (I did a demo at the Product Conference this week that I’ll use as an example) you get 4 DTS Packages. A DTS Package contains what ever logic that you as the creator have decided to put in it. The four packages are:

BA for MBS-NAvision 400 – Initialize
BA for MBS-NAvision 400 – Transfer [CRONUS International Ltd.]
BA for MBS-NAvision 400 – Post Process
BA for MBS-NAvision 400 – Process Cubes

This setup is classic in its layout. First some initialization takes place. In this case tables are truncated. Then the actual transfer of the data is taking place. Then some time variables are set, integrity checks are done and finally the cubes are loaded with data.

There are two major online resources for using DTS in BI. One is Best Practices for Using DTS for Business Intelligence Solutions that in detail describes how best practices can be implemented. SQL Server Accelerator for Business Intelligence (SSABI) is more or less best practices implemented. How DTS is used in SSABI is described in this article.

To give a brief overview of what DTS is made up of it can be summed into the following:

Packages
Packages are containers where the logic is executed. A package can contain any Connection, Task, Global Variable and Work flow that are a part of DTS or developed as Custom Tasks etc. A Package can contain other Packages that can be executed and variables can be passed from outer to inner packages. In this way a certain amount of portability can created in DTS. Packages can be saved in SQL Server, as files and as VB projects from where they can be compiled into executables.

Global Variables
Global Variables in DTS is probably the one feature that I like the most about DTS in SQL 2000. It enables the making of packages that can be executed on several different Servers (Dev/Test/Prod) and allows packages and tasks to be configured at runtime and put into context at runtime. This allows for a high degree of reusability for Packages, Connections and Tasks. It is, however, also the one feature that I know of that has caused most errors produced by humans that do not set Variables correctly or have the set themselves from the environment where they are executed. Variables can be used to set Server names, Connection strings, SQL statements etc. at runtime and can be picked up from INI-files, repositories etc.

Connections
A Connection is just what it sounds like – a connection. DTS doesn’t know if it’s a connection used as a source or target. This means that we from a user perspective can expect the same functionality from all connections since they are all exposed in the same way. Connections can be made to any ODBC or OLE DB provider installed on the machine where the package is running.

Tasks
Tasks in DTS can be split into two categories. Tasks that move or manipulate data in some way and tasks that perform other things like FTP or file system tasks. The first Task is the Transform Data Task which is the one used by BA. This Task can move data from one connection to another. Doing so it can manipulate data with one of the built-in data transformations or with a custom built one. This article describes how to use the Transform Data Task in detail. I realize that this is from Version 7 but it is still very useful. It’s also worth mentioning that the Transform Data Task will try to run bulk insert. This depends on several things like the transformation used. If ActiveX Scripting is chosen it will evaluate data on a row by row basis. The next task is the Execute SQL task. This task can execute any SQL statement against a Connection. The last task is the Data Driven Query. This is useful compared to the data pump since it can perform different actions depending on the outcome of the transformation. This means that it is useful in handling Changing dimension where in some cases data should be updated and others inserted. It is also worth mentioning that this task is slow since it evaluates data row by row and for each row executes a SQL Statement. This makes it even slower than the Transform Data Task. For detailed information on the speed have a look at this presentation given by Gert Drapers at SQL Pass.

Work flow
Work flow is what happens between tasks. In other works this is the instrumentation of tasks. There are three different types. On Success, On Error and On Completion. Each of them is self explanatory and I won’t go into detail with each of them.

Scheduling
Scheduling of DTS Packages can be done in any scheduler that can execute a windows command line. This happens through the utility DTSRUN where a series of parameters can execute a Package together with providing values of any Global Variables in the package. To help generate this command line I recommend looking at the utility DTSRUNUI.

Analysis Services

When creating the sample database mentioned before I end up by having one Database in Analysis Services with one Cube and some shared Dimensions.

The database contains a connection to my relational Database. This Database contains the tables specified in the Configurator that is run from inside Navision 4. These tables are, by the way, truncated and loaded from scratch each time the packages are run. This also means that the cube is loaded from scratch and aggregated again each time a load takes place.

With regards to Analysis Services there is also good reading to find in this article on SSABI. For a general intro to Analysis Services and the concepts of OLAP databases this article is worth reading. It goes into detail on the differences between OLTP and OLAP databases. For operations of Analysis Services the white paper Analysis Services Operations Guide is the resource to read. For optimizing performance the white paper Analysis Services Performance Guide is the resource to read.

Database
a database is the container where cubes and dimensions reside. The database can be compared to a relational database where tables and other things reside. The same can be said about Cubes and dimensions in an Analysis Services Database.

Connections
Connections for analysis services is where data for a cube comes from. Connections can be made to any ODBC or OLE DB source (Please pay attention to the fact that there might be limited availability of drivers for the Itanium 64–bit). One cube can only get data from one connection. This means that it is not possible in Analysis Services 2000 to get data from more than one source for one cube without involving other tools like Linked Servers or DTS.

Cubes
A cube is a collection of a data model (Star Schema, Snowflake or 3NF) with measures, dimensions, partitions and aggregations. A cube can come from not only one connection but also one fact table. This means that if one would like to combine more than one fact table say Actuals and Budget it would be impossible in a cube. For this we have Virtual Cubes. Virtual Cubes are like Views in a relational database. They are a logical representation of the underlying cubes. To be able to combine two cubes in a Virtual Cube there must be at least one shared dimension. This could be any dimension in the two cubes.

Shared & Private Dimensions
Dimensions can be shared or private. A shared dimension is available to all cubes in the database and is needed to create a virtual cube (see above). Private Dimensions are created in a cube and can only be used in the cube in which they were created. This doesn’t mean that a private dimension can’t be visible in a Virtual Cube but the behavior is different from a Shared Dimension. When selecting a member from Private Dimension values of measures from other cubes are not affected. Dimensions can come from a single table, a Snowflake and a Parent-Child table.

Security
Security in Analysis Services can be created down to Cell level. This is comparable to row level security in a Relational Database. It is a built-in functionality of Analysis Services but there are several things to keep in mind when designing this. All performance considerations are discussed in the paper Analysis Services Performance Guide. For Operational implications of security these are discussed in Guide. On how to implement security Mosha has a great collection of information on this topic.

Data Mining
Analysis Services has some Data Mining functionality. I’m not an expert on Data Mining but the guys at www.sqlserverdatamining.com are.

/Claus