Importing Web Logs from different sources in a Web Farm Scenario
Consider the scenario where you have multiple web log sources (let's say you have mulitple web servers for your site) and you want to import those web logs in parallel. This allows requests by a single user served by different sources be counted in a single user's visit.
Here is a sample vb script that illustrates how to invoke the web log import across multiple data sources.
Note: You need to get your data source names from the ServerGroup table in your data warehouse.
(A special thanks to Brian Blum for the script content)
Dim ImportObj 'As Object
Dim error
Dim Shell
set Shell = WScript.CreateObject("WScript.Shell")
Dim ProcEnv
set ProcEnv = Shell.Environment("Process")
Dim localDbServer
localDbServer = ProcEnv("ComputerName")
'Import tracing flags - Not required.
ProcEnv("TraceParseHits") = "true"
ProcEnv("TraceParseCookie") = "false"
ProcEnv("TraceParserStats") = "true"
ProcEnv("TraceInferenceSteps") = "false"
ProcEnv("TraceIEStatistics") = "false"
ProcEnv("TraceFilters") = "false"
ProcEnv("TraceHitStructureOnFilter") = "false"
ProcEnv("TraceIEJobCreate") = "false"
ProcEnv("TraceMaxHits") = "false"
ProcEnv("TraceOpenUserVisits") = "false"
ProcEnv("TraceOverlapTime") = "True"
ProcEnv("BenchMarkParser") = "false"
ProcEnv("BenchMarkIE") = "false"
Dim LogDataSource(1)
LogDataSource(0) = "Log Data Source 1" ' Data Source name from the ServerGroup table
LogDataSource(1) = "Log Data Source 2" ' Data Source name from the ServerGroup table
Set ImportObj = WScript.CreateObject("Commerce.PrivateDTSWebLogImport")
On Error Resume next
error = ImportObj.Init(localDbServer, localDbServer, "startersite_datawarehouse", "dwschema", "", "", "StarterSite")
ReDim LogFiles(1) 'As String
LogFiles(0) = "D:\Logs\S120060901.log"
LogFiles(1) = "D:\Logs\S220060901.log"
error = ImportObj.AddLogDataSource(LogDataSource(0), LogFiles(0), 1)
error = ImportObj.AddLogDataSource(LogDataSource(1), LogFiles(1), 1)
error = ImportObj.Run()
WScript.DisconnectObject ImportObj
Set ImportObj = Nothing
Hope this helps!
-Alan
