MS.COM Operations Tools Team WebLog

Hey - What does this button do?

Data, Data Everywhere

As I've mentioned before, I'm a database guy by trade.  So in this age of incredible stores of deep, rich data about almost everything you could ever want, I constantly amazed at how little we actually use the information available to us.  Generally speaking, the data is used for a brief, specific purpose then never looked at again (although we continue to keep it *just in case*).  So what we end up with is huge stores housing anything from performance to traffic metrics to events to availability and reliability data and wasted opportunity.  Literally terabytes of data (in our environment alone) sit patiently waiting for some enterprising soul to take advantage of it.  Now in defense of most of the people asking to keep this stuff around, they don't actually know what to do with it beyond the scope of their immediate need. 

Now call me a glutton for punishment, but this is where I (and several members of my team) am choosing to spend a great deal of time.   We're still in the early phases of determining how best to make huge datasets of disparate data meaningful and interesting to those who may not necessarily know all of the questions they want to ask, but we have some very cool ideas on how we are going to approach the problem.  Stay tuned...we'll keep you posted!

 

Scott (no pseudonym necessary)

"There is no box"

Published Thursday, September 09, 2004 9:02 PM by mscomts

Comments

 

LTB said:

In my opinion using Topicmaps could give a new view into corporate datastores that would enable workers to actually find and reuse data based on relations between material (and/or contents of it).
September 10, 2004 3:11 AM
 

Zahid Karim Khan said:

I need more instructions about this topic.
Please send me a mail.

Thanking you
September 12, 2004 12:28 PM
 

Markus said:

I worked on a project a few years ago called EdWebb (Entity Database with a Web-Based interface.) At the heart of the project was an object model that sat on top of a fairly abstract data model.

The system was designed so that users could create relationships, through the course of doing their work, between objects such as people, places, projects, tasks, intelligence, etc. Activities, for example, were associated with the people, groups, etc, that they were regarding.

We believed that people generally knew at least one or two pieces of information regarding whatever it is they were looking for, and therefore should be able to find their data in a somewhat non-linear fashion.

Some of the things that we struggled with are:

1. Performance. Although our object model did an excellent job of generating the many-to-many queries, we had some problems expressing what we needed to express using SQL.

2. Recursive references.

3. Presentation. We had a tough time explaining the concept to users and why they would want to use our system versus a less normalized solution.

4. Browser limitations.

Graphical representation of the relationships seems like the most logical solution, but we were fairly limited in presentation because it was a web-based application.

Markus
September 13, 2004 12:34 PM
 

MS COM Operations Tools Team WebLog Data Data Everywhere | Cellulite Creams said:

June 9, 2009 9:41 PM
New Comments to this post are disabled

This Blog

Syndication

Tags

No tags have been created or used yet.

News

All opinions posted here are those of the author(s) and are in no way intended to represent the opinions of our employer. This is provided "AS IS" with no warranties, and confers no rights. Use of included code samples are subject to the terms specified in the Terms of Use.

© 2009 Microsoft Corporation. All rights reserved. Terms of Use  |  Trademarks  |  Privacy Statement
Microsoft
Page view tracker