Forgot your password?
typodupeerror

Comment: A good start (Score 1) 180

by duckett227 (#26526089) Attached to: Visualizing Complex Data Sets?
Of all the products out there, Business Objects strikes me as the best solution to quickly engage you and provide strictly the useful information your looking for. They were also recently acquired by SAP so I would recommend you ask someone at your company what the corporate availability is to their their products. Maybe get in touch with the SAP account executive. If your company doesn't already have the availability to use the product you would probably qualify for some reduced price incentive. The Business Objects GUI is very clean/user friendly and you basically just drag your various dimensions to create a request and qualify respectively. If you can't make sense of your data doing this, then you probably have a poor and heavily denormalized database design. The other issue you address is the poor data quality present in the master data management system, be it the data warehouse etc. These issues plague many of the largest companies out there and can often take many years to sort out. Unfortunately, the best way to fix these issues is typically when someone like you decides to actually use the system for the very reason it was put there. So... ask some good questions and start scratching your head when the answers don't make sense. I can pretty much guarantee you that you will find a goldmine of things that don't make sense if you look hard enough. Its obviously a serious concern when there are data quality issues, however a company almost never knows how bad they are. To put it another way, when it comes to understanding this data your company doesn't know what they don't know yet. Its a continuous process of improvement that is largely driven by the users and their efforts to ask new and innovative questions. The questions to consider would be very industry specific, however try asking some basic queries like how many distinct US states are showing up in the system (if you get 70 thats a big problem), if the same social security number is showing up in a lot of different stores all over the place (fraud detection, criminal activity), if the cost of any products are showing up as 0 or next to nothing (your company is potentially over ordering products that that it thinks are creating huge return percentages, however are actually creating huge losses and inventory costs). To create truly informative strategic reports your going to need a good deal of historical data and a well tuned database system to provide a decent level of service. I am a little concerned as to how your accessing the data though. You state that your manually extracting the data to what I assume to be flat files. Even if your very sparsely qualifying the data and preaggregating much of it, it won't be very useful unless you mean for it to answer a few particular and very similar queries. You need to be tieing your front end tool to the data warehouse or an aggregate cube that is being built off this central system. The cube will likely perform faster unless the data warehouse is well tuned for your requests. Without going into the complicated solutions for such tuning a quick solution is to create a couple aggregate join indexes or denormalize part of the relational system as a last resort for performance. The bottom line is that the greatest value of these multi million dollar database systems is found in the information that is uncovered through asking the right questions, which lead to making better corporate decisions. This may seem obvious, however these systems are used in so many ways from operationalizing complex triggered company processes, to creating customer targeted incentives and market basket analysis/product pricing. When the data quality is bad this typically cascades through these processes and leads to poor decisions and inaccurate forecasting. One very cool solution I have seen is to use MS Excel to access the database. I'm assuming this was using ODBC drivers. You can create some very cool pivot tables and charts/graphs and voila, you save $100,000 and have yourself a pretty cool BI solution! Granted, this is not a BI product, however it can be very powerful as I have seen it work well on Teradata. So that gives you my ideal solution and a free one that can work as well. I really don't recommend looking for some middle ground unless your just poking around and not concerned with creating a long term solution. I don't believe any of the "out of the box" solutions from the major data warehouse vendors are good solutions. Look for a BI/analytics vendor that integrates well with your source/data warehouse system. I know SAS is in a major partnership with Teradata and there is a great deal of integration and R&D going on there. Good Luck :)

A language that doesn't have everything is actually easier to program in than some that do. -- Dennis M. Ritchie

Working...