We have a fact table with 280M rows. Here is a sample structure:
CREATE TABLE fact ( PK BigInt ,Ref1Fk BigInt ,Ref2Fk BigInt ...)
It references a "generic" lookup table, which is structured as follows:
CREATE TABLE lookup ( PK ,LookupType BigInt ,LookupValue varchar(100) )
lookupType defines whether it is Ref1, Ref2, etc...
We are using SSAS 2012 and SQL Server 2012. To simplify the dimension creation, we defined the views, one for each lookupType. The views contain PK and LookupValue (dimension name column).
We then define individual dimensions, one per RefNFk column in the fact table. The goal is to create a cube that is, in its simplest form, will have record counts from the fact table, allowing the user to dissect the data across multiple dimensions. The lookup-based views contain somewhere between 50 and 200 distinct name values.
SSAS and target database are running on the same machine. Windows 2008 R2 SP1, 64GB RAM. Configured as follows: SQL Server Min=16GB, Max=36GB. SSIS Min=50, Total=75. SQL Server 2012 (11.0.3128.0). 2 x Quad core CPU (8 virtual processors).
Source of data is on another SQL Server, connected via a gigabit network.
When dimension is processed, it takes approx. 1+ hours. Processing is invoked from a VisualStudio 2012 (this is where dimensions/cube are defined). Tried doing it from the SSAS connection to SQL Server - result is the same.
First, it gets the data from across the network. It takes a few minutes, after which the network connection is idle. This is followed by disk activity on the SSAS box, where it is writing data out to the store. And then the CPU goes up to approx. 10-12% utilization and stays there, with light spikes in disk i/o, writing to lstore, sstore, ostore... SSAS service does not use up more than 6-8 GB of RAM. SQL Server on the target is also not consuming much memory/resources. As a matter of fact, about 16GB of RAM is free.
So, no disk i/o, no memory usage - just a bit of CPU and a looong time.
Is there a way to somehow speed it up? We have about 10 dimensions for this fact table, and having 1+ hours to process each is a lot. This time does not include creation of the actual cube. Dimensions are created sequentially, so no other activity on the box - and still slow.
Anything on the tuning of the SQL Server / SSAS parameters that can improve performance?
Thank you in advance!