Blue Screen of Death --> Microsoft Power Map

I was fiddling around with Power Map and wanted to see how far I could go ahead with the mapping visualization.......
I basically leveraged the Power Station data file available at:
http://office.microsoft.com/en-us/excel-help/redir/XT104048055.aspx?CTT=5&origin=HA104091224

After downloading this excel file, I started to create Power Map with the basic idea of having 3 layers.
1. The first layer would take into account all the counties based on the Power Transmitted
2. The second layer would consist of all the companies based the Power distribution
3. The third layer considers the power distribution based on Plant Name....

Now this excel file contains roughly about 20,000 rows...... My machine configuration is basically a Windows 8 with an Intel i7 core and 16 GB of RAM. I was good up to step 1 in the steps described above but when I hit step 2 that's when I noticed that my memory just bloated like crazy...... I decided like a crazy person to try step 3 and that's when my machine crashed. Unfortunately I could not take screen shots as well as would not like to reproduce  this issue. But to just give a general idea even playing around with roughly 20,000 rows of data and using the above layering with probably close to 8 GB of free space on my box killed my machine with even the disk usage going to 100%. I decided to work on a smaller subset and show you what exactly I was trying to produce (This is for the state of California with a much lesser subset of counties):


 Step 1 is shown in the above diagram


Step 2 is as shown in the above diagram


The above figure is very close to what I wanted to do but I used a clustered column instead. I would definitely recommend going slow while creating a Power Map. It does provide mind blowing visualization but it is dependent on the amount of memory one can leverage at any given point in time and can blow up if the resultant data set is way too large. The above examples were created using about 500 rows of data. I was way more cautious as I did not want to blow my machine again. But at the end of the day, the map was beyond imagination in terms of visualization of data points. Post a comment on this post if this has happened to you as well!!

Comments

Post a Comment

Popular posts from this blog

Rhino - ETL

Image
Using Rhino-ETL ( a C# based) framework for developing standard ETL's is pretty easy and one can do a lot of fun stuff with the underlying source data. I just wrote up a quick console app to generate data into a text file and push the same data into a table in SQL Server as well as an external file. Here are the following steps (for the external file push): 1. Create a new C# console application solution in Visual Studio. 2. Target the .Net framework as shown in the below screen shot in your project properties:- 3. Create 3 Sub Folders underneath your project as shown in the following screen shot DataObjects --> Contains the class files associated with each and every table/file in your environment. Example:- if your source file contains student data, then you would create a class file called Student with the individual properties (in relation to the properties) exposed (nouns). Operations --> This primarily contains the class files that contain the activities (a...
Read more

Redshift Experience

Image
Big Data - the keyword given to solutions that can handle massive amount of data usually in the petabyte or greater amount. There are several big data solutions out there and all of them have their unique characteristics which can be useful in different scenarios. I was looking into Cloudera's versions of Hadoop like Impala, Sentry and HBase. All these vary based on the use case. For some of my clients I have leveraged Amazon Redshift, Cassandra (and hopefully soon Apache Hadoop). The architecture of these systems differ but the end goal is the storage and processing of vast amounts of data down to second or milli second based result generation. Focusing on this aspect I am going to give a more detailed insight on Redshift which is a node based peta byte scaled database as well as a high level overview of what I recently implemented. Note: The above diagram is from the Redshift Warehousing article ( http://docs.aws.amazon.com/redshift/latest/dg/c_high_level_system_architecture....
Read more

An Agile process evaluation

In the recent past, I have been working as a scrum master for one of the projects. And since there were two folds to this project, a portion being handled onsite as well as a portion being handled offshore, there were initial hiccups that gradually deteriorated away. But here comes the crux of the problem and where an Agile process has to be evaluated.         We had weekly sprints where a subset of tasks were to be completed and we have a daily scrum between the offshore and onsite team. But the onsite team were persistent on a daily mail sent with the changes being addressed. Now the point of the scrum is to take into account who, what and where the changes have to be made based on the product backlog. Now having status mails as part of the Agile process is indeed a flaw in the fundamental flow. After pointing out this fact, I was counter attacked with several points (some of which were valid to have a status update email) and finally left to ...
Read more