├── README.md
├── ScreenShotFlare.png
├── ScreenShotSankey.png
├── cf-stack-mapping-blog-post.md
├── create-exports.json
├── example-imports.json
├── example-imports1.json
├── example-imports2.json
├── flare.csv
├── get_data_flare.py
├── get_data_sankey.py
├── index-flare.html
├── index-sankey.html
├── index.html
├── sankey.js
└── sourcedata-output.json


/README.md:
--------------------------------------------------------------------------------
1 | # cf-cross-stack-mapping
2 | CloudFormation Cross Stack Reference Mapping
3 | 


--------------------------------------------------------------------------------
/ScreenShotFlare.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/1Strategy/cf-cross-stack-mapping/124a9f60a8bc0087f5af8dae880dcde705bb1fe6/ScreenShotFlare.png


--------------------------------------------------------------------------------
/ScreenShotSankey.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/1Strategy/cf-cross-stack-mapping/124a9f60a8bc0087f5af8dae880dcde705bb1fe6/ScreenShotSankey.png


--------------------------------------------------------------------------------
/cf-stack-mapping-blog-post.md:
--------------------------------------------------------------------------------
 1 | # Visualizing CloudFormation Cross Stack References
 2 | ---
 3 | ### Introduction
 4 | Some time ago AWS released a feature for CloudFormation called cross stack references. (https://aws.amazon.com/blogs/aws/aws-cloudformation-update-yaml-cross-stack-references-simplified-substitution/) Cross stack references are very useful when you have parameters or resources that you need to reference in other stacks. This is extremely helpful because you can make your CloudFormation stacks much more modular and easier to maintain. For example, typically when creating infrastructure you would need to create VPC, subnet, route table, NACL, load balancer, EC2 instance, and database resources all in one CloudFormation stack. With cross stack references you can create separate stacks for network resources, application resources, databases, etc. The cross stack references can also be very useful when integrating with an existing environment or CloudFormation template as you can retrieve VPC IDs, Subnet IDs, etc. and use them within a new stack. Cross stack references save a lot of manual work, where previously you would need to look up all the IDs needed and add them to parameters, or create Lambda functions to retrieve the IDs.
 5 | 
 6 | ![Image](http://www.n2ws.com/images/cloudformation.jpg)
 7 | 
 8 | Cross stack references are created by adding an “Export” declaration to a normal CloudFormation stack output and using a new built in function to access them. That new function, Fn::ImportValue, does have a few restrictions that are important to note if you are setting out on creating your first cross-stack reference with it. Namely, the export names must be unique within an account and region, and the function will not allow you to create references across regions. In addition, AWS will not allow you to delete a stack that is being referenced by another stack and your outputs cannot be changed or removed while they are being referenced by a stack.
 9 | 
10 | ### The Problem
11 | There is one problem with cross stack references. When your infrastructure is fairly complex with multiple stacks and many references within those stacks, it can be very difficult to see and understand all the connections and references. Then when it comes time to make changes to a resource or reference it can be hard to see where those changes will have an impact.
12 | 
13 | ### The Solution
14 | To solve the problems with complex cross stack references, I fell back to my tried and trusted friends data, analysis, and visualization. In this case not much analysis was needed. All that's needed, in this case, is some way to get data about all the cross stack references, and then some way to visualize them.
15 | 
16 | ### Getting the Data
17 | To get the necessary data I used the AWS Python SDK Boto3 (https://aws.amazon.com/sdk-for-python/). I first hit the AWS account and retrieve all the CloudFormation exports and the name of the template from which they are being exported. I then iterate over all those exports and retrieve all the places where they are being imported and the name of that template. I then write the output to a json file that will be used for the visualization. Here is a link to the code: (https://github.com/1Strategy/cf-cross-stack-mapping/blob/master/get_data_sankey.py)
18 | 
19 | ### D3 and the Sankey Diagram
20 | The next step was to somehow show all the connections between all the stacks and all the exports and imports. To accomplish this I chose to use a Sankey Diagram (https://en.wikipedia.org/wiki/Sankey_diagram). I chose the Sankey Diagram instead of the usual Network Diagram mainly because I wanted to see the flow of the references between the stacks versus just the connections. Sankey Diagrams are not in your typical visualization tools. So, to show and share the visualization I chose to use the D3 javascript library (https://d3js.org/). D3 is easy to use and it creates a nice visualization that is easy to view and share with any modern web browser. The other nice feature of the D3 Sankey Diagram is that it is interactive, and you can grab the nodes that represent the stacks and move them around as needed. I also experimented with a Flare Chart for a different type of view. However, the Flare Chart lacked a way to show the interconnections that I wanted to be able to see.
21 | 
22 | The code for the visuals can be found here (https://github.com/1Strategy/cf-cross-stack-mapping). The interactive visualizations can be seen here (https://1strategy.github.io/cf-cross-stack-mapping/), and below are example screen shots of each of the visualizations.
23 | 
24 | **Sankey Diagram Example:**
25 | ![Image](https://github.com/1Strategy/cf-cross-stack-mapping/raw/master/ScreenShotSankey.png)
26 | 
27 | **Flare Chart Example:**
28 | ![Image](https://github.com/1Strategy/cf-cross-stack-mapping/raw/master/ScreenShotFlare.png)
29 | 
30 | ### Summary and Conclusions.
31 | In the end a customer can see the complete relationships between all the stacks in their environment. So far the most useful feature has been to see when there are exports that are not being used and removing them to make the cross stack references more efficient and easier to maintain. The other thing this may be useful for is to see an overall picture of your AWS infrastructure and how things are related within it.
32 | 
33 | **Some future considerations:**
34 | Better color coding of the nodes. For instance, one color to denote stacks and another color to denote exports.
35 | 
36 | A drop down to select only the stacks that you want to see. This would be especially helpful when the number of stacks deployed gets very large.
37 | 


--------------------------------------------------------------------------------
/create-exports.json:
--------------------------------------------------------------------------------
 1 | {
 2 |   "AWSTemplateFormatVersion": "2010-09-09",
 3 |   "Description": "AWS CloudFormation Template for creating sample exports to test the visualization",
 4 |   "Parameters": {
 5 |     "VPCID": {
 6 |       "Description": "Enter the VPC ID where the dummy SecurityGroup will be created.",
 7 |       "Type": "String",
 8 |       "Default": "vpc-********"
 9 |     }
10 |   },
11 |   "Resources": {
12 |     "DummySecurityGroup" : {
13 |       "Type": "AWS::EC2::SecurityGroup",
14 |       "Properties": {
15 |         "GroupDescription": "Dummy security group which allowed the creation of a CloudFormation stack that exports values.",
16 |         "VpcId": {"Ref": "VPCID"}
17 |       }
18 |     }
19 |   },
20 |   "Outputs": {
21 |     "Subnet1": {
22 |       "Description": "First example Subnet",
23 |       "Value": "subnet-abcd1234",
24 |       "Export": {"Name":"CreateExports-Subnet1"}
25 |     },
26 |     "Subnet2": {
27 |       "Description": "Second example Subnet",
28 |       "Value": "subnet-abcd5678",
29 |       "Export": {"Name":"CreateExports-Subnet2"}
30 |     },
31 |     "DMZSubnet1": {
32 |       "Description": "First example DMZSubnet",
33 |       "Value": "subnet-abcd4321",
34 |       "Export": {"Name":"CreateExports-DMZSubnet1"}
35 |     },
36 |     "DMZSubnet2": {
37 |       "Description": "Second example DMZSubnet",
38 |       "Value": "subnet-abcd8765",
39 |       "Export": {"Name":"CreateExports-DMZSubnet2"}
40 |     },
41 |     "CoreVpc": {
42 |       "Description": "Core Network VPC ID",
43 |       "Value": {"Ref": "VPCID"},
44 |       "Export": {"Name":"CreateExports-CoreVpc"}
45 |     },
46 |     "DMZVpc": {
47 |       "Description": "DMZ Network VPC ID",
48 |       "Value": {"Ref": "VPCID"},
49 |       "Export": {"Name":"CreateExports-DMZVpc"}
50 |     },
51 |     "CoreServerSecurityGroup": {
52 |       "Description": "Core Server Security Group",
53 |       "Value": "sg-abcd1234",
54 |       "Export" : { "Name" : "CreateExports-CoreServerSecurityGroup" }
55 |     },
56 |     "DMZServerSecurityGroup": {
57 |       "Description": "DMZ Server Security Group",
58 |       "Value": "sg-abcd5678",
59 |       "Export" : { "Name" : "CreateExports-DMZServerSecurityGroup" }
60 |     },
61 |     "S3Endpoint": {
62 |       "Description":"S3 VPC Endpoint",
63 |       "Value": "vpce-abcd1234",
64 |       "Export":{"Name":"CreateExports-S3Endpoint"}
65 |     }
66 |   }
67 | }
68 | 


--------------------------------------------------------------------------------
/example-imports.json:
--------------------------------------------------------------------------------
 1 | {
 2 |   "AWSTemplateFormatVersion": "2010-09-09",
 3 |   "Description": "Example Imports from Create Exports Stack",
 4 |   "Resources": {
 5 |     "TestingCoreSecurityGroup" : {
 6 |       "Type": "AWS::EC2::SecurityGroup",
 7 |       "Properties": {
 8 |         "GroupDescription": "Testing Core Security Group",
 9 |         "VpcId": { "Fn::ImportValue" : "CreateExports-CoreVpc" }
10 |       }
11 |     },
12 |     "TestingDMZSecurityGroup" : {
13 |       "Type": "AWS::EC2::SecurityGroup",
14 |       "Properties": {
15 |         "GroupDescription": "Testing DMZ Security Group",
16 |         "VpcId": { "Fn::ImportValue" : "CreateExports-DMZVpc" }
17 |       }
18 |     }
19 |   },
20 |   "Outputs": {
21 |     "TestingSGOutput": {
22 |       "Description": "Testing SecurityGroup Output",
23 |       "Value": {"Ref":"TestingCoreSecurityGroup"},
24 |       "Export": {"Name":"ExampleImports-SecurityGroup"}
25 |     }
26 |   }
27 | }
28 | 


--------------------------------------------------------------------------------
/example-imports1.json:
--------------------------------------------------------------------------------
 1 | {
 2 |   "AWSTemplateFormatVersion": "2010-09-09",
 3 |   "Description": "Example Imports from Create Exports Stack",
 4 |   "Resources": {
 5 |     "TestingCoreSecurityGroup1" : {
 6 |       "Type": "AWS::EC2::SecurityGroup",
 7 |       "Properties": {
 8 |         "GroupDescription": "Testing Core Security Group 1",
 9 |         "VpcId": { "Fn::ImportValue" : "CreateExports-CoreVpc" }
10 |       }
11 |     },
12 |     "TestingDMZSecurityGroup1" : {
13 |       "Type": "AWS::EC2::SecurityGroup",
14 |       "Properties": {
15 |         "GroupDescription": "Testing DMZ Security Group 1",
16 |         "VpcId": { "Fn::ImportValue" : "CreateExports-DMZVpc" }
17 |       }
18 |     }
19 |   }
20 | }
21 | 


--------------------------------------------------------------------------------
/example-imports2.json:
--------------------------------------------------------------------------------
 1 | {
 2 |   "AWSTemplateFormatVersion": "2010-09-09",
 3 |   "Description": "Example Imports from Create Exports Stack",
 4 |   "Resources": {
 5 |     "TestingCoreSecurityGroup2" : {
 6 |       "Type": "AWS::EC2::SecurityGroup",
 7 |       "Properties": {
 8 |         "GroupDescription": "Testing Core Security Group 2",
 9 |         "VpcId": { "Fn::ImportValue" : "CreateExports-CoreVpc" }
10 |       }
11 |     },
12 |     "TestingDMZSecurityGroup2" : {
13 |       "Type": "AWS::EC2::SecurityGroup",
14 |       "Properties": {
15 |         "GroupDescription": "Testing DMZ Security Group 2",
16 |         "VpcId": { "Fn::ImportValue" : "CreateExports-DMZVpc" }
17 |       }
18 |     }
19 |   }
20 | }
21 | 


--------------------------------------------------------------------------------
/flare.csv:
--------------------------------------------------------------------------------
 1 | id,value
 2 | Templates,
 3 | Templates.Create-Exports-Stack.CreateExports-CoreServerSecurityGroup,
 4 | Templates.Create-Exports-Stack,
 5 | Templates.Create-Exports-Stack.CreateExports-CoreVpc,
 6 | Templates.Create-Exports-Stack.CreateExports-CoreVpc.Example-Imports1,
 7 | Templates.Create-Exports-Stack.CreateExports-CoreVpc.Example-Imports0,
 8 | Templates.Create-Exports-Stack.CreateExports-CoreVpc.Example-Imports2,
 9 | Templates.Create-Exports-Stack.CreateExports-CoreVpc.Example-Imports-New,
10 | Templates.Create-Exports-Stack.CreateExports-DMZServerSecurityGroup,
11 | Templates.Create-Exports-Stack.CreateExports-DMZSubnet1,
12 | Templates.Create-Exports-Stack.CreateExports-DMZSubnet1.Example-Imports-New,
13 | Templates.Create-Exports-Stack.CreateExports-DMZSubnet2,
14 | Templates.Create-Exports-Stack.CreateExports-DMZVpc,
15 | Templates.Create-Exports-Stack.CreateExports-DMZVpc.Example-Imports1,
16 | Templates.Create-Exports-Stack.CreateExports-DMZVpc.Example-Imports0,
17 | Templates.Create-Exports-Stack.CreateExports-DMZVpc.Example-Imports2,
18 | Templates.Create-Exports-Stack.CreateExports-DMZVpc.Example-Imports-New,
19 | Templates.Create-Exports-Stack.CreateExports-S3Endpoint,
20 | Templates.Create-Exports-Stack.CreateExports-Subnet1,
21 | Templates.Create-Exports-Stack.CreateExports-Subnet1.Example-Imports-New,
22 | Templates.Create-Exports-Stack.CreateExports-Subnet2,
23 | Templates.Create-Exports-Stack.CreateExports-Subnet2.Example-Imports-New,
24 | Templates.Example-Imports0.ExampleImports-SecurityGroup,
25 | Templates.Example-Imports0,
26 | Templates.main-vpc.main-vpc-AZAPrivate1SubnetId,
27 | Templates.main-vpc,
28 | Templates.main-vpc.main-vpc-AZAPublic1SubnetId,
29 | Templates.main-vpc.main-vpc-AZBPrivate1SubnetId,
30 | Templates.main-vpc.main-vpc-AZBPublic1SubnetId,
31 | Templates.main-vpc.main-vpc-AZCPrivate1SubnetId,
32 | Templates.main-vpc.main-vpc-AZCPublic1SubnetId,
33 | Templates.main-vpc.main-vpc-VPCID,
34 | 


--------------------------------------------------------------------------------
/get_data_flare.py:
--------------------------------------------------------------------------------
 1 | #!/usr/bin/env python
 2 | import boto3
 3 | import re
 4 | import os
 5 | 
 6 | inuser = boto3.session.Session(profile_name='default')
 7 | client = inuser.client('cloudformation')
 8 | response = client.list_exports()
 9 | f1=open('temp.in', 'w+')
10 | l = response.get('Exports', {})
11 | print >>f1, 'id,value'
12 | print >>f1, 'Templates,'
13 | for x in l:
14 |     s = x['ExportingStackId']
15 |     t = x["Name"]
16 |     print >>f1, '.'.join(['Templates',re.search('.*/(.*)/.*', s).group(1),t]) + ','
17 |     j = re.search('.*/(.*)/.*', s).group(1)
18 |     print >>f1, '.'.join(['Templates',j]) + ','
19 |     try:
20 |         if client.list_imports(ExportName=t):
21 |             cl = client.list_imports(ExportName=t)
22 |             im = cl["Imports"]
23 |             for x in im:
24 |                 print >>f1, '.'.join(['Templates',j,t,x]) + ','
25 |     except:
26 |         pass
27 | #for x in l:
28 | #    s = x['ExportingStackId']
29 | #    print '.'.join(['Templates',re.search('.*/(.*)/.*', s).group(1)]) + ','
30 | f1.close()
31 | 
32 | lines_seen = set()
33 | outfile = open('flare.csv', "w")
34 | for line in open('temp.in', "r"):
35 |    if line not in lines_seen: # not a duplicate
36 |        outfile.write(line)
37 |        lines_seen.add(line)
38 | outfile.close()
39 | 
40 | os.remove('temp.in')
41 | 


--------------------------------------------------------------------------------
/get_data_sankey.py:
--------------------------------------------------------------------------------
 1 | #!/usr/bin/env python
 2 | import boto3
 3 | import re
 4 | 
 5 | f1=open('sourcedata-output.json', 'w+')
 6 | inuser = boto3.session.Session(profile_name='default')
 7 | client = inuser.client('cloudformation')
 8 | response = client.list_exports()
 9 | l = response.get('Exports', {})
10 | print >>f1, '{ "links": ['
11 | output1 = []
12 | for x in l:
13 |     s = x['ExportingStackId']
14 |     t = x["Name"]
15 |     o1 = str('{"source":"' + re.search('.*/(.*)/.*', s).group(1) + '","target":"' + t + '","value":"1.0"}')
16 |     output1.append(o1)
17 |     try:
18 |         if client.list_imports(ExportName=t):
19 |             cl = client.list_imports(ExportName=t)
20 |             im = cl["Imports"]
21 |             for x in im:
22 |                 o2 = str('{"source":"' + t + '","target":"' + x + '","value":"1.0"}')
23 |                 output1.append(o2)
24 |     except:
25 |         pass
26 | print >>f1, ",\n".join(output1)
27 | print >>f1, '], "nodes": ['
28 | output2 = []
29 | for x in l:
30 |     s = x['ExportingStackId']
31 |     o1 = str('{"name":"' + re.search('.*/(.*)/.*', s).group(1) + '"}')
32 |     output2.append(o1)
33 | for x in l:
34 |     o2 = str('{"name":"' + x["Name"] + '"}')
35 |     output2.append(o2)
36 |     t = x["Name"]
37 |     try:
38 |         if client.list_imports(ExportName=t):
39 |             cl = client.list_imports(ExportName=t)
40 |             im = cl["Imports"]
41 |             for z in im:
42 |                 o3 = str('{"name":"' + z + '"}')
43 |                 output2.append(o3)
44 |     except:
45 |         pass
46 | print >>f1, ",\n".join(list(set(output2)))
47 | print >>f1, ']}'
48 | f1.close()
49 | 


--------------------------------------------------------------------------------
/index-flare.html:
--------------------------------------------------------------------------------
 1 | <!DOCTYPE html>
 2 | <meta charset="utf-8">
 3 | <title>FLARE Chart Experiment</title>
 4 | <style>
 5 | 
 6 | .node circle {
 7 |   fill: #999;
 8 | }
 9 | 
10 | .node text {
11 |   font: 10px sans-serif;
12 | }
13 | 
14 | .node--internal circle {
15 |   fill: #555;
16 | }
17 | 
18 | .node--internal text {
19 |   text-shadow: 0 1px 0 #fff, 0 -1px 0 #fff, 1px 0 0 #fff, -1px 0 0 #fff;
20 | }
21 | 
22 | .link {
23 |   fill: none;
24 |   stroke: #555;
25 |   stroke-opacity: 0.4;
26 |   stroke-width: 1.5px;
27 | }
28 | 
29 | </style>
30 | <svg width="1200" height="800"></svg>
31 | <script src="https://d3js.org/d3.v4.min.js"></script>
32 | <script>
33 | 
34 | var svg = d3.select("svg"),
35 |     width = +svg.attr("width"),
36 |     height = +svg.attr("height"),
37 |     g = svg.append("g").attr("transform", "translate(40,0)");
38 | 
39 | var tree = d3.cluster()
40 |     .size([height, width - 320]);
41 | 
42 | var stratify = d3.stratify()
43 |     .parentId(function(d) { return d.id.substring(0, d.id.lastIndexOf(".")); });
44 | 
45 | d3.csv("flare.csv", function(error, data) {
46 |   if (error) throw error;
47 | 
48 |   var root = stratify(data)
49 |       .sort(function(a, b) { return (a.height - b.height) || a.id.localeCompare(b.id); });
50 | 
51 |   tree(root);
52 | 
53 |   var link = g.selectAll(".link")
54 |       .data(root.descendants().slice(1))
55 |     .enter().append("path")
56 |       .attr("class", "link")
57 |       .attr("d", function(d) {
58 |         return "M" + d.y + "," + d.x
59 |             + "C" + (d.parent.y + 100) + "," + d.x
60 |             + " " + (d.parent.y + 100) + "," + d.parent.x
61 |             + " " + d.parent.y + "," + d.parent.x;
62 |       });
63 | 
64 |   var node = g.selectAll(".node")
65 |       .data(root.descendants())
66 |     .enter().append("g")
67 |       .attr("class", function(d) { return "node" + (d.children ? " node--internal" : " node--leaf"); })
68 |       .attr("transform", function(d) { return "translate(" + d.y + "," + d.x + ")"; })
69 | 
70 |   node.append("circle")
71 |       .attr("r", 2.5);
72 | 
73 |   node.append("text")
74 |       .attr("dy", 3)
75 |       .attr("x", function(d) { return d.children ? -8 : 8; })
76 |       .style("text-anchor", function(d) { return d.children ? "end" : "start"; })
77 |       .text(function(d) { return d.id.substring(d.id.lastIndexOf(".") + 1); });
78 | });
79 | 
80 | </script>
81 | 


--------------------------------------------------------------------------------
/index-sankey.html:
--------------------------------------------------------------------------------
  1 | <!DOCTYPE html>
  2 | <meta charset="utf-8">
  3 | <title>SANKEY Chart Experiment</title>
  4 | <style>
  5 | 
  6 | .node rect {
  7 |   cursor: move;
  8 |   fill-opacity: .9;
  9 |   shape-rendering: crispEdges;
 10 | }
 11 | 
 12 | .node text {
 13 |   pointer-events: none;
 14 |   text-shadow: 0 1px 0 #fff;
 15 | }
 16 | 
 17 | .link {
 18 |   fill: none;
 19 |   stroke: #000;
 20 |   stroke-opacity: .2;
 21 | }
 22 | 
 23 | .link:hover {
 24 |   stroke-opacity: .5;
 25 | }
 26 | 
 27 | </style>
 28 | <body>
 29 | 
 30 | <p id="chart">
 31 | 
 32 | <script src="https://d3js.org/d3.v3.js"></script>
 33 | <script src="sankey.js"></script>
 34 | <script>
 35 | 
 36 | var units = "Widgets";
 37 | 
 38 | var margin = {top: 10, right: 10, bottom: 10, left: 10},
 39 |     width = 1200 - margin.left - margin.right,
 40 |     height = 800 - margin.top - margin.bottom;
 41 | 
 42 | var formatNumber = d3.format(",.0f"),    // zero decimal places
 43 |     format = function(d) { return formatNumber(d) + " " + units; },
 44 |     color = d3.scale.category20();
 45 | 
 46 | // append the svg canvas to the page
 47 | var svg = d3.select("#chart").append("svg")
 48 |     .attr("width", width + margin.left + margin.right)
 49 |     .attr("height", height + margin.top + margin.bottom)
 50 |   .append("g")
 51 |     .attr("transform",
 52 |           "translate(" + margin.left + "," + margin.top + ")");
 53 | 
 54 | // Set the sankey diagram properties
 55 | var sankey = d3.sankey()
 56 |     .nodeWidth(36)
 57 |     .nodePadding(10)
 58 |     .size([width, height]);
 59 | 
 60 | var path = sankey.link();
 61 | 
 62 | // load the data
 63 | d3.json("sourcedata-output.json", function(error, graph) {
 64 | //d3.json("temp2.json", function(error, graph) {
 65 | 
 66 |     var nodeMap = {};
 67 |     graph.nodes.forEach(function(x) { nodeMap[x.name] = x; });
 68 |     graph.links = graph.links.map(function(x) {
 69 |       return {
 70 |         source: nodeMap[x.source],
 71 |         target: nodeMap[x.target],
 72 |         value: x.value
 73 |       };
 74 |     });
 75 | 
 76 |   sankey
 77 |       .nodes(graph.nodes)
 78 |       .links(graph.links)
 79 |       .layout(32);
 80 | 
 81 | // add in the links
 82 |   var link = svg.append("g").selectAll(".link")
 83 |       .data(graph.links)
 84 |     .enter().append("path")
 85 |       .attr("class", "link")
 86 |       .attr("d", path)
 87 |       .style("stroke-width", function(d) { return Math.max(1, d.dy); })
 88 |       .sort(function(a, b) { return b.dy - a.dy; });
 89 | 
 90 | // add the link titles
 91 |   link.append("title")
 92 |         .text(function(d) {
 93 |       	return d.source.name + " → " +
 94 |                 d.target.name + "\n" + format(d.value); });
 95 | 
 96 | // add in the nodes
 97 |   var node = svg.append("g").selectAll(".node")
 98 |       .data(graph.nodes)
 99 |     .enter().append("g")
100 |       .attr("class", "node")
101 |       .attr("transform", function(d) {
102 | 		  return "translate(" + d.x + "," + d.y + ")"; })
103 |     .call(d3.behavior.drag()
104 |       .origin(function(d) { return d; })
105 |       .on("dragstart", function() {
106 | 		  this.parentNode.appendChild(this); })
107 |       .on("drag", dragmove));
108 | 
109 | // add the rectangles for the nodes
110 |   node.append("rect")
111 |       .attr("height", function(d) { return d.dy; })
112 |       .attr("width", sankey.nodeWidth())
113 |       .style("fill", function(d) {
114 | 		  return d.color = color(d.name.replace(/ .*/, "")); })
115 |       .style("stroke", function(d) {
116 | 		  return d3.rgb(d.color).darker(2); })
117 |     .append("title")
118 |       .text(function(d) {
119 | 		  return d.name + "\n" + format(d.value); });
120 | 
121 | // add in the title for the nodes
122 |   node.append("text")
123 |       .attr("x", -6)
124 |       .attr("y", function(d) { return d.dy / 2; })
125 |       .attr("dy", ".35em")
126 |       .attr("text-anchor", "end")
127 |       .attr("transform", null)
128 |       .text(function(d) { return d.name; })
129 |     .filter(function(d) { return d.x < width / 2; })
130 |       .attr("x", 6 + sankey.nodeWidth())
131 |       .attr("text-anchor", "start");
132 | 
133 | // the function for moving the nodes
134 |   function dragmove(d) {
135 |     d3.select(this).attr("transform",
136 |         "translate(" + (
137 |         	   d.x = Math.max(0, Math.min(width - d.dx, d3.event.x))
138 |         	) + "," + (
139 |                    d.y = Math.max(0, Math.min(height - d.dy, d3.event.y))
140 |             ) + ")");
141 |     sankey.relayout();
142 |     link.attr("d", path);
143 |   }
144 | });
145 | 
146 | </script>
147 | 
148 | </body>
149 | </html>
150 | 


--------------------------------------------------------------------------------
/index.html:
--------------------------------------------------------------------------------
  1 | <!DOCTYPE html>
  2 | <meta charset="utf-8">
  3 | <title>SANKEY Chart Experiment</title>
  4 | <style>
  5 | 
  6 | .node rect {
  7 |   cursor: move;
  8 |   fill-opacity: .9;
  9 |   shape-rendering: crispEdges;
 10 | }
 11 | 
 12 | .node text {
 13 |   pointer-events: none;
 14 |   text-shadow: 0 1px 0 #fff;
 15 | }
 16 | 
 17 | .link {
 18 |   fill: none;
 19 |   stroke: #000;
 20 |   stroke-opacity: .2;
 21 | }
 22 | 
 23 | .link:hover {
 24 |   stroke-opacity: .5;
 25 | }
 26 | 
 27 | </style>
 28 | <body>
 29 | 
 30 | <p id="chart">
 31 | 
 32 | <script src="https://d3js.org/d3.v3.js"></script>
 33 | <script src="sankey.js"></script>
 34 | <script>
 35 | 
 36 | var units = "Widgets";
 37 | 
 38 | var margin = {top: 10, right: 10, bottom: 10, left: 10},
 39 |     width = 1200 - margin.left - margin.right,
 40 |     height = 800 - margin.top - margin.bottom;
 41 | 
 42 | var formatNumber = d3.format(",.0f"),    // zero decimal places
 43 |     format = function(d) { return formatNumber(d) + " " + units; },
 44 |     color = d3.scale.category20();
 45 | 
 46 | // append the svg canvas to the page
 47 | var svg = d3.select("#chart").append("svg")
 48 |     .attr("width", width + margin.left + margin.right)
 49 |     .attr("height", height + margin.top + margin.bottom)
 50 |   .append("g")
 51 |     .attr("transform",
 52 |           "translate(" + margin.left + "," + margin.top + ")");
 53 | 
 54 | // Set the sankey diagram properties
 55 | var sankey = d3.sankey()
 56 |     .nodeWidth(36)
 57 |     .nodePadding(10)
 58 |     .size([width, height]);
 59 | 
 60 | var path = sankey.link();
 61 | 
 62 | // load the data
 63 | d3.json("sourcedata-output.json", function(error, graph) {
 64 | //d3.json("temp2.json", function(error, graph) {
 65 | 
 66 |     var nodeMap = {};
 67 |     graph.nodes.forEach(function(x) { nodeMap[x.name] = x; });
 68 |     graph.links = graph.links.map(function(x) {
 69 |       return {
 70 |         source: nodeMap[x.source],
 71 |         target: nodeMap[x.target],
 72 |         value: x.value
 73 |       };
 74 |     });
 75 | 
 76 |   sankey
 77 |       .nodes(graph.nodes)
 78 |       .links(graph.links)
 79 |       .layout(32);
 80 | 
 81 | // add in the links
 82 |   var link = svg.append("g").selectAll(".link")
 83 |       .data(graph.links)
 84 |     .enter().append("path")
 85 |       .attr("class", "link")
 86 |       .attr("d", path)
 87 |       .style("stroke-width", function(d) { return Math.max(1, d.dy); })
 88 |       .sort(function(a, b) { return b.dy - a.dy; });
 89 | 
 90 | // add the link titles
 91 |   link.append("title")
 92 |         .text(function(d) {
 93 |       	return d.source.name + " → " +
 94 |                 d.target.name + "\n" + format(d.value); });
 95 | 
 96 | // add in the nodes
 97 |   var node = svg.append("g").selectAll(".node")
 98 |       .data(graph.nodes)
 99 |     .enter().append("g")
100 |       .attr("class", "node")
101 |       .attr("transform", function(d) {
102 | 		  return "translate(" + d.x + "," + d.y + ")"; })
103 |     .call(d3.behavior.drag()
104 |       .origin(function(d) { return d; })
105 |       .on("dragstart", function() {
106 | 		  this.parentNode.appendChild(this); })
107 |       .on("drag", dragmove));
108 | 
109 | // add the rectangles for the nodes
110 |   node.append("rect")
111 |       .attr("height", function(d) { return d.dy; })
112 |       .attr("width", sankey.nodeWidth())
113 |       .style("fill", function(d) {
114 | 		  return d.color = color(d.name.replace(/ .*/, "")); })
115 |       .style("stroke", function(d) {
116 | 		  return d3.rgb(d.color).darker(2); })
117 |     .append("title")
118 |       .text(function(d) {
119 | 		  return d.name + "\n" + format(d.value); });
120 | 
121 | // add in the title for the nodes
122 |   node.append("text")
123 |       .attr("x", -6)
124 |       .attr("y", function(d) { return d.dy / 2; })
125 |       .attr("dy", ".35em")
126 |       .attr("text-anchor", "end")
127 |       .attr("transform", null)
128 |       .text(function(d) { return d.name; })
129 |     .filter(function(d) { return d.x < width / 2; })
130 |       .attr("x", 6 + sankey.nodeWidth())
131 |       .attr("text-anchor", "start");
132 | 
133 | // the function for moving the nodes
134 |   function dragmove(d) {
135 |     d3.select(this).attr("transform",
136 |         "translate(" + (
137 |         	   d.x = Math.max(0, Math.min(width - d.dx, d3.event.x))
138 |         	) + "," + (
139 |                    d.y = Math.max(0, Math.min(height - d.dy, d3.event.y))
140 |             ) + ")");
141 |     sankey.relayout();
142 |     link.attr("d", path);
143 |   }
144 | });
145 | 
146 | </script>
147 | 
148 | </body>
149 | </html>
150 | 


--------------------------------------------------------------------------------
/sankey.js:
--------------------------------------------------------------------------------
  1 | d3.sankey = function() {
  2 |   var sankey = {},
  3 |       nodeWidth = 24,
  4 |       nodePadding = 8,
  5 |       size = [1, 1],
  6 |       nodes = [],
  7 |       links = [];
  8 | 
  9 |   sankey.nodeWidth = function(_) {
 10 |     if (!arguments.length) return nodeWidth;
 11 |     nodeWidth = +_;
 12 |     return sankey;
 13 |   };
 14 | 
 15 |   sankey.nodePadding = function(_) {
 16 |     if (!arguments.length) return nodePadding;
 17 |     nodePadding = +_;
 18 |     return sankey;
 19 |   };
 20 | 
 21 |   sankey.nodes = function(_) {
 22 |     if (!arguments.length) return nodes;
 23 |     nodes = _;
 24 |     return sankey;
 25 |   };
 26 | 
 27 |   sankey.links = function(_) {
 28 |     if (!arguments.length) return links;
 29 |     links = _;
 30 |     return sankey;
 31 |   };
 32 | 
 33 |   sankey.size = function(_) {
 34 |     if (!arguments.length) return size;
 35 |     size = _;
 36 |     return sankey;
 37 |   };
 38 | 
 39 |   sankey.layout = function(iterations) {
 40 |     computeNodeLinks();
 41 |     computeNodeValues();
 42 |     computeNodeBreadths();
 43 |     computeNodeDepths(iterations);
 44 |     computeLinkDepths();
 45 |     return sankey;
 46 |   };
 47 | 
 48 |   sankey.relayout = function() {
 49 |     computeLinkDepths();
 50 |     return sankey;
 51 |   };
 52 | 
 53 |   sankey.link = function() {
 54 |     var curvature = .5;
 55 | 
 56 |     function link(d) {
 57 |       var x0 = d.source.x + d.source.dx,
 58 |           x1 = d.target.x,
 59 |           xi = d3.interpolateNumber(x0, x1),
 60 |           x2 = xi(curvature),
 61 |           x3 = xi(1 - curvature),
 62 |           y0 = d.source.y + d.sy + d.dy / 2,
 63 |           y1 = d.target.y + d.ty + d.dy / 2;
 64 |       return "M" + x0 + "," + y0
 65 |            + "C" + x2 + "," + y0
 66 |            + " " + x3 + "," + y1
 67 |            + " " + x1 + "," + y1;
 68 |     }
 69 | 
 70 |     link.curvature = function(_) {
 71 |       if (!arguments.length) return curvature;
 72 |       curvature = +_;
 73 |       return link;
 74 |     };
 75 | 
 76 |     return link;
 77 |   };
 78 | 
 79 |   // Populate the sourceLinks and targetLinks for each node.
 80 |   // Also, if the source and target are not objects, assume they are indices.
 81 |   function computeNodeLinks() {
 82 |     nodes.forEach(function(node) {
 83 |       node.sourceLinks = [];
 84 |       node.targetLinks = [];
 85 |     });
 86 |     links.forEach(function(link) {
 87 |       var source = link.source,
 88 |           target = link.target;
 89 |       if (typeof source === "number") source = link.source = nodes[link.source];
 90 |       if (typeof target === "number") target = link.target = nodes[link.target];
 91 |       source.sourceLinks.push(link);
 92 |       target.targetLinks.push(link);
 93 |     });
 94 |   }
 95 | 
 96 |   // Compute the value (size) of each node by summing the associated links.
 97 |   function computeNodeValues() {
 98 |     nodes.forEach(function(node) {
 99 |       node.value = Math.max(
100 |         d3.sum(node.sourceLinks, value),
101 |         d3.sum(node.targetLinks, value)
102 |       );
103 |     });
104 |   }
105 | 
106 |   // Iteratively assign the breadth (x-position) for each node.
107 |   // Nodes are assigned the maximum breadth of incoming neighbors plus one;
108 |   // nodes with no incoming links are assigned breadth zero, while
109 |   // nodes with no outgoing links are assigned the maximum breadth.
110 |   function computeNodeBreadths() {
111 |     var remainingNodes = nodes,
112 |         nextNodes,
113 |         x = 0;
114 | 
115 |     while (remainingNodes.length) {
116 |       nextNodes = [];
117 |       remainingNodes.forEach(function(node) {
118 |         node.x = x;
119 |         node.dx = nodeWidth;
120 |         node.sourceLinks.forEach(function(link) {
121 |           nextNodes.push(link.target);
122 |         });
123 |       });
124 |       remainingNodes = nextNodes;
125 |       ++x;
126 |     }
127 | 
128 |     //
129 |     moveSinksRight(x);
130 |     scaleNodeBreadths((size[0] - nodeWidth) / (x - 1));
131 |   }
132 | 
133 |   function moveSourcesRight() {
134 |     nodes.forEach(function(node) {
135 |       if (!node.targetLinks.length) {
136 |         node.x = d3.min(node.sourceLinks, function(d) { return d.target.x; }) - 1;
137 |       }
138 |     });
139 |   }
140 | 
141 |   function moveSinksRight(x) {
142 |     nodes.forEach(function(node) {
143 |       if (!node.sourceLinks.length) {
144 |         node.x = x - 1;
145 |       }
146 |     });
147 |   }
148 | 
149 |   function scaleNodeBreadths(kx) {
150 |     nodes.forEach(function(node) {
151 |       node.x *= kx;
152 |     });
153 |   }
154 | 
155 |   function computeNodeDepths(iterations) {
156 |     var nodesByBreadth = d3.nest()
157 |         .key(function(d) { return d.x; })
158 |         .sortKeys(d3.ascending)
159 |         .entries(nodes)
160 |         .map(function(d) { return d.values; });
161 | 
162 |     //
163 |     initializeNodeDepth();
164 |     resolveCollisions();
165 |     for (var alpha = 1; iterations > 0; --iterations) {
166 |       relaxRightToLeft(alpha *= .99);
167 |       resolveCollisions();
168 |       relaxLeftToRight(alpha);
169 |       resolveCollisions();
170 |     }
171 | 
172 |     function initializeNodeDepth() {
173 |       var ky = d3.min(nodesByBreadth, function(nodes) {
174 |         return (size[1] - (nodes.length - 1) * nodePadding) / d3.sum(nodes, value);
175 |       });
176 | 
177 |       nodesByBreadth.forEach(function(nodes) {
178 |         nodes.forEach(function(node, i) {
179 |           node.y = i;
180 |           node.dy = node.value * ky;
181 |         });
182 |       });
183 | 
184 |       links.forEach(function(link) {
185 |         link.dy = link.value * ky;
186 |       });
187 |     }
188 | 
189 |     function relaxLeftToRight(alpha) {
190 |       nodesByBreadth.forEach(function(nodes, breadth) {
191 |         nodes.forEach(function(node) {
192 |           if (node.targetLinks.length) {
193 |             var y = d3.sum(node.targetLinks, weightedSource) / d3.sum(node.targetLinks, value);
194 |             node.y += (y - center(node)) * alpha;
195 |           }
196 |         });
197 |       });
198 | 
199 |       function weightedSource(link) {
200 |         return center(link.source) * link.value;
201 |       }
202 |     }
203 | 
204 |     function relaxRightToLeft(alpha) {
205 |       nodesByBreadth.slice().reverse().forEach(function(nodes) {
206 |         nodes.forEach(function(node) {
207 |           if (node.sourceLinks.length) {
208 |             var y = d3.sum(node.sourceLinks, weightedTarget) / d3.sum(node.sourceLinks, value);
209 |             node.y += (y - center(node)) * alpha;
210 |           }
211 |         });
212 |       });
213 | 
214 |       function weightedTarget(link) {
215 |         return center(link.target) * link.value;
216 |       }
217 |     }
218 | 
219 |     function resolveCollisions() {
220 |       nodesByBreadth.forEach(function(nodes) {
221 |         var node,
222 |             dy,
223 |             y0 = 0,
224 |             n = nodes.length,
225 |             i;
226 | 
227 |         // Push any overlapping nodes down.
228 |         nodes.sort(ascendingDepth);
229 |         for (i = 0; i < n; ++i) {
230 |           node = nodes[i];
231 |           dy = y0 - node.y;
232 |           if (dy > 0) node.y += dy;
233 |           y0 = node.y + node.dy + nodePadding;
234 |         }
235 | 
236 |         // If the bottommost node goes outside the bounds, push it back up.
237 |         dy = y0 - nodePadding - size[1];
238 |         if (dy > 0) {
239 |           y0 = node.y -= dy;
240 | 
241 |           // Push any overlapping nodes back up.
242 |           for (i = n - 2; i >= 0; --i) {
243 |             node = nodes[i];
244 |             dy = node.y + node.dy + nodePadding - y0;
245 |             if (dy > 0) node.y -= dy;
246 |             y0 = node.y;
247 |           }
248 |         }
249 |       });
250 |     }
251 | 
252 |     function ascendingDepth(a, b) {
253 |       return a.y - b.y;
254 |     }
255 |   }
256 | 
257 |   function computeLinkDepths() {
258 |     nodes.forEach(function(node) {
259 |       node.sourceLinks.sort(ascendingTargetDepth);
260 |       node.targetLinks.sort(ascendingSourceDepth);
261 |     });
262 |     nodes.forEach(function(node) {
263 |       var sy = 0, ty = 0;
264 |       node.sourceLinks.forEach(function(link) {
265 |         link.sy = sy;
266 |         sy += link.dy;
267 |       });
268 |       node.targetLinks.forEach(function(link) {
269 |         link.ty = ty;
270 |         ty += link.dy;
271 |       });
272 |     });
273 | 
274 |     function ascendingSourceDepth(a, b) {
275 |       return a.source.y - b.source.y;
276 |     }
277 | 
278 |     function ascendingTargetDepth(a, b) {
279 |       return a.target.y - b.target.y;
280 |     }
281 |   }
282 | 
283 |   function center(node) {
284 |     return node.y + node.dy / 2;
285 |   }
286 | 
287 |   function value(link) {
288 |     return link.value;
289 |   }
290 | 
291 |   return sankey;
292 | };
293 | 


--------------------------------------------------------------------------------
/sourcedata-output.json:
--------------------------------------------------------------------------------
 1 | { "links": [
 2 | {"source":"Create-Exports-Stack","target":"CreateExports-CoreServerSecurityGroup","value":"1.0"},
 3 | {"source":"Create-Exports-Stack","target":"CreateExports-CoreVpc","value":"4.0"},
 4 | {"source":"CreateExports-CoreVpc","target":"Example-Imports1","value":"1.0"},
 5 | {"source":"CreateExports-CoreVpc","target":"Example-Imports0","value":"1.0"},
 6 | {"source":"CreateExports-CoreVpc","target":"Example-Imports2","value":"1.0"},
 7 | {"source":"CreateExports-CoreVpc","target":"Example-Imports-New","value":"1.0"},
 8 | {"source":"Create-Exports-Stack","target":"CreateExports-DMZServerSecurityGroup","value":"1.0"},
 9 | {"source":"Create-Exports-Stack","target":"CreateExports-DMZSubnet1","value":"1.0"},
10 | {"source":"CreateExports-DMZSubnet1","target":"Example-Imports-New","value":"1.0"},
11 | {"source":"Create-Exports-Stack","target":"CreateExports-DMZSubnet2","value":"1.0"},
12 | {"source":"Create-Exports-Stack","target":"CreateExports-DMZVpc","value":"4.0"},
13 | {"source":"CreateExports-DMZVpc","target":"Example-Imports1","value":"1.0"},
14 | {"source":"CreateExports-DMZVpc","target":"Example-Imports0","value":"1.0"},
15 | {"source":"CreateExports-DMZVpc","target":"Example-Imports2","value":"1.0"},
16 | {"source":"CreateExports-DMZVpc","target":"Example-Imports-New","value":"1.0"},
17 | {"source":"Create-Exports-Stack","target":"CreateExports-S3Endpoint","value":"1.0"},
18 | {"source":"Create-Exports-Stack","target":"CreateExports-Subnet1","value":"1.0"},
19 | {"source":"CreateExports-Subnet1","target":"Example-Imports-New","value":"1.0"},
20 | {"source":"Create-Exports-Stack","target":"CreateExports-Subnet2","value":"1.0"},
21 | {"source":"CreateExports-Subnet2","target":"Example-Imports-New","value":"1.0"},
22 | {"source":"Example-Imports0","target":"ExampleImports-SecurityGroup","value":"1.0"},
23 | {"source":"main-vpc","target":"main-vpc-AZAPrivate1SubnetId","value":"1.0"},
24 | {"source":"main-vpc","target":"main-vpc-AZAPublic1SubnetId","value":"1.0"},
25 | {"source":"main-vpc","target":"main-vpc-AZBPrivate1SubnetId","value":"1.0"},
26 | {"source":"main-vpc","target":"main-vpc-AZBPublic1SubnetId","value":"1.0"},
27 | {"source":"main-vpc","target":"main-vpc-AZCPrivate1SubnetId","value":"1.0"},
28 | {"source":"main-vpc","target":"main-vpc-AZCPublic1SubnetId","value":"1.0"},
29 | {"source":"main-vpc","target":"main-vpc-VPCID","value":"1.0"}
30 | ], "nodes": [
31 | {"name":"CreateExports-CoreVpc"},
32 | {"name":"Create-Exports-Stack"},
33 | {"name":"CreateExports-DMZSubnet2"},
34 | {"name":"main-vpc-AZBPrivate1SubnetId"},
35 | {"name":"CreateExports-DMZServerSecurityGroup"},
36 | {"name":"CreateExports-DMZSubnet1"},
37 | {"name":"Example-Imports2"},
38 | {"name":"CreateExports-Subnet1"},
39 | {"name":"main-vpc-AZCPrivate1SubnetId"},
40 | {"name":"main-vpc-AZBPublic1SubnetId"},
41 | {"name":"CreateExports-Subnet2"},
42 | {"name":"main-vpc-VPCID"},
43 | {"name":"Example-Imports-New"},
44 | {"name":"Example-Imports0"},
45 | {"name":"main-vpc-AZAPrivate1SubnetId"},
46 | {"name":"CreateExports-S3Endpoint"},
47 | {"name":"Example-Imports1"},
48 | {"name":"main-vpc-AZAPublic1SubnetId"},
49 | {"name":"CreateExports-DMZVpc"},
50 | {"name":"main-vpc"},
51 | {"name":"main-vpc-AZCPublic1SubnetId"},
52 | {"name":"CreateExports-CoreServerSecurityGroup"},
53 | {"name":"ExampleImports-SecurityGroup"}
54 | ]}
55 | 


--------------------------------------------------------------------------------