├── .gitignore ├── LICENSE ├── README.md ├── count.tf ├── countable.py └── statecounteditor.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | env/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | wheels/ 24 | *.egg-info/ 25 | .installed.cfg 26 | *.egg 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | 49 | # Translations 50 | *.mo 51 | *.pot 52 | 53 | # Django stuff: 54 | *.log 55 | local_settings.py 56 | 57 | # Flask stuff: 58 | instance/ 59 | .webassets-cache 60 | 61 | # Scrapy stuff: 62 | .scrapy 63 | 64 | # Sphinx documentation 65 | docs/_build/ 66 | 67 | # PyBuilder 68 | target/ 69 | 70 | # Jupyter Notebook 71 | .ipynb_checkpoints 72 | 73 | # pyenv 74 | .python-version 75 | 76 | # celery beat schedule file 77 | celerybeat-schedule 78 | 79 | # SageMath parsed files 80 | *.sage.py 81 | 82 | # dotenv 83 | .env 84 | 85 | # virtualenv 86 | .venv 87 | venv/ 88 | ENV/ 89 | 90 | # Spyder project settings 91 | .spyderproject 92 | .spyproject 93 | 94 | # Rope project settings 95 | .ropeproject 96 | 97 | # mkdocs documentation 98 | /site 99 | 100 | # mypy 101 | .mypy_cache/ 102 | 103 | # MAC 104 | .idea 105 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Coin Graham 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # statecountereditor 2 | Edit the terraform state to remove an item from the count and maintain existing systems. 3 | 4 | ## Purpose 5 | This utility is to assist terraform users when they are utilizing the count functionality and need to make changes that will shift the items in a list. 6 | 7 | ## Start Here 8 | Clone the repo to a folder and open the files. 9 | 10 | ## Explaining Terraform count problem 11 | If you take a look at the example count.tf terraform, you'll see the following definition: 12 | 13 | ``` 14 | variable "test_tags" { 15 | default = [ "ebs1", "ebs2", "ebs3" ] 16 | } 17 | 18 | resource "aws_ebs_volume" "test" { 19 | count = "${length(var.test_tags)}" 20 | availability_zone = "us-east-1a" 21 | type = "gp2" 22 | size = 1 23 | tags { 24 | Name = "${var.test_tags[count.index]}" 25 | } 26 | } 27 | ``` 28 | 29 | This will create three EBS volumes with the name tags ebs1, ebs2, and ebs3 and is a very cool feature in terraform. This allows you to easily create some automation scripts to build a lot of resources. In the terraform state file, they will be listed as "aws_ebs_volume.test.0", "aws_ebs_volume.test.1", "aws_ebs_volume.test.2". The issue is that if in the future you decide that you no longer need ebs2. For clarity, here is the current mapping in our example: 30 | 31 | ``` 32 | Volume to state resource 33 | ebs1 = "aws_ebs_volume.test.0" 34 | ebs2 = "aws_ebs_volume.test.1" 35 | ebs3 = "aws_ebs_volume.test.2" 36 | ``` 37 | 38 | If you remove ebs2 from the array, then ebs3 will shift into ebs2's place in the array. This will cause terraform to see the following when you run terraform plan: 39 | 40 | ``` 41 | Volume to state resource 42 | ebs1 = "aws_ebs_volume.test.0" 43 | ebs3 = "aws_ebs_volume.test.1" (rename ebs2 tag to ebs3) 44 | nothing = "aws_ebs_volume.test.2" (delete ebs3) 45 | ``` 46 | 47 | Here's the actual terraform plan for this: 48 | 49 | ``` 50 | ~ aws_ebs_volume.test.1 51 | tags.Name: "ebs2" => "ebs3" 52 | 53 | - aws_ebs_volume.test.2 54 | ``` 55 | 56 | So while we wanted to remove ebs2, terraform will actually delete ebs3 and rename ebs2 to ebs3. This issue is further complicated by the fact that there is no way to edit the state file for counters with the `terraform state` command. It won't take in commands with the designation like "aws_ebs_volume.test.0". This isn't a huge deal with ebs volumes and tags, but if it were instances and AMIs, all your systems from that item forward would likely be rebuilt in the wash. 57 | 58 | ## One solution 59 | The simplest solution is to open the state file and change it manually. In our example above, you simply need to find "aws_ebs_volume.test.1" and delete it. Then edit all the resource counters going forward and reduce their ending count digit by one. 60 | 61 | ## Enter statecountereditor 62 | So I've created a python script that will introspect the state file and find all the counters that you're using. Then it will prompt you for which one to modify. Once you decide which counter definition you want to modify you'll pick which item you want to remove. The script will then present you with the item for you to validate it is correct. If it is the correct item, that item will be removed and all subsequent items will be reduced one digit to the end. The old state will be saved as "terraform.tfstate.original-%m-%d-%y-%H%M%S" and your new state written as terraform.tfstate. Then you can remove the item from your terraform config and run terraform plan (there should be no changes). 63 | 64 | ## Author 65 | Coin Graham 66 | https://github.com/coingraham/statecountereditor 67 | 68 | 69 | 70 | -------------------------------------------------------------------------------- /count.tf: -------------------------------------------------------------------------------- 1 | # Terraform count state generator 2 | 3 | provider "aws" { 4 | region = "us-east-1" 5 | profile = "firstwatch" 6 | } 7 | 8 | variable "test_tags" { 9 | default = [ "ebs1", "ebs2", "ebs3" ] 10 | } 11 | 12 | resource "aws_ebs_volume" "test" { 13 | count = "${length(var.test_tags)}" 14 | availability_zone = "us-east-1a" 15 | type = "gp2" 16 | size = 1 17 | tags { 18 | Name = "${var.test_tags[count.index]}" 19 | } 20 | } 21 | 22 | variable "another_tags" { 23 | default = [ "ebs1", "ebs2", "ebs3", "ebs4", "ebs5", "ebs6" ] 24 | } 25 | 26 | resource "aws_ebs_volume" "another" { 27 | count = "${length(var.another_tags)}" 28 | availability_zone = "us-east-1a" 29 | type = "gp2" 30 | size = 1 31 | tags { 32 | Name = "${var.another_tags[count.index]}" 33 | } 34 | } 35 | 36 | resource "aws_ebs_volume" "lastly" { 37 | availability_zone = "us-east-1a" 38 | type = "gp2" 39 | size = 1 40 | tags { 41 | Name = "NonCount" 42 | } 43 | } -------------------------------------------------------------------------------- /countable.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | class Countable: 4 | 5 | total = 0 6 | 7 | def __init__(self, resource, name): 8 | Countable.total += 1 9 | self.resource = resource 10 | self.name = name 11 | self.max = 0 12 | 13 | def push(self, number): 14 | number = int(number) 15 | if number > self.max: 16 | self.max = number 17 | 18 | def get_full_name(self): 19 | full_name = self.resource + "." + self.name 20 | return full_name 21 | 22 | def get_max(self): 23 | return int(self.max) + 1 24 | -------------------------------------------------------------------------------- /statecounteditor.py: -------------------------------------------------------------------------------- 1 | import json 2 | import re 3 | import os 4 | import time 5 | from countable import Countable 6 | 7 | 8 | class StateCounterHelper: 9 | 10 | def __init__(self): 11 | 12 | self.countable_dict = {} 13 | self.menu_dict = {} 14 | self.state_file_name = "terraform.tfstate" 15 | self.count_pattern = r"\"(.*\..*\.\d+)\":" 16 | self.json_state = None 17 | 18 | def main(self): 19 | 20 | # Opening statements 21 | print "Running the statecounteditor, please read the README for more information.\n" 22 | 23 | # Read the state file if we can, quit if we cannot 24 | try: 25 | state_file = open(self.state_file_name, "r") 26 | state_info = state_file.read() 27 | 28 | except Exception as read_exception: 29 | print "Could not read state file with error:\n{}".format(read_exception) 30 | return 0 31 | 32 | # Create a json object for the state file and iterator based on the search string 33 | self.json_state = json.loads(state_info) 34 | count_iter = re.finditer(self.count_pattern, state_info) 35 | 36 | # Go through the matches and create counter objects for each. Save in a dict for later 37 | for count in count_iter: 38 | count_group = str(count.group(1)) 39 | count_resource = count_group.split(".")[0] 40 | count_name = count_group.split(".")[1] 41 | count_value = count_group.split(".")[2] 42 | resource_name = ".".join(count_group.split(".")[0:2]) 43 | 44 | if resource_name in self.countable_dict: 45 | self.countable_dict[resource_name]["Countable"].push(count_value) 46 | 47 | else: 48 | countable_obj = Countable(count_resource, count_name) 49 | self.countable_dict[resource_name] = { 50 | "Countable": countable_obj 51 | } 52 | countable_obj.push(count_value) 53 | 54 | # Only proceed if there are countable objects that can be modified 55 | if Countable.total > 0: 56 | countable_id = 1 57 | for resource_name in self.countable_dict: 58 | self.menu_dict[str(countable_id)] = self.countable_dict[resource_name] 59 | countable_id += 1 60 | 61 | os.system('clear') 62 | 63 | # Present the Menu options to decide with resource to modify 64 | print "Please choose the countable resource you want to modify:\n" 65 | 66 | # Build the menu from the countable objects 67 | for menu_item in self.menu_dict.keys(): 68 | countable = self.menu_dict[menu_item]["Countable"] 69 | print "{}. {} with {} items".format(menu_item, countable.get_full_name(), countable.get_max()) 70 | 71 | print "\n0. Quit" 72 | 73 | # Get the user input 74 | choice = raw_input(" >> ") 75 | 76 | # Return the user input 77 | return choice 78 | 79 | # If there are no countable objects return 0 which will effectively quit the program 80 | else: 81 | return 0 82 | 83 | def menu_selection(self, choice): 84 | 85 | # Quit out if the choice is 0 86 | if choice == 0: 87 | print "Quitting." 88 | return 89 | 90 | # Find the countable object for the choice and prompt for which item to remove 91 | if choice in self.menu_dict.keys(): 92 | countable = self.menu_dict[choice]["Countable"] 93 | full_name = countable.get_full_name() 94 | resource_max = countable.get_max() 95 | 96 | # Prompt for which item to remove 97 | print "\nModifying {} with {} items. Which item would you like to remove " \ 98 | "(just put the number and remember it starts with 0)?".format(full_name, resource_max) 99 | 100 | # Get the user input 101 | item_to_remove = raw_input(" >> ") 102 | 103 | # Send the input to the function for removal 104 | self.find_item_to_remove(full_name, item_to_remove, countable) 105 | 106 | def find_item_to_remove(self, full_name, item_to_remove, countable): 107 | 108 | # Setup the name for use later 109 | resource_name = full_name + "." + item_to_remove 110 | 111 | # Check for existance of the item requested, otherwise quit 112 | if resource_name in self.json_state["modules"][0]["resources"]: 113 | resource_definition = self.json_state["modules"][0]["resources"][resource_name] 114 | else: 115 | print "Resource {} does not exist! Exiting.".format(resource_name) 116 | return 117 | 118 | os.system('clear') 119 | 120 | # Get input on the item to remove 121 | print "Do you want to remove {} with the following definition?".format(resource_name) 122 | 123 | print json.dumps(resource_definition, indent=4, sort_keys=True) 124 | 125 | # Get input from the user 126 | answer = raw_input(" Type Yes to Remove, " 127 | "Previous to see the previous item, " 128 | "or Next to see the next item >> ") 129 | 130 | # Move up one item and ask again 131 | if answer == "Next": 132 | item_to_remove = str(int(item_to_remove) + 1) 133 | self.find_item_to_remove(full_name, item_to_remove, countable) 134 | 135 | # Move back one item and ask again 136 | if answer == "Previous": 137 | item_to_remove = str(int(item_to_remove) - 1) 138 | self.find_item_to_remove(full_name, item_to_remove, countable) 139 | 140 | # Advance with the removal of the item 141 | if answer == "Yes": 142 | 143 | # Walk through the items from the selected to the end and move them back one step 144 | for item in range(int(item_to_remove), int(countable.get_max())): 145 | resource_name = full_name + "." + str(item) 146 | next_resource_name = full_name + "." + str(item + 1) 147 | 148 | # If there is a next item, save to this item 149 | if next_resource_name in self.json_state["modules"][0]["resources"]: 150 | self.json_state["modules"][0]["resources"][resource_name] = \ 151 | self.json_state["modules"][0]["resources"][next_resource_name] 152 | 153 | # If there isn't a next item, we're at the end and will delete it. 154 | else: 155 | self.json_state["modules"][0]["resources"].pop(resource_name) 156 | 157 | # Try to rename the existing state file and save the new state file. 158 | try: 159 | # Use time so that the new file name is unique and not overwritten. 160 | time_string = time.strftime("%m-%d-%y-%H%M%S") 161 | original_filename = "terraform.tfstate.original-{}".format(time_string) 162 | os.rename("terraform.tfstate", original_filename) 163 | f = open("terraform.tfstate", "w") 164 | f.write(json.dumps(self.json_state, indent=4, sort_keys=True)) 165 | print "Original state moved to \"{}\" and state file updated.".format(original_filename) 166 | 167 | except Exception as e: 168 | print "Error {} trying to rename the state file.".format(e) 169 | 170 | # If they don't follow instructions quit 171 | else: 172 | print "Please follow the instructions." 173 | return 174 | 175 | 176 | if __name__ == '__main__': 177 | 178 | # Create the base object 179 | my_sch = StateCounterHelper() 180 | 181 | # Get the main resource to modify 182 | my_choice = my_sch.main() 183 | 184 | # Pick the item and do the action 185 | my_sch.menu_selection(my_choice) 186 | --------------------------------------------------------------------------------