├── .github
└── workflows
│ └── sourceguard.yml
├── README.md
├── __init__.py
└── policyCleanUp
└── policyCleanUp.py
/.github/workflows/sourceguard.yml:
--------------------------------------------------------------------------------
1 | name: SourceGuard Code Analysis
2 | on: [push]
3 | jobs:
4 | code-analysis:
5 | runs-on: ubuntu-latest
6 | container:
7 | image: sourceguard/sourceguard-cli
8 | steps:
9 | - name: Scan
10 | uses: CheckPointSW/sourceguard-action@main
11 | with:
12 | SG_CLIENT_ID: ${{ secrets.SG_CLIENT_ID }}
13 | SG_SECRET_KEY: ${{ secrets.SG_SECRET_KEY }}
14 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # PolicyCleanUp
2 | Check Point PolicyCleanUp tool allows automatic cleanup of your policy based on hits count. The tool runs on a policy
3 | and a domain that you named.
4 |
5 | * If a rule was not hit for the number of days that you configured, the rule is a candidate to be disabled.
6 | * If a rule is disabled for the number of days that you configured, the rule is a candidate to be deleted.
7 |
8 | You can adjust the code according to your organization’s policy / needs.
9 |
10 | - This tool can be executed on Management Server / Multi-Domain servers of version of R80.10 and up.
11 |
12 | ## Instructions
13 | Clone the repository with this command:
14 | ```git
15 | git clone https://github.com/CheckPointSW/PolicyCleanUp
16 | ```
17 | or by clicking the _‘Download ZIP’_ button.
18 |
19 | Download and install the [Check Point API Python SDK](https://github.com/CheckPointSW/cp_mgmt_api_python_sdk)
20 | repository, follow the instructions in the SDK repository.
21 |
22 | ## Main Options
23 | *__More options and details can be found with the '-h' option by running:__ python policyCleanUp.py –h*
24 | * [--package , -k] The name of the policy package to clean from zero-hit rules. The default is all
25 | policies.
26 | * [--operation , -op] The operation mode in which the tool runs. The default is plan.
27 |
There are 3 modes:
28 | * plan: tool runs without performing changes in the policy. The output file is in Json format. The output file
29 | holds rules that are candidates for deletion or to be disabled.
30 | * apply: tool runs and makes changes in the policy. The rules are disabled and deleted according to the plan.
31 | At the end of this operation, publish is executed.
32 | * apply without publish: tool runs and makes changes but without publish. In this mode,
33 | you can login from SmartConsole to this session and explore changes before they are published.
34 | * [--import-plan-file , -i] A file that holds output of execution of the ‘plan’ operation.
35 | You can use this file only on apply or apply_without_publish operations. Tip: using this option saves time of the ‘apply’
36 | operation if you already executed a ‘plan’.
37 | * [--disable-after] Time in days in which if there are no hits, the rule is a candidate for disabling.
38 | Default is 180 days. Learn how to change the default in ‘Technical Details’ section.
39 | * [--delete-after] Time in days for a disabled rule to be a candidate for deletion. Only rules that
40 | were disabled by the tool are candidates for deletion. Default is 60 days. Learn how to change the default in ‘Technical
41 | Details’ section.
42 | * [--output-file] Name of the output file. The file is in Json format. This file helps you understand
43 | which rules were affected. If not supplied, the default file name is _‘policyCleanUp-.json’_.
44 | To use it in ‘apply’ operations, pass it as 'import plan file'.
45 |
46 | ## Examples
47 | * Running the tool on a remote management server using username & password:
48 |
```python policyCleanUp.py -m 172.23.78.160 -u James -p MySecretPassword!```
49 |
The tool runs on a remote management server with IP address 172.23.78.160 and the operation is ‘plan’ (default).
50 | * Running the tool on a remote management server using API key:
51 |
```python policyCleanUp.py -m 172.23.78.160 --api-key JpPA+eJ5gekQBY8DF27+ZQ==```
52 | * Running the tool on a Multi-Domain Server for a specific domain and a specific policy package:
53 |
```python policyCleanUp.py -d 172.23.78.152 –k Standard -u James -p MySecretPassword!```
54 | * Running the tool on a Security Management Server with operation plan:
55 |
```python policyCleanUp.py -o plan_output_file.json -op plan -u James -p MySecretPassword!```
56 |
The tool runs in plan mode and creates a json output file named “plan_output_file.json” as noted. This file can
57 | be used later on as an ‘import-plan-file’ for ‘apply’ mode.
58 | * Running the tool on a Security Management Server and applying the import-plan-file:
59 |
```python policyCleanUp.py -i plan_output_file.json -op apply -u James -p MySecretPassword!```
60 |
The tool runs in apply mode. Rules are disabled / deleted according to the import-plan-file “plan_output_file.json”._
61 | * Running the tool on a Security Management Server with operation apply-without-publish, set specific disable/deleted thresholds and session name:
62 |
```python policyCleanUp.py -op apply_without_publish –-root true --disable-after 20 --delete-after 40 --session-name “Policy Cleanup script”```
63 |
The tool runs in ‘apply without publish’ mode which means publish is not executed at the end but changes will be saved n private session.
64 | You can connect to SmartConsole, find the session named “session_name” and explore the changes before publishing the session.
65 | This run will not use default thresholds but instead these are the thresholds:
66 |
- Disabled rules whose last hits were 20 days ago.
67 |
- Deleted rules which were disabled by the tool 40 days ago.
68 |
69 | ## Output
70 | The tool’s output is a file in a Json format that holds the following information:
71 | 1. List of packages that were scanned
72 |
- Each package holds its layers and installation targets
73 |
- Each layer in the package contains:
74 |
- disabled rules, deleted rules and skipped rules (and reason).
75 |
- objects-dictionary (for rules)
76 | 2. List of skipped packages with reasons
77 | 3. Threshold values
78 | 4. Operation mode in which the tool was ran
79 |
80 | Example of Output:
81 | ```Git
82 | {
83 | "operation": "plan",
84 | "packages": [
85 | {
86 | "access-layers": [
87 | {
88 | "delete-rules": {
89 | "rules": [],
90 | "total": 0
91 | },
92 | "disable-rules": {
93 | "rules": [...],
94 | "total": 7
95 | },
96 | "name": "Branch_Office_Policy Network",
97 | "objects-dictionary": [...],
98 | "shared": false,
99 | "skipped-rules": {
100 | "rules": [...],
101 | "total": 3
102 | },
103 | "type": "access-layer",
104 | "uid": "13a747d1-7fda-483b-afca-f8d996a4a574"
105 | },
106 | ],
107 | "installation-targets": [...],
108 | "name": "Branch_Office_Policy",
109 | "type": "package",
110 | "uid": "89368746-46bd-418e-a625-2e848040c76f"
111 | }
112 | ],
113 | "skipped-packages": [
114 | {
115 | "name": "Corporate_Policy",
116 | "skipped-reason": "All package targets are invalid",
117 | "type": "package",
118 | "uid": "e187eb39-f6dd-4ee3-93c3-ce4df3e2e393"
119 | }
120 | ],
121 | "thresholds": {
122 | "delete-after": 6,
123 | "disable-after": 4
124 | }
125 | }
126 |
127 | ```
128 | If you run the tool with plan mode the output can be used as input for import-plan-file for the tool with apply/ apply without publish.
129 |
130 |
131 | ## Technical Details
132 | * The default values for‘--disable-after’ and ‘--delete-after’ are part of the python script (policyCleanUp.py).
133 | To change values, search for the following thresholds in code:
134 | ```python
135 | # Defaults for global disable & delete thresholds
136 | DEFAULT_DISABLE_THRESHOLD = 180
137 | DEFAULT_DELETE_THRESHOLD = 60
138 | ```
139 |
140 |
141 | * Relevant functions in the script that you can change to adjust the logic to your needs:
142 |
1.‘apply_plan’ – a function that applies the changes per rule. If the rule was a candidate for disabling, it calls the ‘disable_rule’ function and if the rule candidate for deletion it calls the ‘delete_rule’ function. In both these functions the changes affect on the rule.
143 |
2.‘rule_should_be_disabled’ – a function that determines if rule should be disabled. A rule should be disabled if it’s last hit date or last modified date (the closest date) is before today’s date minus the threshold. The thresholds are determined by global thresholds or overrides thresholds.
144 |
3.‘rule_should_be_deleted’ – a function that determines if a rule should be deleted. A rule should be deleted if the date it was disabled by the tool is before today’s date minus the threshold. The thresholds are determined by global thresholds or overrides thresholds
145 |
4.‘validate_rule’ – a function that checks if the install-on list contains invalid target and that the rule was not modified after it was installed on targets.
146 |
147 | * Example for a simple code adjustment:
148 |
Objective: As part of rule disabling, set it to the bottom of the rulebase.
149 |
Solution:
150 |
- Find ‘disable_rule’ function.
151 |
- Find the API call that disables the rule (‘set-access-rule’ command).
152 |
- Add ‘new-position’ argument to existing command and set it to bottom.
153 |
154 |
Before:
155 | ```python
156 | # Set rule changes
157 | # *** If you wold like to your logic as part of rule disabling. This is the place ***
158 | # *** For example: set rule position to bottom, you need add to API call 'new-position' parameter with value bottom.***
159 | def disable_rule(rule, layer, api_client):
160 | global DATETIME_NOW
161 | global DATETIME_FORMAT
162 |
163 | # Disable rule & set disabled-time & add comment
164 | set_rule_res = api_client.api_call("set-access-rule",
165 | {"uid": rule['uid'], "layer": layer['uid'], "enabled": "false",
166 | "custom-fields": {"field-3": DATETIME_NOW.strftime(DATETIME_FORMAT)},
167 | "comments": rule['comments'] + " -This rule changed automatically by the policyCleanUp tool"})
168 |
169 | return not is_failure(" Failed to set rule No.{} with UID {}.".format(rule['rule-number'], rule['uid']), set_rule_res)
170 |
171 | ```
172 |
After:
173 | ```python
174 | # Set rule changes
175 | # *** If you wold like to your logic as part of rule disabling. This is the place ***
176 | # *** For example: set rule position to bottom, you need add to API call 'new-position' parameter with value bottom.***
177 | def disable_rule(rule, layer, api_client):
178 | global DATETIME_NOW
179 | global DATETIME_FORMAT
180 |
181 | # Disable rule & set disabled-time & add comment
182 | set_rule_res = api_client.api_call("set-access-rule",
183 | {"uid": rule['uid'], "layer": layer['uid'], "enabled": "false",
184 | "custom-fields": {"field-3": DATETIME_NOW.strftime(DATETIME_FORMAT)},
185 | "comments": rule['comments'] + " -This rule changed automatically by the policyCleanUp tool",
186 | "new-position": "bottom"})
187 |
188 | return not is_failure(" Failed to set rule No.{} with UID {}.".format(rule['rule-number'], rule['uid']), set_rule_res)
189 |
190 | ```
191 |
192 | Notice! The tool uses the custom fields of rule (within SmartConsole - Security Policies > Access Control > Policy > Summary tab)
193 |
194 | ## Development Environment
195 | The tool is developed using Python language 2.7.14 and [Check Point API Python SDK](https://github.com/CheckPointSW/cp_mgmt_api_python_sdk).
196 |
197 |
198 |
199 |
200 |
--------------------------------------------------------------------------------
/__init__.py:
--------------------------------------------------------------------------------
1 | #policyCleanUp_api_python
2 |
--------------------------------------------------------------------------------
/policyCleanUp/policyCleanUp.py:
--------------------------------------------------------------------------------
1 | """
2 |
3 | policyCleanUp.py
4 | version 1.1 (default in lib)
5 |
6 | This tool gives a new utility, Automatic Cleanup Based On Hits utility, to streamline rule bases in a Multi-Domain Security Management environment.
7 | The Automatic Cleanup Based On Hits Utility runs on a policy and domain that you name. It uses Hit Count
8 |
9 |
10 | This tool demonstrates communication with Check Point Management server using Management API Library in Python.
11 | Logout command is called automatically after the work with Management API Library is completed.
12 |
13 | The output format will be similar to the output of show-packages command.
14 | The report(disabled/deleted rules) will be under access-layers.disable-rules | .delete-rules
15 |
16 | Use -h flag for details-usage.
17 |
18 | written by: Check Point software technologies inc.
19 | August 2018
20 |
21 | """
22 |
23 | from __future__ import print_function
24 |
25 | import argparse
26 | import datetime
27 | import json
28 | import os
29 | import sys
30 | import time
31 |
32 | sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
33 |
34 | # A package for reading passwords without displaying them on the console.
35 | import getpass
36 |
37 | # cpapi is a library that handles the communication with the Check Point management server.
38 | from cpapi import APIClient, APIClientArgs
39 |
40 | # Defaults for global disable & delete thresholds
41 | DEFAULT_DISABLE_THRESHOLD = 180
42 | DEFAULT_DELETE_THRESHOLD = 60
43 |
44 | # Script running time in seconds
45 | DATETIME_NOW = datetime.datetime.now().replace(microsecond=0)
46 | DATETIME_NOW_SEC = int(round(time.mktime(DATETIME_NOW.timetuple())))
47 | DATETIME_FORMAT = "%Y-%m-%d %H:%M:%S"
48 | DATETIME_NOW_STR = datetime.datetime.fromtimestamp(DATETIME_NOW_SEC).strftime(DATETIME_FORMAT)
49 |
50 | # Default log file name
51 | DEFAULT_OUTPUT_FILE_NAME = "policyCleanUp-{}.json".format(DATETIME_NOW_SEC)
52 |
53 | # Global domain name
54 | GLOBAL_DOMAIN_NAME = 'Global'
55 |
56 | # Rule install-on all targets constant UID
57 | INSTALL_ON_ALL_TARGETS = "6c488338-8eec-4103-ad21-cd461ac2c476"
58 |
59 | REASON_OF_ALL_TARGETS_INVALID = "Check that Access policy is installed on all targets and " \
60 | "policy wasn't modified after installation date and hitcount is active on targets"
61 |
62 |
63 | # Parse user arguments
64 | def parse_arguments():
65 | # Instantiate the parser
66 | parser = argparse.ArgumentParser(description='\nThis Is Check Point Automatic Cleanup Based On Hits Tool',
67 | formatter_class=argparse.RawTextHelpFormatter)
68 |
69 | # Optional list policies argument
70 | parser.add_argument('--package', '-k',
71 | help='\nName of the policy you want to disable & delete its rules. Default {ALL_POLICIES}',
72 | metavar="")
73 |
74 | # Optional operation argument
75 | parser.add_argument('--operation', '-op', choices=['plan', 'apply', 'apply_without_publish'], default='plan',
76 | metavar="\b",
77 | help='{%(choices)s}\nplan: checking which rules will be disabled/deleted.\napply: disabling & deleting the rules.\n' +
78 | 'apply_without_publish: disabling & deleting the rules without publish.\nDefault {plan}')
79 |
80 | # Optional file argument
81 | parser.add_argument('--import-plan-file', '-i',
82 | help='The name of json file. Apply the \'import-plan-file\' without re-run plan, should be provided in apply/apply_without_publish operations only',
83 | metavar="")
84 |
85 | # Optional management argument
86 | parser.add_argument('--management', '-m', default='127.0.0.1',
87 | help='\nThe management server\'s IP address or hostname. Default {127.0.0.1}', metavar="")
88 |
89 | # Optional domain argument
90 | parser.add_argument('--domain', '-d', help='\nThe name or uid of the Security Management Server domain.',
91 | metavar="")
92 | # Optional port argument
93 | # port is set to None by default, but it gets replaced with 443 if not specified (in lib)
94 | parser.add_argument('--port', help='\nPort of WebAPI server on management server. Default {443}', metavar="")
95 |
96 | # Optional user name argument
97 | parser.add_argument('--user', '-u', dest='username', help='\nManagement administrator user name.', metavar="")
98 |
99 | # Optional password argument
100 | parser.add_argument('--password', '-p', help='\nManagement administrator password.', metavar="")
101 |
102 | # Optional api key argument
103 | parser.add_argument('--api-key', help='\nManagement administrator API key.', metavar="")
104 |
105 | # Optional login as root argument
106 | parser.add_argument('--root', '-r', choices=['true', 'false'],
107 | help='\b{%(choices)s}\nLogin as root. When running on the management server, use this flag with value set to \'true\' to login as Super User administrator.',
108 | metavar=" \b\b")
109 |
110 | # Optional session name argument
111 | parser.add_argument('--session-name',
112 | help='\nSession unique name. Should be provided in apply/apply_without_publish operations only. Default {Policy_clean_up_tool}',
113 | default="Policy_clean_up_tool", metavar="")
114 |
115 | # Optional session description argument
116 | parser.add_argument('--session-description',
117 | help='Session description. Should be provided in apply/apply_without_publish operations only. Default {Current time}',
118 | default=DATETIME_NOW_STR, metavar="")
119 |
120 | # Optional delete threshold argument
121 | parser.add_argument('--delete-after', help='\nTime in days in which rule will be candidate for deletion.',
122 | default=DEFAULT_DELETE_THRESHOLD, metavar="")
123 |
124 | # Optional disable threshold argument
125 | parser.add_argument('--disable-after', help='\nTime in days in which rule will be candidate for disable.',
126 | default=DEFAULT_DISABLE_THRESHOLD, metavar="")
127 |
128 | # Optional output file argument
129 | parser.add_argument('--output-file', '-o', default=DEFAULT_OUTPUT_FILE_NAME, metavar="",
130 | help='\nName of the output Json file that help to understand which rules were affected. You can use it to apply etc as \'import plan file\'.\nDefault {policyCleanUp-.json}')
131 |
132 | # Parse the arguments
133 | args = parser.parse_args()
134 |
135 | # Ask for username, password if needed & add customized logic to parsing
136 | customize_arguments(parser, args)
137 |
138 | return args
139 |
140 |
141 | # Validate arguments from user - checks that the user has entered the required arguments and parsing
142 | def customize_arguments(parser, args):
143 | # The user has not entered username
144 | if args.root is None:
145 | auth_method = ""
146 | if args.username or args.password:
147 | auth_method = "1"
148 | elif args.api_key:
149 | auth_method = "2"
150 |
151 | if args.username is None and args.password is None and args.api_key is None:
152 | if sys.version_info >= (3, 0):
153 | auth_method = input("Select authentication method: \n1. Username & Password \n2. API key\n")
154 | else:
155 | auth_method = raw_input("Select authentication method: \n1. Username & Password \n2. API key\n")
156 |
157 | if auth_method != "1" and auth_method != "2":
158 | print_msg("Invalid authentication method input (Expected 1 or 2)")
159 | exit(1)
160 |
161 | if auth_method == "1":
162 | if args.username is None:
163 | if sys.version_info >= (3, 0):
164 | args.username = input("Username: ")
165 | else:
166 | args.username = raw_input("Username: ")
167 |
168 | # The user has not entered password
169 | if args.password is None:
170 | if sys.stdin.isatty():
171 | args.password = getpass.getpass("Password: ")
172 | else:
173 | print("Attention! Your password will be shown on the screen!")
174 | if sys.version_info >= (3, 0):
175 | args.password = input("Password: ")
176 | else:
177 | args.password = raw_input("Password: ")
178 | else:
179 | if args.api_key is None:
180 | print("Attention! Your API key will be shown on the screen!")
181 | if sys.version_info >= (3, 0):
182 | args.api_key = input("API key: ")
183 | else:
184 | args.api_key = raw_input("API key: ")
185 |
186 | # Plan-file should be provided in apply operations only
187 | if (args.import_plan_file is not None) and (is_apply_operation(args.operation) is False):
188 | print_msg("Error: plan-file should be provided in apply/apply_without_publish operations only.")
189 | exit(1)
190 |
191 | # Session-name and session-description should be provided in apply operations only
192 | if (is_apply_operation(args.operation) is False) and (
193 | is_default_session_details(args.session_name, args.session_description) is False):
194 | print_msg(
195 | "Error: Session-name and Session-description should be provided in apply/apply_without_publish operations only.")
196 | exit(1)
197 |
198 | # If user inserted thresholds, update the "global" thresholds
199 | updateGlobalThreshold(args)
200 |
201 |
202 | # Update global threshold
203 | def updateGlobalThreshold(args):
204 | global DEFAULT_DISABLE_THRESHOLD
205 | DEFAULT_DISABLE_THRESHOLD = int(args.disable_after)
206 |
207 | global DEFAULT_DELETE_THRESHOLD
208 | DEFAULT_DELETE_THRESHOLD = int(args.delete_after)
209 |
210 |
211 | # checks if user inserted details session
212 | def is_default_session_details(session_name, session_description):
213 | if session_name == "Policy_clean_up_tool":
214 | if session_description == DATETIME_NOW_STR:
215 | return True
216 | return False
217 |
218 |
219 | # Get global threshold
220 | def get_global_thresholds():
221 | global DEFAULT_DISABLE_THRESHOLD
222 | global DEFAULT_DELETE_THRESHOLD
223 |
224 | global_thresholds = {}
225 | global_thresholds['disable'] = DEFAULT_DISABLE_THRESHOLD
226 | global_thresholds['delete'] = DEFAULT_DELETE_THRESHOLD
227 |
228 | return global_thresholds
229 |
230 |
231 | # Get list of all the packages information that we should run on. If the user not specify package name - run over all existing packages
232 | def packages_to_run(user_package, client):
233 | packages_to_run = []
234 |
235 | # If package-name isn't specified - run over all existing packages
236 | if user_package is None:
237 |
238 | # Note: api_query returns list of wanted objects received so far from the management server (in contrast to regular api_call that return only a limited number of objects)
239 | show_packages_res = client.api_query("show-packages", details_level="full")
240 | exit_failure("Failed to get all policies information.", show_packages_res)
241 |
242 | packages_to_run = show_packages_res.data.get('packages')
243 |
244 | # No packages at all
245 | if not packages_to_run:
246 | print_msg("There are no existing packages.")
247 | exit(0)
248 |
249 | else:
250 | show_package_res = client.api_call("show-package", {"name": user_package, "details-level": "full"})
251 | exit_failure("Failed to get policy {} information.".format(user_package), show_package_res)
252 |
253 | packages_to_run.append(show_package_res.data)
254 |
255 | return packages_to_run
256 |
257 |
258 | # Get rule's thresholds.
259 | # Disable and deleted thresholds are calculated by global thresholds or overrides thresholds (local override).
260 | # Local thresholds- user can override the global thresholds by adding number to custom-fields per rule.
261 | def get_rule_final_threshold(rule, field, threshold_type, global_thresholds):
262 | # Custom-field1 contains override for disable threshold
263 | local_threshold = rule.get('custom-fields', {}).get(field)
264 | # -1 represents a rule that should be skipped all time.
265 | if global_thresholds.get(threshold_type) == -1:
266 | return None
267 | # Empty value - use global threshold
268 | elif not local_threshold:
269 | return global_thresholds.get(threshold_type)
270 | # Don't touch override
271 | elif local_threshold == "-1":
272 | return None
273 | # Non-numeric value - skip rule
274 | elif local_threshold.isnumeric() is False:
275 | rule['skipped-reason'] = "{} threshold has non-numeric value".format(threshold_type)
276 | return None
277 | else:
278 | int_local_threshold = int(local_threshold)
279 |
280 | # Negative value - skip rule
281 | if int_local_threshold < 1:
282 | rule['skipped-reason'] = "{} threshold is a negative number".format(threshold_type)
283 | return None
284 | # Positive numeric value - use it as override
285 | else:
286 | return int_local_threshold
287 |
288 |
289 | # Get rule last disabled time
290 | def get_rule_disabled_time(rule):
291 | global DATETIME_FORMAT
292 |
293 | # Custum-field3 contains disabled time by the tool
294 | rule_disabled_time = rule.get('custom-fields', {}).get('field-3')
295 |
296 | # No disabled time, skip rule (no warning) - assume the rule didn't disable by the tool
297 | if not rule_disabled_time:
298 | return None
299 |
300 | # Convert last disabled-time to datetime object by DATETIME_FORMAT
301 | try:
302 | return datetime.datetime.strptime(rule_disabled_time, DATETIME_FORMAT)
303 | except (ValueError, TypeError):
304 | return None
305 |
306 |
307 | # Update the valid targets structure for this policy - keep only valid targets
308 | def update_package_valid_targets(package, valid_targets):
309 | package_name = package.get('name')
310 | package_valid_targets = valid_targets['packages-targets'].get(package_name)
311 | if not package_valid_targets:
312 | return
313 |
314 | installation_targets = package.get('installation-targets')
315 |
316 | # Package installation targets is all - only update all-targets-valid
317 | if installation_targets == 'all':
318 | package_valid_targets['all-targets-valid'] = valid_targets.get('all-targets-valid')
319 |
320 | # List of targets uid's, remove all targets that not in installation-targets & update all-targets-valid
321 | else:
322 | # Create set of installation-targets uids
323 | installation_targets_set = set([])
324 | for target in installation_targets:
325 | installation_targets_set.add(target.get('uid'))
326 |
327 | if target.get('uid') not in package_valid_targets['targets']:
328 | package_valid_targets['all-targets-valid'] = False
329 |
330 | # Remove all valid targets that not in installation targets
331 | for valid_target_uid in package_valid_targets['targets']:
332 | if valid_target_uid not in installation_targets_set:
333 | remove_target(valid_targets['packages-targets'], package, valid_target_uid)
334 |
335 |
336 | # Get all gateway targets with hit count on & access-policy installed.
337 | # Return dictionary from packages name to (dictionary of valid targets uid to valid targets)
338 | def valid_packages_targets(packages, client):
339 | # Get all gateways & servers of current domain
340 | # Note: api_query returns list of wanted objects received so far from the management server (in contrast to regular api_call that return only a limited number of objects)
341 | show_gateways_servers_res = client.api_query("show-gateways-and-servers", details_level="full")
342 | exit_failure("Failed to get gateways & servers information.", show_gateways_servers_res)
343 |
344 | # Object to store all valid targets as dictionary from packages
345 | valid_targets = {}
346 |
347 | # True if all the package targets are valid
348 | valid_targets['all-targets-valid'] = True
349 |
350 | # Store dictionary from packages name to (dictionary of valid targets uid to valid targets)
351 | valid_targets['packages-targets'] = {}
352 |
353 | for object in show_gateways_servers_res.data:
354 |
355 | # Check if it is installable target
356 | if is_targetale_object(object) is True:
357 |
358 | if is_valid_package_target(packages, object, client) is True:
359 | add_valid_target(valid_targets['packages-targets'], object)
360 | else:
361 | valid_targets['all-targets-valid'] = False
362 |
363 | # If all domain targets invalid - exit
364 | if not valid_targets['packages-targets']:
365 | print_msg("All domain's targets are invalid. " + REASON_OF_ALL_TARGETS_INVALID)
366 |
367 | # Foreach policy keep only the valid targets
368 | for package in packages:
369 | update_package_valid_targets(package, valid_targets)
370 |
371 | return valid_targets
372 |
373 |
374 | # Check if a package target is valid
375 | def is_valid_package_target(packages, target, client):
376 | # Set of packages names
377 | packages_names_dict = {}
378 | for package in packages:
379 | packages_names_dict[package.get('name')] = package
380 |
381 | policy_name = target.get('policy', {}).get('access-policy-name')
382 |
383 | return (policy_name in packages_names_dict) and is_access_policy_installed(target) and is_installation_updated(
384 | packages_names_dict[policy_name], target) and is_target_hitcount_on(target, client)
385 |
386 |
387 | # Check if policy install
388 | # s after modification
389 | def is_installation_updated(package, target):
390 | policy_installation_time = convert_date_object_to_datetime(
391 | target.get('policy').get('access-policy-installation-date', {}))
392 | policy_last_modify_time = convert_date_object_to_datetime(package.get('meta-info', {}).get('last-modify-time', {}))
393 |
394 | # Install after modify
395 | return (policy_last_modify_time <= policy_installation_time)
396 |
397 |
398 | # Add new valid target to the valid targets dictionary
399 | def add_valid_target(packages_targets, valid_target):
400 | target_policy_name = valid_target['policy']['access-policy-name']
401 |
402 | target_policy = packages_targets.get(target_policy_name)
403 |
404 | if not target_policy:
405 | target_policy = {}
406 | # Flag that save if al the targets are valid
407 | target_policy['all-targets-valid'] = True
408 | # save the minimal installation time of the policy - to check if rule has modified in validate_rule
409 | target_policy['minimal-installation-time'] = convert_date_object_to_datetime(
410 | valid_target.get('policy').get('access-policy-installation-date', {}))
411 | target_policy['targets'] = {}
412 | packages_targets[target_policy_name] = target_policy
413 |
414 | target_policy['targets'][valid_target.get('uid')] = valid_target
415 |
416 | # Update minimal policy installation time
417 | target_installation_time = convert_date_object_to_datetime(
418 | valid_target.get('policy').get('access-policy-installation-date', {}))
419 | if target_installation_time < target_policy['minimal-installation-time']:
420 | target_policy['minimal-installation-time'] = target_installation_time
421 |
422 |
423 | # Delete an existing target that became invalid from the valid targets dictionary
424 | def remove_target(packages_targets, package, target_uid):
425 | package_name = package.get('name')
426 | package_valid_targets = packages_targets.get(package_name, None)
427 |
428 | # Remove target from policy valid targets
429 | if not package_valid_targets:
430 | package_valid_targets.pop(target_uid, None)
431 |
432 | # Remove package with no valid targets
433 | if not package_valid_targets:
434 | packages_targets.pop(package_name, None)
435 |
436 |
437 | # Check if we can install policy on this object
438 | def is_targetale_object(object):
439 | return object.get('network-security-blades', {}).get('firewall')
440 |
441 |
442 | # Check if the target object contains access-policy
443 | def is_access_policy_installed(target):
444 | return target.get('policy', {}).get('access-policy-installed')
445 |
446 |
447 | # Check if the target hit count flag is on
448 | def is_target_hitcount_on(target, client):
449 | # Get target information
450 | ##### These APIs provide direct access to different objects and fields in the database. As a result, when the objects schema change, scripts that relied on specific schema fields may break.#####
451 | ##### When you have the option, always prefer to use the documented APIs and not the generic APIs #####
452 | show_generic_object_res = client.api_call("show-generic-object",
453 | {"uid": target.get('uid'), "details-level": "full"})
454 | if is_failure("Failed to get {} target information".format(target.get('name')), show_generic_object_res):
455 | return False
456 |
457 | # Check if hit count flag in on
458 | if show_generic_object_res.data.get('firewallSetting', {}).get('hitCountFw1Enable') is False:
459 | return False
460 |
461 | return True
462 |
463 |
464 | # Determine the packages the tool should run on (valid packages)
465 | def validate_package(package, valid_targets):
466 | package_name = package.get('name')
467 |
468 | if package_name not in valid_targets['packages-targets']:
469 | package['skipped-reason'] = "All package targets are invalid. " + REASON_OF_ALL_TARGETS_INVALID
470 | return False
471 |
472 | return True
473 |
474 |
475 | # Get res of show access-rulebase command with object-dictionary
476 | def show_access_rulebase(layer, client):
477 | res_of_show_access_rulebase = api_call_command_with_objects_dictionary(client, "show-access-rulebase",
478 | details_level="standard",
479 | container_keys=["rulebase"],
480 | payload={"uid": layer['uid'],
481 | "show-hits": "true",
482 | "hits-settings": {
483 | "from-date": "1970-01-02"}}) # for R80.10 we need to specify from date
484 | if skip_failure(layer, "Failed to get rulebase.", res_of_show_access_rulebase) is True:
485 | return None
486 |
487 | return res_of_show_access_rulebase
488 |
489 |
490 | # It is recommended to use in api_query and not in api_call
491 | # The use of the command 'api_call' in this function is because we want to get the data of rulebase and objects-dictionary at once.
492 | def api_call_command_with_objects_dictionary(client, command, details_level="standard", container_keys=["objects"],
493 | payload=None):
494 | limit = 500 # each time get no more than 500 objects
495 | finished = False # will become true after getting all the data
496 | all_objects = {} # accumulate all the objects from all the API calls
497 |
498 | iterations = 0 # number of times we've made an API call
499 | if payload is None:
500 | payload = {}
501 |
502 | for key in container_keys:
503 | all_objects[key] = []
504 |
505 | # accumulate all the objects-dictionary from all the API calls
506 | all_objects["objects-dictionary"] = {}
507 |
508 | # are we done?
509 | while not finished:
510 |
511 | payload.update({"limit": limit, "offset": iterations * limit, "details-level": details_level})
512 |
513 | api_res = client.api_call(command, payload)
514 |
515 | if api_res.success is False:
516 | return api_res
517 |
518 | total_objects = api_res.data["total"] # total number of objects
519 | received_objects = api_res.data["to"] # number of objects we got so far
520 | for container_key in container_keys:
521 | all_objects[container_key] += api_res.data[container_key]
522 |
523 | all_objects["objects-dictionary"] = add_and_parse_object_dictionary_to_map(api_res.data["objects-dictionary"],
524 | all_objects["objects-dictionary"])
525 |
526 | # did we get all the objects that we're supposed to get
527 | if received_objects == total_objects:
528 | finished = True
529 |
530 | iterations += 1
531 |
532 | return all_objects
533 |
534 |
535 | # Add only new object from object_dictionary that not exists in object_dictionary_as_map
536 | def add_and_parse_object_dictionary_to_map(object_dictionary, object_dictionary_as_map):
537 | for object in object_dictionary:
538 | object_dictionary_as_map[object.get('uid')] = object
539 |
540 | return object_dictionary_as_map
541 |
542 |
543 | # Get full rule-base as a list of rules
544 | def get_rulebase(show_rulebase_res):
545 | rulebase = conceal_sections(show_rulebase_res.get('rulebase'))
546 | return rulebase
547 |
548 |
549 | # Get object-dictionary from show access-rulebase as a map
550 | def get_object_dictionary(show_rulebase_res):
551 | object_dictionary = show_rulebase_res.get('objects-dictionary')
552 | return object_dictionary
553 |
554 |
555 | # Get rule last hit time
556 | # Note: If the rule does not have any hits OR rule modified after the last hit, last hit time will be counted as the last modify time.
557 | def get_rule_last_hit_time(rule):
558 | # Get last modify time & convert it to datetime object
559 | rule_last_modify_time = convert_date_object_to_datetime(rule['meta-info']['last-modify-time'])
560 |
561 | # Get last hit time & convert it to datetime object (if the rule has one hit - at least)
562 | if 'last-date' in rule['hits']:
563 | rule_last_hit_time = convert_date_object_to_datetime(rule['hits']['last-date'])
564 | else:
565 | return rule_last_modify_time
566 |
567 | if rule_last_hit_time > rule_last_modify_time:
568 | return rule_last_hit_time
569 | else:
570 | return rule_last_modify_time
571 |
572 |
573 | # Read plan file as JSON format
574 | def read_plan_json(file):
575 | try:
576 | with open(file, 'r') as import_plan_file:
577 | return json.load(import_plan_file)
578 | except Exception as e:
579 | print('Error: invalid json \'import-plan-file\'. %s' % e)
580 | exit(1)
581 |
582 |
583 | # Write plan to log file in JSON format
584 | def write_plan_json(plan, file):
585 | with open(file, 'w') as output_file:
586 | json.dump(plan, output_file, indent=4, default=jdefault, sort_keys=True)
587 | output_file.write('\n')
588 |
589 |
590 | # Build the structure of the plan output - keep specific keys, add 'skipped', add summary
591 | def build_output_structure(plan, summary, operation):
592 | new_plan = {}
593 | summary['total-candidate-disable-rules'] = 0
594 | summary['total-candidate-delete-rules'] = 0
595 | summary['total-skipped-packages'] = 0
596 | summary['total-skipped-layers'] = 0
597 | summary['total-skipped-rules'] = 0
598 |
599 | new_plan['operation'] = operation
600 | new_plan['thresholds'] = {"delete-after": get_global_thresholds().get("delete"),
601 | "disable-after": get_global_thresholds().get("disable")}
602 | new_plan['packages'] = []
603 | new_plan['skipped-packages'] = []
604 |
605 | # Keys to save
606 | keep_package_keys = ['uid', 'name', 'type']
607 | keep_skipped_package_keys = ['uid', 'name', 'type', 'skipped-reason']
608 | keep_layer_keys = ['uid', 'domain', 'name', 'shared', 'type', 'disable-rules', 'delete-rules', 'skipped-rules']
609 | keep_targets_keys = ['uid', 'name', 'type']
610 |
611 | for package in plan.get('packages', []):
612 |
613 | if 'skipped-reason' in package:
614 | new_package = {key: package.get(key, {}) for key in keep_skipped_package_keys}
615 | new_plan['skipped-packages'].append(new_package)
616 | summary['total-skipped-packages'] += 1
617 | else:
618 | new_package = {key: package.get(key, {}) for key in keep_package_keys}
619 | new_package['access-layers'] = []
620 | new_package['installation-targets'] = []
621 |
622 | for layer in package.get('access-layers', []):
623 | new_layer = {key: layer.get(key, {}) for key in keep_layer_keys}
624 |
625 | if 'skipped-reason' in layer:
626 | new_package['skipped-layers'].append(new_layer)
627 | summary['total-skipped-layers'] += 1
628 | else:
629 | new_layer['disable-rules'] = {'total': len(layer.get('disable-rules', [])),
630 | "rules": layer.get('disable-rules', [])}
631 | new_layer['delete-rules'] = {'total': len(layer.get('delete-rules', [])),
632 | "rules": layer.get('delete-rules', [])}
633 | new_layer['skipped-rules'] = {'total': len(layer.get('skipped-rules', [])),
634 | "rules": layer.get('skipped-rules', [])}
635 | new_layer['objects-dictionary'] = layer.get('objects-dictionary', [])
636 | new_package['access-layers'].append(new_layer)
637 |
638 | summary['total-candidate-disable-rules'] += len(layer.get('disable-rules', []))
639 | summary['total-candidate-delete-rules'] += len(layer.get('delete-rules', []))
640 | summary['total-skipped-rules'] += len(layer.get('skipped-rules', []))
641 |
642 | installation_targets = package.get('installation-targets')
643 | if installation_targets == 'all':
644 | new_package['installation-targets'].append(installation_targets)
645 | else:
646 | for target in package.get('installation-targets', []):
647 | new_target = {key: target.get(key, {}) for key in keep_targets_keys}
648 | new_package['installation-targets'].append(new_target)
649 |
650 | new_plan['packages'].append(new_package)
651 |
652 | return new_plan
653 |
654 |
655 | # Print the summary we built to the user
656 | def print_summary(summary):
657 | print_msg("Plan Summary:")
658 | print_msg(
659 | "Thresholds: {{delete-after: {0} days, disable-after: {1} days}}".format(get_global_thresholds().get("delete"),
660 | get_global_thresholds().get(
661 | "disable")))
662 | print_msg(" {0} {1} candidate to be disabled.".format(summary.get('total-candidate-disable-rules'),
663 | get_appropriate_wording_by_total_number(
664 | summary.get('total-candidate-disable-rules'), "rule",
665 | True)))
666 | print_msg(" {0} {1} candidate to be deleted.".format(summary.get('total-candidate-delete-rules'),
667 | get_appropriate_wording_by_total_number(
668 | summary.get('total-candidate-delete-rules'), "rule",
669 | True)))
670 | print_msg(" {0} {1} skipped. See skipped-packages & skipped-reason in json output file.".format(
671 | summary.get('total-skipped-packages'),
672 | get_appropriate_wording_by_total_number(summary.get('total-skipped-packages'), "package", False)))
673 | print_msg(" {0} {1} skipped. See skipped-layers & skipped-reason in json output file.".format(
674 | summary.get('total-skipped-layers'),
675 | get_appropriate_wording_by_total_number(summary.get('total-skipped-layers'), "layer", False)))
676 | print_msg(" {0} {1} skipped. See skipped-rules & skipped-reason in json output file.".format(
677 | summary.get('total-skipped-rules'),
678 | get_appropriate_wording_by_total_number(summary.get('total-skipped-rules'), "rule", False)))
679 |
680 |
681 | def get_appropriate_wording_by_total_number(total_num, word_to_change, is_present):
682 | if total_num == 1:
683 | if is_present is True:
684 | word_to_change += " is"
685 | else:
686 | word_to_change += " was"
687 | else:
688 | if is_present is True:
689 | word_to_change += "s are"
690 | else:
691 | word_to_change += "s were"
692 |
693 | return word_to_change
694 |
695 |
696 | # Conceal all sections from rulebase - make same
697 | # rulebase without sections
698 | def conceal_sections(rulebase):
699 | new_rulebase = []
700 |
701 | for rule in rulebase:
702 | # Rulebase has section
703 | if 'rulebase' in rule:
704 | new_rulebase.extend(rule.get('rulebase'))
705 | else:
706 | new_rulebase.append(rule)
707 |
708 | return new_rulebase
709 |
710 |
711 | # Apply the plan results
712 | def apply_plan(client, plan):
713 | for package in get_value_by_key_with_validation(plan, 'packages'):
714 | print_msg("Package {}".format(package['name']))
715 |
716 | for layer in get_value_by_key_with_validation(package, 'access-layers'):
717 |
718 | print_msg(" Layer {}".format(layer['name']))
719 |
720 | for rule in get_value_by_key_with_validation(get_value_by_key_with_validation(layer, 'disable-rules'),
721 | 'rules'):
722 | if disable_rule(rule, layer, client) is False:
723 | return False
724 |
725 | for rule in get_value_by_key_with_validation(get_value_by_key_with_validation(layer, 'delete-rules'),
726 | 'rules'):
727 | if delete_rule(rule, layer, client) is False:
728 | return False
729 | return True
730 |
731 |
732 | # get value by key from object with validation
733 | def get_value_by_key_with_validation(object, key):
734 | res_data = object.get(key)
735 | if res_data is None:
736 | print('\n Error: \'import-plan-file\' is corrupted, missing \'{}\' object'.format(key))
737 | exit(1)
738 |
739 | return res_data
740 |
741 |
742 | # Login with user arguments - in non-apply operations, login with read-only
743 | def login(user_args, client):
744 | # Read-only login in non-apply operations
745 | login_read_only = (not is_apply_operation(user_args.operation))
746 | session_details = get_session_details(user_args, login_read_only)
747 | if user_args.root is not None and user_args.root.lower() == 'true':
748 | if user_args.management == '127.0.0.1':
749 | login_res = client.login_as_root(domain=user_args.domain,
750 | payload=dict({"read-only": str(login_read_only)}, **session_details))
751 | else:
752 | print_msg(
753 | "Error: Command contains ambigious parameters. Management server remote ip is unexpected when logging "
754 | "in as root.")
755 | exit(1)
756 | else:
757 | if user_args.api_key and len(user_args.api_key) > 0:
758 | login_res = client.login_with_api_key(user_args.api_key, domain=user_args.domain,
759 | read_only=login_read_only, payload=session_details)
760 | else:
761 | login_res = client.login(user_args.username, user_args.password, domain=user_args.domain,
762 | read_only=login_read_only, payload=session_details)
763 |
764 | exit_failure("Failed to login.", login_res)
765 |
766 |
767 | def get_session_details(user_args, login_read_only):
768 | session_details = {}
769 | if login_read_only is False:
770 | session_details["session-name"] = user_args.session_name
771 | session_details["session-description"] = user_args.session_description
772 |
773 | return session_details
774 |
775 |
776 | # Set rule changes
777 | # *** If you wold like to your logic as part of rule disabling. This is the place ***
778 | # *** For example: set rule position to bottom, you need add to API call 'new-position' parameter with value bottom.***
779 | def disable_rule(rule, layer, api_client):
780 | global DATETIME_NOW
781 | global DATETIME_FORMAT
782 |
783 | # Disable rule & set disabled-time & add comment
784 | set_rule_res = api_client.api_call("set-access-rule",
785 | {"uid": rule['uid'], "layer": layer['uid'], "enabled": "false",
786 | "custom-fields": {"field-3": DATETIME_NOW.strftime(DATETIME_FORMAT)},
787 | "comments": rule[
788 | 'comments'] + " -This rule changed automatically by the policyCleanUp tool"})
789 |
790 | return not is_failure(" Failed to set rule No.{} with UID {}.".format(rule['rule-number'], rule['uid']),
791 | set_rule_res)
792 |
793 |
794 | # delete the rule
795 | def delete_rule(rule, layer, api_client):
796 | delete_rule_res = api_client.api_call("delete-access-rule", {"uid": rule['uid'], "layer": layer['uid']})
797 | return not is_failure(" Failed to delete rule No.{} with UID {}.".format(rule['rule-number'], rule['uid']),
798 | delete_rule_res)
799 |
800 |
801 | # Determine which layers should be skipped
802 | def validate_layer(layer):
803 | if is_global_object(layer) is True:
804 | return False
805 |
806 | return True
807 |
808 |
809 | def get_groups(client):
810 | response = client.api_query(command="show-groups", details_level="uid")
811 | if response.success is False:
812 | print_msg("Failed to get groups. Error: {}".format(response.error_message.encode('utf-8')))
813 | return
814 | return set(response.data)
815 |
816 |
817 | def is_target_valid_group(target_uid, valid_targets, client, groups):
818 | if groups is None or len(groups) == 0:
819 | return False
820 |
821 | if target_uid not in groups:
822 | # target_uid is not a network group
823 | return False
824 |
825 | response = client.api_call(command="show-group", payload={"uid": target_uid})
826 | if response.success is False:
827 | print_msg(
828 | "Failed to get group with UID {}. Error: {}".format(target_uid, response.error_message.encode('utf-8')))
829 | return False
830 |
831 | members = response.data.get("members")
832 | if members is None or len(members) == 0:
833 | # group has no members
834 | return False
835 |
836 | for member in members:
837 | if member.get("uid") not in valid_targets:
838 | return False
839 | return True
840 |
841 |
842 | # Check if the install-on list contains invalid target & rule is updated on targets
843 | def validate_rule(rule, package, valid_targets, client, groups):
844 | global INSTALL_ON_ALL_TARGETS
845 |
846 | package_name = package.get('name')
847 | install_on = rule.get('install-on')
848 | package_valid_targets = valid_targets['packages-targets'].get(package_name)
849 |
850 | # Validate that rule is up-do-date
851 | rule_last_modify_time = convert_date_object_to_datetime(rule['meta-info']['last-modify-time'])
852 | package_installation_time = package_valid_targets['minimal-installation-time']
853 | if rule_last_modify_time > package_installation_time:
854 | rule['skipped-reason'] = "has modified after policy installation"
855 | return False
856 |
857 | # Installation targets all - constant UID
858 | if install_on[0] == INSTALL_ON_ALL_TARGETS:
859 | if package_valid_targets.get('all-targets-valid') is True:
860 | return True
861 | else:
862 | rule['skipped-reason'] = "one of the targets in the install-on list is invalid"
863 | return False
864 |
865 | valid_targets = package_valid_targets.get('targets', {})
866 |
867 | for target_uid in install_on:
868 | if target_uid not in valid_targets:
869 | if not is_target_valid_group(target_uid, valid_targets, client, groups):
870 | rule['skipped-reason'] = "target with UID {} in the install-on list is invalid".format(target_uid)
871 | return False
872 |
873 | return True
874 |
875 |
876 | # Check if domain hit count global property flag is on. If the flag is off - exit
877 | def mgmt_hitcount_on(client, domain):
878 | # Get global properties
879 | ##### These APIs provide direct access to different objects and fields in the database. As a result, when the objects schema change, scripts that relied on specific schema fields may break.#####
880 | ##### When you have the option, always prefer to use the documented APIs and not the generic APIs #####
881 | show_prop_res = client.api_call("show-generic-objects",
882 | {"class-name": "com.checkpoint.objects.classes.dummy.CpmiFirewallProperties",
883 | "details-level": "full"})
884 | exit_failure("Failed to get properties.", show_prop_res)
885 |
886 | domain_name = domain.get('name')
887 |
888 | # Loop over all global properties, and search for current domain hit count flag
889 | for obj in show_prop_res.data.get('objects'):
890 | if (obj['domain']['name'] == domain_name):
891 | if obj.get('enableHitCount') == 0:
892 | print_msg("Hit Count flag is off for domain {}, please turn it on.".format(domain['name']))
893 | exit(0)
894 | else:
895 | return True
896 |
897 |
898 | # get list of all objects from objects dictionary that appear in rule only by uid
899 | def get_objects_of_rule_from_objects_dictionary(rule, object_dictionary):
900 | # accumulate all the objects rule from objects-dictionary that appear by uid
901 | objects_of_rule_to_add = []
902 |
903 | for value in rule.values():
904 | if (isinstance(value, (list, dict))):
905 | for uid in value:
906 | specific_object_to_add = object_dictionary.get(uid)
907 | if (specific_object_to_add is not None):
908 | objects_of_rule_to_add.append(specific_object_to_add)
909 |
910 | else:
911 | specific_object_to_add = object_dictionary.get(value)
912 | if (specific_object_to_add is not None):
913 | objects_of_rule_to_add.append(specific_object_to_add)
914 |
915 | return objects_of_rule_to_add
916 |
917 |
918 | # updates and return the sub dictionary with objects of rule that didn't exist before
919 | def add_to_sub_dictionary(objects_of_rule_to_add, sub_dictionary):
920 | for current_object in objects_of_rule_to_add:
921 | sub_dictionary[current_object.get('uid')] = current_object
922 |
923 | return sub_dictionary
924 |
925 |
926 | # parse map of sub objects dictionary to list of values
927 | def parse_sub_objects_dictionary_map_to_list_of_value(sub_dictionary):
928 | list_values_of_sub_objects_dictionary = []
929 |
930 | for value in sub_dictionary.values():
931 | list_values_of_sub_objects_dictionary.append(value)
932 |
933 | return list_values_of_sub_objects_dictionary
934 |
935 |
936 | # Determine in which conditions the rule should be disabled
937 | # Function checks if rule should be disable based on hit count. Rule would be disabled if it's last hit date or last modified date (The closest date) is former to today's date minus the threshold.
938 | # Note: This function can be adjust according to needs. If you want to change the logic which determines whether rule should be disabled not based on hit count.
939 | def rule_should_be_disabled(rule, global_thresholds):
940 | global DATETIME_NOW
941 |
942 | # Calculate final thresholds
943 | final_disable_threshold = get_rule_final_threshold(rule, 'field-1', 'disable', global_thresholds)
944 | if final_disable_threshold is None:
945 | return False
946 |
947 | # Get rule last hit time
948 | rule_last_hit_time = get_rule_last_hit_time(rule)
949 |
950 | # Disable rule - no hits too long
951 | if rule_last_hit_time + datetime.timedelta(days=final_disable_threshold) < DATETIME_NOW:
952 | return True
953 |
954 | return False
955 |
956 |
957 | # Determine in which conditions the rule should be deleted
958 | def rule_should_be_deleted(rule, global_thresholds):
959 | # Get final delete threshold (after local overrides)
960 | final_delete_threshold = get_rule_final_threshold(rule, 'field-2', 'delete', global_thresholds)
961 | if final_delete_threshold is None:
962 | return False
963 |
964 | # Get rule disabled time by the tool
965 | rule_disabled_time = get_rule_disabled_time(rule)
966 | if rule_disabled_time is None:
967 | return False
968 |
969 | # Delete rule - disabled too long
970 | if rule_disabled_time + datetime.timedelta(days=final_delete_threshold) < DATETIME_NOW:
971 | return True
972 |
973 | return False
974 |
975 |
976 | # Convert checkpoint date reply to datetime object
977 | def convert_date_object_to_datetime(date_object):
978 | date_posix = date_object.get('posix', 0)
979 | # milli-seconds to seconds
980 | date_posix = int(date_posix) / 1000
981 | return datetime.datetime.fromtimestamp(date_posix)
982 |
983 |
984 | # Check if the object is global object
985 | def is_global_object(object):
986 | global GLOBAL_DOMAIN_NAME
987 |
988 | return (object['domain']['name'] == GLOBAL_DOMAIN_NAME)
989 |
990 |
991 | # Determine in which operations the tool should create plan
992 | def is_plan(args):
993 | return (args.import_plan_file is None)
994 |
995 |
996 | # Determine in which operations the tool should disable/delete rules (apply, apply_without_publish)
997 | def is_apply_operation(operation):
998 | return (operation == "apply") or (operation == "apply_without_publish")
999 |
1000 |
1001 | # Determine in which operations the tool should publish changes (apply)
1002 | def is_publish_operation(operation):
1003 | return (operation == "apply")
1004 |
1005 |
1006 | # Check if the API-call failed & skipped reason message
1007 | def skip_failure(skipped_object, error_msg, response):
1008 | if (hasattr(response, 'success')):
1009 | if response.success is False:
1010 | skipped_object['skipped-reason'] = error_msg + " Error: {}".format(response.error_message.encode('utf-8'))
1011 | return True
1012 |
1013 | return False
1014 |
1015 |
1016 | # Check if the API-call failed & print error message
1017 | def is_failure(error_msg, response):
1018 | if response.success is False:
1019 | print_msg(error_msg + " Error: {}".format(response.error_message.encode('utf-8')))
1020 | return True
1021 |
1022 | return False
1023 |
1024 |
1025 | # Check if running on MDS and didn't supply domain
1026 | def check_validation_for_mds(client, domain):
1027 | api_res = client.api_call("show-mdss")
1028 | if api_res.success and int(api_res.data.get('total')) != 0:
1029 | if domain is None:
1030 | print_msg(" Error: You must provide a domain in a Multi-Domain-Management environment.")
1031 | exit(1)
1032 |
1033 |
1034 | # Exit if the API-call failed & print error message
1035 | def exit_failure(error_msg, response):
1036 | if response.success is False:
1037 | print_msg(error_msg + " Error: {}".format(response.error_message))
1038 | exit(1)
1039 |
1040 |
1041 | # Print message with time description
1042 | def print_msg(msg):
1043 | print("[{}] {}".format(datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"), msg))
1044 |
1045 |
1046 | # Default value for non-json serializable objects
1047 | def jdefault(obj):
1048 | return ""
1049 |
1050 |
1051 | def main():
1052 | global DATETIME_NOW
1053 |
1054 | print()
1055 |
1056 | # Store output file information, will be serialize to JSON file
1057 | plan = {}
1058 |
1059 | # Parse user arguments
1060 | user_args = parse_arguments()
1061 |
1062 | client_args = APIClientArgs(server=user_args.management, port=user_args.port)
1063 |
1064 | with APIClient(client_args) as client:
1065 |
1066 | # The API client, would look for the server's certificate SHA1 fingerprint in a file.
1067 | # If the fingerprint is not found on the file, it will ask the user if he accepts the server's fingerprint.
1068 | # In case the user does not accept the fingerprint, exit the program.
1069 | if client.check_fingerprint() is False:
1070 | print_msg("Could not get the server's fingerprint - Check connectivity with the server.")
1071 | exit(1)
1072 |
1073 | # Login to server, (read-only in non-apply operations)
1074 | login(user_args, client)
1075 |
1076 | # Get global thresholds (disable & delete)
1077 | global_thresholds = get_global_thresholds()
1078 |
1079 | # Check if running on MDS and didn't supply domain
1080 | check_validation_for_mds(client, user_args.domain)
1081 |
1082 | # Plan
1083 | if is_plan(user_args):
1084 |
1085 | print_msg("Plan")
1086 |
1087 | # Store list of all packages information that we should run on
1088 | plan['packages'] = packages_to_run(user_args.package, client)
1089 |
1090 | # If management hit count flag in off, hits won't be counted - exit
1091 | mgmt_hitcount_on(client, plan['packages'][0]["domain"])
1092 |
1093 | # Pre-processing to calculate all invalid targets
1094 | print_msg("Validate targets")
1095 | valid_targets = valid_packages_targets(plan['packages'], client)
1096 |
1097 | groups = get_groups(client)
1098 | for package in plan.get('packages'):
1099 | # Skip invalid packages (all installation targets are invalid)
1100 | if validate_package(package, valid_targets) is False:
1101 | continue
1102 |
1103 | print_msg("Package {}".format(package['name']))
1104 |
1105 | # Loop over policy layers
1106 | for layer in package.get('access-layers'):
1107 |
1108 | print_msg(" Layer {}".format(layer['name']))
1109 |
1110 | # Skip invalid layers (global layers)
1111 | if validate_layer(layer) is False:
1112 | continue
1113 |
1114 | layer['skipped-rules'] = []
1115 | layer['disable-rules'] = []
1116 | layer['delete-rules'] = []
1117 | sub_dictionary = {}
1118 |
1119 | show_access_rulebase_res = show_access_rulebase(layer, client)
1120 | object_dictionary_as_map = get_object_dictionary(show_access_rulebase_res)
1121 |
1122 | # Get full rule-base as a list of rules
1123 | rulebase = get_rulebase(show_access_rulebase_res)
1124 | if rulebase is None:
1125 | continue
1126 |
1127 | # Loop over layer rules
1128 | for rule in rulebase:
1129 |
1130 | # Rule is enabled
1131 | if rule.get('enabled') is True:
1132 |
1133 | # Skip rule with invalid target in install-on list OR rule modify after installation
1134 | if validate_rule(rule, package, valid_targets, client, groups) is False:
1135 | if 'skipped-reason' in rule:
1136 | layer['skipped-rules'].append(rule)
1137 | continue
1138 |
1139 | # Check if rule should be disabled - no hits for too long
1140 | if rule_should_be_disabled(rule, global_thresholds) is True:
1141 | objects_of_rule_to_add = get_objects_of_rule_from_objects_dictionary(rule,
1142 | object_dictionary_as_map)
1143 | sub_dictionary = add_to_sub_dictionary(objects_of_rule_to_add, sub_dictionary)
1144 |
1145 | # Add to plan that the rule should be disabled
1146 | layer['disable-rules'].append(rule)
1147 |
1148 | # Rule is disabled
1149 | else:
1150 | # check if the rule should be deleted - disabled too long
1151 | if rule_should_be_deleted(rule, global_thresholds) is True:
1152 | objects_of_rule_to_add = get_objects_of_rule_from_objects_dictionary(rule,
1153 | object_dictionary_as_map)
1154 | sub_dictionary = add_to_sub_dictionary(objects_of_rule_to_add, sub_dictionary)
1155 |
1156 | # Add to plan that the rule should be deleted
1157 | layer['delete-rules'].append(rule)
1158 |
1159 | layer['objects-dictionary'] = parse_sub_objects_dictionary_map_to_list_of_value(sub_dictionary)
1160 |
1161 | summary = {};
1162 | plan = build_output_structure(plan, summary, user_args.operation)
1163 |
1164 | if valid_targets['packages-targets']:
1165 | print_summary(summary)
1166 |
1167 | # Write plan to log file in JSON format
1168 | write_plan_json(plan, user_args.output_file)
1169 |
1170 | print_msg('Plan process has finished.\n')
1171 | print_msg('The output file in: {}'.format(os.path.abspath(user_args.output_file)))
1172 |
1173 | # Get plan-file path as argument
1174 | else:
1175 | plan = read_plan_json(user_args.import_plan_file)
1176 |
1177 | # Apply plan only when the operation is apply/apply_without_publish
1178 | if is_apply_operation(user_args.operation) is True:
1179 |
1180 | print()
1181 | print_msg("Apply plan")
1182 |
1183 | # Disable & Delete rules by the plan. If apply failed in one of them - stop and discard changes
1184 | if apply_plan(client, plan) is False:
1185 | print_msg("Discard")
1186 | discard_res = client.api_call("discard")
1187 | is_failure("Failed to discard changes.", discard_res)
1188 | print()
1189 | exit(1)
1190 |
1191 | # Publish changes only when the operation is apply
1192 | if is_publish_operation(user_args.operation):
1193 |
1194 | print_msg("Publish")
1195 | publish_res = client.api_call("publish")
1196 | is_failure("Failed to publish changes.", publish_res)
1197 |
1198 | else:
1199 |
1200 | print_msg("Continue session in smartconsole")
1201 | continue_smartconsole_res = client.api_call("continue-session-in-smartconsole")
1202 | is_failure("Failed to continue session in smartconsole.", continue_smartconsole_res)
1203 |
1204 | print_msg('Apply process has finished successfully.')
1205 |
1206 | print()
1207 |
1208 |
1209 | if __name__ == "__main__":
1210 | main()
1211 |
--------------------------------------------------------------------------------