5 |
6 | # tfma.Validate
7 |
8 | ```python
9 | tfma.Validate(
10 | *args,
11 | **kwargs
12 | )
13 | ```
14 |
15 |
16 |
17 | Performs validation of alternative evaluations.
18 |
19 | #### Args:
20 |
21 | * `extracts`: PCollection of extracts.
22 | * `alternatives`: Dict of PTransforms (Extracts -> Evaluation) whose
23 | output will be compared for validation purposes (e.g. 'baseline' vs
24 | 'candidate').
25 | * `validators`: List of validators for validating the output from
26 | running the alternatives. The Validation outputs produced by the validators
27 | will be merged into a single output. If there are overlapping output keys,
28 | later outputs will replace earlier outputs sharing the same key.
29 |
30 | #### Returns:
31 |
32 | Validation dict.
33 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/WriteResults.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.WriteResults
7 |
8 | ```python
9 | tfma.WriteResults(
10 | *args,
11 | **kwargs
12 | )
13 | ```
14 |
15 |
16 |
17 | Writes Evaluation or Validation results using given writers.
18 |
19 | #### Args:
20 |
21 | * `evaluation_or_validation`: Evaluation or Validation output.
22 | * `writers`: Writes to use for writing out output.
23 |
24 | #### Raises:
25 |
26 | * `ValueError`: If Evaluation or Validation is empty.
27 |
28 | #### Returns:
29 |
30 | beam.pvalue.PDone.
31 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/compound_key.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.compound_key
7 |
8 | ```python
9 | tfma.compound_key(
10 | keys,
11 | separator=KEY_SEPARATOR
12 | )
13 | ```
14 |
15 | Defined in
16 | [`util.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/util.py).
17 |
18 |
19 |
20 | Returns a compound key based on a list of keys.
21 |
22 | #### Args:
23 |
24 | * `keys`: Keys used to make up compound key.
25 | * `separator`: Separator between keys. To ensure the keys can be parsed
26 | out of any compound key created, any use of a separator within a key will be
27 | replaced by two separators.
28 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/create_keys_key.md:
--------------------------------------------------------------------------------
1 |
5 |
6 | # tfma.default_evaluators
7 |
8 | ```python
9 | tfma.default_evaluators(
10 | eval_shared_model,
11 | desired_batch_size=None,
12 | num_bootstrap_samples=None
13 | )
14 | ```
15 |
16 | Defined in
17 | [`api/model_eval_lib.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/api/model_eval_lib.py).
18 |
19 |
20 |
21 | Returns the default evaluators for use in ExtractAndEvaluate.
22 |
23 | #### Args:
24 |
25 | * `eval_shared_model`: Shared model parameters for EvalSavedModel.
26 | * `desired_batch_size`: Optional batch size for batching in Aggregate.
27 | * `num_bootstrap_samples`: Number of bootstrap samples to draw. If more
28 | than 1, confidence intervals will be computed for metrics. Suggested value
29 | is at least 20.
30 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/default_extractors.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.default_extractors
7 |
8 | ```python
9 | tfma.default_extractors(
10 | eval_shared_model,
11 | slice_spec=None,
12 | desired_batch_size=None,
13 | materialize=True
14 | )
15 | ```
16 |
17 | Defined in
18 | [`api/model_eval_lib.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/api/model_eval_lib.py).
19 |
20 |
21 |
22 | Returns the default extractors for use in ExtractAndEvaluate.
23 |
24 | #### Args:
25 |
26 | * `eval_shared_model`: Shared model parameters for EvalSavedModel.
27 | * `slice_spec`: Optional list of SingleSliceSpec specifying the slices
28 | to slice the data into. If None, defaults to the overall slice.
29 | * `desired_batch_size`: Optional batch size for batching in Aggregate.
30 | * `materialize`: True to have extractors create materialized output.
31 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/default_writers.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.default_writers
7 |
8 | ```python
9 | tfma.default_writers(output_path)
10 | ```
11 |
12 | Defined in
13 | [`api/model_eval_lib.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/api/model_eval_lib.py).
14 |
15 |
16 |
17 | Returns the default writers for use in WriteResults.
18 |
19 | #### Args:
20 |
21 | * `output_path`: Path to store results files under.
22 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/evaluators.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # Module: tfma.evaluators
7 |
8 | Defined in
9 | [`evaluators/__init__.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/evaluators/__init__.py).
10 |
11 |
12 |
13 | Init module for TensorFlow Model Analysis evaluators.
14 |
15 | ## Classes
16 |
17 | [`class Evaluator`](../tfma/evaluators/Evaluator.md): Evaluator(stage_name,
18 | run_after, ptransform)
19 |
20 | ## Functions
21 |
22 | [`AnalysisTableEvaluator(...)`](../tfma/evaluators/AnalysisTableEvaluator.md):
23 | Creates an Evaluator for returning Extracts data for analysis.
24 |
25 | [`MetricsAndPlotsEvaluator(...)`](../tfma/evaluators/MetricsAndPlotsEvaluator.md):
26 | Creates an Evaluator for evaluating metrics and plots.
27 |
28 | [`verify_evaluator(...)`](../tfma/evaluators/verify_evaluator.md): Verifies
29 | evaluator is matched with an extractor.
30 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/evaluators/AnalysisTableEvaluator.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.evaluators.AnalysisTableEvaluator
7 |
8 | ```python
9 | tfma.evaluators.AnalysisTableEvaluator(
10 | key=constants.ANALYSIS_KEY,
11 | run_after=extractor.LAST_EXTRACTOR_STAGE_NAME,
12 | include=None,
13 | exclude=None
14 | )
15 | ```
16 |
17 | Defined in
18 | [`evaluators/analysis_table_evaluator.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/evaluators/analysis_table_evaluator.py).
19 |
20 |
21 |
22 | Creates an Evaluator for returning Extracts data for analysis.
23 |
24 | If both include and exclude are None then tfma.INPUT_KEY extracts will be
25 | excluded by default.
26 |
27 | #### Args:
28 |
29 | * `key`: Name to use for key in Evaluation output.
30 | * `run_after`: Extractor to run after (None means before any
31 | extractors).
32 | * `include`: Keys of extracts to include in output. Keys starting with
33 | '_' are automatically filtered out at write time.
34 | * `exclude`: Keys of extracts to exclude from output.
35 |
36 | #### Returns:
37 |
38 | Evaluator for collecting analysis data. The output is stored under the key
39 | 'analysis'.
40 |
41 | #### Raises:
42 |
43 | * `ValueError`: If both include and exclude are used.
44 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/evaluators/Evaluator.md:
--------------------------------------------------------------------------------
1 |
5 |
6 | # tfma.export.build_parsing_eval_input_receiver_fn
7 |
8 | ```python
9 | tfma.export.build_parsing_eval_input_receiver_fn(
10 | feature_spec,
11 | label_key
12 | )
13 | ```
14 |
15 | Defined in
16 | [`eval_saved_model/export.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/eval_saved_model/export.py).
17 |
18 |
19 |
20 | Build a eval_input_receiver_fn expecting fed tf.Examples.
21 |
22 | Creates a eval_input_receiver_fn that expects a serialized tf.Example fed into a
23 | string placeholder. The function parses the tf.Example according to the provided
24 | feature_spec, and returns all parsed Tensors as features.
25 |
26 | #### Args:
27 |
28 | * `feature_spec`: A dict of string to
29 | `VarLenFeature`/`FixedLenFeature`.
30 | * `label_key`: The key for the label column in the feature_spec. Note
31 | that the label must be part of the feature_spec. If None, does not pass a
32 | label to the EvalInputReceiver (note that label_key must be None and not
33 | simply the empty string for this case).
34 |
35 | #### Returns:
36 |
37 | A eval_input_receiver_fn suitable for use with TensorFlow model analysis.
38 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/exporter.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # Module: tfma.exporter
7 |
8 | Defined in
9 | [`eval_saved_model/exporter.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/eval_saved_model/exporter.py).
10 |
11 |
12 |
13 | `Exporter` class represents different flavors of model export.
14 |
15 | ## Classes
16 |
17 | [`class FinalExporter`](../tfma/exporter/FinalExporter.md): This class exports
18 | the EvalSavedModel in the end.
19 |
20 | [`class LatestExporter`](../tfma/exporter/LatestExporter.md): This class
21 | regularly exports the EvalSavedModel.
22 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/extractors/Extractor.md:
--------------------------------------------------------------------------------
1 |
5 |
6 | # tfma.extractors.Filter
7 |
8 | ```python
9 | tfma.extractors.Filter(
10 | *args,
11 | **kwargs
12 | )
13 | ```
14 |
15 |
16 |
17 | Filters extracts to include/exclude specified keys.
18 |
19 | #### Args:
20 |
21 | * `extracts`: PCollection of extracts.
22 | * `include`: Keys to include in output.
23 | * `exclude`: Keys to exclude from output.
24 |
25 | #### Returns:
26 |
27 | Filtered PCollection of Extracts.
28 |
29 | #### Raises:
30 |
31 | * `ValueError`: If both include and exclude are used.
32 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/extractors/PredictExtractor.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.extractors.PredictExtractor
7 |
8 | ```python
9 | tfma.extractors.PredictExtractor(
10 | eval_shared_model,
11 | desired_batch_size=None,
12 | materialize=True
13 | )
14 | ```
15 |
16 | Defined in
17 | [`extractors/predict_extractor.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/extractors/predict_extractor.py).
18 |
19 |
20 |
21 | Creates an Extractor for TFMAPredict.
22 |
23 | The extractor's PTransform loads and runs the eval_saved_model against every
24 | example yielding a copy of the Extracts input with an additional extract of type
25 | FeaturesPredictionsLabels keyed by tfma.FEATURES_PREDICTIONS_LABELS_KEY.
26 |
27 | #### Args:
28 |
29 | * `eval_shared_model`: Shared model parameters for EvalSavedModel.
30 | * `desired_batch_size`: Optional batch size for batching in Aggregate.
31 | * `materialize`: True to call the FeatureExtractor to add
32 | MaterializedColumn entries for the features, predictions, and labels.
33 |
34 | #### Returns:
35 |
36 | Extractor for extracting features, predictions, labels, and other tensors during
37 | predict.
38 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/extractors/SliceKeyExtractor.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.extractors.SliceKeyExtractor
7 |
8 | ```python
9 | tfma.extractors.SliceKeyExtractor(
10 | slice_spec=None,
11 | materialize=True
12 | )
13 | ```
14 |
15 | Defined in
16 | [`extractors/slice_key_extractor.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/extractors/slice_key_extractor.py).
17 |
18 |
19 |
20 | Creates an extractor for extracting slice keys.
21 |
22 | The incoming Extracts must contain a FeaturesPredictionsLabels extract keyed by
23 | tfma.FEATURES_PREDICTIONS_LABELS_KEY. Typically this will be obtained by calling
24 | the PredictExtractor.
25 |
26 | The extractor's PTransform yields a copy of the Extracts input with an
27 | additional extract pointing at the list of SliceKeyType values keyed by
28 | tfma.SLICE_KEY_TYPES_KEY. If materialize is True then a materialized version of
29 | the slice keys will be added under the key tfma.MATERIALZED_SLICE_KEYS_KEY.
30 |
31 | #### Args:
32 |
33 | * `slice_spec`: Optional list of SingleSliceSpec specifying the slices
34 | to slice the data into. If None, defaults to the overall slice.
35 | * `materialize`: True to add MaterializedColumn entries for the slice
36 | keys.
37 |
38 | #### Returns:
39 |
40 | Extractor for slice keys.
41 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/load_eval_result.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.load_eval_result
7 |
8 | ```python
9 | tfma.load_eval_result(output_path)
10 | ```
11 |
12 | Defined in
13 | [`api/model_eval_lib.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/api/model_eval_lib.py).
14 |
15 |
16 |
17 | Creates an EvalResult object for use with the visualization functions.
18 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/load_eval_results.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.load_eval_results
7 |
8 | ```python
9 | tfma.load_eval_results(
10 | output_paths,
11 | mode
12 | )
13 | ```
14 |
15 | Defined in
16 | [`api/model_eval_lib.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/api/model_eval_lib.py).
17 |
18 |
19 |
20 | Run model analysis for a single model on multiple data sets.
21 |
22 | #### Args:
23 |
24 | * `output_paths`: A list of output paths of completed tfma runs.
25 | * `mode`: The mode of the evaluation. Currently, tfma.DATA_CENTRIC_MODE
26 | and tfma.MODEL_CENTRIC_MODE are supported.
27 |
28 | #### Returns:
29 |
30 | An EvalResults containing the evaluation results serialized at output_paths.
31 | This can be used to construct a time series view.
32 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/make_eval_results.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.make_eval_results
7 |
8 | ```python
9 | tfma.make_eval_results(
10 | results,
11 | mode
12 | )
13 | ```
14 |
15 | Defined in
16 | [`api/model_eval_lib.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/api/model_eval_lib.py).
17 |
18 |
19 |
20 | Run model analysis for a single model on multiple data sets.
21 |
22 | #### Args:
23 |
24 | * `results`: A list of TFMA evaluation results.
25 | * `mode`: The mode of the evaluation. Currently, tfma.DATA_CENTRIC_MODE
26 | and tfma.MODEL_CENTRIC_MODE are supported.
27 |
28 | #### Returns:
29 |
30 | An EvalResults containing all evaluation results. This can be used to construct
31 | a time series view.
32 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/multiple_data_analysis.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.multiple_data_analysis
7 |
8 | ```python
9 | tfma.multiple_data_analysis(
10 | model_location,
11 | data_locations,
12 | **kwargs
13 | )
14 | ```
15 |
16 | Defined in
17 | [`api/model_eval_lib.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/api/model_eval_lib.py).
18 |
19 |
20 |
21 | Run model analysis for a single model on multiple data sets.
22 |
23 | #### Args:
24 |
25 | * `model_location`: The location of the exported eval saved model.
26 | * `data_locations`: A list of data set locations.
27 | * `**kwargs`: The args used for evaluation. See
28 | tfma.run_model_analysis() for details.
29 |
30 | #### Returns:
31 |
32 | A tfma.EvalResults containing all the evaluation results with the same order as
33 | data_locations.
34 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/multiple_model_analysis.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.multiple_model_analysis
7 |
8 | ```python
9 | tfma.multiple_model_analysis(
10 | model_locations,
11 | data_location,
12 | **kwargs
13 | )
14 | ```
15 |
16 | Defined in
17 | [`api/model_eval_lib.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/api/model_eval_lib.py).
18 |
19 |
20 |
21 | Run model analysis for multiple models on the same data set.
22 |
23 | #### Args:
24 |
25 | * `model_locations`: A list of paths to the export eval saved model.
26 | * `data_location`: The location of the data files.
27 | * `**kwargs`: The args used for evaluation. See
28 | tfma.run_model_analysis() for details.
29 |
30 | #### Returns:
31 |
32 | A tfma.EvalResults containing all the evaluation results with the same order as
33 | model_locations.
34 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/post_export_metrics/auc.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.post_export_metrics.auc
7 |
8 | ```python
9 | tfma.post_export_metrics.auc(
10 | *args,
11 | **kwargs
12 | )
13 | ```
14 |
15 | Defined in
16 | [`post_export_metrics/post_export_metrics.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/post_export_metrics/post_export_metrics.py).
17 |
18 |
19 |
20 | This is the function that the user calls.
21 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/post_export_metrics/auc_plots.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.post_export_metrics.auc_plots
7 |
8 | ```python
9 | tfma.post_export_metrics.auc_plots(
10 | *args,
11 | **kwargs
12 | )
13 | ```
14 |
15 | Defined in
16 | [`post_export_metrics/post_export_metrics.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/post_export_metrics/post_export_metrics.py).
17 |
18 |
19 |
20 | This is the function that the user calls.
21 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/post_export_metrics/calibration_plot_and_prediction_histogram.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.post_export_metrics.calibration_plot_and_prediction_histogram
7 |
8 | ```python
9 | tfma.post_export_metrics.calibration_plot_and_prediction_histogram(
10 | *args,
11 | **kwargs
12 | )
13 | ```
14 |
15 | Defined in
16 | [`post_export_metrics/post_export_metrics.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/post_export_metrics/post_export_metrics.py).
17 |
18 |
19 |
20 | This is the function that the user calls.
21 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/post_export_metrics/confusion_matrix_at_thresholds.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.post_export_metrics.confusion_matrix_at_thresholds
7 |
8 | ```python
9 | tfma.post_export_metrics.confusion_matrix_at_thresholds(
10 | *args,
11 | **kwargs
12 | )
13 | ```
14 |
15 | Defined in
16 | [`post_export_metrics/post_export_metrics.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/post_export_metrics/post_export_metrics.py).
17 |
18 |
19 |
20 | This is the function that the user calls.
21 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/post_export_metrics/example_count.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.post_export_metrics.example_count
7 |
8 | ```python
9 | tfma.post_export_metrics.example_count(
10 | *args,
11 | **kwargs
12 | )
13 | ```
14 |
15 | Defined in
16 | [`post_export_metrics/post_export_metrics.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/post_export_metrics/post_export_metrics.py).
17 |
18 |
19 |
20 | This is the function that the user calls.
21 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/post_export_metrics/example_weight.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.post_export_metrics.example_weight
7 |
8 | ```python
9 | tfma.post_export_metrics.example_weight(
10 | *args,
11 | **kwargs
12 | )
13 | ```
14 |
15 | Defined in
16 | [`post_export_metrics/post_export_metrics.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/post_export_metrics/post_export_metrics.py).
17 |
18 |
19 |
20 | This is the function that the user calls.
21 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/post_export_metrics/mean_absolute_error.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.post_export_metrics.mean_absolute_error
7 |
8 | ```python
9 | tfma.post_export_metrics.mean_absolute_error(
10 | *args,
11 | **kwargs
12 | )
13 | ```
14 |
15 | Defined in
16 | [`post_export_metrics/post_export_metrics.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/post_export_metrics/post_export_metrics.py).
17 |
18 |
19 |
20 | This is the function that the user calls.
21 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/post_export_metrics/precision_at_k.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.post_export_metrics.precision_at_k
7 |
8 | ```python
9 | tfma.post_export_metrics.precision_at_k(
10 | *args,
11 | **kwargs
12 | )
13 | ```
14 |
15 | Defined in
16 | [`post_export_metrics/post_export_metrics.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/post_export_metrics/post_export_metrics.py).
17 |
18 |
19 |
20 | This is the function that the user calls.
21 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/post_export_metrics/recall_at_k.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.post_export_metrics.recall_at_k
7 |
8 | ```python
9 | tfma.post_export_metrics.recall_at_k(
10 | *args,
11 | **kwargs
12 | )
13 | ```
14 |
15 | Defined in
16 | [`post_export_metrics/post_export_metrics.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/post_export_metrics/post_export_metrics.py).
17 |
18 |
19 |
20 | This is the function that the user calls.
21 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/post_export_metrics/squared_pearson_correlation.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.post_export_metrics.squared_pearson_correlation
7 |
8 | ```python
9 | tfma.post_export_metrics.squared_pearson_correlation(
10 | *args,
11 | **kwargs
12 | )
13 | ```
14 |
15 | Defined in
16 | [`post_export_metrics/post_export_metrics.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/post_export_metrics/post_export_metrics.py).
17 |
18 |
19 |
20 | This is the function that the user calls.
21 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/types.md:
--------------------------------------------------------------------------------
1 |
5 |
6 | # tfma.unique_key
7 |
8 | ```python
9 | tfma.unique_key(
10 | key,
11 | current_keys,
12 | update_keys=False
13 | )
14 | ```
15 |
16 | Defined in
17 | [`util.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/util.py).
18 |
19 |
20 |
21 | Returns a unique key given a list of current keys.
22 |
23 | If the key exists in current_keys then a new key with _1, _2, ..., etc appended
24 | will be returned, otherwise the key will be returned as passed.
25 |
26 | #### Args:
27 |
28 | * `key`: desired key name.
29 | * `current_keys`: List of current key names.
30 | * `update_keys`: True to append the new key to current_keys.
31 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/validators.md:
--------------------------------------------------------------------------------
1 |
6 |
7 | # Module: tfma.version
8 |
9 | Defined in
10 | [`version.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/version.py).
11 |
12 |
13 |
14 | Contains the version string for this release of TFMA.
15 |
16 | ## Other Members
17 |
18 |
5 |
6 | # Module: tfma.view
7 |
8 | Defined in
9 | [`view/__init__.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/view/__init__.py).
10 |
11 |
12 |
13 | Initializes TFMA's view rendering api.
14 |
15 | ## Functions
16 |
17 | [`render_plot(...)`](../tfma/view/render_plot.md): Renders the plot view as
18 | widget.
19 |
20 | [`render_slicing_metrics(...)`](../tfma/view/render_slicing_metrics.md): Renders
21 | the slicing metrics view as widget.
22 |
23 | [`render_time_series(...)`](../tfma/view/render_time_series.md): Renders the
24 | time series view as widget.
25 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/view/render_plot.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.view.render_plot
7 |
8 | ```python
9 | tfma.view.render_plot(
10 | result,
11 | slicing_spec=None,
12 | label=None
13 | )
14 | ```
15 |
16 | Defined in
17 | [`view/widget_view.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/view/widget_view.py).
18 |
19 |
20 |
21 | Renders the plot view as widget.
22 |
23 | #### Args:
24 |
25 | * `result`: An tfma.EvalResult.
26 | * `slicing_spec`: The slicing spec to identify the slice. Show overall
27 | if unset.
28 | * `label`: A partial label used to match a set of plots in the results.
29 |
30 | #### Returns:
31 |
32 | A PlotViewer object if in Jupyter notebook; None if in Colab.
33 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/view/render_slicing_metrics.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.view.render_slicing_metrics
7 |
8 | ```python
9 | tfma.view.render_slicing_metrics(
10 | result,
11 | slicing_column=None,
12 | slicing_spec=None,
13 | weighted_example_column=None
14 | )
15 | ```
16 |
17 | Defined in
18 | [`view/widget_view.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/view/widget_view.py).
19 |
20 |
21 |
22 | Renders the slicing metrics view as widget.
23 |
24 | #### Args:
25 |
26 | * `result`: An tfma.EvalResult.
27 | * `slicing_column`: The column to slice on.
28 | * `slicing_spec`: The slicing spec to filter results. If neither column
29 | nor spec is set, show overall.
30 | * `weighted_example_column`: Override for the weighted example column.
31 | This can be used when different weights are applied in different aprts of
32 | the model (eg: multi-head).
33 |
34 | #### Returns:
35 |
36 | A SlicingMetricsViewer object if in Jupyter notebook; None if in Colab.
37 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/view/render_time_series.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.view.render_time_series
7 |
8 | ```python
9 | tfma.view.render_time_series(
10 | results,
11 | slice_spec=None,
12 | display_full_path=False
13 | )
14 | ```
15 |
16 | Defined in
17 | [`view/widget_view.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/view/widget_view.py).
18 |
19 |
20 |
21 | Renders the time series view as widget.
22 |
23 | #### Args:
24 |
25 | * `results`: An tfma.EvalResults.
26 | * `slice_spec`: A slicing spec determining the slice to show time
27 | series on. Show overall if not set.
28 | * `display_full_path`: Whether to display the full path to model / data
29 | in the visualization or just show file name.
30 |
31 | #### Returns:
32 |
33 | A TimeSeriesViewer object if in Jupyter notebook; None if in Colab.
34 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/writers.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # Module: tfma.writers
7 |
8 | Defined in
9 | [`writers/__init__.py`](https://github.com/tensorflow/model-analysis/tree/master/tensorflow_model_analysis/writers/__init__.py).
10 |
11 |
12 |
13 | Init module for TensorFlow Model Analysis writers.
14 |
15 | ## Classes
16 |
17 | [`class Writer`](../tfma/writers/Writer.md): Writer(stage_name, ptransform)
18 |
19 | ## Functions
20 |
21 | [`Write(...)`](../tfma/writers/Write.md): Writes given Evaluation or Validation
22 | data using given writer PTransform.
23 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/writers/Write.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 | # tfma.writers.Write
7 |
8 | ```python
9 | tfma.writers.Write(
10 | *args,
11 | **kwargs
12 | )
13 | ```
14 |
15 |
16 |
17 | Writes given Evaluation or Validation data using given writer PTransform.
18 |
19 | #### Args:
20 |
21 | * `evaluation_or_validation`: Evaluation or Validation data.
22 | * `key`: Key for Evaluation or Validation output to write. It is valid
23 | for the key to not exist in the dict (in which case the write is a no-op).
24 | * `ptransform`: PTransform to use for writing.
25 |
26 | #### Raises:
27 |
28 | * `ValueError`: If Evaluation or Validation is empty. The key does not
29 | need to exist in the Evaluation or Validation, but the dict must not be
30 | empty.
31 |
32 | #### Returns:
33 |
34 | beam.pvalue.PDone.
35 |
--------------------------------------------------------------------------------
/g3doc/api_docs/python/tfma/writers/Writer.md:
--------------------------------------------------------------------------------
1 |
38 |
--------------------------------------------------------------------------------
/g3doc/images/tfma-slicing-metrics-browser.gif:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/ageron/model-analysis/34a40c2737c5c4316c235270806eaeb318cd9a1c/g3doc/images/tfma-slicing-metrics-browser.gif
--------------------------------------------------------------------------------
/g3doc/images/tfma-slicing-metrics-browser.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/ageron/model-analysis/34a40c2737c5c4316c235270806eaeb318cd9a1c/g3doc/images/tfma-slicing-metrics-browser.png
--------------------------------------------------------------------------------
/g3doc/images/tfma_pipeline.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/ageron/model-analysis/34a40c2737c5c4316c235270806eaeb318cd9a1c/g3doc/images/tfma_pipeline.png
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2019 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2019 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-bounded-value-bar-chart/demo/index.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-bounded-value-bar-chart/fairness-bounded-value-bar-chart-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
24 |
25 | `;
26 | export {template};
27 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-bounded-value-bar-chart/fairness-bounded-value-bar-chart_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-metric-and-slice-selector/demo/index.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-metric-and-slice-selector/demo/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2019 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 |
17 | (() => {
18 | const element =
19 | document.getElementsByTagName('fairness-metric-and-slice-selector')[0];
20 | element.availableMetrics =
21 | ['post_export_metrics/false_negative_rate',
22 | 'post_export_metrics/false_positive_rate',
23 | 'post_export_metrics/negative_rate', 'post_export_metrics/positive_rate',
24 | 'post_export_metrics/true_negative_rate',
25 | 'post_export_metrics/true_positive_rate', 'accuracy',
26 | 'accuracy_baseline', 'auc', 'auc_precision_recall', 'average_loss',
27 | 'label/mean', 'post_export_metrics/example_count', 'precision',
28 | 'prediction/mean', 'recall', 'totalWeightedExamples'];
29 | })();
30 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-metric-and-slice-selector/fairness-metric-and-slice-selector_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-metric-summary/demo/index.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-metric-summary/fairness-metric-summary_test.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
13 |
14 |
15 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-metrics-board/demo/index.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-metrics-board/fairness-metrics-board_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-metrics-table/demo/index.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-metrics-table/fairness-metrics-table_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-nb-container/demo/index.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-nb-container/fairness-nb-container_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-privacy-container/demo/index.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-privacy-container/demo/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2019 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | (() => {
17 | const element =
18 | document.getElementsByTagName('fairness-privacy-container')[0];
19 | element.omittedSlices = ['Slice:1', 'Slice:2'];
20 | })();
21 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-privacy-container/fairness-privacy-container_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
32 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/frontend/fairness-tensorboard-container/demo/index.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/notebook/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2019 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/notebook/colab/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2019 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/notebook/colab/widget.py:
--------------------------------------------------------------------------------
1 | # Copyright 2019 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Fairness Visualization API for colab."""
15 |
16 |
17 | # The following empty classes are used for
18 | # pytype in tensorflow_model_analysis/addons/fairness/view/widget_view.py.
19 | class FairnessIndicatorViewer(object):
20 | """Empty viewer class for fairness indicator."""
21 | pass
22 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/notebook/visualization.py:
--------------------------------------------------------------------------------
1 | # Copyright 2019 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Fairness Visualization API."""
15 | import sys
16 |
17 |
18 | def _is_colab():
19 | return "google.colab" in sys.modules
20 |
21 |
22 | if _is_colab():
23 | from tensorflow_model_analysis.addons.fairness.notebook.colab.renderer import * # pylint: disable=wildcard-import,g-import-not-at-top
24 | from tensorflow_model_analysis.addons.fairness.notebook.colab.widget import * # pylint: disable=wildcard-import,g-import-not-at-top
25 | else:
26 | pass
27 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/post_export_metrics/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2019 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/addons/fairness/view/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2019 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/api/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/contrib/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/ageron/model-analysis/34a40c2737c5c4316c235270806eaeb318cd9a1c/tensorflow_model_analysis/contrib/__init__.py
--------------------------------------------------------------------------------
/tensorflow_model_analysis/eval_metrics_graph/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/eval_saved_model/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/eval_saved_model/example_trainers/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/evaluators/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Init module for TensorFlow Model Analysis evaluators."""
15 |
16 | from tensorflow_model_analysis.evaluators.analysis_table_evaluator import AnalysisTableEvaluator
17 | from tensorflow_model_analysis.evaluators.evaluator import Evaluation
18 | from tensorflow_model_analysis.evaluators.evaluator import Evaluator
19 | from tensorflow_model_analysis.evaluators.evaluator import verify_evaluator
20 | from tensorflow_model_analysis.evaluators.metrics_and_plots_evaluator import MetricsAndPlotsEvaluator
21 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/evaluators/query_metrics/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Init module for TensorFlow Model Analysis query metrics."""
15 |
16 | from tensorflow_model_analysis.evaluators.query_metrics.min_label_position import MinLabelPositionCombineFn
17 | from tensorflow_model_analysis.evaluators.query_metrics.ndcg import NdcgMetricCombineFn
18 | from tensorflow_model_analysis.evaluators.query_metrics.query_statistics import QueryStatisticsCombineFn
19 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/evaluators/query_metrics/query_types.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Types for query based metrics."""
15 |
16 | from __future__ import absolute_import
17 | from __future__ import division
18 | # Standard __future__ imports
19 | from __future__ import print_function
20 |
21 | from tensorflow_model_analysis import types
22 | from typing import Dict, List, NamedTuple, Text
23 |
24 | # Should contain features, predictions, labels
25 | FPL = Dict[Text, types.DictOfTensorValue]
26 | QueryFPL = NamedTuple('QueryFPL', [('fpls', List[FPL]), ('query_id', Text)])
27 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/export_only/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Init module for TensorFlow Model Analysis export-only modules.
15 |
16 | This module contains only the export-related modules of TFMA, for use in your
17 | Trainer. You may want to use this in your Trainer as it has a smaller import
18 | footprint (it doesn't include Apache Beam which is required for analysis but not
19 | for export). To evaluate your model using TFMA, you should use the full TFMA
20 | module instead, i.e. import tensorflow_model_analysis
21 |
22 | Example usage:
23 | import tensorflow_model_analysis.export_only as tfma_export
24 |
25 | def eval_input_receiver_fn():
26 | ...
27 | return tfma_export.export.EvalInputReceiver(...)
28 |
29 | tfma_export.export.export_eval_saved_model(...)
30 | """
31 |
32 | from tensorflow_model_analysis.eval_saved_model import export
33 | from tensorflow_model_analysis.eval_saved_model import exporter
34 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/BUILD:
--------------------------------------------------------------------------------
1 | licenses(["notice"]) # Apache 2.0
2 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/lib/svg-externs.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2019 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | /**
17 | * A partial externs file for svg for classes / methods used in this project.
18 | * @externs
19 | */
20 | /**
21 | * @constructor
22 | */
23 | function SVGTextContentElement() {}
24 |
25 | /**
26 | * @return {number}
27 | */
28 | SVGTextContentElement.prototype.getComputedTextLength = function() {};
29 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-accuracy-charts/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-accuracy-charts/demo/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | (() => {
17 | const plot = document.body.querySelector('tfma-accuracy-charts');
18 | const input = [];
19 | let precision = 1;
20 | for (let i = 0; i <= 128; i++) {
21 | input.push({
22 | 'truePositives': Math.random(),
23 | 'trueNegatives': Math.random(),
24 | 'falsePositives': Math.random(),
25 | 'falseNegatives': Math.random(),
26 | 'precision': precision,
27 | 'recall': i / 128,
28 | 'threshold': (128 - i) / 128,
29 | });
30 | precision -= Math.random() / 128;
31 | }
32 | plot.data = input;
33 | })();
34 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-accuracy-charts/tfma-accuracy-charts-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
19 |
20 | `;
21 | export {template};
22 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-accuracy-charts/tfma-accuracy-charts_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-array-value/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
Shape [12]
20 |
21 |
Shape [3, 4]
22 |
23 |
Shape [4, 1, 3, 1], click to expand
24 |
25 |
Shape [6, 2], expanded by default
26 |
27 |
28 |
29 |
30 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-array-value/demo/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | (function() {
17 | const data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12];
18 | const data1 = {
19 | 'shape': [12],
20 | 'dataType': 'INT32',
21 | 'int32Values': data
22 | };
23 | const ex1 = document.getElementById('ex1');
24 | ex1.data = JSON.stringify(data1);
25 |
26 | const ex2 = document.getElementById('ex2');
27 | ex2.arrayData = [[1,2,3,4], [5,6,7,8], [9,10,11,12]];
28 |
29 | const data3 = {
30 | 'shape': [4, 1, 3, 1],
31 | 'dataType': 'INT32',
32 | 'int32Values': data
33 | };
34 | const ex3 = document.getElementById('ex3');
35 | ex3.data = JSON.stringify(data3);
36 |
37 | const ex4 = document.getElementById('ex4');
38 | ex4.arrayData = [[1,2], [3,4], [5,6], [7,8], [9,10], [11,12]];
39 | })();
40 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-array-value/tfma-array-value-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
26 |
23 |
24 |
25 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-bounded-value/tfma-bounded-value-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
27 |
29 | `;
30 | export {template};
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-bounded-value/tfma-bounded-value_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
32 |
33 |
34 |
35 |
36 |
37 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-calibration-plot/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-calibration-plot/tfma-calibration-plot-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
19 |
20 | `;
21 | export {template};
22 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-calibration-plot/tfma-calibration-plot_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-config-picker/demo/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | (function() {
17 | const setUp = (id, configs) => {
18 | const picker = document.getElementById(id);
19 | picker.allConfigs = configs;
20 | const output = document.getElementById(id + '-output');
21 |
22 | setInterval(() => {
23 | output.textContent =
24 | 'Config is: \n' + JSON.stringify(picker.selectedConfigs);
25 | }, 500);
26 | };
27 |
28 | setUp('picker1', {
29 | 'output1': [0, 1, 2, 3, 4, 'nci'],
30 | 'output2': [0, 2, 4, 5, 6, 7],
31 | });
32 | setUp('picker2', {
33 | 'output1': ['nci'],
34 | 'output2': ['nci'],
35 | });
36 | setUp('picker3', {
37 | '': [0, 1, 2, 3, 4, 'nci'],
38 | });
39 | setUp('picker4', {
40 | '': ['nci'],
41 | });
42 | })();
43 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-config-picker/tfma-config-picker-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
23 |
24 |
26 |
27 |
28 |
29 |
31 |
32 |
33 | `;
34 | export {template};
35 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-config-picker/tfma-config-picker_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-confusion-matrix-at-thresholds/tfma-confusion-matrix-at-thresholds_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
32 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-gain-chart/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-gain-chart/demo/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | (() => {
17 | const plot = document.body.querySelector('tfma-gain-chart');
18 | const input = [];
19 | const count = 1000;
20 |
21 | let totalPositive = 3 * count;
22 | let totalNegative = count;
23 | let truePositives = 0;
24 | let falsePositives = 0;
25 |
26 | for (let i = 0; i <= count; i++) {
27 | input.push({
28 | 'truePositives': truePositives,
29 | 'falsePositives': falsePositives,
30 | 'trueNegatives': totalNegative - falsePositives,
31 | 'falseNegatives': totalPositive - truePositives,
32 | 'threshold': (count - i) / count,
33 | });
34 |
35 | truePositives = Math.min(
36 | totalPositive, truePositives + Math.round((1 + Math.random()) * 30));
37 | falsePositives = Math.min(
38 | totalNegative, 1 + falsePositives + Math.round(Math.random() * 5));
39 | }
40 |
41 | plot.data = input;
42 | })();
43 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-gain-chart/tfma-gain-chart-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
19 |
20 | `;
21 | export {template};
22 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-gain-chart/tfma-gain-chart_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-google-chart-wrapper/tfma-google-chart-wrapper-template.html:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
23 |
24 | `;
25 | export {template};
26 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-graph-data-filter/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
22 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-graph-data-filter/demo/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | (() => {
17 | const filter = document.getElementById('data-filter');
18 | const metrics = ['weight', 'a', 'b', 'c'];
19 | filter.weightedExamplesColumn = 'weight';
20 | filter.data = new tfma.SingleSeriesGraphData(metrics, [
21 | {
22 | 'slice': 'col:1',
23 | 'metrics': {
24 | 'weight': 100,
25 | 'a': 0.6,
26 | 'b': {'value': 0.61, 'lowerBound': 0.60, 'upperBound': 0.62},
27 | 'c': 0.51,
28 | }
29 | },
30 | {
31 | 'slice': 'col:2',
32 | 'metrics': {
33 | 'weight': 200,
34 | 'a': 0.62,
35 | 'b': {'value': 0.51, 'lowerBound': 0.50, 'upperBound': 0.52},
36 | 'c': 0.52,
37 | }
38 | }
39 | ]);
40 | })();
41 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-graph-data-filter/tfma-graph-data-filter.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-graph-data-filter/tfma-graph-data-filter_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-int64/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-line-chart-grid/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
25 |
26 |
27 |
28 |
Highlight
29 |
Clear selection
30 |
31 |
32 |
33 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-line-chart-grid/tfma-line-chart-grid_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-line-chart/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
22 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-line-chart/demo/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | (() => {
17 | const chart = document.getElementById('chart');
18 | chart.title = 'Demo';
19 | chart.data = [
20 | [{'label': 'x', 'type': 'number'}, {'label': 'y', 'type': 'number'}],
21 | [0, 1], [1, 2], [2, 4], [3, 8], [4, 16]
22 | ];
23 |
24 | const text = document.getElementById('text');
25 | chart.addEventListener('select', (e) => {
26 | text.innerText = 'Row ' + e.detail['point']['row'] + ' selected';
27 | });
28 | chart.addEventListener('clear-selection', () => {
29 | text.innerText = '';
30 | });
31 | })();
32 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-line-chart/tfma-line-chart-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
37 |
28 | Compact matrix should scale up automatically if possible.
29 |
30 |
31 |
32 | Do not render the matrix when there are too many classes.
33 |
34 |
35 |
36 |
37 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-multi-class-confusion-matrix/tfma-multi-class-confusion-matrix-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
21 |
22 | `;
23 | export {template};
24 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-multi-class-confusion-matrix/tfma-multi-class-confusion-matrix.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-multi-class-confusion-matrix/tfma-multi-class-confusion-matrix_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-multi-select/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-multi-select/demo/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | (function() {
17 | const selector = document.getElementById('selector');
18 | selector.items = ['a', 'b', 'c', 'd'];
19 | })();
20 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-multi-select/tfma-multi-select-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
19 |
20 |
21 |
22 | [[item]]
23 |
24 |
25 |
26 |
27 | `;
28 | export {template};
29 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-multi-select/tfma-multi-select.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-multi-select/tfma-multi-select_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-nb-plot/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-nb-plot/tfma-nb-plot-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
23 |
25 |
26 | `;
27 | export {template};
28 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-nb-plot/tfma-nb-plot_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-nb-slicing-metrics/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-nb-slicing-metrics/tfma-nb-slicing-metrics-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
20 |
21 |
23 |
24 | `;
25 | export {template};
26 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-nb-slicing-metrics/tfma-nb-slicing-metrics_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-nb-time-series/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-nb-time-series/tfma-nb-time-series-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
20 |
21 |
22 |
23 | `;
24 | export {template};
25 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-nb-time-series/tfma-nb-time-series_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-plot/tfma-plot_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-precision-recall-curve/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
22 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-precision-recall-curve/demo/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | (() => {
17 | const plot = document.getElementById('plot');
18 | const input = [];
19 | let precision = 1;
20 | for (let i = 0; i <= 128; i++) {
21 | input.push({
22 | 'precision': precision,
23 | 'recall': i / 128,
24 | 'threshold': (128 - i) / 128,
25 | });
26 | precision -= Math.random() / 128;
27 | }
28 | plot.data = input;
29 | })();
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-precision-recall-curve/tfma-precision-recall-curve-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
19 | `;
20 | export {template};
21 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-precision-recall-curve/tfma-precision-recall-curve_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-prediction-distribution/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
22 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-prediction-distribution/demo/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | (() => {
17 | const data = [
18 | {'numWeightedExamples': 0, 'totalWeightedRefinedPrediction': 0},
19 | {'numWeightedExamples': 16, 'totalWeightedRefinedPrediction': 1.5},
20 | {'numWeightedExamples': 1, 'totalWeightedRefinedPrediction': 0.126},
21 | {'numWeightedExamples': 2, 'totalWeightedRefinedPrediction': 0.44},
22 | {'numWeightedExamples': 3, 'totalWeightedRefinedPrediction': 2.1},
23 | {'numWeightedExamples': 1, 'totalWeightedRefinedPrediction': 1}
24 | ];
25 |
26 | const plot = document.getElementById('plot');
27 | plot.data = data;
28 | })();
29 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-prediction-distribution/tfma-prediction-distribution-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
19 |
20 |
21 | `;
22 | export {template};
23 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-prediction-distribution/tfma-prediction-distribution_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-residual-plot/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-residual-plot/tfma-residual-plot-template.html:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
19 |
20 |
21 | `;
22 | export {template};
23 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-residual-plot/tfma-residual-plot_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-roc-curve/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-roc-curve/demo/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | (() => {
17 | const plot = document.getElementById('plot');
18 | const input = [];
19 | let tp;
20 | let fn;
21 |
22 | let fp;
23 | let tn;
24 |
25 | for (let i = 0; i <= 128; i++) {
26 | const val = 1 - (i - 128) * (i - 128) / 128 / 128;
27 | tp = Math.round(128 * val);
28 | fn = 128 - tp;
29 | fp = i;
30 | tn = 128 - i;
31 |
32 | input.push({
33 | 'truePositives': tp,
34 | 'trueNegatives': tn,
35 | 'falsePositives': fp,
36 | 'falseNegatives': fn,
37 | 'threshold': (128 - i) / 128,
38 | });
39 | }
40 | plot.data = input;
41 | })();
42 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-roc-curve/tfma-roc-curve-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
19 |
20 | `;
21 | export {template};
22 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-roc-curve/tfma-roc-curve_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-slice-overview/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-slice-overview/tfma-slice-overview_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-slicing-metrics-browser/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-slicing-metrics-browser/tfma-slicing-metrics-browser_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-time-series-browser/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-time-series-browser/demo/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | (() => {
17 | const makeData = index => tfma.Data.build(
18 | ['a', 'b', 'c'],
19 | [{'metrics': {'a': index + 1, 'b': index * -3, 'c': index * index}}]);
20 |
21 | const browser = document.getElementById('browser');
22 | browser.seriesData = new tfma.SeriesData(
23 | [
24 | {
25 | 'config': {'dataIdentifier': '1', 'modelIdentifier': '0'},
26 | 'data': makeData(0),
27 | },
28 | {
29 | 'config': {'dataIdentifier': '2', 'modelIdentifier': '1'},
30 | 'data': makeData(1),
31 | },
32 | {
33 | 'config': {'dataIdentifier': '1', 'modelIdentifier': '2'},
34 | 'data': makeData(2),
35 | }
36 | ],
37 | true);
38 | })();
39 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-time-series-browser/tfma-time-series-browser-template.html.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | const template = /** @type {!HTMLTemplateElement} */(document.createElement('template'));
17 | template.innerHTML = `
18 |
24 |
25 |
27 |
28 |
30 |
31 |
32 | `;
33 | export {template};
34 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-time-series-browser/tfma-time-series-browser_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-value-at-cutoffs/demo/index.html:
--------------------------------------------------------------------------------
1 |
16 |
17 |
18 |
19 |
20 |
21 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-value-at-cutoffs/demo/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | /**
17 | * @fileoverview Demo JS that provides the precision-at-k component with test
18 | * data.
19 | */
20 |
21 | (function() {
22 | var makeData = function(count) {
23 | const values = [];
24 | for (let i = 0; i <= count; i++) {
25 | values.push({'cutoff': i, 'value': 1 - 0.1 * i});
26 | }
27 | return {'values': values};
28 | };
29 | const precisionTable = document.querySelector('#metric-at-k');
30 | precisionTable.data = JSON.stringify(makeData(8));
31 | }());
32 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/tfma-value-at-cutoffs/tfma-value-at-cutoffs_test.html:
--------------------------------------------------------------------------------
1 |
2 |
17 |
18 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/vulcanize/license_header.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/frontend/vulcanize/tfma.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | import '@polymer/polymer/lib/elements/dom-if.js';
17 | import '@polymer/polymer/lib/elements/dom-repeat.js';
18 |
19 | import '../tfma-nb-plot/tfma-nb-plot.js';
20 | import '../tfma-nb-slicing-metrics/tfma-nb-slicing-metrics.js';
21 | import '../tfma-nb-time-series/tfma-nb-time-series.js';
22 | import '../../addons/fairness/frontend/fairness-nb-container/fairness-nb-container.js';
23 | import '../../addons/fairness/frontend/fairness-tensorboard-container/fairness-tensorboard-container.js';
24 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/math_util.py:
--------------------------------------------------------------------------------
1 | # Copyright 2019 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Math utilities."""
15 |
16 | from __future__ import absolute_import
17 | from __future__ import division
18 | # Standard __future__ imports
19 |
20 | import numpy as np
21 | from scipy import stats
22 | from tensorflow_model_analysis import types
23 |
24 |
25 | def calculate_confidence_interval(
26 | t_distribution_value: types.ValueWithTDistribution):
27 | """Caculate confidence intervals based 95% confidence level."""
28 | alpha = 0.05
29 | std_err = t_distribution_value.sample_standard_deviation / np.sqrt(
30 | t_distribution_value.sample_degrees_of_freedom + 1)
31 | t_stat = stats.t.ppf(1 - (alpha / 2.0),
32 | t_distribution_value.sample_degrees_of_freedom)
33 | upper_bound = t_distribution_value.sample_mean + t_stat * std_err
34 | lower_bound = t_distribution_value.sample_mean - t_stat * std_err
35 | return t_distribution_value.sample_mean, lower_bound, upper_bound
36 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/math_util_test.py:
--------------------------------------------------------------------------------
1 | # Copyright 2019 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Tests for math_util."""
15 |
16 | from __future__ import absolute_import
17 | from __future__ import division
18 | from __future__ import print_function
19 |
20 | import math
21 | import tensorflow as tf
22 | from tensorflow_model_analysis import math_util
23 | from tensorflow_model_analysis import types
24 |
25 |
26 | class MathUtilTest(tf.test.TestCase):
27 |
28 | def testCalculateConfidenceInterval(self):
29 | self.assertEqual(
30 | math_util.calculate_confidence_interval(
31 | types.ValueWithTDistribution(10, 2, 9, 10)),
32 | (10, 8.5692861880948552, 11.430713811905145))
33 | mean, lb, ub = math_util.calculate_confidence_interval(
34 | types.ValueWithTDistribution(-1, -1, -1, -1))
35 | self.assertEqual(mean, -1)
36 | self.assertTrue(math.isnan(lb))
37 | self.assertTrue(math.isnan(ub))
38 |
39 |
40 | if __name__ == '__main__':
41 | tf.test.main()
42 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/model_agnostic_eval/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/notebook/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/notebook/colab/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/notebook/colab/widget.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Visualization API for colab."""
15 |
16 |
17 | # The following empty classes are used for
18 | # pytype in tensorflow_model_analysis/view/widget_view.py.
19 | class SlicingMetricsViewer(object):
20 | """Empty viewer class for slicing metrics."""
21 | pass
22 |
23 |
24 | class TimeSeriesViewer(object):
25 | """Empty viewer class for time series."""
26 | pass
27 |
28 |
29 | class PlotViewer(object):
30 | """Empty viewer class for plot."""
31 | pass
32 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/notebook/jupyter/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/notebook/jupyter/js/lib/embed.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 |
17 | // Entry point for the unpkg bundle containing custom model definitions.
18 | //
19 | // It differs from the notebook bundle in that it does not need to define a
20 | // dynamic baseURL for the static assets and may load some css that would
21 | // already be loaded by the notebook otherwise.
22 |
23 | // Export widget models and views, and the npm package version number.
24 | module.exports = require('./widget.js');
25 | module.exports['version'] = require('../package.json').version;
26 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/notebook/jupyter/js/lib/extension.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | // This file contains the javascript that is run when the notebook is loaded.
17 | // It contains some requirejs configuration and the `load_ipython_extension`
18 | // which is required for any notebook extension.
19 | // Configure requirejs
20 | if (window.require) {
21 | window.require.config({
22 | map: {
23 | '*': {
24 | 'tfma_widget_js': 'nbextensions/tfma_widget_js/index',
25 | }
26 | }
27 | });
28 | }
29 |
30 | // Export the required load_ipython_extension
31 | module.exports = {
32 | load_ipython_extension: function() {}
33 | };
34 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/notebook/jupyter/js/lib/index.js:
--------------------------------------------------------------------------------
1 | /**
2 | * Copyright 2018 Google LLC
3 | *
4 | * Licensed under the Apache License, Version 2.0 (the "License");
5 | * you may not use this file except in compliance with the License.
6 | * You may obtain a copy of the License at
7 | *
8 | * https://www.apache.org/licenses/LICENSE-2.0
9 | *
10 | * Unless required by applicable law or agreed to in writing, software
11 | * distributed under the License is distributed on an "AS IS" BASIS,
12 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | * See the License for the specific language governing permissions and
14 | * limitations under the License.
15 | */
16 | // Entry point for the notebook bundle containing custom model definitions.
17 | //
18 | // Setup notebook base URL
19 | //
20 | // Some static assets may be required by the custom widget javascript. The base
21 | // url for the notebook is not known at build time and is therefore computed
22 | // dynamically.
23 | __webpack_public_path__ =
24 | document.querySelector('body').getAttribute('data-base-url') +
25 | 'nbextensions/tfma_widget_js/';
26 |
27 | // Export widget models and views, and the npm package version number.
28 | module.exports = require('./widget.js');
29 | module.exports['version'] = require('../package.json').version;
30 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/notebook/jupyter/js/package.json:
--------------------------------------------------------------------------------
1 | {
2 | "main": "lib/index.js",
3 | "files": [
4 | "lib/**/*.js",
5 | "dist/*.js"
6 | ],
7 | "scripts": {
8 | "clean": "rimraf dist/",
9 | "prepublish": "webpack",
10 | "test": "echo \"Error: no test specified\" && exit 1"
11 | },
12 | "devDependencies": {
13 | "webpack": "^3.5.5",
14 | "rimraf": "^2.6.1"
15 | },
16 | "dependencies": {
17 | "@jupyter-widgets/base": "^1.0.0",
18 | "lodash": "^4.17.4"
19 | }
20 | }
21 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/notebook/jupyter/tfma_widget/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Initializes TFMA's Jupyter notebook widget."""
15 | from tensorflow_model_analysis.notebook.jupyter.tfma_widget.widget import PlotViewer
16 | from tensorflow_model_analysis.notebook.jupyter.tfma_widget.widget import SlicingMetricsViewer
17 | from tensorflow_model_analysis.notebook.jupyter.tfma_widget.widget import TimeSeriesViewer
18 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/notebook/visualization.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Visualization API."""
15 | import sys
16 |
17 |
18 | def _is_colab():
19 | return "google.colab" in sys.modules
20 |
21 |
22 | if _is_colab():
23 | from tensorflow_model_analysis.notebook.colab.renderer import * # pylint: disable=wildcard-import,g-import-not-at-top
24 | from tensorflow_model_analysis.notebook.colab.widget import * # pylint: disable=wildcard-import,g-import-not-at-top
25 | else:
26 | from tensorflow_model_analysis.notebook.jupyter.tfma_widget import * # pylint: disable=wildcard-import,g-import-not-at-top
27 | from tensorflow_model_analysis.notebook.jupyter.renderer import * # pylint: disable=wildcard-import,g-import-not-at-top
28 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/post_export_metrics/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Init module for TensorFlow Model Analysis post export metrics."""
15 |
16 | from tensorflow_model_analysis.post_export_metrics import metric_keys
17 | from tensorflow_model_analysis.post_export_metrics import metrics
18 | from tensorflow_model_analysis.post_export_metrics import post_export_metrics
19 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/proto/BUILD:
--------------------------------------------------------------------------------
1 | licenses(["notice"]) # Apache 2.0
2 |
3 | package(default_visibility = ["//visibility:public"])
4 |
5 | load("@protobuf_bzl//:protobuf.bzl", "py_proto_library")
6 |
7 | py_proto_library(
8 | name = "config_py_pb2",
9 | srcs = [
10 | "config.proto",
11 | ],
12 | default_runtime = "@com_google_protobuf//:protobuf_python",
13 | protoc = "@com_google_protobuf//:protoc",
14 | deps = [
15 | "@com_google_protobuf//:protobuf_python",
16 | ],
17 | )
18 |
19 | py_proto_library(
20 | name = "metrics_for_slice_py_pb2",
21 | srcs = [
22 | "metrics_for_slice.proto",
23 | ],
24 | default_runtime = "@com_google_protobuf//:protobuf_python",
25 | protoc = "@com_google_protobuf//:protoc",
26 | deps = [
27 | "@com_google_protobuf//:protobuf_python",
28 | ],
29 | )
30 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/proto/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/slicer/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Init module for TensorFlow Model Analysis slicer."""
15 |
16 | from tensorflow_model_analysis.slicer.slicer_lib import deserialize_slice_key
17 | from tensorflow_model_analysis.slicer.slicer_lib import FanoutSlices
18 | from tensorflow_model_analysis.slicer.slicer_lib import serialize_slice_key
19 | from tensorflow_model_analysis.slicer.slicer_lib import SingleSliceSpec
20 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/tools/BUILD:
--------------------------------------------------------------------------------
1 | licenses(["notice"])
2 |
3 | package(default_visibility = [
4 | "//cloud/ml/explainability:__subpackages__",
5 | "//tensorflow_model_analysis:__subpackages__",
6 | ])
7 |
8 | py_binary(
9 | name = "build_docs",
10 | srcs = ["build_docs.py"],
11 | python_version = "PY3",
12 | srcs_version = "PY3",
13 | deps = [
14 | "//tensorflow_model_analysis",
15 | "//third_party/py/absl:app",
16 | "//third_party/py/tensorflow_docs/api_generator:generate_lib",
17 | ],
18 | )
19 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/tools/README.md:
--------------------------------------------------------------------------------
1 | # Tools
2 |
3 | Additional tools and scripts that are not part of the pip package.
4 |
5 | ## build_docs.py
6 |
7 | This is used to generate the api reference docs for tensorflow.org.
8 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/validators/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Init module for TensorFlow Model Analysis validators."""
15 |
16 | from tensorflow_model_analysis.validators.validator import Validation
17 | from tensorflow_model_analysis.validators.validator import Validator
18 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/version.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Contains the version string for this release of TFMA."""
15 |
16 | # Version string for this release of TFMA.
17 | # Note that setup.py reads and uses this version.
18 | VERSION_STRING = '0.15.3'
19 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/view/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Initializes TFMA's view rendering api."""
15 | from tensorflow_model_analysis.view.widget_view import render_plot
16 | from tensorflow_model_analysis.view.widget_view import render_slicing_metrics
17 | from tensorflow_model_analysis.view.widget_view import render_time_series
18 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/writers/__init__.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Init module for TensorFlow Model Analysis writers."""
15 |
16 | from tensorflow_model_analysis.writers.metrics_and_plots_serialization import SerializeMetricsAndPlots
17 | from tensorflow_model_analysis.writers.metrics_and_plots_writer import MetricsAndPlotsWriter
18 | from tensorflow_model_analysis.writers.writer import Write
19 | from tensorflow_model_analysis.writers.writer import Writer
20 |
--------------------------------------------------------------------------------
/tensorflow_model_analysis/writers/writer_test.py:
--------------------------------------------------------------------------------
1 | # Copyright 2018 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # https://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | """Test for writer."""
15 |
16 | from __future__ import division
17 | from __future__ import print_function
18 |
19 | # Standard Imports
20 |
21 | import apache_beam as beam
22 | import tensorflow as tf
23 | from tensorflow_model_analysis.eval_saved_model import testutil
24 | from tensorflow_model_analysis.writers import writer
25 |
26 |
27 | class WriterTest(testutil.TensorflowModelAnalysisTest):
28 |
29 | def testWriteIgnoresMissingKeys(self):
30 | with beam.Pipeline() as pipeline:
31 | test = pipeline | beam.Create(['test'])
32 | # PTransform is None so this will throw exception if it tries to run
33 | _ = {'test': test} | writer.Write('key-does-not-exist', None)
34 |
35 |
36 | if __name__ == '__main__':
37 | tf.test.main()
38 |
--------------------------------------------------------------------------------
/third_party/BUILD:
--------------------------------------------------------------------------------
1 | licenses(["notice"])
2 |
3 | filegroup(
4 | name = "all_files",
5 | srcs = glob(["**"]),
6 | visibility = ["//tensorflow_model_analysis:__subpackages__"],
7 | )
8 |
--------------------------------------------------------------------------------
/third_party/workspace.bzl:
--------------------------------------------------------------------------------
1 | """TensorBoard external dependencies that can be loaded in WORKSPACE files."""
2 |
3 | load("//third_party:polymer.bzl", "tensorflow_model_analysis_polymer_workspace")
4 |
5 | def tensorflow_model_analysis_workspace():
6 | """Download TensorFlow Model Analysis build dependencies."""
7 | tensorflow_model_analysis_polymer_workspace()
8 |
--------------------------------------------------------------------------------