├── .aws ├── .docker │ └── agent │ │ └── Dockerfile ├── .eslintignore ├── .eslintrc ├── .gitignore ├── .npmrc ├── .nvmrc ├── .prettierrc ├── .terraform-version ├── README.md ├── cdktf.json ├── package-lock.json ├── package.json ├── src │ ├── config │ │ └── index.ts │ ├── iam.ts │ └── main.ts └── tsconfig.json ├── .circleci ├── README.md ├── config.yml └── workflows.yml ├── .github ├── CODEOWNERS └── PULL_REQUEST_TEMPLATE.md ├── .gitignore ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── common-utils ├── .coveragerc ├── .docker │ └── main │ │ └── Dockerfile ├── README.md ├── common_utils_build.sh ├── poetry.lock ├── pyproject.toml ├── src │ └── common │ │ ├── __init__.py │ │ ├── cloud │ │ ├── __init__.py │ │ ├── aws_utils.py │ │ └── gcp_utils.py │ │ ├── databases │ │ ├── __init__.py │ │ ├── snowflake_utils.py │ │ └── sqlalchemy_utils.py │ │ ├── deployment │ │ ├── __init__.py │ │ └── worker │ │ │ ├── __init__.py │ │ │ ├── build_image.sh │ │ │ ├── check_version.py │ │ │ ├── cli.py │ │ │ ├── clone_project.sh │ │ │ ├── common_utils_build.sh │ │ │ ├── get_prefect_version.sh │ │ │ └── push_aws_image.sh │ │ ├── settings.py │ │ └── testing_utils.py └── tests │ ├── cloud │ └── unit │ │ ├── test_aws_utils.py │ │ └── test_gcp_utils.py │ ├── conftest.py │ ├── databases │ └── unit │ │ ├── test_snowflake_utils.py │ │ └── test_sqlalchemy_utils.py │ ├── deployment │ └── worker │ │ ├── test_flows │ │ ├── example_flow.py │ │ ├── example_legacy_flow.py │ │ ├── flow_group_2 │ │ │ ├── example_flow.py │ │ │ ├── example_no_jv_flow.py │ │ │ └── no_spec_example_flow.py │ │ └── single_flow │ │ │ └── example_flow.py │ │ └── unit │ │ ├── test_check_version_wk.py │ │ ├── test_cli_wk.py │ │ └── test_deployment_wk.py │ ├── test.json │ └── unit │ ├── test_common.py │ ├── test_settings.py │ └── test_testing_utils.py └── data-products ├── .coveragerc ├── .docker ├── base │ └── Dockerfile └── worker │ └── base │ └── Dockerfile ├── .env.example ├── README.md ├── docs ├── flows │ └── sql_etl │ │ ├── code.md │ │ ├── sql-etl-flow-diagram.jpg │ │ └── usage.md └── index.md ├── mkdocs.yml ├── poetry.lock ├── pyproject.toml ├── src ├── __init__.py ├── article_text_streaming │ ├── __init__.py │ └── article_text_streaming_flow.py ├── braze │ ├── __init__.py │ └── update_flow.py ├── data_retention │ ├── __init__.py │ ├── delete_miscellaneous_table_rows.sql │ ├── delete_raw_user_rows.sql │ ├── delete_snapshot_firehose_rows.sql │ ├── delete_snowplow_events.sql │ ├── delete_stripe_table_rows.sql │ ├── deleted_account_emails.sql │ ├── deleted_account_users.sql │ ├── deleted_accounts_flow.py │ └── snowflake_development_table_clean_up_flow.py ├── firefox_newtab │ ├── run_jobs_flow.py │ └── sql │ │ ├── extras │ │ └── helpers.j2 │ │ ├── firefox_new_tab_impressions_daily │ │ ├── firefox_new_tab_daily_disable_rate_by_feed │ │ │ ├── data.sql │ │ │ └── load.sql │ │ ├── firefox_new_tab_daily_spoc_fill_rate_by_position_feed │ │ │ ├── data.sql │ │ │ └── load.sql │ │ ├── firefox_new_tab_daily_unique_engagement_by_feed │ │ │ ├── data.sql │ │ │ └── load.sql │ │ └── firefox_new_tab_monthly_unique_engagement_by_feed │ │ │ ├── data.sql │ │ │ └── load.sql │ │ ├── firefox_new_tab_impressions_hourly │ │ ├── firefox_new_tab_daily_engagement_by_tile_id_position │ │ │ ├── data.sql │ │ │ └── load.sql │ │ └── firefox_new_tab_daily_engagement_by_tile_id_position_country_locale │ │ │ ├── data.sql │ │ │ └── load.sql │ │ ├── glean_firefox_new_tab_impressions_daily │ │ ├── glean_firefox_new_tab_daily_disable_rate_by_feed │ │ │ ├── data.sql │ │ │ └── load.sql │ │ ├── glean_firefox_new_tab_daily_spoc_fill_rate_by_position_feed │ │ │ ├── data.sql │ │ │ └── load.sql │ │ └── glean_firefox_new_tab_daily_unique_engagement_by_feed │ │ │ ├── data.sql │ │ │ └── load.sql │ │ ├── glean_firefox_new_tab_impressions_hourly │ │ ├── glean_firefox_new_tab_daily_engagement_by_scheduled_corpus_item_id_position_country_locale │ │ │ ├── data.sql │ │ │ └── load.sql │ │ ├── glean_firefox_new_tab_daily_engagement_by_tile_id_position │ │ │ ├── data.sql │ │ │ └── load.sql │ │ └── glean_firefox_new_tab_daily_engagement_by_tile_id_position_country_locale │ │ │ ├── data.sql │ │ │ └── load.sql │ │ └── uapi_interactions_hourly │ │ └── uapi_daily_interactions_by_tile_id_position │ │ ├── data.sql │ │ └── load.sql ├── freestar_revenue_reporting │ ├── __init__.py │ └── freestar_reporting_flow.py ├── new_tab_recommendations │ └── aggregate_engagement_flow.py ├── prospecting │ ├── publisher_features_flow.py │ └── sql │ │ └── publisher_features.sql ├── recommendation_api │ ├── __init__.py │ ├── corpus_candidate_sets_flow.py │ ├── corpus_item_engagement_flow.py │ ├── legacy_candidate_sets_flow.py │ ├── sql │ │ ├── collections_by_recency.sql │ │ ├── labeled_collection_stories.sql │ │ ├── life_hacks.sql │ │ ├── pocket_hits.sql │ │ ├── recommended_by_recency.sql │ │ ├── scheduled_not_syndicated_or_collection.sql │ │ ├── scheduled_syndicated.sql │ │ └── topic.sql │ └── user_impressions_flow.py └── shared │ ├── __init__.py │ ├── api_clients │ ├── __init__.py │ ├── braze │ │ ├── __init__.py │ │ ├── client.py │ │ ├── models.py │ │ ├── pocket_config.py │ │ └── utils.py │ └── sqs.py │ ├── async_utils.py │ ├── blocks │ └── notifications │ │ ├── __init__.py │ │ └── pagerduty.py │ ├── dataclass_utils.py │ ├── feature_store.py │ ├── iteration_utils.py │ ├── models │ ├── __init__.py │ └── corpus_candidate_set_configs.py │ ├── offset_utils.py │ ├── tasks.py │ └── transform_utils.py └── tests ├── article_text_streaming └── unit │ ├── test.json │ └── test_article_text_streaming_flow.py ├── braze └── test_braze.py ├── conftest.py ├── data_retention ├── test_deleted_accounts_flow.py └── test_development_snowflake_schemas_flow.py ├── firefox_newtab └── unit │ ├── sql │ └── test │ │ ├── data.sql │ │ ├── load.sql │ │ ├── sub_folder_1 │ │ ├── data.sql │ │ └── load.sql │ │ └── sub_folder_2 │ │ ├── data.sql │ │ └── load.sql │ └── test_run_jobs_flow.py ├── freestar_revenue_reporting └── unit │ └── test_freestar_reporting_flow.py ├── new_tab_recommendations └── test_aggregate_engagement_flow.py ├── recommendation_api ├── test_corpus_candidate_sets_flow.py ├── test_corpus_item_engagement_flow.py ├── test_legacy_candidate_sets_flow.py └── test_user_impressions_flow.py ├── shared └── unit │ ├── blocks │ └── notifications │ │ └── unit │ │ └── test_pagerduty.py │ ├── test_feature_store.py │ ├── test_offset_utils.py │ └── test_sqs.py ├── test.json └── utils.py /.aws/.docker/agent/Dockerfile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.aws/.docker/agent/Dockerfile -------------------------------------------------------------------------------- /.aws/.eslintignore: -------------------------------------------------------------------------------- 1 | node_modules 2 | dist -------------------------------------------------------------------------------- /.aws/.eslintrc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.aws/.eslintrc -------------------------------------------------------------------------------- /.aws/.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.aws/.gitignore -------------------------------------------------------------------------------- /.aws/.npmrc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.aws/.npmrc -------------------------------------------------------------------------------- /.aws/.nvmrc: -------------------------------------------------------------------------------- 1 | v18.12.1 -------------------------------------------------------------------------------- /.aws/.prettierrc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.aws/.prettierrc -------------------------------------------------------------------------------- /.aws/.terraform-version: -------------------------------------------------------------------------------- 1 | v1.6.6 -------------------------------------------------------------------------------- /.aws/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.aws/README.md -------------------------------------------------------------------------------- /.aws/cdktf.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.aws/cdktf.json -------------------------------------------------------------------------------- /.aws/package-lock.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.aws/package-lock.json -------------------------------------------------------------------------------- /.aws/package.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.aws/package.json -------------------------------------------------------------------------------- /.aws/src/config/index.ts: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.aws/src/config/index.ts -------------------------------------------------------------------------------- /.aws/src/iam.ts: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.aws/src/iam.ts -------------------------------------------------------------------------------- /.aws/src/main.ts: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.aws/src/main.ts -------------------------------------------------------------------------------- /.aws/tsconfig.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.aws/tsconfig.json -------------------------------------------------------------------------------- /.circleci/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.circleci/README.md -------------------------------------------------------------------------------- /.circleci/config.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.circleci/config.yml -------------------------------------------------------------------------------- /.circleci/workflows.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.circleci/workflows.yml -------------------------------------------------------------------------------- /.github/CODEOWNERS: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.github/CODEOWNERS -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.github/PULL_REQUEST_TEMPLATE.md -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/.gitignore -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/CODE_OF_CONDUCT.md -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/CONTRIBUTING.md -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/README.md -------------------------------------------------------------------------------- /common-utils/.coveragerc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/.coveragerc -------------------------------------------------------------------------------- /common-utils/.docker/main/Dockerfile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/.docker/main/Dockerfile -------------------------------------------------------------------------------- /common-utils/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/README.md -------------------------------------------------------------------------------- /common-utils/common_utils_build.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/common_utils_build.sh -------------------------------------------------------------------------------- /common-utils/poetry.lock: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/poetry.lock -------------------------------------------------------------------------------- /common-utils/pyproject.toml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/pyproject.toml -------------------------------------------------------------------------------- /common-utils/src/common/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/__init__.py -------------------------------------------------------------------------------- /common-utils/src/common/cloud/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /common-utils/src/common/cloud/aws_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/cloud/aws_utils.py -------------------------------------------------------------------------------- /common-utils/src/common/cloud/gcp_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/cloud/gcp_utils.py -------------------------------------------------------------------------------- /common-utils/src/common/databases/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /common-utils/src/common/databases/snowflake_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/databases/snowflake_utils.py -------------------------------------------------------------------------------- /common-utils/src/common/databases/sqlalchemy_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/databases/sqlalchemy_utils.py -------------------------------------------------------------------------------- /common-utils/src/common/deployment/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /common-utils/src/common/deployment/worker/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/deployment/worker/__init__.py -------------------------------------------------------------------------------- /common-utils/src/common/deployment/worker/build_image.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/deployment/worker/build_image.sh -------------------------------------------------------------------------------- /common-utils/src/common/deployment/worker/check_version.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/deployment/worker/check_version.py -------------------------------------------------------------------------------- /common-utils/src/common/deployment/worker/cli.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/deployment/worker/cli.py -------------------------------------------------------------------------------- /common-utils/src/common/deployment/worker/clone_project.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/deployment/worker/clone_project.sh -------------------------------------------------------------------------------- /common-utils/src/common/deployment/worker/common_utils_build.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/deployment/worker/common_utils_build.sh -------------------------------------------------------------------------------- /common-utils/src/common/deployment/worker/get_prefect_version.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/deployment/worker/get_prefect_version.sh -------------------------------------------------------------------------------- /common-utils/src/common/deployment/worker/push_aws_image.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/deployment/worker/push_aws_image.sh -------------------------------------------------------------------------------- /common-utils/src/common/settings.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/settings.py -------------------------------------------------------------------------------- /common-utils/src/common/testing_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/src/common/testing_utils.py -------------------------------------------------------------------------------- /common-utils/tests/cloud/unit/test_aws_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/cloud/unit/test_aws_utils.py -------------------------------------------------------------------------------- /common-utils/tests/cloud/unit/test_gcp_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/cloud/unit/test_gcp_utils.py -------------------------------------------------------------------------------- /common-utils/tests/conftest.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/conftest.py -------------------------------------------------------------------------------- /common-utils/tests/databases/unit/test_snowflake_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/databases/unit/test_snowflake_utils.py -------------------------------------------------------------------------------- /common-utils/tests/databases/unit/test_sqlalchemy_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/databases/unit/test_sqlalchemy_utils.py -------------------------------------------------------------------------------- /common-utils/tests/deployment/worker/test_flows/example_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/deployment/worker/test_flows/example_flow.py -------------------------------------------------------------------------------- /common-utils/tests/deployment/worker/test_flows/example_legacy_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/deployment/worker/test_flows/example_legacy_flow.py -------------------------------------------------------------------------------- /common-utils/tests/deployment/worker/test_flows/flow_group_2/example_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/deployment/worker/test_flows/flow_group_2/example_flow.py -------------------------------------------------------------------------------- /common-utils/tests/deployment/worker/test_flows/flow_group_2/example_no_jv_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/deployment/worker/test_flows/flow_group_2/example_no_jv_flow.py -------------------------------------------------------------------------------- /common-utils/tests/deployment/worker/test_flows/flow_group_2/no_spec_example_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/deployment/worker/test_flows/flow_group_2/no_spec_example_flow.py -------------------------------------------------------------------------------- /common-utils/tests/deployment/worker/test_flows/single_flow/example_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/deployment/worker/test_flows/single_flow/example_flow.py -------------------------------------------------------------------------------- /common-utils/tests/deployment/worker/unit/test_check_version_wk.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/deployment/worker/unit/test_check_version_wk.py -------------------------------------------------------------------------------- /common-utils/tests/deployment/worker/unit/test_cli_wk.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/deployment/worker/unit/test_cli_wk.py -------------------------------------------------------------------------------- /common-utils/tests/deployment/worker/unit/test_deployment_wk.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/deployment/worker/unit/test_deployment_wk.py -------------------------------------------------------------------------------- /common-utils/tests/test.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/test.json -------------------------------------------------------------------------------- /common-utils/tests/unit/test_common.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/unit/test_common.py -------------------------------------------------------------------------------- /common-utils/tests/unit/test_settings.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/unit/test_settings.py -------------------------------------------------------------------------------- /common-utils/tests/unit/test_testing_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/common-utils/tests/unit/test_testing_utils.py -------------------------------------------------------------------------------- /data-products/.coveragerc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/.coveragerc -------------------------------------------------------------------------------- /data-products/.docker/base/Dockerfile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/.docker/base/Dockerfile -------------------------------------------------------------------------------- /data-products/.docker/worker/base/Dockerfile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/.docker/worker/base/Dockerfile -------------------------------------------------------------------------------- /data-products/.env.example: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/.env.example -------------------------------------------------------------------------------- /data-products/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/README.md -------------------------------------------------------------------------------- /data-products/docs/flows/sql_etl/code.md: -------------------------------------------------------------------------------- 1 | :::src.sql_etl.run_jobs_flow -------------------------------------------------------------------------------- /data-products/docs/flows/sql_etl/sql-etl-flow-diagram.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/docs/flows/sql_etl/sql-etl-flow-diagram.jpg -------------------------------------------------------------------------------- /data-products/docs/flows/sql_etl/usage.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/docs/flows/sql_etl/usage.md -------------------------------------------------------------------------------- /data-products/docs/index.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/docs/index.md -------------------------------------------------------------------------------- /data-products/mkdocs.yml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/mkdocs.yml -------------------------------------------------------------------------------- /data-products/poetry.lock: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/poetry.lock -------------------------------------------------------------------------------- /data-products/pyproject.toml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/pyproject.toml -------------------------------------------------------------------------------- /data-products/src/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data-products/src/article_text_streaming/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data-products/src/article_text_streaming/article_text_streaming_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/article_text_streaming/article_text_streaming_flow.py -------------------------------------------------------------------------------- /data-products/src/braze/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data-products/src/braze/update_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/braze/update_flow.py -------------------------------------------------------------------------------- /data-products/src/data_retention/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data-products/src/data_retention/delete_miscellaneous_table_rows.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/data_retention/delete_miscellaneous_table_rows.sql -------------------------------------------------------------------------------- /data-products/src/data_retention/delete_raw_user_rows.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/data_retention/delete_raw_user_rows.sql -------------------------------------------------------------------------------- /data-products/src/data_retention/delete_snapshot_firehose_rows.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/data_retention/delete_snapshot_firehose_rows.sql -------------------------------------------------------------------------------- /data-products/src/data_retention/delete_snowplow_events.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/data_retention/delete_snowplow_events.sql -------------------------------------------------------------------------------- /data-products/src/data_retention/delete_stripe_table_rows.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/data_retention/delete_stripe_table_rows.sql -------------------------------------------------------------------------------- /data-products/src/data_retention/deleted_account_emails.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/data_retention/deleted_account_emails.sql -------------------------------------------------------------------------------- /data-products/src/data_retention/deleted_account_users.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/data_retention/deleted_account_users.sql -------------------------------------------------------------------------------- /data-products/src/data_retention/deleted_accounts_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/data_retention/deleted_accounts_flow.py -------------------------------------------------------------------------------- /data-products/src/data_retention/snowflake_development_table_clean_up_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/data_retention/snowflake_development_table_clean_up_flow.py -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/run_jobs_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/run_jobs_flow.py -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/extras/helpers.j2: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/extras/helpers.j2 -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_daily_disable_rate_by_feed/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_daily_disable_rate_by_feed/data.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_daily_disable_rate_by_feed/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_daily_disable_rate_by_feed/load.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_daily_spoc_fill_rate_by_position_feed/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_daily_spoc_fill_rate_by_position_feed/data.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_daily_spoc_fill_rate_by_position_feed/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_daily_spoc_fill_rate_by_position_feed/load.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_daily_unique_engagement_by_feed/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_daily_unique_engagement_by_feed/data.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_daily_unique_engagement_by_feed/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_daily_unique_engagement_by_feed/load.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_monthly_unique_engagement_by_feed/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_monthly_unique_engagement_by_feed/data.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_monthly_unique_engagement_by_feed/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_daily/firefox_new_tab_monthly_unique_engagement_by_feed/load.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_hourly/firefox_new_tab_daily_engagement_by_tile_id_position/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_hourly/firefox_new_tab_daily_engagement_by_tile_id_position/data.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_hourly/firefox_new_tab_daily_engagement_by_tile_id_position/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_hourly/firefox_new_tab_daily_engagement_by_tile_id_position/load.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_hourly/firefox_new_tab_daily_engagement_by_tile_id_position_country_locale/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_hourly/firefox_new_tab_daily_engagement_by_tile_id_position_country_locale/data.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_hourly/firefox_new_tab_daily_engagement_by_tile_id_position_country_locale/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/firefox_new_tab_impressions_hourly/firefox_new_tab_daily_engagement_by_tile_id_position_country_locale/load.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_daily/glean_firefox_new_tab_daily_disable_rate_by_feed/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_daily/glean_firefox_new_tab_daily_disable_rate_by_feed/data.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_daily/glean_firefox_new_tab_daily_disable_rate_by_feed/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_daily/glean_firefox_new_tab_daily_disable_rate_by_feed/load.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_daily/glean_firefox_new_tab_daily_spoc_fill_rate_by_position_feed/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_daily/glean_firefox_new_tab_daily_spoc_fill_rate_by_position_feed/data.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_daily/glean_firefox_new_tab_daily_spoc_fill_rate_by_position_feed/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_daily/glean_firefox_new_tab_daily_spoc_fill_rate_by_position_feed/load.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_daily/glean_firefox_new_tab_daily_unique_engagement_by_feed/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_daily/glean_firefox_new_tab_daily_unique_engagement_by_feed/data.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_daily/glean_firefox_new_tab_daily_unique_engagement_by_feed/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_daily/glean_firefox_new_tab_daily_unique_engagement_by_feed/load.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_hourly/glean_firefox_new_tab_daily_engagement_by_scheduled_corpus_item_id_position_country_locale/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_hourly/glean_firefox_new_tab_daily_engagement_by_scheduled_corpus_item_id_position_country_locale/data.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_hourly/glean_firefox_new_tab_daily_engagement_by_scheduled_corpus_item_id_position_country_locale/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_hourly/glean_firefox_new_tab_daily_engagement_by_scheduled_corpus_item_id_position_country_locale/load.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_hourly/glean_firefox_new_tab_daily_engagement_by_tile_id_position/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_hourly/glean_firefox_new_tab_daily_engagement_by_tile_id_position/data.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_hourly/glean_firefox_new_tab_daily_engagement_by_tile_id_position/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_hourly/glean_firefox_new_tab_daily_engagement_by_tile_id_position/load.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_hourly/glean_firefox_new_tab_daily_engagement_by_tile_id_position_country_locale/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_hourly/glean_firefox_new_tab_daily_engagement_by_tile_id_position_country_locale/data.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_hourly/glean_firefox_new_tab_daily_engagement_by_tile_id_position_country_locale/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/glean_firefox_new_tab_impressions_hourly/glean_firefox_new_tab_daily_engagement_by_tile_id_position_country_locale/load.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/uapi_interactions_hourly/uapi_daily_interactions_by_tile_id_position/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/uapi_interactions_hourly/uapi_daily_interactions_by_tile_id_position/data.sql -------------------------------------------------------------------------------- /data-products/src/firefox_newtab/sql/uapi_interactions_hourly/uapi_daily_interactions_by_tile_id_position/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/firefox_newtab/sql/uapi_interactions_hourly/uapi_daily_interactions_by_tile_id_position/load.sql -------------------------------------------------------------------------------- /data-products/src/freestar_revenue_reporting/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data-products/src/freestar_revenue_reporting/freestar_reporting_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/freestar_revenue_reporting/freestar_reporting_flow.py -------------------------------------------------------------------------------- /data-products/src/new_tab_recommendations/aggregate_engagement_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/new_tab_recommendations/aggregate_engagement_flow.py -------------------------------------------------------------------------------- /data-products/src/prospecting/publisher_features_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/prospecting/publisher_features_flow.py -------------------------------------------------------------------------------- /data-products/src/prospecting/sql/publisher_features.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/prospecting/sql/publisher_features.sql -------------------------------------------------------------------------------- /data-products/src/recommendation_api/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data-products/src/recommendation_api/corpus_candidate_sets_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/recommendation_api/corpus_candidate_sets_flow.py -------------------------------------------------------------------------------- /data-products/src/recommendation_api/corpus_item_engagement_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/recommendation_api/corpus_item_engagement_flow.py -------------------------------------------------------------------------------- /data-products/src/recommendation_api/legacy_candidate_sets_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/recommendation_api/legacy_candidate_sets_flow.py -------------------------------------------------------------------------------- /data-products/src/recommendation_api/sql/collections_by_recency.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/recommendation_api/sql/collections_by_recency.sql -------------------------------------------------------------------------------- /data-products/src/recommendation_api/sql/labeled_collection_stories.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/recommendation_api/sql/labeled_collection_stories.sql -------------------------------------------------------------------------------- /data-products/src/recommendation_api/sql/life_hacks.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/recommendation_api/sql/life_hacks.sql -------------------------------------------------------------------------------- /data-products/src/recommendation_api/sql/pocket_hits.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/recommendation_api/sql/pocket_hits.sql -------------------------------------------------------------------------------- /data-products/src/recommendation_api/sql/recommended_by_recency.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/recommendation_api/sql/recommended_by_recency.sql -------------------------------------------------------------------------------- /data-products/src/recommendation_api/sql/scheduled_not_syndicated_or_collection.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/recommendation_api/sql/scheduled_not_syndicated_or_collection.sql -------------------------------------------------------------------------------- /data-products/src/recommendation_api/sql/scheduled_syndicated.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/recommendation_api/sql/scheduled_syndicated.sql -------------------------------------------------------------------------------- /data-products/src/recommendation_api/sql/topic.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/recommendation_api/sql/topic.sql -------------------------------------------------------------------------------- /data-products/src/recommendation_api/user_impressions_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/recommendation_api/user_impressions_flow.py -------------------------------------------------------------------------------- /data-products/src/shared/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data-products/src/shared/api_clients/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data-products/src/shared/api_clients/braze/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data-products/src/shared/api_clients/braze/client.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/shared/api_clients/braze/client.py -------------------------------------------------------------------------------- /data-products/src/shared/api_clients/braze/models.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/shared/api_clients/braze/models.py -------------------------------------------------------------------------------- /data-products/src/shared/api_clients/braze/pocket_config.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/shared/api_clients/braze/pocket_config.py -------------------------------------------------------------------------------- /data-products/src/shared/api_clients/braze/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/shared/api_clients/braze/utils.py -------------------------------------------------------------------------------- /data-products/src/shared/api_clients/sqs.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/shared/api_clients/sqs.py -------------------------------------------------------------------------------- /data-products/src/shared/async_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/shared/async_utils.py -------------------------------------------------------------------------------- /data-products/src/shared/blocks/notifications/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data-products/src/shared/blocks/notifications/pagerduty.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/shared/blocks/notifications/pagerduty.py -------------------------------------------------------------------------------- /data-products/src/shared/dataclass_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/shared/dataclass_utils.py -------------------------------------------------------------------------------- /data-products/src/shared/feature_store.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/shared/feature_store.py -------------------------------------------------------------------------------- /data-products/src/shared/iteration_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/shared/iteration_utils.py -------------------------------------------------------------------------------- /data-products/src/shared/models/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /data-products/src/shared/models/corpus_candidate_set_configs.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/shared/models/corpus_candidate_set_configs.py -------------------------------------------------------------------------------- /data-products/src/shared/offset_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/shared/offset_utils.py -------------------------------------------------------------------------------- /data-products/src/shared/tasks.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/shared/tasks.py -------------------------------------------------------------------------------- /data-products/src/shared/transform_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/src/shared/transform_utils.py -------------------------------------------------------------------------------- /data-products/tests/article_text_streaming/unit/test.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/article_text_streaming/unit/test.json -------------------------------------------------------------------------------- /data-products/tests/article_text_streaming/unit/test_article_text_streaming_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/article_text_streaming/unit/test_article_text_streaming_flow.py -------------------------------------------------------------------------------- /data-products/tests/braze/test_braze.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/braze/test_braze.py -------------------------------------------------------------------------------- /data-products/tests/conftest.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/conftest.py -------------------------------------------------------------------------------- /data-products/tests/data_retention/test_deleted_accounts_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/data_retention/test_deleted_accounts_flow.py -------------------------------------------------------------------------------- /data-products/tests/data_retention/test_development_snowflake_schemas_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/data_retention/test_development_snowflake_schemas_flow.py -------------------------------------------------------------------------------- /data-products/tests/firefox_newtab/unit/sql/test/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/firefox_newtab/unit/sql/test/data.sql -------------------------------------------------------------------------------- /data-products/tests/firefox_newtab/unit/sql/test/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/firefox_newtab/unit/sql/test/load.sql -------------------------------------------------------------------------------- /data-products/tests/firefox_newtab/unit/sql/test/sub_folder_1/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/firefox_newtab/unit/sql/test/sub_folder_1/data.sql -------------------------------------------------------------------------------- /data-products/tests/firefox_newtab/unit/sql/test/sub_folder_1/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/firefox_newtab/unit/sql/test/sub_folder_1/load.sql -------------------------------------------------------------------------------- /data-products/tests/firefox_newtab/unit/sql/test/sub_folder_2/data.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/firefox_newtab/unit/sql/test/sub_folder_2/data.sql -------------------------------------------------------------------------------- /data-products/tests/firefox_newtab/unit/sql/test/sub_folder_2/load.sql: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/firefox_newtab/unit/sql/test/sub_folder_2/load.sql -------------------------------------------------------------------------------- /data-products/tests/firefox_newtab/unit/test_run_jobs_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/firefox_newtab/unit/test_run_jobs_flow.py -------------------------------------------------------------------------------- /data-products/tests/freestar_revenue_reporting/unit/test_freestar_reporting_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/freestar_revenue_reporting/unit/test_freestar_reporting_flow.py -------------------------------------------------------------------------------- /data-products/tests/new_tab_recommendations/test_aggregate_engagement_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/new_tab_recommendations/test_aggregate_engagement_flow.py -------------------------------------------------------------------------------- /data-products/tests/recommendation_api/test_corpus_candidate_sets_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/recommendation_api/test_corpus_candidate_sets_flow.py -------------------------------------------------------------------------------- /data-products/tests/recommendation_api/test_corpus_item_engagement_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/recommendation_api/test_corpus_item_engagement_flow.py -------------------------------------------------------------------------------- /data-products/tests/recommendation_api/test_legacy_candidate_sets_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/recommendation_api/test_legacy_candidate_sets_flow.py -------------------------------------------------------------------------------- /data-products/tests/recommendation_api/test_user_impressions_flow.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/recommendation_api/test_user_impressions_flow.py -------------------------------------------------------------------------------- /data-products/tests/shared/unit/blocks/notifications/unit/test_pagerduty.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/shared/unit/blocks/notifications/unit/test_pagerduty.py -------------------------------------------------------------------------------- /data-products/tests/shared/unit/test_feature_store.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/shared/unit/test_feature_store.py -------------------------------------------------------------------------------- /data-products/tests/shared/unit/test_offset_utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/shared/unit/test_offset_utils.py -------------------------------------------------------------------------------- /data-products/tests/shared/unit/test_sqs.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/shared/unit/test_sqs.py -------------------------------------------------------------------------------- /data-products/tests/test.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/test.json -------------------------------------------------------------------------------- /data-products/tests/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Pocket/data-flows/HEAD/data-products/tests/utils.py --------------------------------------------------------------------------------