├── src └── cfs │ ├── generate_video_file │ ├── requirements.txt │ └── main.py │ └── generate_tts_files │ ├── requirements.txt │ └── main.py ├── CONTRIBUTING.md ├── variables.tf ├── main.tf ├── cfs.tf ├── LICENSE └── README.md /src/cfs/generate_video_file/requirements.txt: -------------------------------------------------------------------------------- 1 | google-api-python-client 2 | google-cloud-storage 3 | mutagen 4 | google-cloud-pubsub 5 | functions-framework -------------------------------------------------------------------------------- /src/cfs/generate_tts_files/requirements.txt: -------------------------------------------------------------------------------- 1 | google-api-python-client==1.7.9 2 | google-auth-httplib2==0.0.3 3 | google-auth-oauthlib==0.4.0 4 | google-cloud-texttospeech 5 | google-cloud-storage 6 | google-cloud-pubsub 7 | functions-framework -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # How to contribute 2 | 3 | We'd love to accept your patches and contributions to this project. 4 | 5 | ## Before you begin 6 | 7 | ### Sign our Contributor License Agreement 8 | 9 | Contributions to this project must be accompanied by a 10 | [Contributor License Agreement](https://cla.developers.google.com/about) (CLA). 11 | You (or your employer) retain the copyright to your contribution; this simply 12 | gives us permission to use and redistribute your contributions as part of the 13 | project. 14 | 15 | If you or your current employer have already signed the Google CLA (even if it 16 | was for a different project), you probably don't need to do it again. 17 | 18 | Visit to see your current agreements or to 19 | sign a new one. 20 | 21 | ### Review our community guidelines 22 | 23 | This project follows 24 | [Google's Open Source Community Guidelines](https://opensource.google/conduct/). 25 | 26 | ## Contribution process 27 | 28 | ### Code reviews 29 | 30 | All submissions, including submissions by project members, require review. We 31 | use GitHub pull requests for this purpose. Consult 32 | [GitHub Help](https://help.github.com/articles/about-pull-requests/) for more 33 | information on using pull requests. -------------------------------------------------------------------------------- /variables.tf: -------------------------------------------------------------------------------- 1 | # Copyright 2023 Google LLC 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # https://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | # 15 | # -------------------------------------------------- 16 | # Set these before applying the configuration 17 | # -------------------------------------------------- 18 | 19 | variable "gcp_project" { 20 | type = string 21 | description = "Google Cloud Project ID where the artifacts will be deployed" 22 | default = "my-project" 23 | } 24 | 25 | variable "gcp_region" { 26 | type = string 27 | description = "Google Cloud Region" 28 | default = "europe-west1" 29 | } 30 | 31 | variable "ai_dubbing_sa" { 32 | type = string 33 | description = "Service Account for the deployment" 34 | default = "ai-dubbing" 35 | } 36 | 37 | variable "ai_dubbing_bucket_name" { 38 | type = string 39 | description = "GCS bucket used for deployment tasks" 40 | default = "ai_dubbing_bucket" 41 | } 42 | 43 | variable "config_spreadsheet_id" { 44 | type = string 45 | description = "The ID of the config spreadhseet" 46 | default = "my-spreadhseet-id" 47 | } 48 | 49 | variable "execution_schedule" { 50 | type = string 51 | description = "The schedule to execute the process (every 30 min default) " 52 | default = "*/30 * * * *" 53 | } 54 | 55 | variable "config_sheet_name" { 56 | type = string 57 | description = "The name of the sheet" 58 | default = "config" 59 | } 60 | 61 | variable "config_sheet_range" { 62 | type = string 63 | description = "Google Cloud Region" 64 | default = "config!A1:N" 65 | } 66 | 67 | variable "tts_file_column" { 68 | type = string 69 | description = "Column of the TTS file in the config sheet" 70 | default = "K" 71 | } 72 | 73 | variable "final_video_file_column" { 74 | type = string 75 | description = "Column of the video file in the config sheet" 76 | default = "L" 77 | } 78 | 79 | variable "status_column" { 80 | type = string 81 | description = "Column for Status in the config sheet" 82 | default = "M" 83 | } 84 | 85 | variable "last_update_column" { 86 | type = string 87 | description = "Column for last update in the config sheet" 88 | default = "N" 89 | } 90 | 91 | variable "generate_tts_files_trigger_pubsub_topic" { 92 | type = string 93 | description = "The name for the pubsusb topic to trigger the tts generation cloud function" 94 | default = "generate_tts_files_trigger" 95 | } 96 | 97 | variable "generate_video_file_trigger_pubsub_topic" { 98 | type = string 99 | description = "The name for the pubsusb topic to trigger the video generation cloud function" 100 | default = "generate_video_file_trigger" 101 | } -------------------------------------------------------------------------------- /main.tf: -------------------------------------------------------------------------------- 1 | # Copyright 2023 Google LLC 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # https://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | provider "google" { 16 | project = var.gcp_project 17 | region = var.gcp_region 18 | } 19 | 20 | data "google_project" "project" { 21 | provider = google 22 | } 23 | 24 | resource "google_service_account" "service_account" { 25 | account_id = var.ai_dubbing_sa 26 | display_name = "AI Dubbing Service Account" 27 | } 28 | 29 | resource "google_project_service" "enable_cloudbuild" { 30 | project = var.gcp_project 31 | service = "cloudbuild.googleapis.com" 32 | 33 | timeouts { 34 | create = "30m" 35 | update = "40m" 36 | } 37 | 38 | disable_dependent_services = true 39 | depends_on = [google_service_account.service_account] 40 | } 41 | 42 | resource "google_project_service" "enable_texttospeech_api" { 43 | project = var.gcp_project 44 | service = "texttospeech.googleapis.com" 45 | 46 | timeouts { 47 | create = "30m" 48 | update = "40m" 49 | } 50 | 51 | disable_dependent_services = true 52 | depends_on = [google_service_account.service_account] 53 | } 54 | 55 | resource "google_project_service" "enable_sheetsapi" { 56 | project = var.gcp_project 57 | service = "sheets.googleapis.com" 58 | 59 | timeouts { 60 | create = "30m" 61 | update = "40m" 62 | } 63 | 64 | disable_dependent_services = true 65 | depends_on = [google_service_account.service_account] 66 | } 67 | 68 | resource "google_project_service" "enable_cloudfunctions" { 69 | project = var.gcp_project 70 | service = "cloudfunctions.googleapis.com" 71 | 72 | timeouts { 73 | create = "30m" 74 | update = "40m" 75 | } 76 | 77 | disable_dependent_services = true 78 | depends_on = [google_service_account.service_account] 79 | } 80 | 81 | resource "google_project_service" "enable_pubsub" { 82 | project = var.gcp_project 83 | service = "pubsub.googleapis.com" 84 | 85 | timeouts { 86 | create = "30m" 87 | update = "40m" 88 | } 89 | 90 | disable_dependent_services = true 91 | depends_on = [google_service_account.service_account] 92 | } 93 | 94 | resource "google_project_service" "enable_cloudscheduler" { 95 | project = var.gcp_project 96 | service = "cloudscheduler.googleapis.com" 97 | 98 | timeouts { 99 | create = "30m" 100 | update = "40m" 101 | } 102 | 103 | disable_dependent_services = true 104 | depends_on = [google_service_account.service_account] 105 | } 106 | 107 | resource "google_project_iam_member" "permissions_token" { 108 | project = data.google_project.project.project_id 109 | role = "roles/iam.serviceAccountShortTermTokenMinter" 110 | member = "serviceAccount:${google_service_account.service_account.email}" 111 | depends_on = [google_service_account.service_account] 112 | } 113 | 114 | resource "google_project_iam_member" "permissions_gcs" { 115 | project = data.google_project.project.project_id 116 | role = "roles/storage.objectAdmin" 117 | member = "serviceAccount:${google_service_account.service_account.email}" 118 | depends_on = [google_service_account.service_account] 119 | } 120 | 121 | resource "google_project_iam_member" "permissions_pubsub" { 122 | project = data.google_project.project.project_id 123 | role = "roles/pubsub.publisher" 124 | member = "serviceAccount:${google_service_account.service_account.email}" 125 | depends_on = [google_service_account.service_account] 126 | } 127 | 128 | resource "google_pubsub_topic" "generate_tts_files_trigger_topic" { 129 | depends_on = [google_project_service.enable_pubsub] 130 | name = var.generate_tts_files_trigger_pubsub_topic 131 | } 132 | 133 | resource "google_pubsub_topic" "generate_video_file_trigger_topic" { 134 | depends_on = [google_project_service.enable_pubsub] 135 | name = var.generate_video_file_trigger_pubsub_topic 136 | } 137 | 138 | resource "google_cloud_scheduler_job" "job" { 139 | depends_on = [google_pubsub_topic.generate_tts_files_trigger_topic] 140 | name = "ai-dubbing-trigger" 141 | description = "ai-dubbing-trigger" 142 | schedule = "*/5 * * * *" 143 | 144 | pubsub_target { 145 | topic_name = google_pubsub_topic.generate_tts_files_trigger_topic.id 146 | data = base64encode("None") 147 | } 148 | } -------------------------------------------------------------------------------- /cfs.tf: -------------------------------------------------------------------------------- 1 | # Copyright 2023 Google LLC 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # https://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | # 15 | # Generates an archive of the source code compressed as a .zip file. 16 | 17 | resource "google_storage_bucket" "ai_dubbing_bucket" { 18 | project = data.google_project.project.project_id 19 | name = var.ai_dubbing_bucket_name 20 | location = var.gcp_region 21 | force_destroy = false 22 | uniform_bucket_level_access = true 23 | depends_on = [google_service_account.service_account, 24 | google_project_service.enable_cloudfunctions] 25 | } 26 | 27 | data "archive_file" "source_generate_tts_files" { 28 | type = "zip" 29 | source_dir = "src/cfs/generate_tts_files" 30 | output_path = "/tmp/generate_tts_files.zip" 31 | depends_on = [] 32 | } 33 | 34 | data "archive_file" "source_generate_video_file" { 35 | type = "zip" 36 | source_dir = "src/cfs/generate_video_file" 37 | output_path = "/tmp/generate_video_file.zip" 38 | depends_on = [] 39 | } 40 | 41 | # Add source code zip to the Cloud Function's bucket 42 | resource "google_storage_bucket_object" "generate_tts_files_zip" { 43 | source = data.archive_file.source_generate_tts_files.output_path 44 | content_type = "application/zip" 45 | 46 | # Append to the MD5 checksum of the files's content 47 | # to force the zip to be updated as soon as a change occurs 48 | name = "src-${data.archive_file.source_generate_tts_files.output_md5}.zip" 49 | bucket = google_storage_bucket.ai_dubbing_bucket.name 50 | 51 | # Dependencies are automatically inferred so these lines can be deleted 52 | depends_on = [ 53 | google_storage_bucket.ai_dubbing_bucket, # declared in `storage.tf` 54 | ] 55 | } 56 | 57 | # Add source code zip to the Cloud Function's bucket 58 | resource "google_storage_bucket_object" "generate_video_file_zip" { 59 | source = data.archive_file.source_generate_video_file.output_path 60 | content_type = "application/zip" 61 | 62 | # Append to the MD5 checksum of the files's content 63 | # to force the zip to be updated as soon as a change occurs 64 | name = "src-${data.archive_file.source_generate_video_file.output_md5}.zip" 65 | bucket = google_storage_bucket.ai_dubbing_bucket.name 66 | 67 | # Dependencies are automatically inferred so these lines can be deleted 68 | depends_on = [ 69 | google_storage_bucket.ai_dubbing_bucket, # declared in `storage.tf` 70 | ] 71 | } 72 | 73 | # Create the Cloud function triggered by a `Finalize` event on the bucket 74 | resource "google_cloudfunctions_function" "function_generate_tts_files" { 75 | depends_on = [ 76 | google_storage_bucket_object.generate_tts_files_zip, 77 | google_service_account.service_account, 78 | google_project_service.enable_cloudfunctions, 79 | google_project_service.enable_cloudbuild, 80 | google_storage_bucket.ai_dubbing_bucket 81 | ] 82 | name = "generate_tts_files" 83 | runtime = "python38" 84 | 85 | environment_variables = { 86 | GCP_PROJECT = var.gcp_project, 87 | CONFIG_SPREADSHEET_ID = var.config_spreadsheet_id, 88 | CONFIG_SHEET_NAME = var.config_sheet_name, 89 | CONFIG_SHEET_RANGE = var.config_sheet_range, 90 | STATUS_COLUMN = var.status_column, 91 | TTS_FILE_COLUMN = var.tts_file_column, 92 | LAST_UPDATE_COLUMN = var.last_update_column, 93 | GENERATE_VIDEO_TOPIC = var.generate_video_file_trigger_pubsub_topic 94 | } 95 | 96 | # Get the source code of the cloud function as a Zip compression 97 | source_archive_bucket = google_storage_bucket.ai_dubbing_bucket.name 98 | source_archive_object = google_storage_bucket_object.generate_tts_files_zip.name 99 | 100 | # Must match the function name in the cloud function `main.py` source code 101 | entry_point = "main" 102 | service_account_email = google_service_account.service_account.email 103 | available_memory_mb = 2048 104 | timeout = 540 105 | 106 | event_trigger { 107 | event_type = "google.pubsub.topic.publish" 108 | resource = google_pubsub_topic.generate_tts_files_trigger_topic.id 109 | } 110 | } 111 | 112 | # Create the Cloud function triggered by a `Finalize` event on the bucket 113 | resource "google_cloudfunctions_function" "function_generate_video_file" { 114 | depends_on = [ 115 | google_storage_bucket_object.generate_video_file_zip, 116 | google_service_account.service_account, 117 | google_project_service.enable_cloudfunctions, 118 | google_project_service.enable_cloudbuild, 119 | google_storage_bucket.ai_dubbing_bucket 120 | ] 121 | name = "generate_video_file" 122 | runtime = "python38" 123 | 124 | environment_variables = { 125 | GCP_PROJECT = var.gcp_project 126 | CONFIG_SPREADSHEET_ID = var.config_spreadsheet_id, 127 | CONFIG_SHEET_NAME = var.config_sheet_name, 128 | STATUS_COLUMN = var.status_column, 129 | LAST_UPDATE_COLUMN = var.last_update_column, 130 | FINAL_VIDEO_FILE_COLUMN = var.final_video_file_column 131 | } 132 | 133 | # Get the source code of the cloud function as a Zip compression 134 | source_archive_bucket = google_storage_bucket.ai_dubbing_bucket.name 135 | source_archive_object = google_storage_bucket_object.generate_video_file_zip.name 136 | 137 | # Must match the function name in the cloud function `main.py` source code 138 | entry_point = "main" 139 | available_memory_mb = 8192 140 | service_account_email = google_service_account.service_account.email 141 | timeout = 540 142 | 143 | event_trigger { 144 | event_type = "google.pubsub.topic.publish" 145 | resource = google_pubsub_topic.generate_video_file_trigger_topic.id 146 | } 147 | } -------------------------------------------------------------------------------- /src/cfs/generate_tts_files/main.py: -------------------------------------------------------------------------------- 1 | # Copyright 2023 Google LLC 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | # [START main] 16 | 17 | # Config Line Fields 18 | # campaign 19 | # topic 20 | # gcs_bucket 21 | # video_file 22 | # base_audio_file 23 | # text 24 | # voice_id 25 | # millisecond_start_audio 26 | # audio_encoding 27 | # tts_file_url 28 | # final_video_file_url 29 | # status 30 | # last_update 31 | 32 | from typing import Any, Dict, List, Optional 33 | from google.cloud.functions_v1.context import Context 34 | from google.cloud import texttospeech 35 | from google.cloud import storage 36 | from google.cloud import pubsub_v1 37 | from datetime import datetime 38 | 39 | import json 40 | import os.path 41 | import base64 42 | 43 | from googleapiclient.discovery import build 44 | from googleapiclient.errors import HttpError 45 | 46 | # The ID and range of a sample spreadsheet. 47 | GCP_PROJECT = os.getenv('GCP_PROJECT', '') 48 | CONFIG_SPREADSHEET_ID = os.getenv('CONFIG_SPREADSHEET_ID', '') 49 | CONFIG_SHEET_NAME = os.getenv('CONFIG_SHEET_NAME', 'config') 50 | CONFIG_RANGE_NAME = os.getenv('CONFIG_RANGE_NAME', 'config!A1:N') 51 | TTS_FILE_COLUMN = os.getenv('TTS_FILE_COLUMN', 'K') 52 | STATUS_COLUMN = os.getenv('STATUS_COLUMN', 'M') 53 | LAST_UPDATE_COLUMN = os.getenv('LAST_UPDATE_COLUMN', 'N') 54 | GENERATE_VIDEO_TOPIC = os.getenv('GENERATE_VIDEO_TOPIC', 'generate_video_trigger') 55 | 56 | def main(event: Dict[str, Any], context=Optional[Context]): 57 | """ 58 | Reads config and generate TTS files to finally trigger the video generation. 59 | 60 | Args: 61 | event (dict): The dictionary with data specific to this type of event. The 62 | `data` field contains the PubsubMessage message. The `attributes` field 63 | will contain custom attributes if there are any. 64 | context (google.cloud.functions.Context): The Cloud Functions event 65 | metadata. The `event_id` field contains the Pub/Sub message ID. The 66 | `timestamp` field contains the publish time. 67 | """ 68 | del context # unused 69 | del event # unused 70 | lines = _read_config_from_google_sheet(CONFIG_SPREADSHEET_ID,CONFIG_SHEET_NAME) 71 | lines = _generate_tts(lines) 72 | 73 | def _read_config_from_google_sheet(sheet_id, sheet_name) -> List[Dict]: 74 | """ 75 | Reads all the lines in a Google Sheet having the first row the name of the fields and outputs an array of dicts. 76 | 77 | Args: 78 | sheet_id: The ID of the Google Sheet. 79 | sheet_name: The name of the sheet in the Google Sheet. 80 | 81 | Returns: 82 | An array of dicts, where each dict represents a row in the Google Sheet. 83 | """ 84 | creds = None 85 | # The file token.json stores the user's access and refresh tokens, and is 86 | # created automatically when the authorization flow completes for the first 87 | # time. 88 | rows = [] 89 | try: 90 | service = build('sheets', 'v4', credentials=None) 91 | 92 | # Call the Sheets API 93 | sheet = service.spreadsheets() 94 | result = sheet.values().get(spreadsheetId=CONFIG_SPREADSHEET_ID, 95 | range=CONFIG_RANGE_NAME).execute() 96 | values = result.get('values', []) 97 | 98 | # Create an array of dicts, where each dict represents a row in the Google Sheet. 99 | if not values: 100 | print('No values') 101 | return rows 102 | 103 | headers = values[0] 104 | i = 2 105 | for row in values[1:]: 106 | if row: 107 | new_row = {field: value for field, value in zip(headers, row)} 108 | new_row['index'] = i 109 | rows.append(new_row) 110 | i = i + 1 111 | 112 | except HttpError as err: 113 | print(err) 114 | 115 | return rows 116 | 117 | def _generate_tts(lines: List[Dict]) -> List[Dict]: 118 | """ 119 | For each line makes a call to Google TTS AI to generate the audio files and store them in GCS 120 | 121 | Args: 122 | lines: Dict object containing all 123 | sheet_name: The name of the sheet in the Google Sheet. 124 | 125 | Returns: 126 | An array of dicts, where each dict represents a row in the Google Sheet. 127 | """ 128 | for line in lines: 129 | if line: 130 | try: 131 | today = datetime.today().strftime('%Y%m%d') 132 | file_name = f"output/{today}/{_build_file_name(line)}" 133 | _tts_api_call(line, file_name) 134 | line['status'] = 'TTS OK' 135 | line['tts_file_url'] = file_name 136 | _call_video_generation(line) 137 | 138 | except Exception as e: 139 | line['status'] = e 140 | line['tts_file_url'] = 'N/A' 141 | print(e) 142 | 143 | _update_sheet_line(line) 144 | 145 | return lines 146 | 147 | 148 | def _tts_api_call(line: Dict, file_name: str): 149 | """ 150 | It call the TTS API with the parameters received in the line parameter 151 | 152 | Args: 153 | line: Dict object containing the fields to generate the tts audio file 154 | """ 155 | 156 | # Instantiates a client 157 | client = texttospeech.TextToSpeechClient() 158 | # Set the text input to be synthesized 159 | synthesis_input = texttospeech.SynthesisInput(ssml=line['text']) 160 | 161 | # Build the voice request, select the language code ("en-US") and the ssml 162 | # voice gender ("neutral") 163 | voice_id = line['voice_id'].split('##')[0] 164 | language_code = line['voice_id'][:5] 165 | voice = texttospeech.VoiceSelectionParams(language_code=language_code, name=voice_id) 166 | 167 | # Select the type of audio file you want returned 168 | audio_config = texttospeech.AudioConfig( 169 | audio_encoding=eval('texttospeech.AudioEncoding.' + line['audio_encoding']) 170 | ) 171 | 172 | # Perform the text-to-speech request on the text input with the selected 173 | # voice parameters and audio file type 174 | response = client.synthesize_speech( 175 | input=synthesis_input, voice=voice, audio_config=audio_config 176 | ) 177 | 178 | # The response's audio_content is binary. 179 | # _write_to_local_file(file_name, response) 180 | 181 | _write_to_gcs(line['gcs_bucket'], file_name, response) 182 | 183 | def _write_to_gcs(gcs_bucket: str, file_name: str, response): 184 | """ 185 | Writes the contents of response into file_name as binary in the gcs_bucket 186 | 187 | Args: 188 | gcs_bucket: name of the gcs bucket 189 | file_name: the name of the file to write 190 | response: the object containing the binary data to write 191 | """ 192 | 193 | storage_client = storage.Client() 194 | bucket = storage_client.bucket(gcs_bucket) 195 | blob = bucket.blob(file_name) 196 | 197 | # Mode can be specified as wb/rb for bytes mode. 198 | # See: https://docs.python.org/3/library/io.html 199 | with blob.open("wb") as f: 200 | f.write(response.audio_content) 201 | 202 | def _build_file_name(line: Dict) -> str: 203 | """ 204 | It builds the file name based on the configuration fields 205 | 206 | Args: 207 | line: Dict object containing the fields to generate the tts audio file 208 | 209 | Returns: 210 | A string with the name in low case 211 | """ 212 | 213 | name = ( 214 | f"{line['campaign']}" 215 | f"-{line['topic']}" 216 | f"-{line['voice_id']}" 217 | f".{line['audio_encoding']}" 218 | ) 219 | 220 | return name.lower() 221 | 222 | def _call_video_generation(line: Dict): 223 | """ 224 | It triggers the video generation call function passing all the info required 225 | 226 | Args: 227 | line: Dict object containing the fields to generate the video file 228 | """ 229 | _send_pub_sub(line, GENERATE_VIDEO_TOPIC) 230 | 231 | def _update_sheet_line(line: Dict): 232 | """ 233 | Updates the line in the Google Spreadsheet defined in the global variabl. 234 | 235 | Args: 236 | line: Dict containing all the relevant info. 237 | """ 238 | 239 | try: 240 | service = build('sheets', 'v4', credentials=None) 241 | index = line['index'] 242 | status_range = f"{CONFIG_SHEET_NAME}!{STATUS_COLUMN}{index}:{STATUS_COLUMN}{index}" 243 | tts_file_range = f"{CONFIG_SHEET_NAME}!{TTS_FILE_COLUMN}{index}:{TTS_FILE_COLUMN}{index}" 244 | last_update_range = f"{CONFIG_SHEET_NAME}!{LAST_UPDATE_COLUMN}{index}:{LAST_UPDATE_COLUMN}{index}" 245 | 246 | body1 = { 247 | 'values' : [[str(line['status'])],], 248 | 'majorDimension' : 'COLUMNS' 249 | } 250 | 251 | body2 = { 252 | 'values' : [[str(line['tts_file_url'])],], 253 | 'majorDimension' : 'COLUMNS' 254 | } 255 | 256 | now = datetime.now() 257 | body3 = { 258 | 'values' : [[str(now.strftime("%Y/%m/%d, %H:%M:%S"))],], 259 | 'majorDimension' : 'COLUMNS' 260 | } 261 | # Call the Sheets API 262 | 263 | sheet = service.spreadsheets() 264 | sheet.values().update(spreadsheetId=CONFIG_SPREADSHEET_ID, 265 | range=status_range, 266 | valueInputOption='RAW', 267 | body=body1).execute() 268 | 269 | sheet.values().update(spreadsheetId=CONFIG_SPREADSHEET_ID, 270 | range=tts_file_range, 271 | valueInputOption='RAW', 272 | body=body2).execute() 273 | 274 | sheet.values().update(spreadsheetId=CONFIG_SPREADSHEET_ID, 275 | range=last_update_range, 276 | valueInputOption='RAW', 277 | body=body3).execute() 278 | except HttpError as err: 279 | print(err) 280 | 281 | 282 | def _send_pub_sub(message: Dict, topic: str): 283 | """ 284 | It sends a message to the pubsub topic 285 | 286 | Args: 287 | message: Dict object containing the info to send to the topic 288 | topic: string containing the name of the topic 289 | """ 290 | publisher = pubsub_v1.PublisherClient() 291 | topic_path = publisher.topic_path(GCP_PROJECT, topic) 292 | msg_json = json.dumps(message) 293 | 294 | unused_msg_id = publisher.publish( 295 | topic_path, 296 | data=bytes(msg_json, 'utf-8'), 297 | ).result() 298 | 299 | if __name__ == '__main__': 300 | 301 | msg_data = {} 302 | msg_data = base64.b64encode(bytes(json.dumps(msg_data).encode('utf-8'))) 303 | print(msg_data) 304 | main( 305 | event={ 306 | 'data': msg_data, 307 | 'attributes': { 308 | } 309 | }, 310 | context=None) 311 | 312 | # [END main] -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | 2 | Apache License 3 | Version 2.0, January 2004 4 | http://www.apache.org/licenses/ 5 | 6 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 7 | 8 | 1. Definitions. 9 | 10 | "License" shall mean the terms and conditions for use, reproduction, 11 | and distribution as defined by Sections 1 through 9 of this document. 12 | 13 | "Licensor" shall mean the copyright owner or entity authorized by 14 | the copyright owner that is granting the License. 15 | 16 | "Legal Entity" shall mean the union of the acting entity and all 17 | other entities that control, are controlled by, or are under common 18 | control with that entity. For the purposes of this definition, 19 | "control" means (i) the power, direct or indirect, to cause the 20 | direction or management of such entity, whether by contract or 21 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 22 | outstanding shares, or (iii) beneficial ownership of such entity. 23 | 24 | "You" (or "Your") shall mean an individual or Legal Entity 25 | exercising permissions granted by this License. 26 | 27 | "Source" form shall mean the preferred form for making modifications, 28 | including but not limited to software source code, documentation 29 | source, and configuration files. 30 | 31 | "Object" form shall mean any form resulting from mechanical 32 | transformation or translation of a Source form, including but 33 | not limited to compiled object code, generated documentation, 34 | and conversions to other media types. 35 | 36 | "Work" shall mean the work of authorship, whether in Source or 37 | Object form, made available under the License, as indicated by a 38 | copyright notice that is included in or attached to the work 39 | (an example is provided in the Appendix below). 40 | 41 | "Derivative Works" shall mean any work, whether in Source or Object 42 | form, that is based on (or derived from) the Work and for which the 43 | editorial revisions, annotations, elaborations, or other modifications 44 | represent, as a whole, an original work of authorship. For the purposes 45 | of this License, Derivative Works shall not include works that remain 46 | separable from, or merely link (or bind by name) to the interfaces of, 47 | the Work and Derivative Works thereof. 48 | 49 | "Contribution" shall mean any work of authorship, including 50 | the original version of the Work and any modifications or additions 51 | to that Work or Derivative Works thereof, that is intentionally 52 | submitted to Licensor for inclusion in the Work by the copyright owner 53 | or by an individual or Legal Entity authorized to submit on behalf of 54 | the copyright owner. For the purposes of this definition, "submitted" 55 | means any form of electronic, verbal, or written communication sent 56 | to the Licensor or its representatives, including but not limited to 57 | communication on electronic mailing lists, source code control systems, 58 | and issue tracking systems that are managed by, or on behalf of, the 59 | Licensor for the purpose of discussing and improving the Work, but 60 | excluding communication that is conspicuously marked or otherwise 61 | designated in writing by the copyright owner as "Not a Contribution." 62 | 63 | "Contributor" shall mean Licensor and any individual or Legal Entity 64 | on behalf of whom a Contribution has been received by Licensor and 65 | subsequently incorporated within the Work. 66 | 67 | 2. Grant of Copyright License. Subject to the terms and conditions of 68 | this License, each Contributor hereby grants to You a perpetual, 69 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 70 | copyright license to reproduce, prepare Derivative Works of, 71 | publicly display, publicly perform, sublicense, and distribute the 72 | Work and such Derivative Works in Source or Object form. 73 | 74 | 3. Grant of Patent License. Subject to the terms and conditions of 75 | this License, each Contributor hereby grants to You a perpetual, 76 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 77 | (except as stated in this section) patent license to make, have made, 78 | use, offer to sell, sell, import, and otherwise transfer the Work, 79 | where such license applies only to those patent claims licensable 80 | by such Contributor that are necessarily infringed by their 81 | Contribution(s) alone or by combination of their Contribution(s) 82 | with the Work to which such Contribution(s) was submitted. If You 83 | institute patent litigation against any entity (including a 84 | cross-claim or counterclaim in a lawsuit) alleging that the Work 85 | or a Contribution incorporated within the Work constitutes direct 86 | or contributory patent infringement, then any patent licenses 87 | granted to You under this License for that Work shall terminate 88 | as of the date such litigation is filed. 89 | 90 | 4. Redistribution. You may reproduce and distribute copies of the 91 | Work or Derivative Works thereof in any medium, with or without 92 | modifications, and in Source or Object form, provided that You 93 | meet the following conditions: 94 | 95 | (a) You must give any other recipients of the Work or 96 | Derivative Works a copy of this License; and 97 | 98 | (b) You must cause any modified files to carry prominent notices 99 | stating that You changed the files; and 100 | 101 | (c) You must retain, in the Source form of any Derivative Works 102 | that You distribute, all copyright, patent, trademark, and 103 | attribution notices from the Source form of the Work, 104 | excluding those notices that do not pertain to any part of 105 | the Derivative Works; and 106 | 107 | (d) If the Work includes a "NOTICE" text file as part of its 108 | distribution, then any Derivative Works that You distribute must 109 | include a readable copy of the attribution notices contained 110 | within such NOTICE file, excluding those notices that do not 111 | pertain to any part of the Derivative Works, in at least one 112 | of the following places: within a NOTICE text file distributed 113 | as part of the Derivative Works; within the Source form or 114 | documentation, if provided along with the Derivative Works; or, 115 | within a display generated by the Derivative Works, if and 116 | wherever such third-party notices normally appear. The contents 117 | of the NOTICE file are for informational purposes only and 118 | do not modify the License. You may add Your own attribution 119 | notices within Derivative Works that You distribute, alongside 120 | or as an addendum to the NOTICE text from the Work, provided 121 | that such additional attribution notices cannot be construed 122 | as modifying the License. 123 | 124 | You may add Your own copyright statement to Your modifications and 125 | may provide additional or different license terms and conditions 126 | for use, reproduction, or distribution of Your modifications, or 127 | for any such Derivative Works as a whole, provided Your use, 128 | reproduction, and distribution of the Work otherwise complies with 129 | the conditions stated in this License. 130 | 131 | 5. Submission of Contributions. Unless You explicitly state otherwise, 132 | any Contribution intentionally submitted for inclusion in the Work 133 | by You to the Licensor shall be under the terms and conditions of 134 | this License, without any additional terms or conditions. 135 | Notwithstanding the above, nothing herein shall supersede or modify 136 | the terms of any separate license agreement you may have executed 137 | with Licensor regarding such Contributions. 138 | 139 | 6. Trademarks. This License does not grant permission to use the trade 140 | names, trademarks, service marks, or product names of the Licensor, 141 | except as required for reasonable and customary use in describing the 142 | origin of the Work and reproducing the content of the NOTICE file. 143 | 144 | 7. Disclaimer of Warranty. Unless required by applicable law or 145 | agreed to in writing, Licensor provides the Work (and each 146 | Contributor provides its Contributions) on an "AS IS" BASIS, 147 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 148 | implied, including, without limitation, any warranties or conditions 149 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 150 | PARTICULAR PURPOSE. You are solely responsible for determining the 151 | appropriateness of using or redistributing the Work and assume any 152 | risks associated with Your exercise of permissions under this License. 153 | 154 | 8. Limitation of Liability. In no event and under no legal theory, 155 | whether in tort (including negligence), contract, or otherwise, 156 | unless required by applicable law (such as deliberate and grossly 157 | negligent acts) or agreed to in writing, shall any Contributor be 158 | liable to You for damages, including any direct, indirect, special, 159 | incidental, or consequential damages of any character arising as a 160 | result of this License or out of the use or inability to use the 161 | Work (including but not limited to damages for loss of goodwill, 162 | work stoppage, computer failure or malfunction, or any and all 163 | other commercial damages or losses), even if such Contributor 164 | has been advised of the possibility of such damages. 165 | 166 | 9. Accepting Warranty or Additional Liability. While redistributing 167 | the Work or Derivative Works thereof, You may choose to offer, 168 | and charge a fee for, acceptance of support, warranty, indemnity, 169 | or other liability obligations and/or rights consistent with this 170 | License. However, in accepting such obligations, You may act only 171 | on Your own behalf and on Your sole responsibility, not on behalf 172 | of any other Contributor, and only if You agree to indemnify, 173 | defend, and hold each Contributor harmless for any liability 174 | incurred by, or claims asserted against, such Contributor by reason 175 | of your accepting any such warranty or additional liability. 176 | 177 | END OF TERMS AND CONDITIONS 178 | 179 | APPENDIX: How to apply the Apache License to your work. 180 | 181 | To apply the Apache License to your work, attach the following 182 | boilerplate notice, with the fields enclosed by brackets "[]" 183 | replaced with your own identifying information. (Don't include 184 | the brackets!) The text should be enclosed in the appropriate 185 | comment syntax for the file format. We also recommend that a 186 | file or class name and description of purpose be included on the 187 | same "printed page" as the copyright notice for easier 188 | identification within third-party archives. 189 | 190 | Copyright [yyyy] [name of copyright owner] 191 | 192 | Licensed under the Apache License, Version 2.0 (the "License"); 193 | you may not use this file except in compliance with the License. 194 | You may obtain a copy of the License at 195 | 196 | http://www.apache.org/licenses/LICENSE-2.0 197 | 198 | Unless required by applicable law or agreed to in writing, software 199 | distributed under the License is distributed on an "AS IS" BASIS, 200 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 201 | See the License for the specific language governing permissions and 202 | limitations under the License. -------------------------------------------------------------------------------- /src/cfs/generate_video_file/main.py: -------------------------------------------------------------------------------- 1 | # Copyright 2023 Google LLC 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | # [START main] 16 | 17 | from google.cloud import storage 18 | from mutagen.mp3 import MP3 19 | from typing import Any, Dict, Optional 20 | from google.cloud.functions_v1.context import Context 21 | from google.cloud import pubsub_v1 22 | from googleapiclient.discovery import build 23 | from googleapiclient.errors import HttpError 24 | from datetime import datetime 25 | 26 | import os 27 | import string 28 | import random 29 | import json 30 | import base64 31 | 32 | GCP_PROJECT = os.getenv('GCP_PROJECT', '') 33 | CONFIG_SPREADSHEET_ID = os.getenv('CONFIG_SPREADSHEET_ID', '') 34 | CONFIG_SHEET_NAME = os.getenv('CONFIG_SHEET_NAME', 'config') 35 | CONFIG_RANGE_NAME = os.getenv('CONFIG_RANGE_NAME', 'config!A1:N') 36 | FINAL_VIDEO_FILE_COLUMN = os.getenv('FINAL_VIDEO_FILE_COLUMN', 'L') 37 | STATUS_COLUMN = os.getenv('STATUS_COLUMN', 'M') 38 | LAST_UPDATE_COLUMN = os.getenv('LAST_UPDATE_COLUMN', 'N') 39 | 40 | def _get_mp3_length(path: str): 41 | """Returns the length of a MP3 file in seconds. 42 | 43 | Args: 44 | path: Path to the MP3 file. 45 | """ 46 | try: 47 | audio = MP3(path) 48 | length = audio.info.length 49 | return length 50 | except: 51 | return None 52 | 53 | 54 | def _copy_file_from_gcs(gcs_bucket: str, source_blob_name: str, destination_local_filename: str): 55 | """Copies a file from Google Cloud Storage to a temporary local filename. 56 | 57 | Args: 58 | gcs_bucket: string containing the bucket name. 59 | source_blob_name: Name of the source blob containing the file. 60 | destination_local_filename: Name of the local file that will be written. 61 | """ 62 | storage_client = storage.Client() 63 | bucket = storage_client.bucket(gcs_bucket) 64 | blob = bucket.blob(source_blob_name) 65 | blob.download_to_filename(destination_local_filename) 66 | 67 | 68 | def _copy_file_to_gcs(gcs_bucket: str, source_local_filename: str, destination_blob_name: str): 69 | """Copies a file to Google Cloud Storage from a temporary local filename. 70 | 71 | Args: 72 | gcs_bucket: string containing the bucket name. 73 | source_local_filename: Name of the local file that will be copied. 74 | destination_blob_name: Name of the blob to be created in GCS. 75 | """ 76 | storage_client = storage.Client() 77 | bucket = storage_client.bucket(gcs_bucket) 78 | blob = bucket.blob(destination_blob_name) 79 | print(gcs_bucket) 80 | print(source_local_filename) 81 | print(destination_blob_name) 82 | print('Checking if blob exists') 83 | if blob.exists(): 84 | print('Deleting existing target file') 85 | blob.delete() 86 | blob.upload_from_filename(source_local_filename) 87 | 88 | 89 | def _mix_video_and_speech(config: Dict): #, video_file, speech_file,destination_video_name, voice_delay): 90 | """Mixes a generated speech file with the audio of the specified video. 91 | 92 | Args: 93 | config: Dictionary containing the configuration information. 94 | """ 95 | # Generate a 10 characters long random string (rnd) 96 | random_string = ''.join(random.choices( 97 | string.ascii_lowercase + string.digits, k=12)) 98 | 99 | # Copy video_file from blob to /tmp/video[rnd].mp4 100 | source_video_input = '/tmp/video_{random_string}.mp4'.format( 101 | random_string=random_string) 102 | input_video_file = f"{config['video_file']}" 103 | print(input_video_file) 104 | _copy_file_from_gcs(config['gcs_bucket'], input_video_file, source_video_input) 105 | 106 | # Copy speech_file from blob to /tmp/speech[rnd].mp3 107 | source_speech_input = '/tmp/speech_{random_string}.mp4'.format( 108 | random_string=random_string) 109 | _copy_file_from_gcs(config['gcs_bucket'], config['tts_file_url'], source_speech_input) 110 | 111 | # Generate and run mix command to generate the output video file 112 | generated_video_file = '/tmp/output_{random_string}.mp4'.format( 113 | random_string=random_string) 114 | 115 | audio_reduction = 0.9 116 | if config['base_audio_vol_percent']: 117 | audio_reduction = config['base_audio_vol_percent'] 118 | 119 | # Calculate when the original audio should be adjusted during and 120 | # after the voice dub section 121 | audio_down_start = int(config['millisecond_start_audio'])/1000 122 | voice_dub_length = _get_mp3_length(source_speech_input) 123 | print('Voice Dub length is {voice_dub_length}'.format( 124 | voice_dub_length=voice_dub_length)) 125 | audio_down_end = audio_down_start + voice_dub_length 126 | if config['base_audio_file']: 127 | # Copy base_audio_file from blob to /tmp/base_audio_[rnd].{sound_extension} 128 | sound_extension = config['base_audio_file'].split(".")[1] 129 | source_audio_input = f"/tmp/base_audio_{random_string}.{sound_extension}" 130 | _copy_file_from_gcs(config['gcs_bucket'], config['base_audio_file'], source_audio_input) 131 | ffmpeg_mix_command = ( 132 | "ffmpeg " 133 | "-loglevel error " 134 | f"-i {source_video_input} -i {source_audio_input} -i {source_speech_input} " 135 | "-filter_complex " 136 | f"\"[2:a] adelay={config['millisecond_start_audio']}|{config['millisecond_start_audio']} [voice_dub];" 137 | f"[1:a] volume={audio_reduction}:enable='between(t,{audio_down_start},{audio_down_end})' [original_audio];" 138 | f"[original_audio] volume=0.9:enable='gt(t,{audio_down_end})' [original_audio];" 139 | f"[voice_dub][original_audio] amix=duration=longest [audio_out]" 140 | "\" " 141 | f"-map 0:v -map \"[audio_out]\" -y {generated_video_file}" 142 | ) 143 | else: 144 | ffmpeg_mix_command = ( 145 | "ffmpeg " 146 | "-loglevel error " 147 | f"-i {source_video_input} -i {source_speech_input} " 148 | "-filter_complex " 149 | f"\"[1:a] adelay={0}|{0} [voice_dub];" 150 | f"[0:a] volume={audio_reduction}:enable='between(t,{audio_down_start},{audio_down_end})' [original_audio];" 151 | f"[original_audio] volume=0.9:enable='gt(t,{audio_down_end})' [original_audio];" 152 | f"[voice_dub][original_audio] amix=duration=longest [audio_out]" 153 | "\" " 154 | f"-map 0:v -map \"[audio_out]\" -y {generated_video_file}" 155 | ) 156 | print(f'Running command: {ffmpeg_mix_command}') 157 | try: 158 | os.system(ffmpeg_mix_command) 159 | print('Copying output file to GCS') 160 | today = datetime.today().strftime('%Y%m%d') 161 | target_video_file_name = f"output/{today}/{_build_file_name(config)}" 162 | print(target_video_file_name) 163 | # Copy the generated video file to the target GCS bucket 164 | _copy_file_to_gcs(config['gcs_bucket'], generated_video_file, target_video_file_name) 165 | config['status'] = 'Video OK' 166 | config['final_video_file_url'] = f"gs://{config['gcs_bucket']}/{target_video_file_name}" 167 | 168 | except Exception as e: 169 | config['status'] = e 170 | config['final_video_file_url'] = "N/A" 171 | 172 | _update_sheet_line(config) 173 | 174 | # Cleanup temp files 175 | print('Cleaning up') 176 | os.remove(source_video_input) 177 | os.remove(source_speech_input) 178 | os.remove(generated_video_file) 179 | 180 | def main(event: Dict[str, Any], context=Optional[Context]): 181 | """Mixes a generated speech audio file into an input video. 182 | Args: 183 | event (dict): The dictionary with data specific to this type of event. The 184 | `data` field contains the PubsubMessage message. The `attributes` field 185 | will contain custom attributes if there are any. 186 | context (google.cloud.functions.Context): The Cloud Functions event 187 | metadata. The `event_id` field contains the Pub/Sub message ID. The 188 | `timestamp` field contains the publish time. 189 | """ 190 | 191 | del context # Unused 192 | data = base64.b64decode(event['data']).decode('utf-8') 193 | config = json.loads(data) 194 | _mix_video_and_speech(config) 195 | _update_sheet_line(config) 196 | print('Process completed') 197 | return 'done' 198 | 199 | 200 | def _build_file_name(config: Dict) -> str: 201 | """ 202 | It builds the file name based on the configuration fields 203 | 204 | Args: 205 | config: Dict object containing the fields to generate the video file 206 | 207 | Returns: 208 | A string with the name in low case 209 | """ 210 | 211 | name = ( 212 | f"{config['campaign']}" 213 | f"-{config['topic']}" 214 | f"-{config['voice_id']}.mp4" 215 | ) 216 | 217 | return name.lower() 218 | 219 | def _update_sheet_line(line: Dict): 220 | """ 221 | Updates the line in the Google Spreadsheet defined in the global variabl. 222 | 223 | Args: 224 | line: Dict containing all the relevant info. 225 | """ 226 | 227 | try: 228 | service = build('sheets', 'v4', credentials=None) 229 | index = line['index'] 230 | status_range = f'{CONFIG_SHEET_NAME}!{STATUS_COLUMN}{index}:{STATUS_COLUMN}{index}' 231 | final_video_file_range = f'{CONFIG_SHEET_NAME}!{FINAL_VIDEO_FILE_COLUMN}{index}:{FINAL_VIDEO_FILE_COLUMN}{index}' 232 | last_update_range = f"{CONFIG_SHEET_NAME}!{LAST_UPDATE_COLUMN}{index}:{LAST_UPDATE_COLUMN}{index}" 233 | 234 | body1 = { 235 | 'values' : [[str(line['status'])],], 236 | 'majorDimension' : 'COLUMNS' 237 | } 238 | 239 | body2 = { 240 | 'values' : [[str(line['final_video_file_url'])],], 241 | 'majorDimension' : 'COLUMNS' 242 | } 243 | 244 | now = datetime.now() 245 | body3 = { 246 | 'values' : [[now.strftime("%Y/%m/%d, %H:%M:%S")],], 247 | 'majorDimension' : 'COLUMNS' 248 | } 249 | # Call the Sheets API 250 | 251 | sheet = service.spreadsheets() 252 | sheet.values().update(spreadsheetId=CONFIG_SPREADSHEET_ID, 253 | range=status_range, 254 | valueInputOption='RAW', 255 | body=body1).execute() 256 | 257 | sheet.values().update(spreadsheetId=CONFIG_SPREADSHEET_ID, 258 | range=final_video_file_range, 259 | valueInputOption='RAW', 260 | body=body2).execute() 261 | 262 | sheet.values().update(spreadsheetId=CONFIG_SPREADSHEET_ID, 263 | range=last_update_range, 264 | valueInputOption='RAW', 265 | body=body3).execute() 266 | 267 | except HttpError as err: 268 | print(err) 269 | 270 | if __name__ == '__main__': 271 | config = { 272 | 'source_bucket_name': 'videodub_test_input', 273 | 'target_bucket': 'videodub_test_output' 274 | } 275 | 276 | msg_data = {'campaign': 'summer', 277 | 'topic': 'outdoor', 278 | 'video_file': 'input/bumper_master.mp4', 279 | 'base_audio_file': 'input/soundtrack_bumper.wav', 280 | 'text': '\n Here are SSML', 281 | 'voice_id': 'en-US-Standard-I', 282 | 'millisecond_start_audio': '0', 283 | 'audio_encoding': 'MP3', 284 | 'gcs_bucket': 'videodub_test_input', 285 | 'index': 1, 286 | 'status': 'OK', 287 | 'tts_file_url': 'output/20230419/summer-outdoor-en-us-standard-i##male.mp3'} 288 | 289 | msg_data = base64.b64encode(bytes(json.dumps(msg_data).encode('utf-8'))) 290 | 291 | main( 292 | event={ 293 | 'data': msg_data, 294 | 'attributes': { 295 | 'forwarded': 'true' 296 | } 297 | }, 298 | context=None) 299 | 300 | # [END main] -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## README 2 | 3 | # Context 4 | 5 | AI Dubbing allows you to create localized videos using the same video base and adding translations using Google AI Powered TextToSpeech API 6 | 7 | # Pre-requisites: 8 | 9 | * Google Cloud 10 | * Google Workspace (Google Spreadsheets) 11 | * Google Cloud user with privileges over all the APIs listed in the config (ideally Owner role), so it’s possible to grant some privileges to the Service Account automatically. \ 12 | 13 | * Latest version of Terraform installed \ 14 | 15 | * Python version >= 3.8.1 installed 16 | * Python Virtualenv installed \ 17 | 18 | 19 | Roles that will be automatically granted to the service account during the installation process: 20 | 21 | "roles/iam.serviceAccountShortTermTokenMinter" 22 | 23 | "roles/storage.objectAdmin" 24 | 25 | "roles/pubsub.publisher" 26 | 27 | # Installation Steps: 28 | 29 | 30 | 31 | 1. Open a shell \ 32 | 33 | 2. Clone the [git repository](https://github.com/google/ai_video_dubbing) 34 | 35 | 3. Open a text editor and configure the following installation variables in the file _“variables.tf”_ 36 | 37 | variable "gcp\_project" { 38 | type = string 39 | description = "Google Cloud Project ID where the artifacts will be deployed" 40 | default = "my-project" 41 | } 42 | 43 | variable "gcp\_region" { 44 | type = string 45 | description = "Google Cloud Region" 46 | default = "my-gcp-region" 47 | } 48 | 49 | variable "ai\_dubbing\_sa" { 50 | type = string 51 | description = "Service Account for the deployment" 52 | default = "ai-dubbing" 53 | } 54 | 55 | variable "ai\_dubbing\_bucket\_name" { 56 | type = string 57 | description = "GCS bucket used for deployment tasks" 58 | default = "my-bucket" 59 | } 60 | 61 | variable "config\_spreadsheet\_id" { 62 | type = string 63 | description = "The ID of the config spreadhseet" 64 | default = "my-google-sheets-sheet-id" 65 | } 66 | 67 | #Do not set this value to a high frequency, since executions might overlap 68 | 69 | variable "execution\_schedule" { 70 | type = string 71 | description = "The schedule to execute the process (every 30 min default) " 72 | default = "\*/30 \* \* \* \*" 73 | } 74 | 75 | 4. _Now execute ”terraform apply” \ 76 | _ 77 | 5. Type “yes” and hit return when the system asks for confirmation_ \ 78 | _ 79 | 80 | # Generated Cloud Artifacts 81 | 82 | * Service Account: if it not exists, it will be created as per the values in the configuration 83 | * Cloud Scheduler: ai-dubbing-trigger 84 | * Cloud Functions: generate\_tts\_file, generate\_video\_file. Both triggered by pub/sub 85 | * Cloud Pub/Sub topics: generate\_tts\_files\_trigger, generate\_video\_file\_trigger 86 | 87 | # Output 88 | 89 | Every generated artifact will be stored in the supplied GCS bucket under the output/YYYYMMDD folder, where YYYY represents the year, MM the month and DD the day of the processing date 90 | 91 | ## Audio Files 92 | 93 | The generated TTS audio files will be stored as mp3 94 | 95 | {campaign}-{topic}-{voice\_id}.mp3 96 | 97 | Audio url: gs://{gcs\_bucket}/output/{YYYYMMDD}/{campaign}-{topic}-{voice\_id}.mp3 98 | 99 | ## Video Files 100 | 101 | The generated video field will be stored as mp4 102 | 103 | {campaign}-{topic}-{voice\_id}.mp4 104 | 105 | Video url: gs://{gcs\_bucket}/output/{YYYYMMDD}/{campaign}-{topic}-{voice\_id}.mp4 106 | 107 | # 108 | 109 | # How to activate 110 | 111 | Most of the effort will be on building the first SSML text and adapting the timings to the video. Once that task is mastered, video creation will be done in a breeze! 112 | 113 | You can use the [web-based SSML-Editor](https://actions-on-google-labs.github.io/nightingale-ssml-editor/?%3Cspeak%3E%3Cseq%3E%0A%09%3Cmedia%20xml%3Aid%3D%22track-0%22%20begin%3D%220s%22%20soundLevel%3D%22%2B0dB%22%3E%0A%09%09%3Cspeak%3E%3Cp%3EI%20will%20read%20this%20text%20for%20you%3C%2Fp%3E%3C%2Fspeak%3E%0A%09%3C%2Fmedia%3E%0A%3C%2Fseq%3E%3C%2Fspeak%3E) for this purpose, and then export each SSML file. 114 | 115 | ## What’s required for video generation 116 | 117 | * A file containing a base video without music 118 | * A file containing the music for the video 119 | 120 | ## Configure the input 121 | 122 | 1. Create a [copy of the configuration spreadsheet](https://docs.google.com/spreadsheets/d/1lGZ_uwdqXLPUlgYtZaY-orFkw5YJ4SgsGrHDzGEVHzg/copy) 123 | 124 | 2. Configure the fields in the sheet “config” following the instructions 125 | 126 | 127 | 128 | 131 | 133 | 135 | 137 | 139 | 141 | 142 | 143 | 145 | 147 | 149 | 151 | 153 | 155 | 156 | 157 | 159 | 161 | 163 | 165 | 167 | 169 | 170 | 171 | 173 | 175 | 177 | 179 | 181 | 183 | 184 | 185 | 187 | 189 | 191 | 193 | 195 | 197 | 198 | 199 | 201 | 203 | 205 | 207 | 209 | 211 | 212 | 213 | 215 | 217 | 219 | 221 | 241 | 243 | 244 | 245 | 247 | 249 | 251 | 253 | 255 | 257 | 258 | 259 | 261 | 263 | 265 | 267 | 271 | 273 | 274 | 275 | 277 | 279 | 281 | 283 | 285 | 287 | 288 | 289 | 291 | 293 | 295 | 297 | 299 | 300 | 301 | 302 | 304 | 306 | 308 | 310 | 312 | 314 | 315 | 316 | 318 | 320 | 322 | 324 | 326 | 328 | 329 | 330 | 332 | 334 | 336 | 338 | 340 | 342 | 343 |
129 | Field Name 130 | Type 132 | Mandatory 134 | Description 136 | Sample Value 138 | Notes 140 |
campaign 144 | Input 146 | Yes 148 | A string to generate the name of the video 150 | summer 152 | 154 |
topic 158 | Input 160 | Yes 162 | A string to generate the name of the video 164 | outdoor 166 | 168 |
gcs_bucket 172 | Input 174 | Yes 176 | The bucket where video_file and base_audio_file could be located (the service account must be granted access). We recommend to use the same gcs_bucket as for the output 178 | videodub_test_input 180 | 182 |
video_file 186 | Input 188 | Yes 190 | The location of the master video file within the gcs_bucket 192 | input/videos/bumper_base_video.mp4 194 | 196 |
base_audio_file 200 | Input 202 | No 204 | The location of the base audio file within the gcs_bucket 206 | input/audios/bumper_base_audio.mp3 208 | 210 |
text 214 | Input 216 | Yes 218 | The SSML text to convert to speech 220 | 222 |

223 | 224 |

225 | 226 |

227 | 228 |

229 | Find your own style in the constantly renewed catalog of the <emphasis level="strong">somewhere.com online shop</emphasis></prosody> 230 |

231 | 232 |

233 | Design what you love</prosody> 234 |

235 | 236 |

237 | 238 |

239 | 240 |

Check SSML supported syntax 242 |
voice_id 246 | Input 248 | Yes 250 | The id of the voice to use 252 | en-GB-Wavenet-C##FEMALE 254 | Check voices here 256 |
millisecond_start_audio 260 | Input 262 | No 264 | Millisecond of the video when the audio must start. This could be also accomplished using TTS 266 |

268 | 0

269 | 270 | 		272 |
audio_encoding 276 | Input 278 | Yes 280 | The audio encoding available 282 | MP3 284 | At the moment only MP3 is supported 286 |
base_audio_vol_percent 290 | Input 292 | Yes 294 | Modifies the volume of the base audio (whether in the base video or in the base audio file) 296 | 0.6 298 |
final_video_file_url 303 | Output 305 | N/A 307 | The location of the generated video file with the base audio and speech 309 | gs://videodub_tester/output/20230420/summer-outdoor-en-gb-wavenet-c##female.mp4 311 | 313 |
status 317 | Output 319 | N/A 321 | The status of the process 323 | Video OK 325 | 327 |
last_update 331 | Output 333 | N/A 335 | The last time the row was modified by the automatic process 337 | 2023/04/20, 12:25:16 339 | 341 |
344 | 345 | ## Trigger the generation process 346 | 347 | Once all the configuration is set in the spreadsheet, the process will run every X minutes, as defined by the execution\_schedule. 348 | 349 | The “Status” column will change its contents, the possible values are: 350 | 351 | * “TTS OK”: audio file generated correctly 352 | * “Video OK”: video file generated correctly 353 | * Other value: an error occurred 354 | 355 | When all the cells in the status column would display “Video OK”, the process will be completed 356 | 357 | When all the cells display “Video OK” or different from “TTS OK”, the process will be completed but it might have errors \ 358 | 359 | Just download the videos from gs://{gcs\_bucket}/output/{YYYYMMDD} and make the best use of them. 360 | 361 | Note: 362 | 363 | For the initial tests, the scheduled execution period might be too long. The recommendation in these kinds of situations is just to disable the schedule and run it on demand. To do that: 364 | 1. Go to the Cloud Scheduler tab in your Google Cloud project and 365 | 2. Check the box next to “ai-dubbing-trigger” 366 | 3. Click on “Force Run” 367 | --------------------------------------------------------------------------------