Additional features

- Added functionality to regenerate Demand Gen text assets using a desired target language
- Added the total duration of the variant beneath the segments list, so that users can preview the total duration when manually selecting/deselecting segments

Change-Id: I089c52305a8647ac2f7fb7e46dc062d2442ff303

README.md

@@ -33,8 +33,10 @@ limitations under the License.
 Update to the latest version by running `npm run update-app` after pulling the latest changes from the repository via `git pull --rebase --autostash`; you would need to redploy the *UI* for features marked as `frontend`, and *GCP components* for features marked as `backend`.
 
 * [December 2024]
-  * `frontend`: Added possibility to load previously rendered videos via a new dropdown. You can now also specify a name for each render job which will be displayed alongside the render timestamp.
+  * `frontend`: Added functionality to generate Demand Gen text assets in a desired target language. Read more [here](#6-output-videos).
+  * `frontend`: Added possibility to load previously rendered videos via a new dropdown. You can now also specify a name for each render job which will be displayed alongside the render timestamp. Read more [here](#43-loading-previously-rendered-videos).
   * `frontend`: Added checkbox to select/deselect all segments during variants preview.
+  * `frontend`: During variants generation, users can now preview the total duration of their variant directly as they are selecting/deselecting segments.
   * `frontend`: The ABCDs evaluation section per variant may now additionally include recommendations on how to improve the video's content to make it more engaging.
   * `frontend`: Improved user instruction following for the variants generation prompt and simplified the input process; you no longer need a separate checkbox to include or exclude elements - just specify your requirements directly in the prompt.
   * `frontend` + `backend`: Added support for Gemini 2.0.
@@ -84,7 +86,7 @@ Please make sure you have fulfilled all prerequisites mentioned under [Requireme
     * First, enter your GCP Project ID.
     * Then select whether you would like to deploy GCP components (defaults to `Yes`) and the UI (also defaults to `Yes`).
         * When deploying GCP components, you will be prompted to enter an optional [Cloud Function region](https://cloud.google.com/functions/docs/locations) (defaults to `us-central1`) and an optional [GCS location](https://cloud.google.com/storage/docs/locations) (defaults to `us`).
-        * When deploying the UI, you will be asked if you are a Google Workspace user and if you want others in your Workspace domin to access your deployed web app (defaults to `No`). By default, the web app is only accessible by you, and that is controlled by the [web app access settings](https://developers.google.com/apps-script/manifest/web-app-api-executable#webapp) in the project's [manifest file](./ui/appsscript.json), which defaults to `MYSELF`. If you answer `Yes` here, this value will be changed to `DOMAIN` to allow other individuals within your organisation to access the web app without having to deploy it themselves.
+        * When deploying the UI, you will be asked if you are a Google Workspace user and if you want others in your Workspace domain to access your deployed web app (defaults to `No`). By default, the web app is only accessible by you, and that is controlled by the [web app access settings](https://developers.google.com/apps-script/manifest/web-app-api-executable#webapp) in the project's [manifest file](./ui/appsscript.json), which defaults to `MYSELF`. If you answer `Yes` here, this value will be changed to `DOMAIN` to allow other individuals within your organisation to access the web app without having to deploy it themselves.
 
     > Note: If you have already run the script and you notice that there was a typo in one of the inputs, or if you would like to change the Cloud region, GCS location, or the UI access settings for Google Workspace users, you **must** first run `git reset --hard` before rerunning `npm start`.
 
@@ -234,12 +236,13 @@ Users are now ready for combination. They can view the A/V segments and generate
   * Users can then click `Generate` to generate variants accordingly, which will query language models on Vertex AI with a detailed script of the video to generate potential variants that fulfill the optional user-provided prompt and target duration.
 * Generated variants are displayed in tabs - one per tab - and both the *video preview* and *segments list* views are updated to preselect the A/V segments of the variant currently being viewed. Clicking on the video's play button in the *video preview* mode will preview only those preselected segments.
 
-  <center><img src='./img/variants.png' width="600px" alt="Vigenair UI: Variants preview" /></center>
+  <center><img src='./img/variants.png' width="800px" alt="Vigenair UI: Variants preview" /></center>
   <br />
 
   Each variant has the following information:
   * A title which is displayed in the variant's tab.
   * A duration, which is also displayed in the variant's tab.
+  * The total duration is also displayed below the segments list, so that users can preview the variant's duration as they select/deselect segments.
   * The list of A/V segments that make up the variant.
   * A description of the variant and what is happening in it.
   * An LLM-generated Score, from 1-5, representing how well the variant adheres to the input rules and guidelines, which default to a subset of [YouTubes ABCDs](https://www.youtube.com/ads/abcds-of-effective-video-ads/). Users are strongly encouraged to update this section of the generation prompt in [config.ts](ui/src/config.ts) to refer to their own brand voice and creative guidelines.
@@ -270,7 +273,15 @@ Users are now ready for combination. They can view the A/V segments and generate
 
   <center><img src='./img/reorder-segments.gif' alt="Vigenair's segment reordering feature" /></center>
 
-#### 4.3. Render Queue
+#### 4.3. Loading Previously Rendered Videos
+
+Users may also choose to skip the variants generation and rendering process and directly display previously rendered videos using the "Load rendered videos" dropdown.
+
+<center><img src='./img/load-rendered.png' width="600px" alt="Vigenair UI: Load previously rendered videos" /></center>
+
+The values displayed in the dropdown represent the optional custom name that the user may have provided when building their render queue (see [Rendering](#5-rendering)), along with the date and time of rendering (which is added automatically, so users don't need to manually input this information).
+
+#### 4.4. Render Queue
 
 Desired variants can be added to the render queue along with the their associated render settings:
 
@@ -284,13 +295,13 @@ Desired variants can be added to the render queue along with the their associate
 
 #### 5. Rendering
 
-Clicking on the `Render` button inside the render queue will render the variants in their desired formats and settings via the Combiner service Cloud Function (writing `render.json` to GCS, which serves as the input to the service, and the output is a `combos.json` file. Both files, along with the *rendered* variants, are stored in a `<timestamp>-combos` subfolder below the root video folder).
+Clicking on the `Render` button inside the render queue will render the variants in their desired formats and settings via the Combiner service Cloud Function (writing `render.json` to GCS, which serves as the input to the service, and the output is a `combos.json` file. Both files, along with the *rendered* variants, are stored in a `<timestamp>-combos` subfolder below the root video folder). Users may also optionally specify a name for the render queue, which will be displayed in the "Load rendered videos" dropdown (see [Loading Previosuly Rendered Videos](#43-loading-previously-rendered-videos)).
 
 <center><img src='./img/rendering.png' width="600px" alt="Vigenair UI: Rendering videos" /></center>
 
 #### 6. Output Videos
 
-The UI continuously queries GCS for updates. Once a `combos.json` is available, the final videos - in their different formats and along with all associated assets - will be displayed. Users can preview the final videos and select the ones they would like to upload into Google Ads / YouTube. Users may also share a page of the Web App containing the rendered videos and associated image & text assets via the dedicated "share" icon in the top-right corner of the "Rendered videos" panel.
+The UI continuously queries GCS for updates. Once a `combos.json` is available, the final videos - in their different formats and along with all associated assets - will be displayed. Users can preview the final videos and select the ones they would like to upload into Google Ads / YouTube. Users may also share a page of the Web App containing the rendered videos and associated image & text assets via the dedicated "share" icon in the top-right corner of the "Rendered videos" panel. Finally, users may also regenerate Demand Gen text assets, either in bulk or individually, using the auto-detected language of the video or by specifying a desired target language.
 
 <center><img src='./img/rendered.png' width="600px" alt="Vigenair UI: Rendered videos display with 'share' icon" /></center>
 <center><img src='./img/rendered-assets.png' width="600px" alt="Vigenair UI: Rendered image and text assets" /></center>

ui/src/config.ts

@@ -108,34 +108,37 @@ export const CONFIG = {
 
 
     `,
-    textAssetGenerationPrompt: `You are a leading digital marketer and an expert at crafting high-performing search ad headlines and descriptions that captivate users and drive conversions.
+    textAssetsGenerationPrompt: `You are a leading digital marketer and an expert at crafting high-performing search ad headlines and descriptions that captivate users and drive conversions.
     Follow these instructions in order:
 
     1. **Analyze the Video**: Carefully analyze the video ad to identify the brand, key products or services, unique selling points, and the core message conveyed.
-    2. **Target Audience**: Consider the target audience of the video ad. What are their interests, needs, and pain points? How can the search ad resonate with them?
-    3. **Unwanted Example**: Here's an example of a Headline and Description that I DO NOT want you to generate: Headline: {{headline}} Description: {{description}}
-    4. **Craft a Headline and a Description**: Generate a compelling search ad headline and description based on your analysis. Adhere to these guidelines:
-        - **Headline (Max 40 Characters)**:
+    2. **Target Audience**: Consider the target audience of the video ad. What are their interests, needs, and pain points? How can the search ads resonate with them?
+    {{badExamplePromptPart}}
+    3. **Craft Headlines and a Descriptions**: Generate {{desiredCount}} compelling search ad headlines and descriptions based on your analysis. Adhere to these guidelines:
+        - **Headlines (Max 40 Characters)**:
             - Include the brand name or a relevant keyword.
             - Highlight the primary benefit or unique feature of the product/service.
             - Create a sense of urgency or exclusivity.
             - Use action words and power words to grab attention.
             - Avoid overselling and nebulous claims.
             - Do not output any question marks or exclamation marks.
-        - **Description (Max 90 Characters)**:
+        - **Descriptions (Max 90 Characters)**:
             - Expand on the headline, providing additional details or benefits.
             - Include a strong call to action (e.g. "Shop now", "Learn more", "Sign up").
             - Use keywords strategically for better targeting.
             - Maintain a clear and concise message.
             - Avoid overselling and nebulous claims.
             - Do not output more than one question mark or exclamation mark.
-    4. **Output Format**: Output the following components in this exact format:
+    4. **Output Format**: For each generated search ad, output the following components in this exact format:
     Headline: The generated headline.
     Description: The accompanying description.
 
+    Separate each search ad you output by the value: "## Ad".
     Output in {{videoLanguage}}.
     `,
   },
+  textAssetsBadExamplePromptPart:
+    "3. **Unwanted Example**: Here's an example of a Headline and Description that I DO NOT want you to generate: Headline: {{headline}} Description: {{description}}",
   defaultVideoLanguage: 'English',
   defaultVideoWidth: 1280,
   defaultVideoHeight: 720,

ui/src/generation.ts

@@ -24,6 +24,9 @@ import {
 } from './ui/src/app/api-calls/api-calls.service.interface';
 import { VertexHelper } from './vertex';
 
+const GENERATE_TEXT_ASSETS_REGEX =
+  /.*Headline\s?:\**(?<headline>.*)\n+\**Description\s?:\**(?<description>.*)/ims;
+
 export interface AvSegment {
   av_segment_id: number;
   description: string;
@@ -57,9 +60,7 @@ export class GenerationHelper {
     gcsFolder: string,
     settings: GenerationSettings
   ): string {
-    const videoLanguage =
-      (StorageManager.loadFile(`${gcsFolder}/language.txt`, true) as string) ||
-      CONFIG.defaultVideoLanguage;
+    const videoLanguage = GenerationHelper.getVideoLanguage(gcsFolder);
     const duration = settings.duration;
     const expectedDurationRange =
       GenerationHelper.calculateExpectedDurationRange(duration);
@@ -77,6 +78,13 @@ export class GenerationHelper {
     return generationPrompt;
   }
 
+  static getVideoLanguage(gcsFolder: string): string {
+    return (
+      (StorageManager.loadFile(`${gcsFolder}/language.txt`, true) as string) ||
+      CONFIG.defaultVideoLanguage
+    );
+  }
+
   static calculateExpectedDurationRange(duration: number): string {
     const durationFraction = 20 / 100;
     const expectedDurationRange = `${duration - duration * durationFraction}-${duration + duration * durationFraction}`;
@@ -251,15 +259,18 @@ export class GenerationHelper {
   }
 
   static generateTextAsset(
-    gcsFolder: string,
     variantVideoPath: string,
-    textAsset: VariantTextAsset
+    textAsset: VariantTextAsset,
+    textAssetLanguage: string
   ): VariantTextAsset {
-    const videoLanguage =
-      (StorageManager.loadFile(`${gcsFolder}/language.txt`, true) as string) ||
-      CONFIG.defaultVideoLanguage;
-    const generationPrompt = CONFIG.vertexAi.textAssetGenerationPrompt
-      .replace('{{videoLanguage}}', videoLanguage)
+    const generationPrompt = CONFIG.vertexAi.textAssetsGenerationPrompt
+      .replace('{{videoLanguage}}', textAssetLanguage)
+      .replace('{{desiredCount}}', '1')
+      .replace('3. ', '4. ')
+      .replace(
+        '{{badExamplePromptPart}}',
+        CONFIG.textAssetsBadExamplePromptPart
+      )
       .replace('{{headline}}', textAsset.headline)
       .replace('{{description}}', textAsset.description);
 
@@ -268,9 +279,8 @@ export class GenerationHelper {
       `gs:/${decodeURIComponent(variantVideoPath)}`
     );
     AppLogger.info(`GenerateTextAsset Response: ${response}`);
-    const regex =
-      /.*Headline\s?:\**(?<headline>.*)\n+\**Description\s?:\**(?<description>.*)/ims;
-    const matches = response.match(regex);
+    const result = response.split('## Ad').filter(Boolean)[0];
+    const matches = result.match(GENERATE_TEXT_ASSETS_REGEX);
     if (matches) {
       const { headline, description } = matches.groups as {
         headline: string;
@@ -286,4 +296,51 @@ export class GenerationHelper {
       throw new Error(message);
     }
   }
+
+  static generateTextAssets(
+    variantVideoPath: string,
+    textAssetsLanguage: string
+  ) {
+    const count = 5;
+    const generationPrompt = CONFIG.vertexAi.textAssetsGenerationPrompt
+      .replace('{{videoLanguage}}', textAssetsLanguage)
+      .replace('{{desiredCount}}', String(count))
+      .replace('{{badExamplePromptPart}}\n    ', '');
+
+    const textAssets: VariantTextAsset[] = [];
+    let iteration = 0;
+
+    while (textAssets.length < count) {
+      iteration++;
+      const response = VertexHelper.generate(
+        generationPrompt,
+        `gs:/${decodeURIComponent(variantVideoPath)}`
+      );
+      AppLogger.info(`GenerateTextAssets Response: ${response}`);
+
+      const results = response.split('## Ad').filter(Boolean);
+
+      for (const result of results) {
+        const matches = result.match(GENERATE_TEXT_ASSETS_REGEX);
+        if (matches) {
+          const { headline, description } = matches.groups as {
+            headline: string;
+            description: string;
+          };
+          textAssets.push({
+            headline: String(headline).trim(),
+            description: String(description).trim(),
+          });
+          if (textAssets.length === count) {
+            break;
+          }
+        } else {
+          AppLogger.warn(
+            `WARNING - Received an incomplete response for iteration #${iteration} from the API!\nResponse: ${response}`
+          );
+        }
+      }
+    }
+    return textAssets;
+  }
 }

ui/src/index.ts

@@ -180,14 +180,25 @@ function getWebAppUrl(): string {
 
 // eslint-disable-next-line @typescript-eslint/no-unused-vars
 function regenerateTextAsset(
-  gcsFolder: string,
   variantVideoPath: string,
-  textAsset: VariantTextAsset
+  textAsset: VariantTextAsset,
+  textAssetLanguage: string
 ): VariantTextAsset {
   return GenerationHelper.generateTextAsset(
-    gcsFolder,
     variantVideoPath,
-    textAsset
+    textAsset,
+    textAssetLanguage
+  );
+}
+
+// eslint-disable-next-line @typescript-eslint/no-unused-vars
+function generateTextAssets(
+  variantVideoPath: string,
+  textAssetsLanguage: string
+): VariantTextAsset[] {
+  return GenerationHelper.generateTextAssets(
+    variantVideoPath,
+    textAssetsLanguage
   );
 }
 
@@ -209,6 +220,11 @@ function storeApprovalStatus(
   return true;
 }
 
+// eslint-disable-next-line @typescript-eslint/no-unused-vars
+function getVideoLanguage(gcsFolder: string) {
+  return GenerationHelper.getVideoLanguage(gcsFolder);
+}
+
 // eslint-disable-next-line @typescript-eslint/no-unused-vars
 function doGet(e: GoogleAppsScript.Events.DoGet) {
   const output = HtmlService.createTemplateFromFile('ui')

ui/src/ui/src/app/api-calls/api-calls.mock.service.ts

@@ -152,9 +152,9 @@ export class ApiCallsService implements ApiCalls {
     return of('');
   }
   regenerateTextAsset(
-    gcsFolder: string,
     variantVideoPath: string,
-    textAsset: VariantTextAsset
+    textAsset: VariantTextAsset,
+    textAssetLanguage: string
   ): Observable<VariantTextAsset> {
     return new Observable(subscriber => {
       setTimeout(() => {
@@ -183,4 +183,34 @@ export class ApiCallsService implements ApiCalls {
       }, 1000);
     });
   }
+  getVideoLanguage(gcsFolder: string): Observable<string> {
+    return new Observable(subscriber => {
+      this.ngZone.run(() => {
+        subscriber.next('German');
+        subscriber.complete();
+      });
+    });
+  }
+  generateTextAssets(
+    variantVideoPath: string,
+    textAssetsLanguage: string
+  ): Observable<VariantTextAsset[]> {
+    return new Observable(subscriber => {
+      setTimeout(() => {
+        this.ngZone.run(() => {
+          const textAssets = [];
+          for (let i = 0; i < 5; i++) {
+            textAssets.push({
+              headline: `NEW headline ${i + 1} in ${textAssetsLanguage}.`,
+              description: `NEW description ${i + 1} in ${textAssetsLanguage}`,
+              approved: true,
+              editable: false,
+            });
+          }
+          subscriber.next(textAssets);
+          subscriber.complete();
+        });
+      }, 1000);
+    });
+  }
 }

ui/src/ui/src/app/api-calls/api-calls.service.interface.ts

@@ -183,12 +183,17 @@ export interface ApiCalls {
   getGcsFolderPath(folder: string): Observable<string>;
   getWebAppUrl(): Observable<string>;
   regenerateTextAsset(
-    gcsFolder: string,
     variantVideoPath: string,
-    textAsset: VariantTextAsset
+    textAsset: VariantTextAsset,
+    textAssetLanguage: string
   ): Observable<VariantTextAsset>;
   storeApprovalStatus(
     gcsFolder: string,
     combos: RenderedVariant[]
   ): Observable<boolean>;
+  getVideoLanguage(gcsFolder: string): Observable<string>;
+  generateTextAssets(
+    variantVideoPath: string,
+    textAssetsLanguage: string
+  ): Observable<VariantTextAsset[]>;
 }