Generative metadata using Open AI (POC)

This optional module is part of release 24.4 and is still in the proof-of-concept (POC) stage.

Introduction

Generative metadata allows for creating instructions for an AI model to generate multiple metadata fields. Any metadata field can be generated by providing a good hint to the AI model. For the POC we have created the following instructions. During the POC phase, these instructions are stored as custom properties of the Plugins.

{
  "Fields": {
    "Dynamic.GeneratedPocTitle": {
      "Hint": "Generate a prosaic title"
    },
    "Dynamic.GeneratedPocDescription": {
      "Hint": "Generate a description"
    },
    "Descriptive.Keywords.Keyword": {
      "Hint": "Generate a list of keywords as JSON array. All keywords should start with a capital letter. Generate around 10 keywords.",
      "Type": "array"
    },
    "Descriptive.Position.Location": {
      "Hint": "Generate the geocoordinates formatted as longitude and latitude seperated by comma"
    },
    "Dynamic.GeneratedPocIsPoorQuality": {
      "Hint": "Generate whether there is image has poor quality or is blurry",
      "Type": "boolean"
    },
    "Dynamic.GeneratedPocGdprHasFace": {
      "Hint": "Generate whether there is a human face on the image",
      "Type": "boolean"
    }
  }

However, MediaHaven is already prepared to handle many other questions you can ask of AI vision models. For example, is the picture taken outside or inside, is it nighttime, are there animals in the picture, etc. The language of the generated metadata is in the default locale of the organisation.

Activation

This feature will be automatically activated on mh-uat.mediahaven.com for the organisation mh-uatB & mh-uatc during the sprint release 24.4/4. However, it still requires the Open AI API key to be configured → Steps 3/4.

Create all the following field definitions linked with the dotted key you specify in the instructions (for UAT this is automatically done for the fields provided above)
Enable the module GENERATIVE_METADATA_OPEN_API_POC for the customer’s organisation. See the REST document for information on how to do that.
Obtain the Open AI API key for this environment. The development Open AI API key is stored in LastPass.
Update the plugin GENERATIVE_METADATA_OPEN_API_POC for the property Secret with the above Open AI API key. See Postman “Update secret for plugin”

Caveats

The metadata is only generated once at ingest time