Index versus Database

Introduction

The set of Metadata Fields from a record is stored in both the database and in the index. Because searching is much more efficient in the index, it removes pressure from the database.

Persistence

  • When saving a record in the API, its metadata is synchronously saved in the database and the API call only returns once the database transaction is completed

  • Simultaneously a message is sent to the index queues, which in turn caused the metadata to be indexed in Apache Solr

Asynchronicity

  • Because the Apache Solr index is updated asynchronously, it takes some time before GET requests in the API see the change.

  • Under normal circumstances and with adequate hardware this delay is below 300 ms.

  • To prevent polling the API using GET requests to see if an update came through you can use Webhooks to be notified asynchronously

Indexation

  • The same metadata is stored both in the database and in the index

  • The only exception is the full set of technical EXIF metadata, extracted during Metadata Harvest & Mining, which is only stored in the database

  • Additionally, specific Field Definitions of the type ofSimple Fields can be marked as indexed, allowing you to make search queries as documented in the MediaHaven Rest API