Index versus Database

Introduction

The set of https://mediahaven.atlassian.net/wiki/spaces/CS/pages/42074134 from a record is stored in both the database and in the index. Because searching is much more efficient in the index, it removes pressure from the database.

Persistence

  • When saving a record in the API, its metadata is synchronously saved in the database and the API call only returns once the database transaction is completed

  • Simultaneously a message is sent to the index queues, which in turn caused the metadata to be indexed in Apache Solr

Asynchronicity

  • Because the Apache Solr index is updated asynchronously, it takes some time before GET requests in the API see the change.

  • Under normal circumstances and with adequate hardware this delay is below 300 ms.

  • To prevent polling the API using GET requests to see if an update came through you can use https://mediahaven.atlassian.net/wiki/spaces/CS/pages/1704329251 to be notified asynchronously

Indexation

  • The same metadata is stored both in the database and in the index

  • The only exception is the full set of technical EXIF metadata, extracted during https://mediahaven.atlassian.net/wiki/spaces/CS/pages/1730019345, which is only stored in the database

  • Additionally, specific of the type of can be marked as indexed, allowing you to make search queries as documented in the