Skip to end of banner
Go to start of banner

Pools

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Current »

Introduction

Files archived by MediaHaven / DigiHaven are written to storage using a particular number of copies.

For example

  • 1 copy: This case is not advised as it goes against our archival mandate

  • 2 copies: The most common case where we write a main and backup copy on disk

  • 3 copies: The case at Meemoo where we write to A, B and C tapes

Concepts

The storage concepts in the engine have a tree structure with progressive information, each concept is a child concept of the previous.

  • Super clusters : groups pools which are mirrored across all organisations (future concept)

  • Cluster groups: groups pools across all organisations having the same role (one of the mirrored copies)

  • Clusters: pool from a cluster group for a specific organisation

  • Storage Pools: pool corresponding with a protocol

Example

  • Super cluster for all LTO-8 tapes named “tape_8” with contains all A, B and C tapes

  • Cluster group for all LTO-8 archive tapes named “tape_archive_8”

  • Cluster for the archive LTO-8 tapes of the organisation vrt named “vrt_tape_archive_8”

  • Storage Pool for the tape A00154L8 belonging to the cluster “vrt_tape_archive_8”

Tree

In this diagram, the 3rd copy, namely VAULT/C tapes are omitted.

Properties

Concept

Property

Examples

Description

Cluster group

cluster group ID

Unique label for the cluster group

Cluster group

type

MOB, BROWSE, CACHE

  • MOB: Original file

  • BROWSE: Preview file

  • CACHE: Cached file used by the worker daemon

Cluster group

role

  • ARCHIVE: 1st copy

  • BACKUP: 2nd copy

  • VAULT: 3rd copy

  • DISTRIBUTION: linked with an external system such as YouTube

  • TRANSIENT: non definitive storage such a GPFS buffer

  • EXTERNAL: linked with another MediaHaven installation (syncer)

Cluster group

tape_like

A%L8 IM%L6

Pattern to which the tape barcodes correspond

Cluster group

Free space: calculate

  • If true, calculate the free space for all pools in this cluster groups

Cluster group

Free space: storage path

mh-dev-backup.mediahaven.com:/mnt/STORAGE/POOL2

Server specified by the URI to contact to determine the free space parameters of the volume mounted on the path specified by the URI

Cluster group

Free space: add queued space

Whether to add the queued space to the free space, must be enabled when the ingest and archive storage share the same physical volume

Cluster

cluster ID

vrt_tape_archive_8

Unique label for the cluster

Cluster

organisation ID

100

  • restricts the cluster group to particular organisation

  • multiple clusters for the same cluster group and organisation is allowed

Storage Pool

storage pool ID

100154

Unique ID (integer) for the storage pool

Storage Pool

ID in cluster

1

When allocating storage it selects mirrored copies across pools of multiple cluster groups having the same ID in cluster

Storage Pool

Total Space

Total storage in bytes

Storage Pool

Free Space

Remaining storage in bytes

Storage Pool

Keep Free Space

  • Storage space in base to keep free as guideline

  • Setting keep free space to total space, makes the pool illegible for writing

Storage Pool

Queued space

Free space likely to be consumed in the future, i.e. when scheduling jobs to write to tape

Storage Pool

Is Online

true false

When false this pool is currently unavailable, for example when a tape is removed from the tape library

Storage Pool

Protocol, Credentials, Server, Port, Path

ftp://user:***@server:port/path

Forms together an URI

Storage Pool

Class

HOT COOL

Object storage can write files in different modes

  • No labels