The Ingest Monitoring offers an overview of the items for your organisation in MediaHaven. Each item in MediaHaven has a list identifiers and a particular status and is linked with a series of events. Firstly we describe these concepts of status, events and identifiers. Secondly we describe how these concepts are used in the design of the ingest into MediaHaven and the representation in three views in the MediaHaven Ingest Monitoring.
ArchiveStatus
Value
Remark
The item has been detected by the tape ingest server but has not yet been transferred to the ingest server.
The item has been detected on the ingest server and is being processed.
The item is not successfully processed. The events for this item contain one or more events and these contain the reason for the failure.
The item is successfully archived on two or more disks. For files where the ultimate location is tape, this status will not appear.
The item is successfully archived on N tapes. Depending on the installation N can be 1 (archive), 2 (backup) or 3 (vault).
Only used for files of the type:
Ensemble (Collection, Set, Newspaper): all files in the ensemble have either ArchiveStatus , or .
Metadata Only: the ArchiveStatus is immediately upon creation
Events
MediaHaven logs the complete history of an item as a list of successive Premis events. The outcome of an event is either good or bad .
Here we offer an overview of event types.
Event
Description
The item has been created in MediaHaven. From this point on the three identifiers (see below) are defined.
The item has been detected by either the tape ingest server or by the standard ingest service
(tape ingest only) The item has been successfully packed into a ZIP archive.
The metadata of the item has been updated in the database.
The file has been manually published by the user or has been automatically published.
Event generated by the standerd ingest service to typically log validation errors (e.g. MD5 checksum, archive validation) with outcome .
The service has started the transcoding and storage allocation of the item.
The service has finished the transcoding and storage allocation of the item. For installations where the ultimate location is not tape, the item will have the status from this point onward.
(For installations that write to 1 or more tapes) The item has been successfully written to an tape. The comment of the event contains the barcode of the tape. If the item has been successfully written to all required tapes, the item will have the status after the batch of tapes writes is complete.
(For installations that write to 2 or more tapes) The item has been successfully written to a tape. The comment of the event contains the barcode of the tape. If the item has been successfully written to all required tapes, the item will have the status after the batch of tapes writes is complete.
(For installations that write to 3 tapes) The item has been successfully written to a tape. The comment of the event contains the barcode of the tape. If the item has been successfully written to all required tapes, the item will have the status after the batch of tapes writes is complete.
An export of the item has been requested by a user. The event can have outcome if permission was denied.
The export of the item has been completed with outcome or .
Examples of the sequence of events
An item submitted using the standard online ingest and stored on disk: →
An item submitted using the standard online ingest with an MD5 checksum failure: →
An item submitted using the standard online ingest and stored on an archive and backup tape: →
An item submitted using the standard online ingest and but which could not be written to tape due to an internal error: →
An item read from an ingest tape and stored on an archive and backup tape: →
An item read from an ingest tape but with an MD5 checksum failure on the ingest tape: →
Identification
In order to unique identify items in the system MediaHaven uses three identifiers:
Umid (aka MediaObjectId): This is a 64 character hex string uniquely representing the item in MediaHaven.
FragmentId: This is a 96 character hex string representing a fragment of an item (e.g. a clip from a video, a page from a document). An item has always at least one fragment, termed the main fragment.
ExternalId: (Advanced installations only) The field contains the identifier of the item provided by the customer. The ingest flow is tightly coupled with this field: it will reject or accept a item based on whether the ExternalId already exists in the system. In the design section below we go into detail about this.
Views
The Ingest Monitoring offers three views:
SIPs: Offers an overview of items coupled with an ExternalId, showing only the most recent submission of each ExternalId.
items: Offers a complete history of items submitted to MediaHaven, including failed or deleted submissions.
Events: Offers the complete history of events in MediaHaven. When restricted to events of a single item, it shows the complete history of events that occurred to that item.
Design
The ingest views are designed around the following philosophy:
In the view you see the state of the most recent submission of items with an ExternalId. While in the view you see the the complete backlog of the submissions of an item into MediaHaven, including submissions. When you present an item with an ExternalId to the system and it already exists with status , or and the old item is not deleted, the new item is rejected and an event or is generated on the existing item. If the ExternalId already existed with status , the old item is deleted automatically and a new item is d. If the ExternalId already existed but was deleted, the new item is also accepted.
Let's us explain this by an example using the ExternalId .
SIPs
The screenshot below shows in the view the item with ExternalId with status that was successfully archived on .
Files
The screenshot below shows in the view the complete history of submissions of the item . The most recent submission has status , while the previous submissions have status and have been deleted (showed by a shaded or striped colour). In the example the item has been submitted 8 times. Notice that each submission has its own FragmentId while they share the same ExternalId. Due to design of the ingest, there can be exactly only one non-deleted item for the same ExternalId.
Events (succes)
If we look into the view of the successful submission we see the following events. Notice that the item has been successfully written to the tape and tape . Afterwards it has been exported to an tape .
Events (failure)
If we look into the view of a submission we see for example the following events.
We notice that the event with outcome . The comment contains the following error:
METS: referred file does not exist: OriginalData/510_2242_000_00049_000/510_2242_000_00049_000_0_0001.tif
The submitted item was an archive and the METS file linked with it, described a file which was not found inside the archive.
,
or
.
