Transfer assets

Overview

Typically you will need to transfer video assets from an on-premise source to a location where the encoder/packager can access them. This is necessary for OPF's content workflows to work.

To do this, you must install the Command File Ingester (CFI) module, typically on the file server that stores the assets. The benefits of using CFI are as follows:

• It allows efficient transfers when the source and destination file systems are on different hosts from OPF, by eliminating the need to make an intermediate copy of each file.

• No OPF resources are required to control and monitor the transfers.

• Long (slow) transfers are not aborted if the Workflow module has to be restarted before the copy is complete.

Workflow

The following diagram shows the interactions between the main components:

Here:

1. The Content Workflow Manager (CWM) generates an XML file containing the details of the file operation that the CFI is to perform. This XML command file is written into a directory that the CFI polls for commands that it is to execute.

2. The CWM then polls a status directory (usually the same location where it wrote the XML command file) for a status file that indicates the final status of the requested operation.

3. Once the CFI has received a command, it executes the operation. When the operation ends, whether successfully or not, it writes out a status feedback file. This status feedback file has the same name as the original XML command file, but with an added suffix that indicates success or failure.

Supported file transfer operations

The following table shows the file transfer operations and protocols that CFI supports:

Prerequisites

CFI must be installed on the file server where the source files or located (or on a machine that can access this file server). Ask your NAGRA representative to install CFI.

Main steps

• Specify the processing device

• Specify the source device

• Specify the destination device

• Create and submit the transfer profile

Specify a processing device

To specify the processing device:

1. Construct the request payload so that it looks like this:

   JSON

   {
    "name": "cfi device name",
    "id": "cfi_device1",
    "workflow": {
       "locked" : "false",
       "pendingJobs": 0,
       "maxJobsAllowed": 10,
       "accessPoints": [
           {
               "endPoint": "sftp://user:pwd@host:22/relativePath/",
               "type": "command" //this is same as the polling url
           },
           {
               "endPoint": "ftp://user:pwd@host:port/relativePath/status/",
               "type": "status"
           }
       ]
    }
   }

   The full list of fields is available in Device attributes.

2. Send a POST request to BCM's devices API with the above payload:

   BASH

   https://<host>:<port>/businessConfig/admin/v1/devices/

Specify the source device

To specify the source device:

1. Construct the request payload so that it looks like this:

   JSON

   {
       "name": "source device name",
       "id": "source_device_id_1",
       "workflow": {
           "locked" : "false",
           "pendingJobs": 0,
           "maxJobsAllowed": 10,
           "accessPoints": [{
               "endPoint": "file:///data/ingest/asset/",
               "type": "local"
           }]
       }
   }

   The full list of fields is available in Device attributes.

2. Send a POST request to BCM's devices API with the above payload:

   BASH

   https://<host>:<port>/businessConfig/admin/v1/devices/

Specify the destination device

To specify the destination device (S3 in this case):

1. Construct the request payload so that it looks like this:

   JSON

   {
    "name": "s3 device name",
    "id": "destination_device_id_1",
    "metadataSet": {
       "s3bucket": "",
       "accessKey": "",
       "accessSecret": ""
     },
    "workflow": {
       "locked" : "false",
       "accessPoints": [
           {
               "endPoint": "as3://aws_server/relativePath/",
               "type": "s3"
           }
       ]
    }
   }

   The full list of fields is available in  Device attributes.

2. Send a POST request to BCM's devices API  with the above payload:

   CODE

   https://<host>:<port>/businessConfig/admin/v1/devices/

Create and submit a transfer profile

To configure a transfer profile:

1. Construct a profile that looks like this example:

   JSON

   {
    "id": "profile_id_1",
    "name": "Transfer Asset with CFI workflow profile",
    "type": "ContentPreparation",
    "businessRulesProperties": {
       "sourceAssetType": "Reference",
       "sourceAssetDeviceLinkType": "source",
       "destinationAssetType": "HD",
       "destinationAssetDeviceLinkType": "source"
    },
    "workflow": {
       "workflowVersion": "TransferAssetWithCFI",
        "deviceUsedForResourceAllocation": "processingDevice",
       "sourceDevice": {
           "accessPointType": "local"
       },
       "destinationDevice": {
           "id": "destination_device_id_1",
           "accessPointType": "s3",
           "relativePath" : ""
       },
       "processingDevice": {
           "id": "processing_device_id_1",
           "accessPointType": "command"
       },
   
       "workflowInputProperties": {
           "sourceUri": "",
           "destinationUri": "",
           "processingUri": "",
           "cfiLocation": "sourceDevice",
           "checkForSourceAssetPresence": true,
           "removeSource": false,
           "overrideDestination": "true",//
           not being used for now
           "useProgressFilenameForDestination": "true",
           "successExtension": ".success",
           "errorExtension": ".error",
           "statisticsExtension": ".statistics",
           "proxyAddress": "<proxy>:<port>",//if there is no proxy.then this should be empty
           "createDestinationDirectoryStructure": false,
           "missingSourceErrorPattern": "non existing file",
           "jobCompletionTimeout": 600,
           "loopBackTimer": 10,
           "numberOfRetries": 3,
           "retryDelay": 120,
           "numberOfRetriesForRescheduling" : 3,
           "rescheduleDelay" : 120
       },
       "workflowOutputProperties": {
           "fullDestinationUri": ""
       }
    }
   }

   The full list of fields is available in Profile attributes.

2. Submit the profile to BCM by sending a POST request to BCM's profiles API with the profile as the request payload:

   CODE

   https://<host>:<port>/businessConfig/admin/v1/profiles/