Transfer assets

Overview

Typically you will need to transfer video assets from an on-premise source to a location where the encoder/packager can access them. This is necessary for OPF's content workflows to work.

To do this, you must install the Command File Ingester (CFI) module, typically on the file server that stores the assets. The benefits of using CFI are as follows:

It allows efficient transfers when the source and destination file systems are on different hosts from OPF, by eliminating the need to make an intermediate copy of each file.
No OPF resources are required to control and monitor the transfers.
Long (slow) transfers are not aborted if the Workflow module has to be restarted before the copy is complete.

Workflow

The following diagram shows the interactions between the main components:

Here:

The Content Workflow Manager (CWM) generates an XML file containing the details of the file operation that the CFI is to perform. This XML command file is written into a directory that the CFI polls for commands that it is to execute.
The CWM then polls a status directory (usually the same location where it wrote the XML command file) for a status file that indicates the final status of the requested operation.
Once the CFI has received a command, it executes the operation. When the operation ends, whether successfully or not, it writes out a status feedback file. This status feedback file has the same name as the original XML command file, but with an added suffix that indicates success or failure.

Supported file transfer operations

The following table shows the file transfer operations and protocols that CFI supports:

Protocol	POST	PUT	GET	MPUT		MPOST			MGET
Protocol	POST	PUT	GET	Non-recursive	Recursive	Non-recursive	Recursive		Non-recursive	Recursive
httpakamai	Supported	Supported	Supported	Supported	Supported	Supported		Supported	Not yet supported	Not yet supported
FTP, FTPS	Will not be supported	Supported	Supported	Supported	Supported	Will not be supported			Supported	Supported
Amazon S3 (AS3)	Supported Not tested	Supported Not tested	Supported Not tested	Supported Not tested	Supported Not tested	Supported Not tested		Supported Not tested	Not yet supported	Not yet supported

Prerequisites

CFI must be installed on the file server where the source files or located (or on a machine that can access this file server). Ask your NAGRA representative to install CFI.

Main steps

Specify a processing device

To specify the processing device:

Construct the request payload so that it looks like this:

CODE

{
 "name": "cfi device name",
 "id": "cfi_device1",
 "workflow": {
    "locked" : "false",
    "pendingJobs": 0,
    "maxJobsAllowed": 10,
    "accessPoints": [
        {
            "endPoint": "sftp://user:pwd@host:22/relativePath/",
            "type": "command" //this is same as the polling url
        },
        {
            "endPoint": "ftp://user:pwd@host:port/relativePath/status/",
            "type": "status"
        }
    ]
 }
}

The full list of fields is available in Device attributes.

Send a POST request to the BCM's devices API with the above payload:
```
https://<host>:<port>/businessConfig/admin/v1/devices/
```

Specify the source device

To specify the source device:

Construct the request payload so that it looks like this:

CODE

{
    "name": "source device name",
    "id": "source_device_id_1",
    "workflow": {
        "locked" : "false",
        "pendingJobs": 0,
        "maxJobsAllowed": 10,
        "accessPoints": [{
            "endPoint": "file:///data/ingest/asset/",
            "type": "local"
        }]
    }
}

The full list of fields is available in Device attributes.

Send a POST request to the BCM's devices API with the above payload:
```
https://<host>:<port>/businessConfig/admin/v1/devices/
```

Specify the destination device

To specify the destination device (S3 in this case):

Construct the request payload so that it looks like this:

CODE

{
 "name": "s3 device name",
 "id": "destination_device_id_1",
 "metadataSet": {
    "s3bucket": "",
    "accessKey": "",
    "accessSecret": ""
  },
 "workflow": {
    "locked" : "false",
    "accessPoints": [
        {
            "endPoint": "as3://aws_server/relativePath/",
            "type": "s3"
        }
    ]
 }
}

The full list of fields is available in Device attributes.

Send a POST request to the BCM's devices API with the above payload:
```
https://<host>:<port>/businessConfig/admin/v1/devices/
```

Create and submit a transfer profile

To configure a transfer profile:

Construct a profile that looks like this example:

CODE

{
 "id": "profile_id_1",
 "name": "Transfer Asset with CFI workflow profile",
 "type": "ContentPreparation",
 "businessRulesProperties": {
    "sourceAssetType": "Reference",
    "sourceAssetDeviceLinkType": "source",
    "destinationAssetType": "HD",
    "destinationAssetDeviceLinkType": "source"
 },
 "workflow": {
    "workflowVersion": "TransferAssetWithCFI",
     "deviceUsedForResourceAllocation": "processingDevice",
    "sourceDevice": {
        "accessPointType": "local"
    },
    "destinationDevice": {
        "id": "destination_device_id_1",
        "accessPointType": "s3",
        "relativePath" : ""
    },
    "processingDevice": {
        "id": "processing_device_id_1",
        "accessPointType": "command"
    },

    "workflowInputProperties": {
        "sourceUri": "",
        "destinationUri": "",
        "processingUri": "",
        "cfiLocation": "sourceDevice",
        "checkForSourceAssetPresence": true,
        "removeSource": false,
        "overrideDestination": "true",//
        not being used for now
        "useProgressFilenameForDestination": "true",
        "successExtension": ".success",
        "errorExtension": ".error",
        "statisticsExtension": ".statistics",
        "proxyAddress": "<proxy>:<port>",//if there is no proxy.then this should be empty
        "createDestinationDirectoryStructure": false,
        "missingSourceErrorPattern": "non existing file",
        "jobCompletionTimeout": 600,
        "loopBackTimer": 10,
        "numberOfRetries": 3,
        "retryDelay": 120,
        "numberOfRetriesForRescheduling" : 3,
        "rescheduleDelay" : 120
    },
    "workflowOutputProperties": {
        "fullDestinationUri": ""
    }
 }
}

The full list of fields is available in Profile attributes.

Submit the profile to BCM by sending a POST request to the BCM's profiles API with the profile as the request payload:
```
https://<host>:<port>/businessConfig/admin/v1/profiles/
```