Roles

These are the roles that are needed:

  • Admin
  • Read storage
  • Write storage
  • Reindex storage

Links

  • AWS S3
  • Backblaze B2
  • Google Cloud Storage
  • Microsoft Azure
  • IBM Cloud Object Storage
  • Wasabi
  • Cloudian
  • Creating a Storage

    In iconik, you can create a storage for files or exports, however, it is not possible to add keyframes or proxies storages.

    1. Click on ADMIN in the top navigation.
    2. Choose Storages from the left navigation bar.
    3. Click on + NEW STORAGE to open the form to create a storage.
    4. Choose a name, storage purpose, and storage type. If you're unsure about what different purposes and types do read about it here. The name must be unique for the iconik account.
    5. Click to continue with the creation.
    6. Fill in the form.
    7. Click to create the storage.
    8. If storage did not succeed it may still create a storage where it is possible to change settings to make it work.

    Please note: If you are adding an existing bucket, and it contains a large amount of data (typically 10s of Terrabytes) or a large numbers (several 100,000s or more) and you want to bring all the files in as assets into iconik, it might make sense to contact your support to make sure that we can preemptively add enough resources to make the operation timely

    *Important: By enabling the delete flag on a storage, you will be enabling users and administrators with that role to delete your files from your bucket*

    Form fields for adding a storage

    If unsure what the different fields in the forms mean it is described below. The types are listed on the Storage Overview Page.

    If selected File Storage Type

    • Growing Files Threshold - The time, in seconds, that the ISG will wait while a file updates in the file system. The default is 10 seconds and works in the majority of all cases but might need to be increased if necessary.
    • Scan Interval - The time interval in which the ISG looks for new and updated files in the file system.
    • Mount Point - The root storage which ISG will be scanning from.
    • Scan Directories - If left empty the ISG will scan all files in all subdirectories from the root. If wanting to include only part of the files the paths of these will be added here. To add more than one press the + icon to add more paths.
    • Asset Versions Suffix - Suffix to use for creating new versions of assets. Default is _v#.
    • Allow Access Outside Scan Directories - Enable this if you want ISG to be allowed to read and write files outside of the listed Scan directories (Useful for some transfers or API integrations).
    • Group Access - The selected group will have full access on all assets created by the ISG for this storage.
    • Local Keyframe Creation - If enabled, creates the keyframe locally instead of in the cloud. Requires that the local proxy creation is enabled and that there exists at least one local transcoder.
    • Local Proxy Creation - If enabled, creates the proxy locally instead of in the cloud. Requires that the local keyframe creation is enabled and that there exists at least one local transcoder.
    • Upload Original Files - If enabled allows the ISG to upload the original files to the cloud, otherwise, the files will only be contained in the local storage.
    • Create Collections for Directories - If enabled iconik will create a collection for each folder from the root. It will create sub-collections for each sub-folder.
    • System Defined Storage - If selected it is only possible to change the read, write and delete alternatives. It is not possible to change the other storage specifications.
    • Read - If enabled, the assets in the storage will have read permission enabled.
    • Write - If enabled, the assets in the storage will have write permission enabled.
    • Delete - If enabled, the assets in the storage will have delete permission enabled. Please note, that if this is set to true when an asset is deleted in iconik it will also be removed from the storage upon purging
    • Scan - If enabled, the storage will continue to scan for new files on the local storage.
    • Collection Directory Mapping - If enabled, will create iconik collections for directories found.
    • Include file extension in title - If enabled titles will include the file extension.
    • Filename is external id - Will populate the external ID field on the asset with the filename of the file.
    • Sidecar metadata required - Requires that a sidecar file is present to import a file for iconik Storage Gateway Storages.
    • Sidecar metadata view - The Metadata View that is used to show the sidecar metadata when using the ISG.
    • Ignore Files Patterns - The ISG will ignore to scan the files which contain the pattern from this field. Examples:
      • *exclude* (match file/folder with substring exclude)
      • *.pdf (match file/folder that ends on .pdf)
      • You can also use regular expressions: re:/^.*\.png$/ (match file/folder with .png ending)
    • Don't Transcode Patterns - The ISG will not transcode which contain the pattern from this field.
    • Folder name tags metadata view - Will map folders to Tags in a tag field that is in the view specified here.
    • Folder name tags metadata Field Name - Select the Field from the view specified above.
    • Consider identical files the same - Compares the checksum of the file found to all files in iconik, if it matches it will create a new file entry for the file on the existing asset.
    • Create Asset for each Directory - If enabled, the ISG will create an asset for each directory instead of each file. Requires both that original and transcode in directory patterns are defined.
    • Original in Directory Patterns - Defines which files that shall be the original file.
    • Transcode in Directory Patterns - Defines which files the ISG shall create proxies from.

    If selected Amazon S3 Storage Type

    • Access Key - The access key for the amazon S3 storage you wish to use.
    • Secret Key - The secret key for the amazon S3 storage you wish to use.
    • Bucket - The name of the S3 bucket which you wishes to use.
    • Path - If left empty, iconik will use the whole bucket. Otherwise defines which part of the bucket the storage should start at.
    • Region - The Amazon region which the bucket is located at. (If the address is https://s3.eu-central-1.amazonaws.com/bucketname then the endpoint is https://s3.eu-central-1.amazonaws.com and the region is eu-central-1). Note that if you are using another S3 compatible storage provider, such as IBM or Wasabi, the URLs and regions are different. Please refer to the corresponding vendors documentation.
    • Endpoint - The endpoint for your bucket. Is the first part of your buckets web address. (If address is https://s3.eu-central-1.amazonaws.com/bucketname then the endpoint is https://s3.eu-central-1.amazonaws.com). Note that if you are using another S3 compatible storage provider, such as IBM or Wasabi, the URLs and regions are different. Please refer to the corresponding vendors documentation.
    • System Defined Storage - If selected, it is only possible to change the read, write and delete alternatives. It is not possible to change the other storage specifications.
    • Read - If enabled, the assets in the storage will have read permission enabled.
    • Write - If enabled, the assets in the storage will have write permission enabled.
    • Delete - If enabled, the assets in the storage will have delete permission enabled. Please note, that if this is set to true, that when an asset is deleted in iconik it will also be removed from this storage upon Purging
    • Scan - If enabled, an eye will be visible next to the storage in the list. Click on the eye to scan through the cloud storage and add the new files to iconik.

    Permissions

    The access key you use need to have the necessary permissions to the bucket content and to the bucket itself to be able to set the CORS settings.

    Example: { "Version": "2012-10-17", "Id": "Policy12345576", "Statement": [ { "Sid": "Stmt123456", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::1233456778:user/demoUser" }, "Action": "s3:*", "Resource": [ "arn:aws:s3:::iconik-demo", "arn:aws:s3:::iconik-demo/*" ] } ] }

    If you are setting up a third party bucket we strongly recommend you read our knowledgebase article Controlling Costs as using iconik with third party buckets may cause costs for you.

    If selected Google Cloud Storage Type

    • Service Account JSON - The Service account JSON for the Google Cloud Storage account that you wish to use.
    • Bucket - The name of the Google Cloud Storage bucket you wish to use.
    • Path - If left empty, iconik will use the whole bucket. Otherwise defines which part of the bucket the storage should start at.
    • Project - The Google Cloud Project name which the bucket resides.
    • System Defined Storage - If selected, it is only possible to change the read, write and delete alternatives. It is not possible to change the other storage specifications.
    • Read - If enabled, the assets in the storage will have read permission enabled.
    • Write - If enabled, the assets in the storage will have write permission enabled.
    • Delete - If enabled, the assets in the storage will have delete permission enabled. Please note, that if this is set to true, that when an asset is deleted in iconik it will also be removed from this storage when Purging
    • Scan - If enabled, an eye will be visible next to the storage in the list. Click on the eye to scan through the cloud storage and add the new files to iconik.

    Permissions

    The service account you are using need to have the following permissions to work correctly: storage.buckets.get storage.buckets.update storage.objects.list storage.objects.create storage.objects.delete storage.objects.get

    The eastest way to achieve this is to add the roles Storage Legacy Bucket Owner, and Storage Legacy Bucket Reader. Note that Bucket Owner is not enough on its own.

    If you are setting up a third party bucket we strongly recommend you read our knowledgebase article Controlling Costs as using iconik with third party buckets may cause costs for you.

    If selected Backblaze B2 Storage Type

    • Key Type - Choose between Backblaze B2 Master Application Key and Application Key
    • keyId - The ID for the Application key set with your Backblaze B2 account.
    • applicationKey - The application key for your Backblaze account.
    • Bucket Name - The name of the Backblaze B2 bucket you wish to use.
    • Bucket ID - The ID of the Backblaze B2 bucket you wish to use.
    • Path - If left empty, iconik will use the whole bucket. Otherwise defines which part of the bucket the storage should start at.
    • Read - If enabled, the assets in the storage will have read permission enabled.
    • Write - If enabled, the assets in the storage will have write permission enabled.
    • Delete - If enabled, the assets in the storage will have delete permission enabled. Please note, that if this is set to true, that when an asset is deleted in iconik it will also be removed from this storage when Purging
    • Scan - If enabled, an eye will be visible next to the storage in the list. Click on the eye to scan through the cloud storage and add the new files to iconik.

    Permissions

    The key you are using need to have full access to all buckets to be able to set CORS correctly.

    If you are setting up a third party bucket we strongly recommend you read our knowledgebase article Controlling Costs as using iconik with third party buckets may cause costs for you.

    If you selected Azure

    • Connection String - The connection string used to access the bucket. You can read more about connection string and where to find it at Azure documentation page.
    • Container Name - The container name. You can read more about how to create new container at Azure documentation page.

    If selected Portal Storage Type

    • System Defined Storage - If selected, it is only possible to change the read, write and delete alternatives. It is not possible to change the other storage specifications.
    • Read - If enabled, the assets in the storage will have read permission enabled.
    • Write - If enabled, the assets in the storage will have write permission enabled.
    • Delete - If enabled, the assets in the storage will have delete permission enabled.

    Learn More