Preview Feature

​

The Cribl Terraform provider is a Preview feature that is still being developed. We do not recommend using it in a production environment, because the feature might not be fully tested or optimized for performance, and related documentation could be incomplete.

Please continue to submit feedback through normal Cribl support channels, but assistance might be limited while the feature remains in Preview.

Use the cribl-tf-config-exporter CLI tool to export an existing Cribl configuration to Terraform HashiCorp Configuration Language (HCL) format and migrate to Terraform-managed infrastructure as code.

The cribl-tf-config-exporter tool operates in three phases:

1. Discovery: Uses the Cribl API to identify all supported Cribl resource types, build an inventory of resource to export, and collect resource metadata.

2. Conversion: Uses the Cribl API to retrieve the complete configuration for each discovered resource and converts the configuration to Terraform-enabled models.

3. HCL generation: Produces organized Terraform HCL files, module structure files, provider configuration, import blocks, and a README.md with usage instructions.

The output is ready-to-use Terraform module set that represents the Cribl configuration.

Prerequisites

​

To install and use cribl-tf-config-exporter, make sure that you meet the following prerequisites:

• Read-only access on a Cribl.Cloud or on-prem deployment.

  • Because cribl-tf-config-exporter uses the Cribl API for only read operations, it does not require greater access rights on Cribl deployments. cribl-tf-config-exporter does not make any changes to Cribl deployments.

• Terraform version 1.5 or newer.

• Cribl Terraform provider (criblio/criblio) version 1.20.138 or newer with properly configured authentication.

Install cribl-tf-config-exporter

​

1. In the GitHub repository for the Cribl Terraform provider, select the latest cribl-tf-config-exporter release.

2. On the release page, under Assets, download the archive for your platform:

   Platform	Archive File Name Example
   macOS (Intel)	cribl-tf-config-exporter_<version>_darwin_amd64.zip
   macOS (Apple Silicon M1/M2/M3)	cribl-tf-config-exporter_<version>_darwin_arm64.zip
   Linux (x86-64)	cribl-tf-config-exporter_<version>_linux_amd64.zip
   Linux (ARM64)	cribl-tf-config-exporter_<version>_linux_arm64.zip
   Windows	cribl-tf-config-exporter_<version>_windows_amd64.zip

   cribl-tf-config-exporter is built with Go and released via goreleaser as platform-dependent binaries. For Linux, one binary is provided per architecture (x86-64 and ARM64), not per distribution. The Linux binaries run on glibc-based distributions such as Debian, RHEL, and Ubuntu as long as the architecture matches.

3. Extract the archive locally and run the binary with no arguments to display usage information:

   • macOS or Linux: ./cribl-tf-config-exporter
   • Windows: cribl-tf-config-exporter.exe
     
     

   To run cribl-tf-config-exporter without the ./ prefix, add the extracted directory to your PATH or copy the binary to a directory in your PATH.

Resolve macOS Security Warning

​

When you install and run cribl-tf-config-exporter on macOS, you may see a security warning that it cannot be opened because it is from an unidentified developer. The process may also be killed with no dialog. This occurs because the binary is not signed by Apple.

To remove quarantine and resolve the security warning, run the following command:

xattr -d com.apple.quarantine ./cribl-tf-config-exporter

If you prefer, you can allow cribl-tf-config-exporter to run for a single use as follows:

1. Right-click cribl-tf-config-exporter.

2. Select Open.

3. Select Open again to confirm.

Authenticate

​

Use one of the supported authentication methods for the Cribl Terraform provider for cribl-tf-config-exporter authentication.

For Cribl.Cloud deployments, you can use the --org-id and --workspace-id options to specify the Organization ID and Workspace ID instead of listing them in the provider block, credentials file, or environment variables. For example:

cribl-tf-config-exporter export --org-id <orgId> --workspace-id <workspaceId> --output-dir ./tf

For on-prem deployments, you can use the --server-url option to specify the server URL instead of listing it as an environment variable or in a credentials file. For example:

cribl-tf-config-exporter export --server-url <https://hostname:port> --output-dir ./tf

If you use the --org-id and --workspace-id options (Cribl.Cloud) or the --server-url option (on-prem), the values you specify take precedence over any values provided in the provider block, environment variables, or credentials file that you use for Terraform authentication. You must still configure a provider block, environment variables, or credentials file to provide the other values that are required for authentication as shown in the instructions for authenticating in Cribl.Cloud or on-prem deployments.

Sub-Commands and Options

​

cribl-tf-config-exporter export

​

Export Cribl configuration resources to Terraform HCL files and generate main.tf, variables.tf, and import.tf for Terraform operations.

Options for cribl-tf-config-exporter export

​

Use the following options to control which resources to include in the export and how the resources are organized in the output directory. Usage examples demonstrate common commands using these options.

Output Directory Structure

​

By default, the output directory structure organizes resources into subdirectories based on Worker Groups, Edge Fleets, and Cribl Search as follows:

• Worker Groups: <output-dir>/<group_id>/<resource-type>/
• Edge Fleets: <output-dir>/<fleet_id>/<resource-type>/
• Cribl Search (including the default_search group): <output-dir>/search/<resource-type>/

Resources with no Worker Group, Edge Fleet, or Cribl Search context, such as lakehouse and Cribl Lake datasets, are organized under the global subdirectory: <output-dir>/global/<resource-type>.

If you use the --flat option, all resources are organized directly under <output-dir>/<resource-type>/, with no categorization by Worker Group, Edge Fleet, or Cribl Search subdirectories. The global/ subdirectory is also omitted in flat mode.

cribl-tf-config-exporter version

​

Display the current version and build information for cribl-tf-config-exporter.

Usage Examples

​

Prepare to Export and Migrate

​

As you prepare to export and migrate your Cribl configuration to Terraform, consider the following workflow approaches to help ensure a seamless process:

• Before you actually export and migrate resources, run your cribl-tf-config-exporter export commands with the --dry-run option. Review the results to confirm resource counts and types and verify that discovery works as expected. This gives you a chance to refine and update your commands as needed.

• Migrate incrementally to reduce the scope of each command and make validation more manageable. After you validate the outcome of each command, continue with another subset of resources in subsequent commands. For example:

  • Use the --include option to focus on a subset of resource types in each command.

  • Use the --group option for large deployments to restrict each command to a specific Worker Group or Edge Fleet.

• Specify a dedicated output directory such as ./tf to keep exported files organized and separate from other files. Isolating files in a dedicated output directory facilitates review and makes version control more straightforward.

• Pin the Cribl Terraform provider version in providers.tf in your output directory (for example, version = ">= 1.20.138"). The providers.tf file is generated along with other Terraform files when you run cribl-tf-config-exporter export. Pinning a provider version helps prevent unexpected changes in functionality and behavior, and it maintains a consistent infrastructure state when you run terraform apply across different environments.

• Use environment variables or a terraform.tfvars file to keep secrets, credentials, and sensitive information out of committed files. If you use terraform.tfvars, make sure to add it to your .gitignore file.

Export Resources and Migrate to Terraform Management

​

The following procedure generally describes how to use cribl-tf-config-exporter to export Cribl resources and migrate Cribl configurations to Terraform management.

See Copy a Configuration to a New Workspace in Cribl.Cloud and Migrate from an On-Prem Deployment to Cribl.Cloud for more detailed procedures.

1. Run the cribl-tf-config-exporter export command to generate your Cribl configuration and Terraform files.

2. In the output directory, review the generated main.tf, variables.tf, and import.tf files as well as any other generated HCL files. Confirm that the output directory contains the expected resources.

   • The count of discovered resources may not match the count of exported resources. Discovery counts all resources that cribl-tf-config-exporter finds via the Cribl API. Export skips the excluded resources, and some default resources are explicitly omitted, so cribl-tf-config-exporter may discover more resources than it exports.

   • If needed, update the generated import.tf file to manage default resources.

3. In your command line interface, navigate to the output directory.

4. Run terraform validate to check syntax accuracy and validity. You may need to manually adjust certain resources in complex configurations.

5. Run terraform init to initialize the Terraform project in the output directory.

6. Run terraform plan to verify the configuration.

7. Run terraform import (or terraform apply if you are using import blocks) to bring resources under Terraform management.

The first time you run terraform plan, you may see a diff for certain resources (such as criblio_group and criblio_source). This diff does not mean that the existing Cribl resources are actually changing. Instead, it means that Terraform is reconciling its state with the live configuration of Cribl resources. After you run terraform apply, the state is updated and subsequent terraform plan executions should show no drift.

Copy a Configuration to a New Workspace in Cribl.Cloud

​

Follow these steps to copy a Cribl configuration from a source Workspace to a new target Workspace in Cribl.Cloud (for example, from a dev Workspace to a staging Workspace in the same Organization, or from a Workspace in Organization A to a Workspace in Organization B).

The examples depict authentication using environment variables, but you can use any of the supported authentication methods for Cribl.Cloud.

1. Export Cribl resources from the source Workspace:

   # Set environment variables for source Workspace
   export CRIBL_ORGANIZATION_ID="source-org-id"
   export CRIBL_WORKSPACE_ID="source-workspace-id"
   export CRIBL_CLIENT_ID="your-client-id"
   export CRIBL_CLIENT_SECRET="your-client-secret"
   
   # Run the cribl-tf-config-exporter CLI tool
   cribl-tf-config-exporter export --output-dir ./tf

   Copy

2. Point the Terraform provider at the target Workspace using environment variables as shown below or by updating providers.tf:

   # Set environment variables for target Workspace
   export CRIBL_ORGANIZATION_ID="target-org-id"
   export CRIBL_WORKSPACE_ID="target-workspace-id"
   export CRIBL_CLIENT_ID="your-client-id"
   export CRIBL_CLIENT_SECRET="your-client-secret"

   Copy

3. Replace import.tf with a version that contains only import blocks for default resources. See Manage default Resources for details.

4. Apply the exported Cribl resources to the target Workspace:

   cd tf && terraform init && terraform plan && terraform apply

   Copy

Because cribl-tf-config-exporter exports to standard Terraform HCL, the same Terraform modules work against any Workspace. To target a different Workspace, change only the authentication credentials and Workspace IDs and replace import.tf with a version that includes only default resources.

Migrate from an On-Prem Deployment to Cribl.Cloud

​

Follow these steps to migrate a Cribl configuration from an on-prem deployment to a new Cribl.Cloud deployment.

On-prem deployments export only global resources like Sources, Destinations, Routes, Pipelines, Packs, and Worker Groups and Edge Fleets. Although you can use cribl-tf-config-exporter to migrate most of your on-prem configuration, you must configure Cribl.Cloud-only resources for Cribl Search, Cribl Lake, and Lakehouse separately in the new Cribl.Cloud Workspace.

In addition, some resource configurations differ between on-prem deployments and Cribl.Cloud. For example, on-prem Worker Groups include onPrem = true, which is onPrem = false for Cribl.Cloud Worker Groups. You must configure any such resources separately when migrating from on-prem to Cribl.Cloud.

The examples depict authentication using environment variables with automatic token management, but you can use any of the supported authentication methods for Cribl.Cloud.

1. Export Cribl resources from the on-prem deployment:

   # Set environment variables for the on-prem deployment
   export CRIBL_ONPREM_SERVER_URL="https://cribl.example.com"
   export CRIBL_ONPREM_USERNAME="admin"
   export CRIBL_ONPREM_PASSWORD="admin"
   
   # Run the cribl-tf-config-exporter CLI tool
   cribl-tf-config-exporter export --server-url https://cribl.example.com --output-dir ./tf

   Copy

2. Update the environment variables to provide the Cribl.Cloud Organization ID, Workspace ID, and authentication credentials:

   # Set environment variables for the Cribl.Cloud Organization and Workspace
   export CRIBL_ORGANIZATION_ID="your-org-id"
   export CRIBL_WORKSPACE_ID="your-cloud-workspace-id"
   export CRIBL_CLIENT_ID="your-client-id"
   export CRIBL_CLIENT_SECRET="your-client-secret"

   Copy

3. Replace import.tf with a version that contains only import blocks for default resources. See Manage default Resources for details.

4. Apply the exported Cribl resources to Cribl.Cloud:

   cd tf && terraform init && terraform plan && terraform apply

   Copy

Manage Sensitive Data

​

When cribl-tf-config-exporter generates Terraform HCL files, it uses placeholders for sensitive data like tokens, secrets, and certificates. These placeholders are variable references that indicate where to supply the actual values. The README.md that cribl-tf-config-exporter generates explains how to provide the actual values via terraform.tfvars or environment variables. This ensures that sensitive data is not committed to version control and aligns with best practices for managing sensitive data in Terraform.

Manage default Resources

​

The import.tf file that is generated when you run cribl-tf-config-exporter export includes import blocks for all exported resources, including:

• User-created resources.

• Default resources that exist in every Cribl environment, like the default Worker Group and Route.

If you are migrating to a new Cribl environment (such as a new Workspace or a fresh Cribl.Cloud instance), you must update the generated import.tf file so that it includes only import blocks for default resources that already exist in the new Cribl environment. These import blocks for default resources must have matching module addresses in main.tf. In addition, import.tf must not include import blocks for any user-created resources that do not yet exist in the new environment.

If import.tf does not contain import blocks for default resources with matching module addresses in main.tf, Terraform will try to create these resources from main.tf and fail because the default resources already exist in the new environment. If import.tf contains import blocks for user-created resources, the import will fail because the resources do not exist in the new environment.

If terraform plan or terraform apply returns an already exists error for a resource, add an import block for that resource in import.tf and retry.

The following table lists common default resources. It is not a complete list because default resources vary by Cribl versions, deployment types (Cribl.Cloud or on-prem), and the Cribl products that are in use.

If you use the --flat option to omit categorization by Worker Group, Edge Node, or Cribl Search in the output directory structure, the module addresses differ. Follow this example as a template for import.tf when migrating to a new Cribl environment:

# Import blocks for default resources that exist in every environment.
# Include only import blocks for default resources that match module addresses in main.tf.
# Module addresses follow the layout: <group_id>_<type> (for example, default_group, default_routes).
# Add import blocks for additional resources if terraform plan fails with "already exists" error.
import {
  to = module.default_group.criblio_group.default
  id = "default"
}
import {
  to = module.default_fleet_group.criblio_group.default_fleet
  id = "default_fleet"
}
import {
  to = module.default_routes.criblio_routes.default
  id = "default"
}
import {
  to = module.default_search_routes.criblio_routes.default_search
  id = "default_search"
}
import {
  to = module.default_fleet_routes.criblio_routes.default_fleet
  id = "default_fleet"
}

# Import blocks for default resources that exist in every environment.
# Include only import blocks for default resources that match module addresses in main.tf.
# Module addresses follow the layout: <group_id>_<type> (for example, default_group, default_routes).
# Add import blocks for additional resources if terraform plan fails with "already exists" error.
import {
  to = module.group.criblio_group.group_default
  id = "default"
}
import {
  to = module.group.criblio_group.group_default_fleet
  id = "default_fleet"
}
import {
  to = module.routes.criblio_routes.routes_default_default
  id = "default"
}
import {
  to = module.routes.criblio_routes.routes_default_search_default_search
  id = "default_search"
}
import {
  to = module.routes.criblio_routes.routes_default_fleet_default_fleet
  id = "default_fleet"
}

Excluded Resources

​

cribl-tf-config-exporter does not export the following resource types:

• criblio_commit

• criblio_deploy

• criblio_group_system_settings

• criblio_key

• criblio_lakehouse_dataset_connection

• criblio_lookup_file

• criblio_mapping_ruleset

• criblio_notification

• criblio_pack_lookups

• criblio_search_usage_group

• criblio_workspace

Exports include only user-created resources. cribl-tf-config-exporter does not export the following built-in and default resources:

Enterprise Resource Support

​

If the Cribl deployment does not have an Enterprise license, cribl-tf-config-exporter skips all Enterprise resources.

Resource Support for On-Prem Deployments

​

In on-prem deployments, cribl-tf-config-exporter skips the following resource types because they are available only in Cribl.Cloud:

• Search resources (criblio_search_*)

• Lake resources (criblio_cribl_lake_*)

• Lakehouse resources

In on-prem deployments, errors for these types are expected and do not cause the export to fail.