Configure a Self-Hosted Agent

Configuration Methods

Configuring an agent can be done in two ways:

• Via the command line.

The command must be in the following format:

For Windows Grapes.Agent.exe config --key1 value1 --key2 value2

For Linux

dotnet Grapes.Agent.dll config --key1 value1 --key2 value2

Note: The command line does not allow managing all configuration options.

• Via the configuration file.

The configuration file is editable using a simple text editor. It is in JSON format.

The file is called config.json and is located in the “config” directory of the installation.

Note: The agent must be restarted after a configuration file modification.

Configuration Options

Data Storage

Grapes Managed Storage

"DatasetStorage": {
  "Mode": "GrapesStorage",
  "Configuration": {
    "TempFolder": "../_temp"
  }
}

This is the default storage mode.

File System Storage

"DatasetStorage": {
  "Mode": "FileSystem",
  "Configuration": {
    "TempFolder": "../_temp"
  },
  "FileSystem": {
    "DataFolder": "../data"
  }
}

Storage in Azure Storage Account

"DatasetStorage": {
  "Mode": "AzureStorage",
  "Configuration": {
    "TempFolder": "../_temp"
  },
  "AzureStorage": {
    "StorageConnectionString": "{{Your connection string here}}}",
    "ContainerName": "{{Data container name}}"
  }
}

If you need a different storage mode, feel free to contact support

Logs

The agent logs are based on the [Serilog] (https://github.com/serilog/serilog) library.

By default, two outputs are defined at the log level:

• Console
• File (in the logs directory)

You can change this configuration by editing the config.json file and adding this section:

"Serilog": {
  "MinimumLevel": {
    "Default": "Info",
    "Override": {
      "Microsoft": "Warning"
    }
  },
  "WriteTo": [
    {
      "Name": "File",
      "Args": {
        "path": "../logs/log-.txt",
        "rollingInterval": "Day",
        "restrictedToMinimumLevel": "Debug"
      }
    },
    {
      "Name": "Console",
      "Args": {
        "theme": "Serilog.Sinks.SystemConsole.Themes.AnsiConsoleTheme::Code, Serilog.Sinks.Console",
        "outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj} {NewLine}{Exception}",
        "restrictedToMinimumLevel": "Information"
      }
    }
  ]
}

Note: If you need to use other types of log outputs, feel free to contact support

Parallelization

Starting from version 1.07

By default, the number of simultaneous operations executed by an agent is not limited. The agent uses the number of threads provided by the underlying scheduler.

An operation refers to tasks like exporting a table. Indeed, the grapes agent can execute only one task at a time (e.g., exporting a dataset), but it can parallelize the execution of certain sub-operations.

However, it is possible to control the degree of parallelization of the agent’s operations.

"Workers": {
  "DefaultMaxDegreeOfParallelism": -1
}

The parameter value can be:

• -1: This is the default value.
• >0: A fixed number of simultaneous operations