Skip to main content

Temporal Server security

Work in progress

This guide is a work in progress. Some sections may be incomplete. Information may change at any time.

Overview​

A secured Temporal server has its network communication encrypted and has authentication and authorization protocols set up for API calls made to it. Without these, your server could be accessed by unwanted entities.

What is documented on this page are the built-in opt-in security measures that come with Temporal. However users may also choose to design their own security architecture with reverse proxies or run unsecured instances inside a VPC environment.

Server Samples​

The https://github.com/temporalio/samples-server repo offers two examples, which are further explained below:

  • TLS: how to configure Transport Layer Security (TLS) to secure network communication with and within a Temporal cluster.
  • Authorizer: how to inject a low-level authorizer component that can control access to all API calls.

Encryption in transit with mTLS​

Temporal supports Mutual Transport Layer Security (mTLS) as a way of encrypting network traffic between the services of a cluster and also between application processes and a Cluster. Self-signed or properly minted certificates can be used for mTLS. mTLS is set in Temporal's TLS configuration. The configuration includes two sections such that intra-Cluster and external traffic can be encrypted with different sets of certificates and settings:

  • internode: Configuration for encrypting communication between nodes in the cluster.
  • frontend: Configuration for encrypting the Frontend's public endpoints.

A customized configuration can be passed using either the WithConfig or WithConfigLoader Server options.

See TLS configuration reference for more details.

Encryption at rest with Data Converter​

A Data Converter is a Temporal SDK component that encodes and decodes data entering and exiting a Temporal Server.

Data Converter encodes and decodes data

Data is encoded before it is sent to a Temporal Server, and it is decoded when it is received from a Temporal Server.

The main pieces of data that run through the Data Converter are arguments and return values:

  • The Client:
    • Encodes Workflow, Signal, and Query arguments.
    • Decodes Workflow and Query return values.
  • The Worker:
    • Decodes Workflow, Signal, and Query arguments.
    • Encodes Workflow and Query return values.
    • Decodes and encodes Activity arguments and return values.

Each piece of data (like a single argument or return value) is encoded as a Payload Protobuf message, which consists of binary data and key-value metadata.

Default Data Converter​

Each Temporal SDK includes a default Data Converter. In most SDKs, the default converter supports binary, JSON, and Protobufs. (In SDKs that cannot determine parameter types at runtime—like TypeScript—Protobufs aren't included in the default converter.) It tries to encode values in the following order:

  • Null
  • Binary
  • Protobuf JSON
  • JSON

For example:

  • If a value is an instance of a Protobuf message, it will be encoded with proto3 JSON.
  • If a value isn't null, binary, or a Protobuf, it will be encoded as JSON. If any part of it is not serializable as JSON (for example, a Date—see JSON data types), an error will be thrown.

The default converter also supports decoding binary Protobufs.

Custom Data Converter​

Applications can create their own custom Data Converters to alter the format (for example using MessagePack instead of JSON) or add compression or encryption.

To use a custom Data Converter, provide it in the following contexts:

Custom Data Converters are not applied to all data:

Payload Codecs​

In TypeScript, Go, and Python, data conversion happens in two stages:

  1. A Payload Converter converts a value into a Payload.
  2. A Payload Codec transforms an array of Payloads (for example, a list of Workflow arguments) into another array of Payloads.

The Payload Codec is an optional step that happens between the wire and the Payload Converter:

Temporal Server <--> Wire <--> Payload Codec <--> Payload Converter <--> User code

Common Payload Codec transformations are compression and encryption.

In codec implementations, we recommended running the function (whether it be compressing, encrypting, etc) on the entire input Payload, and putting the result in a new Payload's data field. That way, the input Payload's headers are preserved. See, for example:

Encryption​

Doing encryption in a custom Data Converter ensures that all application data is encrypted during the following actions:

  • Being sent to/from Temporal Server.
  • Moving inside Temporal Server.
  • Stored by Temporal Server.

Then data exists unencrypted in memory only on the Client and in the Worker Process that is executing Workflows and Activities on hosts that the application developer controls.

Our encryption samples use AES GCM with 256-bit keys:

Codec Server​

A Codec Server is an HTTP server that runs data from tctl or the Web UI through a Payload Codec.

By default, tctl and the Web UI use the Default Data Converter without a Payload Codec. If you use a Payload Codec with your SDK, you may not be able to understand the Payload data displayed in the Web UI/tctl (for example, it may be encrypted or compressed). In order to convert the data to its original format, you can configure the Web UI/tctl to use a Codec Server that uses your Payload Codec.

Use case: tctl​

Suppose that you want to view Workflow History. This information needs to be decoded before it can be viewed.

You can use tctl workflow showid to view a Workflow Execution Event History.

tctl workflow showid <workflowID>

With a Codec Server, Payloads that are part of the Event History will be sent to the Codec Server to be decoded before being deserialized by the Default Data Converter and displayed in your terminal.

Use case: Web UI​

Workflow Execution Event History is available in the Web UI. Payload information for each Event is captured within Event 'input' and 'result' fields. Without a Codec Server, this information remains encoded.

Passing these Payloads through a Codec Server returns decoded results to the Web UI.

Codec Server setup​

To use a Codec Server, first run it with your Payload Codec and then configure tctl and the Web UI to use it.

Run the server​

A Codec Server is an HTTP server that implements two endpoints:

  • POST /encode
  • POST /decode

Each endpoint receives and responds with a JSON body that has a payloads property with an array of Payloads. The endpoints run the Payloads through a Payload Codec before returning them.

Sample Codec Servers:

Configure tctl​

Once the Codec Server is started, for example on http://localhost:8888, provide it to tctl using the --codec_endpoint global option:

tctl --codec_endpoint 'http://localhost:8888' workflow show --wid workflow-id-123

Configure the Web UI​

Once the Codec Server is started, there are two ways to provide it to the Web UI:

In the UI​

Data Encoder icon

Select the icon with an up-down arrow on the bottom left of the screen. This action displays the codec endpoint dialog.

Enter the URL and port number for your codec endpoint. Exit the dialog, go back to the previous page, and refresh the page.

The button should now be light blue, and your Payloads should be displayed in a readable format.

In the config file​

The codec endpoint can be specified in the configuration file:

codec:
    endpoint: {{ default .Env.TEMPORAL_CODEC_ENDPOINT "{namespace}"}}

Authentication​

There are a few authentication protocols available to prevent unwanted access such as authentication of servers, clients, and users.

Servers​

To prevent spoofing and MITM attacks you can specify the serverName in the client section of your respective mTLS configuration. This enables established connections to authenticate the endpoint, ensuring that the server certificate presented to any connecting Client has the appropriate server name in its CN property. It can be used for both internode and frontend endpoints.

More guidance on mTLS setup can be found in the samples-server repo, and you can reach out to us for further guidance.

Client connections​

To restrict a client's network access to cluster endpoints you can limit it to clients with certificates issued by a specific Certificate Authority (CA). Use the clientCAFiles/ clientCAData and requireClientAuth properties in both the internode and frontend sections of the mTLS configuration.

Users​

To restrict access to specific users, authentication and authorization is performed through extensibility points and plugins as described in the Authorization section below.

Authorization​

note

Information regarding Authorizer and ClaimMapper has been moved to another location.

Temporal offers two plugin interfaces for implementing API call authorization:

The authorization and claim mapping logic is customizable, making it available to a variety of use cases and identity schemes. When these are provided the frontend invokes the implementation of these interfaces before executing the requested operation.

See https://github.com/temporalio/samples-server/blob/main/extensibility/authorizer for a sample implementation.

Single sign-on integration​

Temporal can be integrated with a single sign-on (SSO) experience by utilizing the ClaimMapper and Authorizer plugins. The default JWT ClaimMapper implementation can be used as is or as a base for a custom implementation of a similar plugin.

Temporal Web​

To enable SSO for the Temporal Web UI edit the web service's configuration per the Temporal Web README.