| --- |
| id: functions-worker |
| title: Deploy and manage functions worker |
| sidebar_label: "Functions Worker" |
| original_id: functions-worker |
| --- |
| |
| Pulsar `functions-worker` is a logic component to run Pulsar Functions in cluster mode. Two options are available, and you can select either of the two options based on your requirements. |
| - [run with brokers](#run-Functions-worker-with-brokers) |
| - [run it separately](#run-Functions-worker-separately) in a different broker |
| |
| :::note |
| |
| The `--- Service Urls---` lines in the following diagrams represent Pulsar service URLs that Pulsar client and admin use to connect to a Pulsar cluster. |
| |
| ::: |
| |
| ## Run Functions-worker with brokers |
| |
| The following diagram illustrates the deployment of functions-workers running along with brokers. |
| |
|  |
| |
| To enable functions-worker running as part of a broker, you need to set `functionsWorkerEnabled` to `true` in the `broker.conf` file. |
| |
| ```conf |
| |
| functionsWorkerEnabled=true |
| |
| ``` |
| |
| When you set `functionsWorkerEnabled` to `true`, it means that you start functions-worker as part of a broker. You need to configure the `conf/functions_worker.yml` file to customize your functions_worker. |
| |
| Before you run Functions-work with broker, you have to configure Functions-worker, and then start it with brokers. |
| |
| ### Configure Functions-Worker to run with brokers |
| In this mode, since `functions-worker` is running as part of broker, most of the settings already inherit from your broker configuration (for example, configurationStore settings, authentication settings, and so on). |
| |
| Pay attention to the following required settings when configuring functions-worker in this mode. |
| |
| - `numFunctionPackageReplicas`: The number of replicas to store function packages. The default value is `1`, which is good for standalone deployment. For production deployment, to ensure high availability, set it to be more than `2` . |
| - `pulsarFunctionsCluster`: Set the value to your Pulsar cluster name (same as the `clusterName` setting in the broker configuration). |
| |
| If authentication is enabled on the BookKeeper cluster, configure the following BookKeeper authentication settings. |
| |
| - `bookkeeperClientAuthenticationPlugin`: the BookKeeper client authentication plugin name. |
| - `bookkeeperClientAuthenticationParametersName`: the BookKeeper client authentication plugin parameters name. |
| - `bookkeeperClientAuthenticationParameters`: the BookKeeper client authentication plugin parameters. |
| |
| ### Start Functions-worker with broker |
| |
| Once you have configured the `functions_worker.yml` file, you can start or restart your broker. |
| |
| And then you can use the following command to verify if `functions-worker` is running well. |
| |
| ```bash |
| |
| curl <broker-ip>:8080/admin/v2/worker/cluster |
| |
| ``` |
| |
| After entering the command above, a list of active function workers in the cluster is returned. The output is something similar as follows. |
| |
| ```json |
| |
| [{"workerId":"<worker-id>","workerHostname":"<worker-hostname>","port":8080}] |
| |
| ``` |
| |
| ## Run Functions-worker separately |
| |
| This section illustrates how to run `functions-worker` as a separate process in separate machines. |
| |
|  |
| |
| > Note |
| In this mode, make sure `functionsWorkerEnabled` is set to `false`, so you won't start `functions-worker` with brokers by mistake. |
| |
| ### Configure Functions-worker to run separately |
| |
| To run function-worker separately, you have to configure the following parameters. |
| |
| #### Worker parameters |
| |
| - `workerId`: The type is string. It is unique across clusters, used to identify a worker machine. |
| - `workerHostname`: The hostname of the worker machine. |
| - `workerPort`: The port that the worker server listens on. Keep it as default if you don't customize it. |
| - `workerPortTls`: The TLS port that the worker server listens on. Keep it as default if you don't customize it. |
| |
| #### Function package parameter |
| |
| - `numFunctionPackageReplicas`: The number of replicas to store function packages. The default value is `1`. |
| |
| #### Function metadata parameter |
| |
| - `pulsarServiceUrl`: The Pulsar service URL for your broker cluster. |
| - `pulsarWebServiceUrl`: The Pulsar web service URL for your broker cluster. |
| - `pulsarFunctionsCluster`: Set the value to your Pulsar cluster name (same as the `clusterName` setting in the broker configuration). |
| |
| If authentication is enabled for your broker cluster, you *should* configure the authentication plugin and parameters for the functions worker to communicate with the brokers. |
| |
| - `clientAuthenticationPlugin` |
| - `clientAuthenticationParameters` |
| |
| #### Security settings |
| |
| If you want to enable security on functions workers, you *should*: |
| - [Enable TLS transport encryption](#enable-tls-transport-encryption) |
| - [Enable Authentication Provider](#enable-authentication-provider) |
| - [Enable Authorization Provider](#enable-authorization-provider) |
| |
| **Enable TLS transport encryption** |
| |
| To enable TLS transport encryption, configure the following settings. |
| |
| ``` |
| |
| tlsEnabled: true |
| tlsCertificateFilePath: /path/to/functions-worker.cert.pem |
| tlsKeyFilePath: /path/to/functions-worker.key-pk8.pem |
| tlsTrustCertsFilePath: /path/to/ca.cert.pem |
| |
| ``` |
| |
| For details on TLS encryption, refer to [Transport Encryption using TLS](security-tls-transport.md). |
| |
| **Enable Authentication Provider** |
| |
| To enable authentication on Functions Worker, configure the following settings. |
| > Note |
| Substitute the *providers list* with the providers you want to enable. |
| |
| ``` |
| |
| authenticationEnabled: true |
| authenticationProviders: [ provider1, provider2 ] |
| |
| ``` |
| |
| For *SASL Authentication* provider, add `saslJaasClientAllowedIds` and `saslJaasBrokerSectionName` |
| under `properties` if needed. |
| |
| ``` |
| |
| properties: |
| saslJaasClientAllowedIds: .*pulsar.* |
| saslJaasBrokerSectionName: Broker |
| |
| ``` |
| |
| For *Token Authentication* prodivder, add necessary settings under `properties` if needed. |
| See [Token Authentication](security-token-admin.md) for more details. |
| |
| ``` |
| |
| properties: |
| tokenSecretKey: file://my/secret.key |
| # If using public/private |
| # tokenPublicKey: file:///path/to/public.key |
| |
| ``` |
| |
| **Enable Authorization Provider** |
| |
| To enable authorization on Functions Worker, you need to configure `authorizationEnabled` and `configurationStoreServers`. The authentication provider connects to `configurationStoreServers` to receive namespace policies. |
| |
| ```yaml |
| |
| authorizationEnabled: true |
| configurationStoreServers: <configuration-store-servers> |
| |
| ``` |
| |
| You should also configure a list of superuser roles. The superuser roles are able to access any admin API. The following is a configuration example. |
| |
| ```yaml |
| |
| superUserRoles: |
| - role1 |
| - role2 |
| - role3 |
| |
| ``` |
| |
| #### BookKeeper Authentication |
| |
| If authentication is enabled on the BookKeeper cluster, you should configure the BookKeeper authentication settings as follows: |
| |
| - `bookkeeperClientAuthenticationPlugin`: the plugin name of BookKeeper client authentication. |
| - `bookkeeperClientAuthenticationParametersName`: the plugin parameters name of BookKeeper client authentication. |
| - `bookkeeperClientAuthenticationParameters`: the plugin parameters of BookKeeper client authentication. |
| |
| ### Start Functions-worker |
| |
| Once you have finished configuring the `functions_worker.yml` configuration file, you can use the following command to start a `functions-worker`: |
| |
| ```bash |
| |
| bin/pulsar functions-worker |
| |
| ``` |
| |
| ### Configure Proxies for Functions-workers |
| |
| When you are running `functions-worker` in a separate cluster, the admin rest endpoints are split into two clusters. `functions`, `function-worker`, `source` and `sink` endpoints are now served |
| by the `functions-worker` cluster, while all the other remaining endpoints are served by the broker cluster. |
| Hence you need to configure your `pulsar-admin` to use the right service URL accordingly. |
| |
| In order to address this inconvenience, you can start a proxy cluster for routing the admin rest requests accordingly. Hence you will have one central entry point for your admin service. |
| |
| If you already have a proxy cluster, continue reading. If you haven't setup a proxy cluster before, you can follow the [instructions](http://pulsar.apache.org/docs/en/administration-proxy/) to |
| start proxies. |
| |
|  |
| |
| To enable routing functions related admin requests to `functions-worker` in a proxy, you can edit the `proxy.conf` file to modify the following settings: |
| |
| ```conf |
| |
| functionWorkerWebServiceURL=<pulsar-functions-worker-web-service-url> |
| functionWorkerWebServiceURLTLS=<pulsar-functions-worker-web-service-url> |
| |
| ``` |
| |
| ## Compare the Run-with-Broker and Run-separately modes |
| |
| As described above, you can run Function-worker with brokers, or run it separately. And it is more convenient to run functions-workers along with brokers. However, running functions-workers in a separate cluster provides better resource isolation for running functions in `Process` or `Thread` mode. |
| |
| Use which mode for your cases, refer to the following guidelines to determine. |
| |
| Use the `Run-with-Broker` mode in the following cases: |
| - a) if resource isolation is not required when running functions in `Process` or `Thread` mode; |
| - b) if you configure the functions-worker to run functions on Kubernetes (where the resource isolation problem is addressed by Kubernetes). |
| |
| Use the `Run-separately` mode in the following cases: |
| - a) you don't have a Kubernetes cluster; |
| - b) if you want to run functions and brokers separately. |
| |
| ## Troubleshooting |
| |
| **Error message: Namespace missing local cluster name in clusters list** |
| |
| ``` |
| |
| Failed to get partitioned topic metadata: org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: Namespace missing local cluster name in clusters list: local_cluster=xyz ns=public/functions clusters=[standalone] |
| |
| ``` |
| |
| The error message prompts when either of the cases occurs: |
| - a) a broker is started with `functionsWorkerEnabled=true`, but the `pulsarFunctionsCluster` is not set to the correct cluster in the `conf/functions_worker.yaml` file; |
| - b) setting up a geo-replicated Pulsar cluster with `functionsWorkerEnabled=true`, while brokers in one cluster run well, brokers in the other cluster do not work well. |
| |
| **Workaround** |
| |
| If any of these cases happens, follow the instructions below to fix the problem: |
| |
| 1. Get the current clusters list of `public/functions` namespace. |
| |
| ```bash |
| |
| bin/pulsar-admin namespaces get-clusters public/functions |
| |
| ``` |
| |
| 2. Check if the cluster is in the clusters list. If the cluster is not in the list, add it to the list and update the clusters list. |
| |
| ```bash |
| |
| bin/pulsar-admin namespaces set-clusters --cluster=<existing-clusters>,<new-cluster> public/functions |
| |
| ``` |
| |
| 3. Set the correct cluster name in `pulsarFunctionsCluster` in the `conf/functions_worker.yml` file. |
