blob: ba96681a3acaad706891cde05af3ba6c61af7daa [file]
.. ################################################################################
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
################################################################################
========
AI / LLM
========
Model Providers
---------------
Model providers configure connections and options for AI functions.
``OpenAIProvider`` and ``TritonProvider`` provide typed configurations for Flink's
built-in providers. See their API references below for parameters and defaults.
Use ``options`` to supply Java provider option names and string values directly.
Use ``set_model_provider`` to register a configuration. Pass a provider alone to
use its identifier as the name, or supply a name to distinguish configurations.
Use ``list_model_providers`` to inspect registered names:
.. code-block:: python
import pyflink.dataframe as pf
provider = pf.TritonProvider(endpoint="http://localhost:8000")
pf.set_model_provider("inference", provider)
pf.list_model_providers() # ["inference"]
A single registered provider is selected automatically. When using several
providers, choose a default with ``set_default_model_provider("inference")``.
Registrations are shared within the Python process.
For other providers available in the application's classpath, use ``GenericProvider``
with provider-specific option names and string values, or subclass ``ModelProvider``
to create a reusable configuration.
Prediction
----------
Use ``df.llm.predict`` to infer a result for each row and append the output columns.
Supply one or more column names in model input order. The provider determines which
input types and number of columns it supports.
.. code-block:: python
pf.set_model_provider("chat", pf.OpenAIProvider(
endpoint="https://api.openai.com/v1/chat/completions", api_key="key", retry_num=3))
questions = pf.from_records([{"question": "What is Apache Flink?"}])
answers = questions.llm.predict(
"question", provider="chat", model="gpt-6-luna")
The default output is one ``STRING`` column named ``output``. Set ``output_type``
to a SQL type string or a ``DataType`` to declare the provider's native output schema.
A ``DataType.struct(...)`` expands its top-level fields into named output columns.
Output field names must be unique and must not already occur in the input DataFrame.
The provider validates the output schema it supports.
.. code-block:: python
# Name the prediction column "answer" instead of the default "output".
answers = questions.llm.predict(
"question", provider="chat", model="gpt-6-luna",
output_type=pf.DataType.struct({"answer": pf.DataType.string()}))
You can pass in extra arguments to override the selected provider's constructor settings:
.. code-block:: python
# Override the provider's retry_num parameter for this call.
answers = questions.llm.predict(
"question", provider="chat", model="gpt-6-luna", retry_num=0)
.. currentmodule:: pyflink.dataframe
.. autosummary::
:toctree: api/
ModelProvider
OpenAIProvider
TritonProvider
GenericProvider
set_model_provider
set_default_model_provider
list_model_providers
.. automethod:: pyflink.dataframe.llm.LLMAccessor.predict