Google Git
Sign in
apache / paimon-python / 107ab37470380ab905d9819472134c0b4d5060bb
commit107ab37470380ab905d9819472134c0b4d5060bb[log] [tgz]
authoryuzelin <33053040+yuzelin@users.noreply.github.com>Fri Aug 16 11:55:28 2024 +0800
committerGitHub <noreply@github.com>Fri Aug 16 11:55:28 2024 +0800
tree087912938bea03035ca660486d0404462bfb7c9e
parent8e079857424ce9e652ac2e6e021211d3fd0111f3 [diff]
Add base implementation framework (#3)
23 files changed
tree: 087912938bea03035ca660486d0404462bfb7c9e
  1. .github/
  2. dev/
  3. java_based_implementation/
  4. paimon_python_api/
  5. .asf.yaml
  6. .gitignore
  7. copyright.txt
  8. LICENSE
  9. NOTICE
  10. README.md
  11. setup.py
  12. tox.ini
README.md

Paimon

License

This repo is for Apache Paimon Python SDK.

Development Notice

Checkstyle

Flake8 is used to enforce some coding guidelines.

  1. Install flake8 for your Python interpreter using pip install flake8.
  2. In PyCharm go to “Settings” → “Tools” → “External Tools”.
  3. Select the “+” button to add a new external tool.
  4. Set “Name” to “flake8”.
  5. Set “Description” to “Code Style Check”.
  6. Set “Program” to the path of your Python interpreter, e.g. /usr/bin/python.
  7. Set “Arguments” to -m flake8 --config=tox.ini.
  8. Set “Working Directory” to $ProjectFileDir$.

You can verify the setup by right-clicking on any file or folder in the flink-python project and running “External Tools” → “flake8”.

Usage

Java-Based Implementation

We can use py4j to leverage Java code to read Paimon data. This section describes how to use this implementation.

Build paimon-python-java-bridge

cd java_based_implementation/paimon-python-java-bridge/
mvn clean install -DskipTests

The built target is java-based-implementation/paimon-python-java-bridge/target/paimon-python-java-bridge-.jar

Set Environment Variables

py4j need to access a JVM, so we should set JVM arguments (optional) and Java classpath. A convenient way is using os packages to set environment variables which only affect current process.

import os

os.environ['PYPAIMON_JAVA_CLASSPATH'] = '/path/to/paimon-python-java-bridge-<version>.jar'
os.environ['_PYPAIMON_JVM_ARGS'] = 'jvm_arg1 jvm_arg2 ...'

API Reference

TODO