blob: 8717fa7228505643d2eea20e0d46e22e2ccd4015 [file] [log] [blame] [view]
# Apache OpenDALâ„¢ Python binding
## Installation
```bash
pip install opendal
```
## Local Usage
Developer must set two required arguments to work with files locally:
- `scheme`: which should be specified as `fs`
- `root`: where OpenDAl considers the root of the directory for operations will be.
For example in the following operator:
`opendal.Operator("fs", root="/foo")`
OpenDAL considers `/foo` to be the root of all paths, and that means that we can access paths inside of `/foo` without specifying anything else.
If `/foo` happens to contain the file `baz.txt`, we can simply call `.list("/")`
We can see this in the following example:
```python
from opendal import Operator
from pathlib import Path
import os
os.makedirs("foo", exist_ok=True)
file = Path("foo", "bar.txt")
file.touch()
op = Operator("fs", root=Path("foo").name)
for file in op.list("/"):
print(file)
```
When running we get the following output:
```
/
bar.txt
```
If we want full access to our file system, we can specify a root of `"/"`, but note that for any operations you will always need to specify the full path to a file or directory.
### Reading
There are two ways to read data using OpenDAL. One way is to use the `read` method on an operator:
```python
from opendal import Operator
from pathlib import Path
import os
os.makedirs("foo", exist_ok=True)
file = Path("foo", "bar.txt")
file.touch()
file.write_text("baz")
op = Operator("fs", root=Path("foo").name)
data = op.read(file.name)
print(data)
```
Yields the following:
```
b'baz'
```
Note that the output is bytes, but we can get it as a string by simply calling `.decode()` on the data we read. All reads with OpenDAL return bytes.
Now lets use the `open` method on an operator:
```python
from opendal import Operator
from pathlib import Path
import os
os.makedirs("foo", exist_ok=True)
file = Path("foo", "bar.txt")
file.touch()
file.write_text("baz")
op = Operator("fs", root=Path("foo").name)
with op.open(file.name, "rb") as f:
print(f.read())
```
This again yields
```
b'baz'
```
Again, note that all reads with OpenDAL return bytes, so specifying a mode of `"r"` will raise an exception.
### Writing
Now that we know how to read data, let's replace the `Pathlib` code above with OpenDAL using the `write` method:
```python
from opendal import Operator
from pathlib import Path
op = Operator("fs", root=Path("foo").name)
op.write("baz.txt", "my amazing data".encode())
print(op.read("baz.txt").decode())
```
This yields the following:
```
my amazing data
```
And again, but using the `open` method:
```python
from opendal import Operator
from pathlib import Path
op = Operator("fs", root=Path("foo").name)
with op.open("baz.txt", "wb") as f:
f.write("my amazing data".encode())
print(op.read("baz.txt").decode())
```
This again yields:
```
my amazing data
```
Again, note that all writing happens in bytes and a mode of `"w"` will raise an exception.
## Async
OpenDAL supports async operation on all operator methods. One can simply replace the `Operator` with an `AsyncOperator` and await on method calls. The below example illustrates this behavior:
Standard API:
```python
import opendal
op = opendal.Operator("fs", root="/tmp")
op.write("test.txt", b"Hello World")
print(op.read("test.txt"))
print(op.stat("test.txt").content_length)
```
Async API equivalent:
```python
import asyncio
async def main():
op = opendal.AsyncOperator("fs", root="/tmp")
await op.write("test.txt", b"Hello World")
print(await op.read("test.txt"))
asyncio.run(main())
```