jupytext: text_representation: extension: .md format_name: myst kernelspec: name: python3 display_name: Python 3

Basic Operations

In this section, you will learn how to display essential details of DataFrames using specific functions.

from datafusion import SessionContext
import random

ctx = SessionContext()
df = ctx.from_pydict({
    "nrs": [1, 2, 3, 4, 5],
    "names": ["python", "ruby", "java", "haskell", "go"],
    "random": random.sample(range(1000), 5),
    "groups": ["A", "A", "B", "C", "B"],
})
df

Use {py:func}~datafusion.dataframe.DataFrame.limit to view the top rows of the frame:

df.limit(2)

Display the columns of the DataFrame using {py:func}~datafusion.dataframe.DataFrame.schema:

df.schema()

The method {py:func}~datafusion.dataframe.DataFrame.to_pandas uses pyarrow to convert to pandas DataFrame, by collecting the batches, passing them to an Arrow table, and then converting them to a pandas DataFrame.

df.to_pandas()

{py:func}~datafusion.dataframe.DataFrame.describe shows a quick statistic summary of your data:

df.describe()