{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Introduction to StreamPipes Python\n",
    "\n",
    "### Why there is an extra Python library for StreamPipes?\n",
    "[Apache StreamPipes](https://streampipes.apache.org/) aims to enable non-technical users to connect and analyze IoT data streams.\n",
    "To this end, it provides an easy-to-use and convenient user interface that allows one to connect to an IoT data source and create some visual\n",
    "graphs within a few minutes. <br>\n",
    "Although this is the main use case of Apache StreamPipes, it can also provide great value for people who are eager to work on data analysis or data science with IoT data, but don't we do get in touch with all the hassle associated with extracting data from devices in a suitable format.\n",
    "In this scenario, StreamPipes helps you connect to your data source and extract the data for you.\n",
    "You then can make the data available outside StreamPipes by writing it into an external source, such as a database, Kafka, etc.\n",
    "While this requires another component, you can also extract your data directly from StreamPipes programmatically using the StreamPipes API.\n",
    "For convenience, we also provide you with a StreamPipes client both available for Java and Python.\n",
    "Specifically with StreamPipes Python, we want to address the amazing data analytics and data science community in Python and benefit from the great universe of Python libraries out there.\n",
    "\n",
    "<br>\n",
    "\n",
    "### How to install StreamPipes Python?\n",
    "Simply use the following `pip` command:"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "%pip install streampipes\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "If you want to have the current development state you can also execute:"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "\n",
    "%pip install git+https://github.com/apache/streampipes.git#subdirectory=streampipes-client-python"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "The corresponding documentation can be found [here](\n",
    "https://streampipes.apache.org/docs/docs/python/dev/).\n",
    "<br>\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### How to prepare the tutorials\n",
    "In case you want to reproduce the first two tutorials exactly on your end, you need to create a simple pipeline in StreamPipes like demonstrated below.\n",
    "\n",
    "![tutorial-preparation](https://raw.githubusercontent.com/apache/streampipes/dev/streampipes-client-python/docs/img/tutorial-preparation.gif)\n",
    "\n",
    "<br>"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### How to configure the Python client\n",
    "In order to access the resources available in StreamPipes, one must be able to authenticate against the backend.\n",
    "For this purpose, the client sofar only supports the authentication via an API token that can be generated via the StreamPipes UI, as you see below.\n",
    "\n",
    "![how-to-get-api-key](https://raw.githubusercontent.com/apache/streampipes/dev/streampipes-client-python/docs/img/how-to-get-api-key.gif)\n",
    "\n",
    "<br>\n",
    "\n",
    "Having generated the API token, one can directly start initializing a client instance as follows:"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "from streampipes.client import StreamPipesClient\n",
    "from streampipes.client.config import StreamPipesClientConfig\n",
    "from streampipes.client.credential_provider import StreamPipesApiKeyCredentials"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "config = StreamPipesClientConfig(\n",
    "    credential_provider=StreamPipesApiKeyCredentials(\n",
    "        username=\"test@streampipes.apache.org\",\n",
    "        api_key=\"API-KEY\",\n",
    "    ),\n",
    "    host_address=\"localhost\",\n",
    "    https_disabled=True,\n",
    "    port=80\n",
    ")"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Please be aware that connecting to StreamPipes via a `https` connection is currently not supported by the Python client.\n",
    "\n",
    "Providing secrets like the `api_key` as plaintext in the source code is an anti-pattern.\n",
    "This is why the StreamPipes client also supports passing the required secrets as environment variables.\n",
    "To do so, you must initialize the credential provider like the following:"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "StreamPipesApiKeyCredentials.from_env(username_env=\"USER\", api_key_env=\"API-KEY\")"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Please note that you pass the names of the environment variables.\n",
    "To ensure that the above code works, you must set the environment variables with the same name you specified in `from_env`.\n",
    "In this scenario this would look like the following:"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "%export USER=\"<USERNAME>\"\n",
    "%export API_KEY=\"<API-KEY>\""
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Having the `config` ready, we can now initialize the actual client."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "client = StreamPipesClient(client_config=config)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "That's already it. You can check if everything works out by using the following command:"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2023-02-24 17:05:49,398 - streampipes.endpoint.endpoint - [INFO] - [endpoint.py:167] [_make_request] - Successfully retrieved all resources.\n",
      "2023-02-24 17:05:49,457 - streampipes.endpoint.endpoint - [INFO] - [endpoint.py:167] [_make_request] - Successfully retrieved all resources.\n",
      "\n",
      "Hi there!\n",
      "You are connected to a StreamPipes instance running at http://localhost:80.\n",
      "The following StreamPipes resources are available with this client:\n",
      "1x DataLakeMeasures\n",
      "1x DataStreams\n"
     ]
    }
   ],
   "source": [
    "client.describe()"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "This prints you a short textual description of the connected StreamPipes instance to the console.\n",
    "\n",
    "<br>\n",
    "\n",
    "The created `client` instance serves as the central point of interaction with StreamPipes.\n",
    "You can invoke a variety of commands directly on this object.\n",
    "\n",
    "Are you curious now how you actually can get data out of StreamPipes and make use of it with Python?\n",
    "Then check out the next tutorial on [extracting Data from the StreamPipes data lake](../2-extracting-data-from-the-streampipes-data-lake).\n",
    "\n",
    "<br>\n",
    "\n",
    "Thanks for reading this introductory tutorial.\n",
    "We hope you like it and would love to receive some feedback from you.\n",
    "Just go to our [GitHub discussion page](https://github.com/apache/streampipes/discussions) and let us know your impression.\n",
    "We'll read and react to them all, we promise!"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [],
   "metadata": {
    "collapsed": false
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  },
  "license": "https://www.apache.org/licenses/LICENSE-2.0"
 },
 "nbformat": 4,
 "nbformat_minor": 0
}