<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
-->

<body>
Apache Edgent is an SDK for developing and executing streaming analytics at the <i>edge</i>.
<P>
<em>
Apache Edgent is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
</em>
</P>
<H1>Edgent</H1>
<OL>
<LI><a href="#overview">Overview</A></LI>
<LI><a href="#model">Programming Model</A></LI>
<LI><a href="#start">Getting Started</A></LI>
</OL>
<a name="overview"></a>
<H2>Overview</H2>
Apache Edgent is an SDK for developing and executing streaming analytics at the <i>edge</i>.
Edgent provides a stream processing style programming model for 
composing stream processing graphs and a micro-services style runtime for
executing the graphs.  Some analytics and a rich collection
of connectors are provided. Users can easily develop and use their 
own analytics and connectors.  
A framework for testing processing graphs is also included.
<P>
Edgent is focusing on two edge cases:
<UL>
<LI>Internet of Things (IoT) - Widely distributed and/or mobile devices and gateways.</LI>
<LI>Enterprise Embedded - Edge analytics within an enterprise, such as local analytic applications of each system in a machine room, or error log analytics in application servers.</LI>
</UL>
In both cases Edgent applications analyze live data and
send results of that analytics and/or data intermittently
to back-end systems for deeper analysis. An Edgent application
can use analytics to decide when to send information to back-end systems,
such as when the behaviour of the system is outside normal parameters
(e.g. an engine running too hot).
<BR>
Edgent applications do not send data continually
to back-end systems as the cost of communication may be high
(e.g. cellular networks) or bandwidth may be limited.
<P>
Edgent applications communicate with back-end systems through
some form of message hub as there may be millions of edge devices.
Edgent supports these message hubs:
<UL>
<LI> MQTT - Messaging standard for IoT</LI>
<LI> IBM Watson IoT Platform - Cloud based service providing a device model on top of MQTT</LI>
<LI> Apache Kafka - Enterprise message bus</LI>
</UL> 
</P>
<P>
Back-end analytic systems are used to perform analysis on information from Edgent applications that cannot be performed at the edge. Such analysis may be:
<UL>
<LI>Running complex analytic algorithms than require more resources (cpu, memory etc.) than are available at the edge. </LI>
<LI>Maintaining more state per device that can exist at the edge, e.g. hours of state for patients' medical sensors. </LI>
<LI>Correlating device information with multiple data sources:
<UL>
<LI> Weather data</LI>
<LI> Social media data</LI>
<LI> Data of record (e.g patients' medical histories, trucking manifests).</LI>
<LI> Other devices </LI>
<LI>etc.</LI>
</UL>
</LI>
</UL>
<BR>
Back-end systems can interact or control devices based upon their analytics, by sending commands to specific devices, e.g. reduce maximum engine revs to reduce chance of failure before the next scheduled service, or send an alert of an accident ahead.
</P>
<a name="model"></a>
<H2>Programming Model</H2>
Edgent applications are streaming applications in which each <em>tuple</em>
(data item or event) in a <em>stream</em> of data is processed as it occurs.
Additionally, you can process <em>windows</em> (logical subsets) of data.
For example, you could analyze the last 90 seconds of data from a sensor to identify trends in the data
<P>
<H3>Topology functional API</H3>
<H4>Overview</H4>
The primary api is {@link Topology} which uses a functional
model to build a topology of {@link TStream streams} for an application.
<BR>
{@link TStream TStream} is a declaration of a stream of tuples, an application will create streams that source data (e.g. sensor readings) and then apply functions that transform those streams into derived streams, for example simply filtering a stream containg engine temperator readings to a derived stream that only contains readings thar are greater than 100&deg;C.
<BR>
An application terminates processing for a stream by <em>sinking</em> it. Sinking effectively terminates a stream by applying processing to each tuple on the stream (as it occurs) that does not produce a result. Typically this sinking is transmitting the tuple to an external system, for example the messgae hub to send the data to a back-end system, or locally sending the data to a user interface.
</P>
<P>
This programming style is typical for streaming systems and similar APIs are supported by systems such as Apache Flink, Apache Spark Streaming, IBM Streams and Java 8 streams.
</P>
<H4>Functions</H4>
Edgent supports Java 8 and it is encouraged to use Java 8 as functions can be easily and clearly written using lambda expressions.
<H4>Arbitrary Topology</H4>
Simple applications may just be a pipeline of streams, for example, logically:
<BR>
{@code source --> filter --> transform --> aggregate --> send to MQTT}
<BR>
However Edgent allows arbitrary topologies including:
<UL>
<LI>Multiple source streams in an application</LI>
<LI>Multiple sinks in an application </LI>
<LI>Multiple processing including sinks against a stream (fan-out)</LI>
<LI>Union of streams (fan-in)  </LI>
<LI>Correlation of streams by allowing streams to be joined</LI>
</UL>
<H3>Graph API</H3>
<H4>Overview</H4>
The {@link Graph graph} API is a lower-level API that the
topology api is built on top of. A graph consists of
{@link org.apache.edgent.oplet.Oplet oplet} invocations connected by streams.
The oplet invocations contain the processing applied to each tuple
on streams connected to their input ports. Processing by the oplet
submits tuples to its output ports for subsequent processing
by downstream connected oplet invocations.
<a name="start"></a>
<H2>Getting Started</H2>
<P>
A number of sample Java applications are provided that demonstrate use of Edgent.
General information about Edgent Application development and 
some development utilities are included.
See <a href="https://edgent.apache.org/docs/edgent-getting-started">Getting Started</a>.
</body>