index.html

---
layout: default
title: Overview
notitle: true
---
<div class="jumbotron">
<h4>Apache Crail (Incubating) is an open source user-level I/O architecture for the Apache data processing ecosystem designed from ground up for high-performance storage and networking hardware</h4>
</div>

<br>
<div>
<center>
<img src="{{ site.base }}/img/crail_stack2.png" align="middle" style="width: 340px; text-align:center;">
</center>
</div>
<br>

<div class="row">
<div class="col-lg-4">
<h2>Open</h2>
<p align="justify">Crail is open source and integrates seamlessly with the Apache ecosystem, such as Spark, Flink, Parquet, etc. Crail is based on Crail Store -- a high-performance distributed data store for temporary data -- and a series of modules interfacing with the compute engine. Crail modules provide standard interfaces (e.g. HDFS) and can be loaded transparently at runtime (e.g., Spark shuffle).</p>
<p><a class="btn btn-default" href="overview/index.html#spark">Learn more &raquo;</a></p>
</div>
<div class="col-lg-4">
<h2>Fast</h2>
<p align="justify">Crail is built explictly for user-level I/O (RDMA, NVMef, etc.), allowing storage and networking hardware to directly access I/O memory within the data processing engine. Bypassing OS and JVM during data access enables delivering bare-metal I/O performance to analytics workloads. For example, Crail achieves data access at rates close to the 100Gb/s network limit with latencies below 10 us.</p>
<p><a class="btn btn-default" href="overview/index.html#overview">Learn more &raquo;</a></p>
</div>
<div class="col-lg-4">
<h2>Heterogeneous</h2>
<p align="justify">Crail orchestrates I/O operations across different storage tiers including DRAM, flash or GPU memory. Aside from providing fine grained control as to which storage tier is used when storing data, Crail also supports horizontal tiering where higher performing storage resources are filled up across the cluster prior to using lower performing tiers -- making effective use of the storage hardware.</p>
<p><a class="btn btn-default" href="overview/index.html#fs">Learn more &raquo;</a></p>
</div>
</div>

<h2>News</h2>
<ul class="news list-unstyled">
{% for post in site.categories.news limit: site.front_page_news %}
    {% if post.shortnews %}
        <li class="shortnews">
            <span class="date">{{ post.date | date: "%B %-d, %Y" }}</span>
            {{ post.content }}
        </li>
    {% else %}
        <li class="bloglink">
            <span class="date">{{ post.date | date: "%B %-d, %Y" }}</span>
            <a href="{{ post.url }}">&raquo; {{ post.title }}</a>
        </li>
    {% endif %}
{% endfor %}
</ul>
{% assign numposts = site.categories.news | size %}
{% if numposts >= site.front_page_news %}
<p><a href="{{ site.base }}/news/">Older posts&hellip;</a></p>
{% endif %}