Separate benchmark tasks into jobs. (#4404)

* Separate benchmark tasks into jobs.

To enable more granular re-run in Concourse, split the Benchmarks into
independently triggerable jobs in the main pipeline.
Extract variables from the template into the jinja.variables file.
Make better use of YML anchors.

Co-authored-by: Robert Houghton <rhoughton@pivotal.io>
Co-authored-by: Helena Bales <hbales@pivotal.io>

* empty to trigger pr

(cherry picked from commit 1fcadf987706af7515a64869e088e1573695dae0)
2 files changed
tree: 7caaba9710e5bc598beba3edf98ee47a7aed6a44
  1. .github/
  2. boms/
  3. buildSrc/
  4. ci/
  5. dev-tools/
  6. docker/
  7. etc/
  8. extensions/
  9. geode-assembly/
  10. geode-book/
  11. geode-common/
  12. geode-concurrency-test/
  13. geode-connectors/
  14. geode-core/
  15. geode-cq/
  16. geode-docs/
  17. geode-dunit/
  18. geode-experimental-driver/
  19. geode-http-service/
  20. geode-junit/
  21. geode-log4j/
  22. geode-logging/
  23. geode-lucene/
  24. geode-management/
  25. geode-memcached/
  26. geode-old-client-support/
  27. geode-old-versions/
  28. geode-protobuf/
  29. geode-protobuf-messages/
  30. geode-pulse/
  31. geode-rebalancer/
  32. geode-redis/
  33. geode-serialization/
  34. geode-unsafe/
  35. geode-wan/
  36. geode-web/
  37. geode-web-api/
  38. geode-web-management/
  39. gradle/
  40. static-analysis/
  41. .gitattributes
  42. .gitignore
  43. build.gradle
  44. BUILDING.md
  45. gradle.properties
  46. gradlew
  47. gradlew.bat
  48. KEYS
  49. LICENSE
  50. NOTICE
  51. README.md
  52. settings.gradle
  53. TESTING.md
README.md

Build Status License Maven Central homebrew Docker Pulls

Contents

  1. Overview
  2. How to Get Apache Geode
  3. Main Concepts and Components
  4. Location of Directions for Building from Source
  5. Geode in 5 minutes
  6. Application Development
  7. Documentation
  8. Wiki
  9. How to Contribute?
  10. Export Control

Overview

Apache Geode is a data management platform that provides real-time, consistent access to data-intensive applications throughout widely distributed cloud architectures.

Apache Geode pools memory, CPU, network resources, and optionally local disk across multiple processes to manage application objects and behavior. It uses dynamic replication and data partitioning techniques to implement high availability, improved performance, scalability, and fault tolerance. In addition to being a distributed data container, Apache Geode is an in-memory data management system that provides reliable asynchronous event notifications and guaranteed message delivery.

Apache Geode is a mature, robust technology originally developed by GemStone Systems. Commercially available as GemFireâ„¢, it was first deployed in the financial sector as the transactional, low-latency data engine used in Wall Street trading platforms. Today Apache Geode technology is used by hundreds of enterprise customers for high-scale business applications that must meet low latency and 24x7 availability requirements.

How to Get Apache Geode

You can download Apache Geode from the website, run a Docker image, or install with homebrew on OSX. Application developers can load dependencies from Maven Central.

Maven

<dependencies>
    <dependency>
        <groupId>org.apache.geode</groupId>
        <artifactId>geode-core</artifactId>
        <version>$VERSION</version>
    </dependency>
</dependencies>

Gradle

dependencies {
  compile "org.apache.geode:geode-core:$VERSION"
}

Main Concepts and Components

Caches are an abstraction that describe a node in an Apache Geode distributed system.

Within each cache, you define data regions. Data regions are analogous to tables in a relational database and manage data in a distributed fashion as name/value pairs. A replicated region stores identical copies of the data on each cache member of a distributed system. A partitioned region spreads the data among cache members. After the system is configured, client applications can access the distributed data in regions without knowledge of the underlying system architecture. You can define listeners to receive notifications when data has changed, and you can define expiration criteria to delete obsolete data in a region.

Locators provide clients with both discovery and server load balancing services. Clients are configured with locator information, and the locators maintain a dynamic list of member servers. The locators provide clients with connection information to a server.

Apache Geode includes the following features:

  • Combines redundancy, replication, and a “shared nothing” persistence architecture to deliver fail-safe reliability and performance.
  • Horizontally scalable to thousands of cache members, with multiple cache topologies to meet different enterprise needs. The cache can be distributed across multiple computers.
  • Asynchronous and synchronous cache update propagation.
  • Delta propagation distributes only the difference between old and new versions of an object (delta) instead of the entire object, resulting in significant distribution cost savings.
  • Reliable asynchronous event notifications and guaranteed message delivery through optimized, low latency distribution layer.
  • Data awareness and real-time business intelligence. If data changes as you retrieve it, you see the changes immediately.
  • Integration with Spring Framework to speed and simplify the development of scalable, transactional enterprise applications.
  • JTA compliant transaction support.
  • Cluster-wide configurations that can be persisted and exported to other clusters.
  • Remote cluster management through HTTP.
  • REST APIs for REST-enabled application development.
  • Rolling upgrades may be possible, but they will be subject to any limitations imposed by new features.

Building this Release from Source

See BUILDING.md for instructions on how to build the project.

Running Tests

See TESTING.md for instructions on how to run tests.

Geode in 5 minutes

Geode requires installation of JDK version 1.8. After installing Apache Geode, start a locator and server:

$ gfsh
gfsh> start locator
gfsh> start server

Create a region:

gfsh> create region --name=hello --type=REPLICATE

Write a client application (this example uses a Gradle build script):

build.gradle

apply plugin: 'java'
apply plugin: 'application'

mainClassName = 'HelloWorld'

repositories { mavenCentral() }
dependencies {
  compile 'org.apache.geode:geode-core:1.4.0'
  runtime 'org.slf4j:slf4j-log4j12:1.7.24'
}

src/main/java/HelloWorld.java

import java.util.Map;
import org.apache.geode.cache.Region;
import org.apache.geode.cache.client.*;

public class HelloWorld {
  public static void main(String[] args) throws Exception {
    ClientCache cache = new ClientCacheFactory()
      .addPoolLocator("localhost", 10334)
      .create();
    Region<String, String> region = cache
      .<String, String>createClientRegionFactory(ClientRegionShortcut.CACHING_PROXY)
      .create("hello");

    region.put("1", "Hello");
    region.put("2", "World");

    for (Map.Entry<String, String>  entry : region.entrySet()) {
      System.out.format("key = %s, value = %s\n", entry.getKey(), entry.getValue());
    }
    cache.close();
  }
}

Build and run the HelloWorld example:

$ gradle run

The application will connect to the running cluster, create a local cache, put some data in the cache, and print the cached data to the console:

key = 1, value = Hello
key = 2, value = World

Finally, shutdown the Geode server and locator:

gfsh> shutdown --include-locators=true

For more information see the Geode Examples repository or the documentation.

Application Development

Apache Geode applications can be written in these client technologies:

The following libraries are available external to the Apache Geode project:

Export Control

This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.

The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.

The following provides more details on the included cryptographic software: