| /*- |
| * Copyright (C) 2002, 2018, Oracle and/or its affiliates. All rights reserved. |
| * |
| * This file was distributed by Oracle as part of a version of Oracle Berkeley |
| * DB Java Edition made available at: |
| * |
| * http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/downloads/index.html |
| * |
| * Please see the LICENSE file included in the top-level directory of the |
| * appropriate version of Oracle Berkeley DB Java Edition for a copy of the |
| * license and additional information. |
| */ |
| |
| package com.sleepycat.je; |
| |
| import java.util.Collection; |
| |
| /** |
| * @hidden |
| * For internal use only. |
| * |
| * Provides a way to create an association between primary and secondary |
| * databases that is not limited to a one-to-many association. |
| * <p> |
| * By implementing this interface, a secondary database may be associated with |
| * one or more primary databases. For example, imagine an application that |
| * wishes to partition data from a single logical "table" into multiple primary |
| * databases to take advantage of the performance benefits of {@link |
| * Environment#removeDatabase} for a single partition, while at the same having |
| * a single index database for the entire "table". For read operations via a |
| * secondary database, JE reads the primary key from the secondary database and |
| * then must read the primary record from the primary database. The mapping |
| * from a primary key to its primary database is performed by the user's |
| * implementation of the {@link #getPrimary} method. |
| * <p> |
| * In addition, a primary database may be divided into subsets of keys where |
| * each subset may be efficiently indexed by zero or more secondary databases. |
| * This effectively allows multiple logical "tables" per primary database. |
| * When a primary record is written, the secondaries for its logical "table" |
| * are returned by the user's implementation of {@link #getSecondaries}. |
| * During a primary record write, because {@link #getSecondaries} is called to |
| * determine the secondary databases, only the key creators/extractors for |
| * those secondaries are invoked. For example, if you have a single primary |
| * database with 10 logical "tables", each of which has 5 distinct secondary |
| * databases, when a record is written only 5 key creators/extractors will be |
| * invoked, not 50. |
| * <p> |
| * <b>Configuring a SecondaryAssociation</b> |
| * <p> |
| * When primary and secondary databases are associated using a {@code |
| * SecondaryAssociation}, the databases must all be configured with the same |
| * {@code SecondaryAssociation} instance. A common error is to forget to |
| * configure the {@code SecondaryAssociation} on the primary database. |
| * <p> |
| * A {@code SecondaryAssociation} is configured using {@link |
| * DatabaseConfig#setSecondaryAssociation} and {@link |
| * SecondaryConfig#setSecondaryAssociation} for a primary and secondary |
| * database, respectively. When calling {@link |
| * Environment#openSecondaryDatabase}, null must be passed for the |
| * primaryDatabase parameter when a {@code SecondaryAssociation} is configured. |
| * <p> |
| * Note that when a {@code SecondaryAssociation} is configured, {@code true} |
| * may not be passed to the {@link SecondaryConfig#setAllowPopulate} method. |
| * Population of new secondary databases in an existing {@code |
| * SecondaryAssociation} is done differently, as described below. |
| * <p> |
| * <b>Adding and removing secondary associations</b> |
| * <p> |
| * The state information defining the association between primary and secondary |
| * databases is maintained by the application, and made available to JE by |
| * implementing the {@code SecondaryAssociation} interface. The application |
| * may add/remove databases to/from the association without any explicit |
| * coordination with JE. However, certain rules need to be followed to |
| * maintain data integrity. |
| * <p> |
| * In the simplest case, there is a fixed (never changing) set of primary and |
| * secondary databases and they are added to the association at the time they |
| * are created, before writing or reading records. In this case, since reads |
| * and writes do not occur concurrently with changes to the association, no |
| * special rules are needed. For example: |
| * <ol> |
| * <li>Open the databases.</li> |
| * <li>Add databases to the association.</li> |
| * <li>Begin writing and reading.</li> |
| * </ol> |
| * <p> |
| * In other cases, primary and secondary databases may be added to an already |
| * established association, along with concurrent reads and writes. The rules |
| * for doing so are described below. |
| * <p> |
| * Adding an empty (newly created) primary database is no more complex than in |
| * the simple case above, assuming that writes to the primary database do not |
| * proceed until after it has been added to the association. |
| * <ol> |
| * <li>Open the new primary database.</li> |
| * <li>Add the primary database to the association such that {@link |
| * #getSecondaries} returns the appropriate list when passed a key in the |
| * new primary database, and {@link #getPrimary} returns the new database |
| * when passed a key it contains.</li> |
| * <li>Begin writing to the new primary database.</li> |
| * </ol> |
| * <p> |
| * Using the procedure above for adding a primary database, records will be |
| * indexed in secondary databases as they are added to the primary database, |
| * and will be immediately available to secondary index queries. |
| * <p> |
| * Alternatively, records can be added to the primary database without making |
| * them immediately available to secondary index queries by using the following |
| * procedure. Records will be indexed in secondary databases as they are added |
| * but will not be returned by secondary queries until after the last step. |
| * <ol> |
| * <li>Open the new primary database.</li> |
| * <li>Modify the association such that {@link #getSecondaries} returns the |
| * appropriate list when passed a key in the new primary database, but |
| * {@link #getPrimary} returns null when passed a key it contains.</li> |
| * <li>Write records in the new primary database.</li> |
| * <li>Modify the association such that {@link #getPrimary} returns the new |
| * database when passed a key it contains.</li> |
| * </ol> |
| * <p> |
| * The following procedure should be used for removing an existing (non-empty) |
| * primary database from an association. |
| * <ol> |
| * <li>Remove the primary database from the association, such that {@link |
| * #getPrimary} returns null for all keys in the primary.</li> |
| * <li>Disable read and write operations on the database and ensure that |
| * all in-progress operations have completed.</li> |
| * <li>At this point the primary database may be closed and removed |
| * (e.g., with {@link Environment#removeDatabase}), if desired.</li> |
| * <li>For each secondary database associated with the removed primary |
| * database, {@link SecondaryDatabase#deleteObsoletePrimaryKeys} should be |
| * called to process all secondary records.</li> |
| * <li>At this point {@link #getPrimary} may throw an exception (rather |
| * than return null) when called for a primary key in the removed database, |
| * if desired.</li> |
| * </ol> |
| * <p> |
| * The following procedure should be used for adding a new (empty) secondary |
| * database to an association, and to populate the secondary incrementally |
| * from its associated primary database(s). |
| * <ol> |
| * <li>Open the secondary database and call {@link |
| * SecondaryDatabase#startIncrementalPopulation}. The secondary may not be |
| * used (yet) for read operations.</li> |
| * <li>Add the secondary database to the association, such that the |
| * collection returned by {@link #getSecondaries} includes the new |
| * database, for all primary keys that should be indexed.</li> |
| * <li>For each primary database associated with the new secondary |
| * database, {@link Database#populateSecondaries} should be called to |
| * process all primary records.</li> |
| * <li>Call {@link SecondaryDatabase#endIncrementalPopulation}. The |
| * secondary database may now be used for read operations.</li> |
| * </ol> |
| * <p> |
| * The following procedure should be used for removing an existing (non-empty) |
| * secondary database from an association. |
| * <ol> |
| * <li>Remove the secondary database from the association, such that it is |
| * not included in the collection returned by {@link #getSecondaries}.</li> |
| * <li>Disable read and write operations on the database and ensure that |
| * all in-progress operations have completed.</li> |
| * <li>The secondary database may now be closed and removed, if |
| * desired.</li> |
| * </ol> |
| * <p> |
| * <b>Other Implementation Requirements</b> |
| * <p> |
| * The implementation must use data structures for storing and accessing |
| * association information that provide <em>happens-before</em> semantics. In |
| * other words, when the association is changed, other threads calling the |
| * methods in this interface must see the changes as soon as the change is |
| * complete. |
| * <p> |
| * The implementation should use non-blocking data structures to hold |
| * association information to avoid blocking on the methods in this interface, |
| * which may be frequently called from many threads. |
| * <p> |
| * The simplest way to meet the above two requirements is to use concurrent |
| * structures such as those provided in the java.util.concurrent package, e.g., |
| * ConcurrentHashMap. |
| * <p> |
| * For an example implementation, see: |
| * test/com/sleepycat/je/test/SecondaryAssociationTest.java |
| */ |
| public interface SecondaryAssociation { |
| |
| /* |
| * Implementation note on concurrent access and latching. |
| * |
| * When adding/removing primary/secondary DBs to an existing association, |
| * concurrency issues arise. This section explains how they are handled, |
| * assuming the rules described in the javadoc below are followed when a |
| * custom association is configured. |
| * |
| * There are two cases: |
| * |
| * 1) A custom association is NOT configured. An internal association is |
| * created that manages the one-many association between a primary and its |
| * secondaries. |
| * |
| * 2) A custom association IS configured. It is used internally. |
| * |
| * For case 1, no custom association, DB addition and removal take place in |
| * the Environment.openSecondaryDatabase and SecondaryDatabase.close |
| * methods, respectively. These methods, and the changes to the internal |
| * association, are protected by acquiring the secondary latch |
| * (EnvironmentImpl.getSecondaryAssociationLock) exclusively. All write |
| * operations that use the association -- puts and deletes -- acquire this |
| * latch shared. Therefore, write operations and changes to the |
| * association do not take place concurrently. Read operations via a |
| * secondary DB/cursor do not acquire this latch, so they can take place |
| * concurrently with changes to the association; however, read operations |
| * only call getPrimary and they expect it may return null at any time (the |
| * operation ignores the record in that case). It is impossible to close |
| * a primary while it is being read via a secondary, because secondaries |
| * must be closed before their associated primary. (And the application is |
| * responsible for finishing reads before closing any DB.) |
| * |
| * For case 2, a custom association, the secondary latch does not provide |
| * protection because modifications to the association take place in the |
| * application domain, not in the Environment.openSecondaryDatabase and |
| * SecondaryDatabase.close methods. Instead, protection is provided by the |
| * rules described in the javadoc here. Namely: |
| * |
| * Primary/secondary DBs are added to the association AFTER opening them |
| * (of course). The DBs will not be be accessed until they are added to |
| * the association. |
| * |
| * Primary/secondary DBs are removed from the association BEFORE closing |
| * them. The application is responsible for ensuring they are no longer |
| * accessed before they are closed. |
| * |
| * When a primary DB is added, its secondaries will be kept in sync as |
| * soon as writing begins, since it must be added to the association |
| * before writes are allowed. |
| * |
| * When a secondary DB is added, the incremental population procedure |
| * ensures that it is fully populated before being accessed. |
| */ |
| |
| /** |
| * Returns true if there are no secondary databases in the association. |
| * |
| * This method is used by JE to optimize for the case where a primary |
| * has no secondary databases. This allows a {@code SecondaryAssociation} |
| * to be configured on a primary database even when it has no associated |
| * secondaries, with no added overhead, and to allow for the possibility |
| * that secondaries will be added later. |
| * <p> |
| * For example, when a primary database has no secondaries, no internal |
| * latching is performed by JE during writes. JE determines that no |
| * secondaries are present by calling {@code isEmpty}, prior to doing the |
| * write operation. |
| * |
| * @throws RuntimeException if an unexpected problem occurs. The exception |
| * will be thrown to the application, which should then take appropriate |
| * action. |
| */ |
| boolean isEmpty(); |
| |
| /** |
| * Returns the primary database for the given primary key. This method |
| * is called during read operations on secondary databases that are |
| * configured with this {@code SecondaryAssociation}. |
| * |
| * This method should return null when the primary database has been |
| * removed from the association, or when it should not be included in the |
| * results for secondary queries. In this case, the current operation will |
| * treat the secondary record as if it does not exist, i.e., it will be |
| * skipped over. |
| * |
| * @throws RuntimeException if an unexpected problem occurs. The exception |
| * will be thrown to the application, which should then take appropriate |
| * action. |
| */ |
| Database getPrimary(DatabaseEntry primaryKey); |
| |
| /** |
| * Returns the secondary databases associated with the given primary key. |
| * This method is called during write operations on primary databases that |
| * are configured with this {@code SecondaryAssociation}. |
| * |
| * @throws RuntimeException if an unexpected problem occurs. The exception |
| * will be thrown to the application, which should then take appropriate |
| * action. |
| */ |
| Collection<SecondaryDatabase> getSecondaries(DatabaseEntry primaryKey); |
| } |
