| // Licensed to the Apache Software Foundation (ASF) under one |
| // or more contributor license agreements. See the NOTICE file |
| // distributed with this work for additional information |
| // regarding copyright ownership. The ASF licenses this file |
| // to you under the Apache License, Version 2.0 (the |
| // "License"); you may not use this file except in compliance |
| // with the License. You may obtain a copy of the License at |
| // |
| // http://www.apache.org/licenses/LICENSE-2.0 |
| // |
| // Unless required by applicable law or agreed to in writing, |
| // software distributed under the License is distributed on an |
| // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| // KIND, either express or implied. See the License for the |
| // specific language governing permissions and limitations |
| // under the License. |
| |
| [[prior_release_notes]] |
| = Apache Kudu Prior Version Release Notes |
| |
| :author: Kudu Team |
| :imagesdir: ./images |
| :icons: font |
| :toc: left |
| :toclevels: 3 |
| :doctype: book |
| :backend: html5 |
| :sectlinks: |
| :experimental: |
| |
| This section reproduces the release notes for new features and incompatible |
| changes in prior releases of Apache Kudu. |
| |
| |
| NOTE: The list of known issues and limitations for prior releases are not |
| reproduced on this page. Please consult the |
| link:http://kudu.apache.org/releases/[documentation of the appropriate release] |
| for a list of known issues and limitations. |
| |
| [[rn_1.7.0_upgrade_notes]] |
| == Upgrade Notes |
| |
| * Upgrading directly from Kudu 1.6.0 is supported and no special upgrade steps |
| are required. A rolling upgrade of the server side will _not_ work because |
| the default replica management scheme changed, and running masters and tablet |
| servers with different replica management schemes is not supported, see |
| <<rn_1.7.0_incompatible_changes>> for details. However, mixing client and |
| server sides of different versions is not a problem. You can still |
| update your clients before your servers or vice versa. |
| When upgrading to Kudu 1.7, it is required to first shut down all Kudu processes |
| across the cluster, then upgrade the software on all servers, then restart |
| the Kudu processes on all servers in the cluster. |
| |
| [[rn_1.7.0_obsoletions]] |
| == Obsoletions |
| |
| * The `tcmalloc_contention_time` metric, which previously tracked the amount |
| of time spent in memory allocator lock contention, has been removed. |
| |
| [[rn_1.7.0_deprecations]] |
| == Deprecations |
| |
| * Support for Java 7 has been deprecated since Kudu 1.5.0 and may be removed in |
| the next major release. |
| |
| [[rn_1.7.0_new_features]] |
| == New features |
| |
| * Kudu now supports the decimal column type. The decimal type is a numeric data type |
| with fixed scale and precision suitable for financial and other arithmetic |
| calculations where the imprecise representation and rounding behavior of float and |
| double make those types impractical. The decimal type is also useful for integers |
| larger than int64 and cases with fractional values in a primary key. |
| See link:schema_design.html#decimal[Decimal Type] for more details. |
| |
| * The strategy Kudu uses for automatically healing tablets which have lost a |
| replica due to server or disk failures has been improved. The new re-replication |
| strategy, or replica management scheme, first adds a replacement tablet replica |
| before evicting the failed one. With the previous replica management scheme, |
| the system first evicts the failed replica and then adds a replacement. The new |
| replica management scheme allows for much faster recovery of tablets in |
| scenarios where one tablet server goes down and then returns back shortly after |
| 5 minutes or so. The new scheme also provides substantially better overall |
| stability on clusters with frequent server failures. |
| (see link:https://issues.apache.org/jira/browse/KUDU-1097[KUDU-1097]). |
| |
| * The `kudu fs update_dirs` tool now supports removing directories. Unless the |
| `--force` flag is specified, Kudu will not allow the removal of a directory |
| across which tablets are configured to spread data. If specified, all tablet |
| replicas configured to use that directory will fail upon starting up and be |
| replicated elsewhere, provided a majority exists elsewhere. |
| |
| * Users can use the new `--fs_metadata_dir` to specify the directory in which |
| to place tablet-specific metadata. It is recommended, although not |
| necessary, that this be placed on a high-performance drive with high |
| bandwidth and low latency, e.g. a solid-state drive. If not specified, |
| metadata will be placed in the directory specified by `--fs_wal_dir`, or in |
| the directory specified by the first entry of `--fs_data_dirs` if metadata |
| already exists there from a pre-Kudu 1.7 deployment. Kudu will not |
| automatically move existing metadata based on this configuration. |
| |
| * Kudu 1.7 introduces a new scan read mode READ_YOUR_WRITES. Users can specify |
| READ_YOUR_WRITES when creating a new scanner in C++, Java and Python clients. |
| If this mode is used, the client will perform a read such that it follows all |
| previously known writes and reads from this client. Reads in this mode ensure |
| read-your-writes and read-your-reads session guarantees, while minimizing |
| latency caused by waiting for outstanding write transactions to complete. |
| Note that this is still an experimental feature which may be stabilized in |
| future releases. |
| |
| * The tablet server web UI scans dashboard (/scans) has been improved with |
| several new features, including: showing the most recently completed scans, |
| a pseudo-SQL scan descriptor that concisely shows the selected columns and |
| applied predicates, and more complete and better documented scan statistics. |
| |
| * Kudu daemons now expose a web page `/stacks` which dumps the current stack |
| trace of every thread running in the server. This information can be helpful |
| when diagnosing performance issues. |
| |
| == Optimizations and improvements |
| |
| * By default, each tablet replica will now stripe data blocks across 3 data |
| directories instead of all data directories. This decreases the likelihood |
| that any given tablet will be affected in the event of a single disk failure. |
| No substantial performance impact is expected due to this feature based on |
| link:https://github.com/apache/kudu/commit/60276c54a221d554287c6645df7df542fe6d6443[performance testing]. |
| This change only affects new replicas created after upgrading to Kudu 1.7. |
| |
| * Kudu servers previously offered the ability to enable a separate metrics log |
| which stores periodic snapshots of all metrics available on a server. This |
| functionality is now available as part of a more general “diagnostics log” |
| which is enabled by default. The diagnostics log includes periodic dumps of |
| server metrics as well as collections of thread stack traces. The default |
| configuration ensures that no more than 640MB of diagnostics logs are retained, |
| and typically the space consumption is significantly less due to compression. |
| The format and contents of this log file are documented in the |
| link:administration.html[Administration guide]. |
| |
| * The handling of errors in the synchronous Java client has been improved so that, |
| when an exception is thrown, the stack trace indicates the correct location |
| where the client function was invoked rather than a call stack of an internal |
| worker thread. The original call stack from the worker thread is available as |
| a “suppressed exception”. |
| |
| * The logging of errors in the Java client has been improved to exclude exception |
| stack traces for expected scenarios such as failure to connect to a server in a |
| cluster. Instead, only a single line informational message will be logged in |
| such cases to aid in debugging. |
| |
| * The Java client now uses a predefined prioritized list of TLS ciphers when |
| establishing an encrypted connection to Kudu servers. This cipher list matches |
| the list of ciphers preferred for server-to-server communication and ensures |
| that the most efficient and secure ciphers are preferred. When the Kudu client |
| is running on Java 8 or newer, this provides a substantial speed-up to read |
| and write performance. |
| |
| * Reporting for the `kudu cluster ksck` tool has been updated so tablets and |
| tables with on-going tablet copies are shown as "recovering". Additional |
| reporting changes have been made to make various common scenarios, |
| particularly tablet copies, less alarming. |
| |
| * The performance of inserting rows containing many string or binary columns has |
| been improved, especially in the case of highly concurrent write workloads. |
| |
| * By default, Spark tasks that scan Kudu will now be able to scan non-leader |
| replicas. This allows Spark to more easily schedule kudu-spark tasks local to |
| the data. Users can disable this behavior by passing 'leader_only' to the |
| 'kudu.scanLocality' option." |
| |
| * The number of OS threads used in the steady state and during bursts of |
| activity (such as in Raft leader elections triggered by a node failure) has |
| been drastically reduced and should no longer exceed the value of `ulimit -u`. |
| As such, it should no longer be necessary to increase the value of `ulimit -u` |
| (or of /proc/sys/kernel/threads-max) in order to run a Kudu tablet server in |
| most cases. |
| (see link:https://issues.apache.org/jira/browse/KUDU-1913[KUDU-1913]). |
| |
| * An issue where sparse column predicates could cause excessive data-block reads |
| has been fixed. Previously in certain scans with sparsely matching predicates |
| on multiple columns, Kudu would read and decode the same data blocks many times. |
| The improvement typically results in a 5-10x performance increase for the |
| affected scans. |
| (see link:https://issues.apache.org/jira/browse/KUDU-2231[KUDU-2231]). |
| |
| * The efficiency and on-disk size of large updated values has been improved. |
| This will improve update-heavy workloads which overwrite large (1KiB+) values. |
| (see link:https://issues.apache.org/jira/browse/KUDU-2253[KUDU-2253]). |
| |
| |
| [[rn_1.7.0_fixed_issues]] |
| == Fixed Issues |
| |
| * Fixed a scenario where the on-disk data of a tablet server was completely |
| erased and and a new tablet server was started on the same host. This issue |
| could prevent tablet replicas previously hosted on the server from being |
| evicted and re-replicated. |
| Tablets now immediately evict replicas that respond with a different server |
| UUID than expected. |
| (see link:https://issues.apache.org/jira/browse/KUDU-1613[KUDU-1613]). |
| |
| * Fixed a rare race condition when connecting to masters during their |
| startup which might cause a client to get a response without a CA certificate |
| and/or authentication token. This would cause the client to fail to authenticate |
| with other servers in the cluster. The leader master now always sends a CA |
| certificate and an authentication token (when applicable) to a Kudu client |
| with a successful ConnectToMaster response. |
| (see link:https://issues.apache.org/jira/browse/KUDU-1927[KUDU-1927]). |
| |
| * The Kudu Java client now will retry a connection if no master is discovered as a |
| leader, and the user has a valid authentication token. This avoids failure |
| in recoverable cases when masters are in the process of the very first leader |
| election after starting up. |
| (see link:https://issues.apache.org/jira/browse/KUDU-2262[KUDU-2262]). |
| |
| * The Java client will now automatically attempt to re-acquire Kerberos |
| credentials from the ticket cache when the prior credentials are about to |
| expire. This allows client instances to persist longer than the expiration |
| time of a single Kerberos ticket so long as some other process renews the |
| credentials in the ticket cache. Documentation on interacting with Kerberos |
| authentication has been added to the Javadoc for the `AsyncKuduClient` class. |
| (see link:https://issues.apache.org/jira/browse/KUDU-2264[KUDU-2264]). |
| |
| * Follower masters are now able to verify authentication tokens even if they have never |
| been a leader. Prior to this fix, if a follower master had never been a leader, |
| clients would be unable to authenticate to that master, resulting in spurious |
| error messages being logged. |
| (see link:https://issues.apache.org/jira/browse/KUDU-2265[KUDU-2265]). |
| |
| * Fixed a tablet server crash when a tablet replica is deleted during a scan. |
| (see link:https://issues.apache.org/jira/browse/KUDU-2295[KUDU-2295]). |
| |
| * The evaluation order of predicates in scans with multiple predicates has been |
| made deterministic. Due to a bug, this was not necessarily the case previously. |
| Predicates are applied in most to least selective order, with ties broken by |
| column index. The evaluation order may change in the future, particularly when |
| better column statistics are made available internally. |
| (see link:https://issues.apache.org/jira/browse/KUDU-2312[KUDU-2312]). |
| |
| * Previously, the `kudu tablet change_config move_replica` tool required all |
| tablet servers in the cluster to be available when performing a move. This |
| restriction has been relaxed: only the tablet server that will receive a replica |
| of the tablet being moved and the hosts of the tablet's existing replicas need to be |
| available for the move to occur. |
| (see link:https://issues.apache.org/jira/browse/KUDU-2331[KUDU-2331]). |
| |
| * Fixed a bug in the Java client which prevented the client from locating the |
| new leader master after a leader failover in the case that the previous leader |
| either remained online or restarted quickly. This bug resulted in the client |
| timing out operations with errors indicating that there was no leader master. |
| (see link:https://issues.apache.org/jira/browse/KUDU-2343[KUDU-2343]). |
| |
| * The Unix process username of the client is now included inside the exported |
| security credentials, so that the effective username of clients who import |
| credentials and subsequently use unauthenticated (SASL PLAIN) connections |
| matches the client who exported the security credentials. For example, this is |
| useful to let the Spark executors know which username to use if the Spark |
| driver has no authentication token. This change only affects clusters with |
| encryption disabled using `--rpc-encryption=disabled`. |
| (see link:https://issues.apache.org/jira/browse/KUDU-2259[KUDU-2259]). |
| |
| [[rn_1.7.0_wire_compatibility]] |
| == Wire Protocol compatibility |
| |
| Kudu 1.7.0 is wire-compatible with previous versions of Kudu: |
| |
| * Kudu 1.7 clients may connect to servers running Kudu 1.0 or later. If the client uses |
| features that are not available on the target server, an error will be returned. |
| * Rolling upgrade between Kudu 1.6 and Kudu 1.7 servers is believed to be possible |
| though has not been sufficiently tested. Users are encouraged to shut down all nodes |
| in the cluster, upgrade the software, and then restart the daemons on the new version. |
| * Kudu 1.0 clients may connect to servers running Kudu 1.7 with the exception of the |
| below-mentioned restrictions regarding secure clusters. |
| |
| The authentication features introduced in Kudu 1.3 place the following limitations |
| on wire compatibility between Kudu 1.7 and versions earlier than 1.3: |
| |
| * If a Kudu 1.7 cluster is configured with authentication or encryption set to "required", |
| clients older than Kudu 1.3 will be unable to connect. |
| * If a Kudu 1.7 cluster is configured with authentication and encryption set to "optional" |
| or "disabled", older clients will still be able to connect. |
| |
| [[rn_1.7.0_incompatible_changes]] |
| == Incompatible Changes in Kudu 1.7.0 |
| |
| * The newly introduced replica management scheme is not compatible with the |
| old scheme, so it's not possible to run pre-1.7 Kudu masters with |
| 1.7 Kudu tablet servers or vice versa. This is a server-side |
| incompatibility only and it does not affect client compatibility. In other words, |
| Kudu clients of prior versions are compatible with upgraded Kudu clusters. |
| |
| ** Kudu masters of 1.7 version will not register Kudu tablet servers of 1.6 |
| and prior versions. |
| ** Kudu tablet servers of 1.7 version will not work with Kudu masters of 1.6 |
| and prior versions. |
| |
| * The format of the previously-optional metrics log has changed to include a |
| human-readable timestamp on each line. The path of the log file has also |
| changed with the word “diagnostics” replacing the word “metrics” in the file |
| name. The metrics log has been optimized to only include those metrics which |
| have changed in between successive samples, and to not include entity attributes |
| such as tablet partition information in the log. |
| (see link:https://issues.apache.org/jira/browse/KUDU-2297[KUDU-2297]). |
| |
| [[rn_1.7.0_client_compatibility]] |
| === Client Library Compatibility |
| |
| * The Kudu 1.7 Java client library is API- and ABI-compatible with Kudu 1.6. Applications |
| written against Kudu 1.6 will compile and run against the Kudu 1.7 client library and |
| vice-versa. |
| |
| * The Kudu 1.7 {cpp} client is API- and ABI-forward-compatible with Kudu 1.6. |
| Applications written and compiled against the Kudu 1.6 client library will run without |
| modification against the Kudu 1.7 client library. Applications written and compiled |
| against the Kudu 1.7 client library will run without modification against the Kudu 1.6 |
| client library. |
| |
| * The Kudu 1.7 Python client is API-compatible with Kudu 1.6. Applications |
| written against Kudu 1.6 will continue to run against the Kudu 1.7 client |
| and vice-versa. |
| |
| * Kudu 1.7 clients that attempt to create a table with a decimal column on a |
| target server running Kudu 1.6 or earlier will receive an error response. |
| Similarly Kudu clients running Kudu 1.6 or earlier will result in an error |
| when attempting to access any table containing containing a decimal |
| column. |
| |
| [[rn_1.6.0_upgrade_notes]] |
| == Upgrade Notes |
| |
| * Upgrading directly from Kudu 1.5.0 is supported and no special upgrade steps |
| are required. A rolling upgrade may work, however it has not been tested. |
| When upgrading Kudu, it is recommended to first shut down all Kudu processes |
| across the cluster, then upgrade the software on all servers, then restart |
| the Kudu processes on all servers in the cluster. |
| |
| [[rn_1.6.0_obsoletions]] |
| == Obsoletions |
| |
| * Support for Spark 1 (kudu-spark_2.10) has been **removed** in Kudu 1.6.0 and |
| now only Spark 2 is supported. Spark 1 support was deprecated in Kudu 1.5.0. |
| |
| [[rn_1.6.0_deprecations]] |
| == Deprecations |
| |
| * Support for Java 7 has been deprecated since Kudu 1.5.0 and may be removed in |
| the next major release. |
| |
| [[rn_1.6.0_new_features]] |
| == New features |
| |
| * Tablet servers' tolerance of disk failures is now enabled by default and has |
| been extended to handle data directory failures at runtime. In the event of |
| a disk failure at runtime, any tablets with data on a failed disk will be |
| shut down and restarted on another tablet server. There is a configurable |
| tradeoff between a newly added tablet's tolerance to disk failures and its |
| ability to parallelize reads via the experimental |
| `--fs_target_data_dirs_per_tablet` flag. Tablets that are spread across fewer |
| disks are less likely to be affected by a disk failure, at the cost of |
| reduced parallelism. By default, tablets are striped across all available |
| disks. Note that the first configured data directory and the WAL directory |
| cannot currently tolerate disk failures. This will be further improved in |
| future Kudu releases. |
| |
| * Kudu servers can now adopt new data directories via the new |
| `kudu fs update_dirs` tool. The new directory will be used by new tablet |
| replicas only. Note that removing directories is not yet supported |
| (see link:https://issues.apache.org/jira/browse/KUDU-2202[KUDU-2202]). |
| |
| * Kudu servers have two new flags to control webui TLS/HTTPS |
| settings: `--webserver_tls_ciphers` and `--webserver_tls_min_protocol`. |
| These flags allow the advertised TLS ciphers and TLS protocol versions to be |
| configured. Additionally, the webserver now excludes insecure legacy ciphers |
| by default |
| (see link:https://issues.apache.org/jira/browse/KUDU-2190[KUDU-2190]). |
| |
| == Optimizations and improvements |
| |
| * Kudu servers can now tolerate short interruptions in NTP clock |
| synchronization. NTP synchronization is still required when any Kudu daemon |
| starts up. If NTP synchronization is not available, diagnostic information |
| is now logged to help pinpoint the issue |
| (see link:https://issues.apache.org/jira/browse/KUDU-1578[KUDU-1578]). |
| |
| * Tablet server startup time has been improved significantly on servers |
| containing large numbers of blocks. |
| |
| * The log block manager now performs disk data deletion in batches. This |
| optimization can significantly reduce the time taken to delete data on a tablet. |
| |
| * The usage of sensitive data redaction flag has been slightly changed. By |
| setting `--redact=log` flag, redaction will be disabled in the web UI but |
| retained for server logs. Alternatively, `--redact=none` can be used to |
| disable redaction completely. |
| |
| * The Spark DataSource integration now can take advantage of scan locality |
| for better scan performance, the scan will take place at the closest replica |
| instead of going to the leader. |
| |
| * Various optimizations were made to reduce the 99th percentile latency of |
| writes on the tablet server. This can also improve throughput on certain |
| write workloads, particularly on larger clusters. |
| |
| * Kudu may now be configured to ignore system-wide auth_to_local mappings |
| configured in /etc/krb5.conf by setting the configuration flag |
| `--use_system_auth_to_local=false` |
| (see link:https://issues.apache.org/jira/browse/KUDU-2198[KUDU-2198]). |
| |
| * The performance of the compaction scheduler has been improved. In |
| previous versions, certain types of time series workloads were found to |
| cause compaction scheduling to take tens of seconds. These workloads now |
| schedule compactions an order of magnitude more efficiently. |
| |
| * The compaction scheduler has been improved to avoid running a compaction |
| when the benefit of that compaction is extremely small. |
| |
| * Tablet servers now consider the health of all replicas of a tablet before |
| deciding to evict one. This can improve stability of the Kudu cluster after |
| experiencing multiple simultaneous daemon failures |
| (see link:https://issues.apache.org/jira/browse/KUDU-2048[KUDU-2048]). |
| |
| * Several performance improvements have been made to the Kudu master, |
| particularly in concurrency of clients opening tables. This should improve |
| performance in highly concurrent workloads. |
| |
| * The on-disk size metric for a tablet now includes all data and metadata. |
| Previously, it excluded WAL segments and consensus metadata |
| (see link:https://issues.apache.org/jira/browse/KUDU-1755[KUDU-1755]). |
| |
| * Added verbose mode for the 'kudu cluster ksck' command to enable output |
| of detailed information on the cluster's metadata, even when no errors are |
| detected. |
| |
| [[rn_1.6.0_fixed_issues]] |
| == Fixed Issues |
| |
| * HybridTime timestamp propagation now works in the Java client when using scan |
| tokens (see link:https://issues.apache.org/jira/browse/KUDU-1411[KUDU-1411]). |
| |
| * Fixed an error message commonly found in tablet server logs indicating that |
| operations were being read "from the future" |
| (see link:https://issues.apache.org/jira/browse/KUDU-1078[KUDU-1078]). |
| |
| * Tombstoned tablets no longer report metrics |
| (see link:https://issues.apache.org/jira/browse/KUDU-2044[KUDU-2044]). |
| |
| * Fixed a bug in the C++ client which could cause tablets to be erroneously |
| pruned, or skipped, during certain scans, resulting in fewer results than |
| expected being returned from queries. The bug only affected tables whose range |
| partition columns are a proper prefix of the primary key |
| (see link:https://issues.apache.org/jira/browse/KUDU-2173[KUDU-2173]). |
| |
| * Published Kudu Java artifacts are now fully compatible with JRE 7 and JRE 8. |
| There was previously a bug in the release process which made them compatible |
| only with JRE 8 |
| (see link:https://issues.apache.org/jira/browse/KUDU-2188[KUDU-2188]). |
| |
| * Fixed a typo in the list of default TLS ciphers used by Kudu servers. As a |
| result, two additional cipher suites are now available: |
| ** ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(128) Mac=SHA256 |
| ** AES256-GCM-SHA384 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(256) Mac=AEAD |
| |
| [[rn_1.6.0_wire_compatibility]] |
| == Wire Protocol compatibility |
| |
| Kudu 1.6.0 is wire-compatible with previous versions of Kudu: |
| |
| * Kudu 1.6 clients may connect to servers running Kudu 1.0 or later. If the client uses |
| features that are not available on the target server, an error will be returned. |
| * Rolling upgrade between Kudu 1.5 and Kudu 1.6 servers is believed to be possible |
| though has not been sufficiently tested. Users are encouraged to shut down all nodes |
| in the cluster, upgrade the software, and then restart the daemons on the new version. |
| * Kudu 1.0 clients may connect to servers running Kudu 1.6 with the exception of the |
| below-mentioned restrictions regarding secure clusters. |
| |
| The authentication features introduced in Kudu 1.3 place the following limitations |
| on wire compatibility between Kudu 1.6 and versions earlier than 1.3: |
| |
| * If a Kudu 1.6 cluster is configured with authentication or encryption set to "required", |
| clients older than Kudu 1.3 will be unable to connect. |
| * If a Kudu 1.6 cluster is configured with authentication and encryption set to "optional" |
| or "disabled", older clients will still be able to connect. |
| |
| [[rn_1.6.0_incompatible_changes]] |
| == Incompatible Changes in Kudu 1.6.0 |
| |
| [[rn_1.6.0_client_compatibility]] |
| === Client Library Compatibility |
| |
| * The Kudu 1.6 Java client library is API- and ABI-compatible with Kudu 1.5. Applications |
| written against Kudu 1.5 will compile and run against the Kudu 1.6 client library and |
| vice-versa. |
| |
| * The Kudu 1.6 {cpp} client is API- and ABI-forward-compatible with Kudu 1.5. |
| Applications written and compiled against the Kudu 1.5 client library will run without |
| modification against the Kudu 1.6 client library. Applications written and compiled |
| against the Kudu 1.6 client library will run without modification against the Kudu 1.5 |
| client library. |
| |
| * The Kudu 1.6 Python client is API-compatible with Kudu 1.5. Applications |
| written against Kudu 1.5 will continue to run against the Kudu 1.6 client |
| and vice-versa. |
| |
| [[rn_1.5.0]] |
| == Release notes specific to 1.5.0 |
| |
| [[rn_1.5.0_upgrade_notes]] |
| == Upgrade Notes |
| |
| * Kudu 1.5 now enables the optional ability to compute, store, and verify |
| checksums on all pieces of data stored on a server by default. Due to |
| storage format changes, downgrading to versions 1.3 or earlier is not |
| supported and will result in an error. |
| |
| * Spark 2.2+ requires Java 8 at runtime even though Kudu Spark 2.x integration |
| is Java 7 compatible. Spark 2.2 is the default dependency version as of |
| Kudu 1.5.0. |
| |
| * The kudu-spark-tools module has been renamed to kudu-spark2-tools_2.11 in |
| order to include the Spark and Scala base versions. This matches the pattern |
| used in the kudu-spark module and artifacts. |
| |
| * To improve security, world-readable Kerberos keytab files are no longer |
| accepted by default. Set `--allow_world_readable_credentials=true` to override |
| this behavior. See |
| link:https://issues.apache.org/jira/browse/KUDU-1955[KUDU-1955] for additional |
| details. |
| |
| [[rn_1.5.0_deprecations]] |
| == Deprecations |
| |
| * Support for Java 7 is deprecated as of Kudu 1.5.0 and may be removed in the |
| next major release. |
| |
| * Support for Spark 1 (kudu-spark_2.10) is deprecated as of Kudu 1.5.0 and may |
| be removed in the next minor release. |
| |
| [[rn_1.5.0_new_features]] |
| == New features |
| |
| * Tablet servers are now optionally able to tolerate disk failures at |
| startup. This feature is experimental; by default, Kudu will crash if it |
| experiences a disk failure. When enabled, tablets with any data on the failed |
| disk will not be opened and will be replicated as needed. To enable this, set |
| the `--crash_on_eio` flag to `false`. Additionally, there is a configurable |
| tradeoff between a newly added tablet's tolerance to disk failures and its |
| parallelization of I/O via the `--fs_target_data_dirs_per_tablet` flag. |
| Tablets that are spread across fewer disks are less likely to be affected by a |
| disk failure, at the cost of reduced parallelism. Note that the first |
| configured data directory and the WAL directory cannot currently tolerate disk |
| failures, and disk failures during run-time are still fatal. |
| |
| * Kudu server web UIs have a new configuration dashboard (/config) which |
| provides a high level summary of important security configuration values, such |
| as whether RPC authentication is required, or web server HTTPS encryption is |
| enabled. Other types of configuration will be added in future releases. |
| |
| * The `kudu` command line tool has two new features: `kudu tablet change_config |
| move_replica` and `kudu local_replica data_size`. The 'tablet change_config |
| move_replica' tool moves a tablet replica from one tablet server to another, |
| under the condition that the tablet is healthy. An operator can use this tool to |
| rebalance tablet replicas between tablet servers. The 'local_replica data size' |
| tool summarizes the space usage of a tablet, breaking it down by type of file, |
| column, and rowset. |
| |
| * kudu-client-tools now supports exporting CSV files and importing |
| Apache Parquet files. This feature is unstable and may change APIs and |
| functionality in future releases. |
| |
| * kudu-spark-tools now supports importing and exporting CSV, Apache Avro and |
| Apache Parquet files. This feature is unstable and may change APIs and |
| functionality in future releases. |
| |
| [[rn_1.5.0_optimizations_improvements]] |
| == Optimizations and improvements |
| |
| * The log block manager now performs disk synchronization in batches. |
| This optimization can significantly reduce the time taken to copy tablet data |
| from one server to another; in one case tablet copy time is reduced by 35%. |
| It also improves the general performance of flushes and compactions. |
| |
| * A new feature referred to as "tombstoned voting" is added to the Raft |
| consensus subsystem to allow tablet replicas in the `TABLET_DATA_TOMBSTONED` |
| state to vote in tablet leader elections. This feature increases Kudu's |
| stability and availability by improving the likelihood that Kudu will be able |
| to self-heal in more edge-case scenarios, such as when tablet copy operations |
| fail. See link:https://issues.apache.org/jira/browse/KUDU-871[KUDU-871] for |
| details. |
| |
| * The tablet on-disk size metric has been made more accurate. Previously, the |
| metric included only REDO deltas; it now counts all deltas. Additionally, the |
| metric includes the size of bloomfiles, ad hoc indexes, and the tablet |
| superblock. WAL segments and consensus metadata are still not counted. The |
| latter is very small compared to the size of data, but the former may be |
| significant depending on the workload (this will be resolved in a future |
| release). |
| |
| * The number of threads used by the Kudu tablet server has been further reduced. |
| Previously, each follower tablet replica used a dedicated thread to detect |
| leader tablet replica failures, and each leader replica used one dedicated |
| thread per follower to send Raft heartbeats to that follower. The work |
| performed by these dedicated threads has been reassigned to other threads. |
| Other improvements were made to facilitate better thread sharing by tablets. |
| For the purpose of capacity planning, expect the Kudu tablet server to create |
| one thread for every five "cold" (i.e. those not servicing writes) tablets, |
| and an additional three threads for every "hot" tablet. This will be further |
| improved upon in future Kudu releases. |
| |
| [[rn_1.5.0_fixed_issues]] |
| == Fixed Issues |
| |
| * The Java Kudu client now automatically requests new authentication tokens |
| after expiration. As a result, long-lived Java clients are now supported. See |
| link:https://issues.apache.org/jira/browse/KUDU-2013[KUDU-2013] for more |
| details. |
| |
| * Multiple Kerberos compatibility bugs have been fixed, including support |
| for environments with disabled reverse DNS, FreeIPA compatibility, principal |
| names including uppercase characters, and hosts without a FQDN. |
| |
| * A bug in the binary prefix decoder which could cause a tablet server 'check' |
| assertion crash has been fixed. The crash could only be triggered in very |
| specific scenarios; see |
| link:https://issues.apache.org/jira/browse/KUDU-2085[KUDU-2085] for additional |
| details. |
| |
| [[rn_1.5.0_wire_compatibility]] |
| == Wire Protocol compatibility |
| |
| Kudu 1.5.0 is wire-compatible with previous versions of Kudu: |
| |
| * Kudu 1.5 clients may connect to servers running Kudu 1.0 or later. If the client uses |
| features that are not available on the target server, an error will be returned. |
| * Rolling upgrade between Kudu 1.4 and Kudu 1.5 servers is believed to be possible |
| though has not been sufficiently tested. Users are encouraged to shut down all nodes |
| in the cluster, upgrade the software, and then restart the daemons on the new version. |
| * Kudu 1.0 clients may connect to servers running Kudu 1.5 with the exception of the |
| below-mentioned restrictions regarding secure clusters. |
| |
| The authentication features introduced in Kudu 1.3 place the following limitations |
| on wire compatibility between Kudu 1.5 and versions earlier than 1.3: |
| |
| * If a Kudu 1.5 cluster is configured with authentication or encryption set to "required", |
| clients older than Kudu 1.3 will be unable to connect. |
| * If a Kudu 1.5 cluster is configured with authentication and encryption set to "optional" |
| or "disabled", older clients will still be able to connect. |
| |
| [[rn_1.5.0_incompatible_changes]] |
| == Incompatible Changes in Kudu 1.5.0 |
| |
| [[rn_1.5.0_client_compatibility]] |
| === Client Library Compatibility |
| |
| * The Kudu 1.5 Java client library is API- and ABI-compatible with Kudu 1.4. Applications |
| written against Kudu 1.4 will compile and run against the Kudu 1.5 client library and |
| vice-versa, unless one of the following newly added APIs is used: |
| |
| * The Kudu 1.5 {cpp} client is API- and ABI-forward-compatible with Kudu 1.4. |
| Applications written and compiled against the Kudu 1.4 client library will run without |
| modification against the Kudu 1.5 client library. Applications written and compiled |
| against the Kudu 1.5 client library will run without modification against the Kudu 1.4 |
| client library. |
| |
| * The Kudu 1.5 Python client is API-compatible with Kudu 1.4. Applications |
| written against Kudu 1.4 will continue to run against the Kudu 1.5 client |
| and vice-versa. |
| |
| [[rn_1.4.0]] |
| == Release notes specific to 1.4.0 |
| |
| [[rn_1.4.0_upgrade_notes]] |
| == Upgrade Notes |
| |
| * The Maintenance Manager now fully uses the threads it's given (see the improvements |
| described further below), so it's now able to generate a lot more IO by flushing and |
| compacting more often. Generally, the recommended ratio of MM threads to data |
| directories is 1:3; operators of clusters above that ratio should be mindful of |
| this when upgrading. |
| |
| [[rn_1.4.0_new_features]] |
| == New features |
| |
| * The C++ and Java client libraries now support the ability to alter the |
| storage attributes (e.g. encoding and compression) and default value |
| of existing columns. Additionally, it is now possible to rename |
| a column which is part of a table's primary key. |
| |
| * The C++ client library now includes an experimental `KuduPartitioner` API which may |
| be used to efficiently map rows to their associated partitions and hosts. |
| This may be used to achieve better locality or distribution of writes |
| in client applications. |
| |
| * The Java client library now supports enabling fault tolerance on scanners. |
| Fault tolerant scanners are able to transparently recover from concurrent |
| server crashes at the cost of some performance overhead. See the Java |
| API documentation for more details on usage. |
| |
| * The `kudu` command line tool now includes a new advanced administrative |
| command `kudu remote_replica unsafe_change_config`. This command may be used |
| to force a tablet to perform an unsafe change of its Raft replication |
| configuration. This can be used to recover from scenarios such as a loss |
| of a majority of replicas, at the risk of losing edits. |
| |
| * The `kudu` command line tool now includes the `kudu fs check` command |
| which performs various offline consistency checks on the local on-disk |
| storage of a Kudu Tablet Server or Master. In addition to detecting |
| various inconsistencies or corruptions, it can also detect and remove |
| data blocks that are no longer referenced by any tablet but were not |
| fully removed from disk due to a crash or a bug in prior versions of Kudu. |
| |
| * The `kudu` command line tool can now be used to list the addresses and |
| identifiers of the servers in the cluster using either `kudu master list` |
| or `kudu tserver list`. |
| |
| * Kudu 1.4 now includes the optional ability to compute, store, and verify |
| checksums on all pieces of data stored on a server. Prior versions only |
| performed checksums on certain portions of the stored data. This feature |
| is not enabled by default since it makes a backward-incompatible change |
| to the on-disk formats and thus prevent downgrades. Kudu 1.5 will enable |
| the feature by default. |
| |
| == Optimizations and improvements |
| |
| * `kudu cluster ksck` now detects and reports new classes of |
| inconsistencies and issues. In particular, it is better able to |
| detect cases where a configuration change such as a replica eviction |
| or addition is pending but is unable to be committed. It also now |
| properly detects and reports cases where a tablet has no elected |
| leader. |
| |
| * The default size for Write Ahead Log (WAL) segments has been reduced |
| from 64MB to 8MB. Additionally, in the case that all replicas of a |
| tablet are fully up to date and data has been flushed from memory, |
| servers will now retain only a single WAL segment rather than |
| two. These changes are expected to reduce the average consumption of |
| disk space on the configured WAL disk by 16x, as well as improve the |
| startup speed of tablet servers by reducing the number and size of |
| WAL segments that need to be re-read. |
| |
| * The default on-disk storage system used by Kudu servers (Log Block Manager) |
| has been improved to compact its metadata and remove dead containers. |
| This compaction and garbage collection occurs only at startup. Thus, the |
| first startup after upgrade is expected to be longer than usual, and |
| subsequent restarts should be shorter. |
| |
| * The usability of the Kudu web interfaces has been improved, |
| particularly for the case where a server hosts many tablets or a |
| table has many partitions. Pages that list tablets now include |
| a top-level summary of tablet status and show the complete list |
| under a toggleable section. |
| |
| * The Maintenance Manager has been improved to improve utilization of the |
| configured maintenance threads. Previously, maintenance work would |
| only be scheduled a maximum of 4 times per second, but now maintenance |
| work will be scheduled immediately whenever any configured thread is |
| available. This can improve the throughput of write-heavy workloads. |
| |
| * The Maintenance Manager will now aggressively schedule flushes of |
| in-memory data when memory consumption crosses 60% of the configured |
| process-wide memory limit. The backpressure mechanism which begins |
| to throttle client writes has been accordingly adjusted to not begin |
| throttling until reaching 80% of the configured limit. These two |
| changes together result in improved write throughput, more consistent |
| latency, and fewer timeouts due to memory exhaustion. |
| |
| * Many performance improvements were made to write performance. Applications |
| which send large batches of writes to Kudu should see substantially |
| improved throughput in Kudu 1.4. |
| |
| * Several improvements were made to reduce the memory consumption of |
| Kudu Tablet Servers which hold large volumes of data. The specific |
| amount of memory saved varies depending on workload, but the expectation |
| is that approximately 350MB of excess peak memory usage has been eliminated |
| per TB of data stored. |
| |
| * The number of threads used by the Kudu Tablet Server has been reduced. |
| Previously, each tablet used a dedicated thread to append to its WAL. |
| Those threads now automatically stop running if there is no activity |
| on a given tablet for a short period of time. |
| |
| [[rn_1.4.0_fixed_issues]] |
| == Fixed Issues |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-2020[KUDU-2020] |
| Fixed an issue where re-replication after a failure would proceed |
| significantly slower than expected. This bug caused many tablets |
| to be unnecessarily copied multiple times before successfully |
| being considered re-replicated, resulting in significantly more |
| network and IO bandwidth usage than expected. Mean time to recovery |
| on clusters with large amounts of data is improved by up to 10x by this |
| fix. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1982[KUDU-1982] |
| Fixed an issue where the Java client would call `NetworkInterface.getByInetAddress` |
| very often, causing performance problems particularly on Windows |
| where this function can be quite slow. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1755[KUDU-1755] |
| Improved the accuracy of the `on_disk_size` replica metrics to |
| include the size consumed by bloom filters, primary key indexes, |
| and superblock metadata, and delta files. Note that, because the size |
| metric is now more accurate, the reported values are expected to |
| increase after upgrading to Kudu 1.4. This does not indicate that |
| replicas are using more space after the upgrade; rather, it is |
| now accurately reporting the amount of space that has always been |
| used. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1192[KUDU-1192] |
| Kudu servers will now periodically flush their log messages to disk |
| even if no `WARNING`-level messages have been logged. This makes it |
| easier to tail the logs to see progress output during normal startup. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1999[KUDU-1999] |
| Fixed the ability to run Spark jobs in "cluster" mode against |
| Kudu clusters secured by Kerberos. |
| |
| |
| [[rn_1.4.0_wire_compatibility]] |
| == Wire Protocol compatibility |
| |
| Kudu 1.4.0 is wire-compatible with previous versions of Kudu: |
| |
| * Kudu 1.4 clients may connect to servers running Kudu 1.0 or later. If the client uses |
| features that are not available on the target server, an error will be returned. |
| * Kudu 1.0 clients may connect to servers running Kudu 1.4 with the exception of the |
| below-mentioned restrictions regarding secure clusters. |
| * Rolling upgrade between Kudu 1.3 and Kudu 1.4 servers is believed to be possible |
| though has not been sufficiently tested. Users are encouraged to shut down all nodes |
| in the cluster, upgrade the software, and then restart the daemons on the new version. |
| |
| The authentication features introduced in Kudu 1.3 place the following limitations |
| on wire compatibility between Kudu 1.4 and versions earlier than 1.3: |
| |
| * If a Kudu 1.4 cluster is configured with authentication or encryption set to "required", |
| clients older than Kudu 1.3 will be unable to connect. |
| * If a Kudu 1.4 cluster is configured with authentication and encryption set to "optional" |
| or "disabled", older clients will still be able to connect. |
| |
| [[rn_1.4.0_incompatible_changes]] |
| == Incompatible Changes in Kudu 1.4.0 |
| |
| * Kudu servers, by default, will now only allow unencrypted or unauthenticated connections |
| from trusted subnets, which are private networks (127.0.0.0/8,10.0.0.0/8,172.16.0.0/12, |
| 192.168.0.0/16,169.254.0.0/16) and local subnets of all local network interfaces. |
| Unencrypted or unauthenticated connections from publicly routable IPs will be rejected, |
| even if encryption and authentication are not configured. |
| + |
| The trusted subnets can be configured using the `--trusted_subnets` flag, which can be set |
| to IP blocks represented in CIDR notation separated by comma. Set it to '0.0.0.0/0' to |
| allow unauthenticated connections from all remote IP addresses. However, if network access |
| is not otherwise restricted by a firewall, malicious users may be able to gain unauthorized |
| access. This can be mitigated if authentication and encryption are configured to be |
| required. |
| |
| [[rn_1.4.0_client_compatibility]] |
| === Client Library Compatibility |
| * The Kudu 1.4 Java client library is API- and ABI-compatible with Kudu 1.3. Applications |
| written against Kudu 1.3 will compile and run against the Kudu 1.4 client library and |
| vice-versa, unless one of the following newly added APIs is used: |
| ** `[Async]KuduScannerBuilder.setFaultTolerant(...)` |
| ** New methods in `AlterTableOptions`: `removeDefault`, `changeDefault`, `changeDesiredBlockSize`, |
| `changeEncoding`, `changeCompressionAlgorithm` |
| ** `KuduClient.updateLastPropagatedTimestamp` |
| ** `KuduClient.getLastPropagatedTimestamp` |
| ** New getters in `PartialRow`: `getBoolean`, `getByte`, `getShort`, `getInt`, `getLong`, |
| `getFloat`, `getDouble`, `getString`, `getBinaryCopy`, `getBinary`, `isNull`, |
| `isSet`. |
| |
| |
| * The Kudu 1.4 {cpp} client is API- and ABI-forward-compatible with Kudu 1.3. |
| Applications written and compiled against the Kudu 1.3 client library will run without |
| modification against the Kudu 1.4 client library. Applications written and compiled |
| against the Kudu 1.4 client library will run without modification against the Kudu 1.3 |
| client library unless they use one of the following new APIs: |
| ** `KuduPartitionerBuilder` |
| ** `KuduPartitioner |
| ** `KuduScanner::SetRowFormatFlags` (unstable API) |
| ** `KuduScanBatch::direct_data`, `KuduScanBatch::indirect_data` (unstable API) |
| |
| * The Kudu 1.4 Python client is API-compatible with Kudu 1.3. Applications |
| written against Kudu 1.3 will continue to run against the Kudu 1.4 client |
| and vice-versa. |
| |
| [[rn_1.3.0]] |
| == Release notes specific to 1.3.0 |
| |
| [[rn_1.3.0_new_features]] |
| == New features |
| |
| * Kudu 1.3 adds support for strong authentication based on Kerberos. This optional feature |
| allows users to authenticate themselves using Kerberos tickets, and also provides |
| mutual authentication of servers using Kerberos credentials stored in keytabs. This |
| feature is optional, but recommended for deployments requiring security. |
| |
| * Kudu 1.3 adds support for encryption of data on the network using Transport Layer Security |
| (TLS). Kudu will now use TLS to encrypt all network traffic between clients and servers as |
| well as any internal traffic among servers, with the exception of traffic determined to |
| be within a localhost network connection. Encryption is enabled by default whenever it can |
| be determined that both the client and server support the feature. |
| |
| * Kudu 1.3 adds coarse-grained service-level authorization of access to the cluster. |
| The operator may set up lists of permitted users who may act as administrators and |
| as clients of the cluster. Combined with the strong authentication feature described |
| above, this can enable a secure environment for some use cases. Note that fine-grained |
| access control (e.g. table-level or column-level) is not yet supported. |
| |
| * Kudu 1.3 adds a background task to tablet servers which removes historical versions of |
| data which have fallen behind the configured data retention time. This reduces disk space |
| usage in all workloads, but particularly in those with a higher volume of updates or |
| upserts. |
| |
| * Kudu now incorporates Google Breakpad, a library which writes crash reports in |
| the case of a server crash. These reports can be found within the configured log directory, |
| and can be useful during bug diagnosis. |
| |
| |
| == Optimizations and improvements |
| |
| * Kudu servers will now change the file permissions of data directories and contained |
| data files based on a new configuration flag `--umask`. As a result, after upgrading, |
| permissions on disk may be more restrictive than in previous versions. The new default |
| configuration improves data security. |
| |
| * Kudu's web UI will now redact strings which may include sensitive user data. For example, |
| the monitoring page which shows in-progress scans no longer includes the scanner predicate |
| values. The tracing and RPC diagnostics endpoints no longer include contents of RPCs which |
| may include table data. |
| |
| * By default, Kudu now reserves 1% of each configured data volume as free space. If a volume |
| is seen to have less than 1% of disk space free, Kudu will stop writing to that volume |
| to avoid completely filling up the disk. |
| |
| * The default encoding for numeric columns (int, float, and double) has been changed |
| to `BIT_SHUFFLE`. The default encoding for binary and string columns has been |
| changed to `DICT_ENCODING`. Dictionary encoding automatically falls back to the old |
| default (`PLAIN`) when cardinality is too high to be effectively encoded. |
| + |
| These new defaults match the default behavior of other storage mechanisms such as |
| Apache Parquet and are likely to perform better out of the box. |
| |
| * Kudu now uses `LZ4` compression when writing its Write Ahead Log (WAL). This improves |
| write performance and stability for many use cases. |
| |
| * Kudu now uses `LZ4` compression when writing delta files. This can improve both |
| read and write performance as well as save substantial disk usage, especially |
| for workloads involving a high number of updates or upserts containing compressible |
| data. |
| |
| * The Kudu API now supports the ability to express `IS NULL` and `IS NOT NULL` predicates |
| on scanners. The Spark DataSource integration will take advantage of these new |
| predicates when possible. |
| |
| * Both {cpp} and Java clients have been optimized to prune partitions more effectively |
| when performing scans using the `IN (...)` predicate. |
| |
| * The exception messages produced by the Java client are now truncated to a maximum length |
| of 32KB. |
| |
| |
| [[rn_1.3.0_fixed_issues]] |
| == Fixed Issues |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1893[KUDU-1893] |
| Fixed a critical bug in which wrong results would be returned when evaluating |
| predicates applied to columns added using the `ALTER TABLE` operation. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1905[KUDU-1905] |
| Fixed a crash after inserting a row sharing a primary key with a recently-deleted |
| row in tables where the primary key is comprised of all of the columns. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1899[KUDU-1899] |
| Fixed a crash after inserting a row with an empty string as the single-column |
| primary key. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1904[KUDU-1904] |
| Fixed a potential crash when performing random reads against a column using RLE |
| encoding and containing long runs of NULL values. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1853[KUDU-1853] |
| Fixed an issue where disk space could be leaked on servers which experienced an error |
| during the process of copying tablet data from another server. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1856[KUDU-1856] |
| Fixed an issue in which disk space could be leaked by Kudu servers storing data on |
| partitions using the XFS file system. Any leaked disk space will be automatically |
| recovered upon upgrade. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1888[KUDU-1888], |
| link:https://issues.apache.org/jira/browse/KUDU-1906[KUDU-1906] |
| Fixed multiple issues in the Java client where operation callbacks would never be |
| triggered, causing the client to hang. |
| |
| |
| [[rn_1.3.0_wire_compatibility]] |
| == Wire Protocol compatibility |
| |
| Kudu 1.3.0 is wire-compatible with previous versions of Kudu: |
| |
| * Kudu 1.3 clients may connect to servers running Kudu 1.0. If the client uses features |
| that are not available on the target server, an error will be returned. |
| * Kudu 1.0 clients may connect to servers running Kudu 1.3 with the exception of the |
| below-mentioned restrictions regarding secure clusters. |
| * Rolling upgrade between Kudu 1.2 and Kudu 1.3 servers is believed to be possible |
| though has not been sufficiently tested. Users are encouraged to shut down all nodes |
| in the cluster, upgrade the software, and then restart the daemons on the new version. |
| |
| The authentication features newly introduced in Kudu 1.3 place the following limitations |
| on wire compatibility with older versions: |
| |
| * If a Kudu 1.3 cluster is configured with authentication or encryption set to "required", |
| older clients will be unable to connect. |
| * If a Kudu 1.3 cluster is configured with authentication and encryption set to "optional" |
| or "disabled", older clients will still be able to connect. |
| |
| |
| [[rn_1.3.0_incompatible_changes]] |
| == Incompatible Changes in Kudu 1.3.0 |
| |
| * Due to storage format changes in Kudu 1.3, downgrade from Kudu 1.3 to earlier versions |
| is not supported. After upgrading to Kudu 1.3, attempting to restart with an earlier |
| version will result in an error. |
| |
| * In order to support running MapReduce and Spark jobs on secure clusters, these |
| frameworks now connect to the cluster at job submission time to retrieve authentication |
| credentials which can later be used by the tasks to be spawned. This means that |
| the process submitting jobs to Kudu clusters must have direct access to that cluster. |
| |
| * The embedded web servers in Kudu processes now specify the `X-Frame-Options: DENY` HTTP |
| header which prevents embedding Kudu web pages in HTML `iframe` elements. |
| |
| [[rn_1.3.0_client_compatibility]] |
| === Client Library Compatibility |
| |
| * The Kudu 1.3 Java client library is API- and ABI-compatible with Kudu 1.2. Applications |
| written against Kudu 1.2 will compile and run against the Kudu 1.3 client library and |
| vice-versa, unless one of the following newly added APIs is used: |
| ** `[Async]KuduClient.exportAuthenticationCredentials(...)` (unstable API) |
| ** `[Async]KuduClient.importAuthenticationCredentials(...)` (unstable API) |
| ** `[Async]KuduClient.getMasterAddressesAsString()` |
| ** `KuduPredicate.newIsNotNullPredicate()` |
| ** `KuduPredicate.newIsNullPredicate()` |
| |
| * The Kudu 1.3 {cpp} client is API- and ABI-forward-compatible with Kudu 1.2. |
| Applications written and compiled against the Kudu 1.2 client library will run without |
| modification against the Kudu 1.3 client library. Applications written and compiled |
| against the Kudu 1.3 client library will run without modification against the Kudu 1.2 |
| client library unless they use one of the following new APIs: |
| ** `kudu::DisableOpenSSLInitialization()` |
| ** `KuduClientBuilder::import_authentication_credentials(...)` |
| ** `KuduClient::ExportAuthenticationCredentials(...)` |
| ** `KuduClient::NewIsNotNullPredicate(...)` |
| ** `KuduClient::NewIsNullPredicate(...)` |
| |
| * The Kudu 1.3 Python client is API-compatible with Kudu 1.2. Applications |
| written against Kudu 1.2 will continue to run against the Kudu 1.3 client |
| and vice-versa. |
| |
| |
| [[rn_1.2.0]] |
| == Release notes specific to 1.2.0 |
| |
| [[rn_1.2.0_new_features]] |
| == New features |
| |
| * Kudu clients and servers now redact user data such as cell values |
| from log messages, Java exception messages, and `Status` strings. |
| User metadata such as table names, column names, and partition |
| bounds are not redacted. |
| + |
| Redaction is enabled by default, but may be disabled by setting the new |
| `log_redact_user_data` flag to `false`. |
| |
| * Kudu's ability to provide consistency guarantees has been substantially |
| improved: |
| |
| ** Replicas now correctly track their "safe timestamp". This timestamp |
| is the maximum timestamp at which reads are guaranteed to be |
| repeatable. |
| |
| ** A scan created using the `SCAN_AT_SNAPSHOT` mode will now |
| either wait for the requested snapshot to be "safe" at the replica |
| being scanned, or be re-routed to a replica where the requested |
| snapshot is "safe". This ensures that all such scans are repeatable. |
| |
| ** Kudu Tablet Servers now properly retain historical data when a row |
| with a given primary key is inserted and deleted, followed by the |
| insertion of a new row with the same key. Previous versions of Kudu |
| would not retain history in such situations. This allows the server |
| to return correct results for snapshot scans with a timestamp in the |
| past, even in the presence of such "reinsertion" scenarios. |
| |
| ** The Kudu clients now automatically retain the timestamp of their latest |
| successful read or write operation. Scans using the `READ_AT_SNAPSHOT` mode |
| without a client-provided timestamp automatically assign a timestamp |
| higher than the timestamp of their most recent write. Writes also propagate |
| the timestamp, ensuring that sequences of operations with causal dependencies |
| between them are assigned increasing timestamps. Together, these changes |
| allow clients to achieve read-your-writes consistency, and also ensure |
| that snapshot scans performed by other clients return causally-consistent |
| results. |
| |
| * Kudu servers now automatically limit the number of log files. |
| The number of log files retained can be configured using the |
| `max_log_files` flag. By default, 10 log files will be retained |
| at each severity level. |
| |
| == Optimizations and improvements |
| |
| * The logging in the Java and {cpp} clients has been substantially quieted. |
| Clients no longer log messages in normal operation unless there |
| is some kind of error. |
| |
| * The {cpp} client now includes a `KuduSession::SetErrorBufferSpace` |
| API which can limit the amount of memory used to buffer |
| errors from asynchronous operations. |
| |
| * The Java client now fetches tablet locations from the Kudu Master |
| in batches of 1000, increased from batches of 10 in prior versions. |
| This can substantially improve the performance of Spark and Impala |
| queries running against Kudu tables with large numbers of tablets. |
| |
| * Table metadata lock contention in the Kudu Master was substantially |
| reduced. This improves the performance of tablet location lookups on |
| large clusters with a high degree of concurrency. |
| |
| * Lock contention in the Kudu Tablet Server during high-concurrency |
| write workloads was also reduced. This can reduce CPU consumption and |
| improve performance when a large number of concurrent clients are writing |
| to a smaller number of a servers. |
| |
| * Lock contention when writing log messages has been substantially reduced. |
| This source of contention could cause high tail latencies on requests, |
| and when under high load could contribute to cluster instability |
| such as election storms and request timeouts. |
| |
| * The `BITSHUFFLE` column encoding has been optimized to use the `AVX2` |
| instruction set present on processors including Intel(R) Sandy Bridge |
| and later. Scans on `BITSHUFFLE`-encoded columns are now up to 30% faster. |
| |
| * The `kudu` tool now accepts hyphens as an alternative to underscores |
| when specifying actions. For example, `kudu local-replica copy-from-remote` |
| may be used as an alternative to `kudu local_replica copy_from_remote`. |
| |
| [[rn_1.2.0_fixed_issues]] |
| == Fixed Issues |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1508[KUDU-1508] |
| Fixed a long-standing issue in which running Kudu on `ext4` file systems |
| could cause file system corruption. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1399[KUDU-1399] |
| Implemented an LRU cache for open files, which prevents running out of |
| file descriptors on long-lived Kudu clusters. By default, Kudu will |
| limit its file descriptor usage to half of its configured `ulimit`. |
| |
| * link:http://gerrit.cloudera.org:8080/5192[Gerrit #5192] |
| Fixed an issue which caused data corruption and crashes in the case that |
| a table had a non-composite (single-column) primary key, and that column |
| was specified to use `DICT_ENCODING` or `BITSHUFFLE` encodings. If a |
| table with an affected schema was written in previous versions of Kudu, |
| the corruption will not be automatically repaired; users are encouraged |
| to re-insert such tables after upgrading to Kudu 1.2 or later. |
| |
| * link:http://gerrit.cloudera.org:8080/5541[Gerrit #5541] |
| Fixed a bug in the Spark `KuduRDD` implementation which could cause |
| rows in the result set to be silently skipped in some cases. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1551[KUDU-1551] |
| Fixed an issue in which the tablet server would crash on restart in the |
| case that it had previously crashed during the process of allocating |
| a new WAL segment. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1764[KUDU-1764] |
| Fixed an issue where Kudu servers would leak approximately 16-32MB of disk |
| space for every 10GB of data written to disk. After upgrading to Kudu |
| 1.2 or later, any disk space leaked in previous versions will be |
| automatically recovered on startup. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1750[KUDU-1750] |
| Fixed an issue where the API to drop a range partition would drop any |
| partition with a matching lower _or_ upper bound, rather than any partition |
| with matching lower _and_ upper bound. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1766[KUDU-1766] |
| Fixed an issue in the Java client where equality predicates which compared |
| an integer column to its maximum possible value (e.g. `Integer.MAX_VALUE`) |
| would return incorrect results. |
| |
| * link:https://issues.apache.org/jira/browse/KUDU-1780[KUDU-1780] |
| Fixed the `kudu-client` Java artifact to properly shade classes in the |
| `com.google.thirdparty` namespace. The lack of proper shading in prior |
| releases could cause conflicts with certain versions of Google Guava. |
| |
| * link:http://gerrit.cloudera.org:8080/5327[Gerrit #5327] |
| Fixed shading issues in the `kudu-flume-sink` Java artifact. The sink |
| now expects that Hadoop dependencies are provided by Flume, and properly |
| shades the Kudu client's dependencies. |
| |
| * Fixed a few issues using the Python client library from Python 3. |
| |
| |
| [[rn_1.2.0_wire_compatibility]] |
| == Wire Protocol compatibility |
| |
| Kudu 1.2.0 is wire-compatible with previous versions of Kudu: |
| |
| * Kudu 1.2 clients may connect to servers running Kudu 1.0. If the client uses features |
| that are not available on the target server, an error will be returned. |
| * Kudu 1.0 clients may connect to servers running Kudu 1.2 without limitations. |
| * Rolling upgrade between Kudu 1.1 and Kudu 1.2 servers is believed to be possible |
| though has not been sufficiently tested. Users are encouraged to shut down all nodes |
| in the cluster, upgrade the software, and then restart the daemons on the new version. |
| |
| [[rn_1.2.0_incompatible_changes]] |
| == Incompatible Changes in Kudu 1.2.0 |
| |
| * The replication factor of tables is now limited to a maximum of 7. In addition, |
| it is no longer allowed to create a table with an even replication factor. |
| |
| * The `GROUP_VARINT` encoding is now deprecated. Kudu servers have never supported |
| this encoding, and now the client-side constant has been deprecated to match the |
| server's capabilities. |
| |
| === New Restrictions on Data, Schemas, and Identifiers |
| |
| Kudu 1.2.0 introduces several new restrictions on schemas, cell size, and identifiers: |
| |
| Number of Columns:: By default, Kudu will not permit the creation of tables with |
| more than 300 columns. We recommend schema designs that use fewer columns for best |
| performance. |
| |
| Size of Cells:: No individual cell may be larger than 64KB. The cells making up a |
| a composite key are limited to a total of 16KB after the internal composite-key encoding |
| done by Kudu. Inserting rows not conforming to these limitations will result in errors |
| being returned to the client. |
| |
| Valid Identifiers:: Identifiers such as column and table names are now restricted to |
| be valid UTF-8 strings. Additionally, a maximum length of 256 characters is enforced. |
| |
| [[rn_1.2.0_client_compatibility]] |
| === Client Library Compatibility |
| |
| * The Kudu 1.2 Java client is API- and ABI-compatible with Kudu 1.1. Applications |
| written against Kudu 1.1 will compile and run against the Kudu 1.2 client and |
| vice-versa. |
| |
| * The Kudu 1.2 {cpp} client is API- and ABI-forward-compatible with Kudu 1.1. |
| Applications written and compiled against the Kudu 1.1 client will run without |
| modification against the Kudu 1.2 client. Applications written and compiled |
| against the Kudu 1.2 client will run without modification against the Kudu 1.1 |
| client unless they use one of the following new APIs: |
| ** `kudu::DisableSaslInitialization()` |
| ** `KuduSession::SetErrorBufferSpace(...)` |
| |
| * The Kudu 1.2 Python client is API-compatible with Kudu 1.1. Applications |
| written against Kudu 1.1 will continue to run against the Kudu 1.2 client |
| and vice-versa. |
| |
| |
| [[rn_1.1.0]] |
| == Release notes specific to 1.1.0 |
| |
| [[rn_1.1.0_new_features]] |
| == New features |
| |
| * The Python client has been brought up to feature parity with the Java and {cpp} clients |
| and as such the package version will be brought to 1.1 with this release (from 0.3). A |
| list of the highlights can be found below. |
| ** Improved Partial Row semantics |
| ** Range partition support |
| ** Scan Token API |
| ** Enhanced predicate support |
| ** Support for all Kudu data types (including a mapping of Python's `datetime.datetime` to |
| `UNIXTIME_MICROS`) |
| ** Alter table support |
| ** Enabled Read at Snapshot for Scanners |
| ** Enabled Scanner Replica Selection |
| ** A few bug fixes for Python 3 in addition to various other improvements. |
| |
| * IN LIST predicate pushdown support was added to allow optimized execution of filters which |
| match on a set of column values. Support for Spark, Map Reduce and Impala queries utilizing |
| IN LIST pushdown is not yet complete. |
| |
| * The Java client now features client-side request tracing in order to help troubleshoot timeouts. |
| Error messages are now augmented with traces that show which servers were contacted before the |
| timeout occurred instead of just the last error. The traces also contain RPCs that were |
| required to fulfill the client's request, such as contacting the master to discover a tablet's |
| location. Note that the traces are not available for successful requests and are not |
| programmatically queryable. |
| |
| == Optimizations and improvements |
| |
| * Kudu now publishes JAR files for Spark 2.0 compiled with Scala 2.11 along with the |
| existing Spark 1.6 JAR compiled with Scala 2.10. |
| |
| * The Java client now allows configuring scanners to read from the closest replica instead of |
| the known leader replica. The default remains the latter. Use the relevant `ReplicaSelection` |
| enum with the scanner's builder to change this behavior. |
| |
| * Tablet servers use a new policy for retaining write-ahead log (WAL) segments. |
| Previously, servers used the 'log_min_segments_to_retain' flag to prioritize |
| any flushes which were retaining log segments past the configured value (default 2). |
| This policy caused servers to flush in-memory data more frequently than necessary, |
| limiting write performance. |
| + |
| The new policy introduces a new flag 'log_target_replay_size_mb' which |
| determines the threshold at which write-ahead log retention will prioritize flushes. |
| The new flag is considered experimental and users should not need to modify |
| its value. |
| + |
| The improved policy has been seen to improve write performance in some use cases |
| by a factor of 2x relative to the old policy. |
| |
| * Kudu's implementation of the Raft consensus algorithm has been improved to include |
| a "pre-election" phase. This can improve the stability of tablet leader election |
| in high-load scenarios, especially if each server hosts a high number of tablets. |
| |
| * Tablet server start-up time has been substantially improved in the case that |
| the server contains a high number of tombstoned tablet replicas. |
| |
| === Command line tools |
| |
| * The tool `kudu tablet leader_step_down` has been added to manually force a leader to step down. |
| * The tool `kudu remote_replica copy` has been added to manually copy a replica from |
| one running tablet server to another. |
| * The tool `kudu local_replica delete` has been added to delete a replica of a tablet. |
| * The `kudu test loadgen` tool has been added to replace the obsoleted |
| `insert-generated-rows` standalone binary. The new tool is enriched with |
| additional functionality and can be used to run load generation tests against |
| a Kudu cluster. |
| |
| == Wire protocol compatibility |
| |
| Kudu 1.1.0 is wire-compatible with previous versions of Kudu: |
| |
| * Kudu 1.1 clients may connect to servers running Kudu 1.0. If the client uses the new |
| 'IN LIST' predicate type, an error will be returned. |
| * Kudu 1.0 clients may connect to servers running Kudu 1.1 without limitations. |
| * Rolling upgrade between Kudu 1.0 and Kudu 1.1 servers is believed to be possible |
| though has not been sufficiently tested. Users are encouraged to shut down all nodes |
| in the cluster, upgrade the software, and then restart the daemons on the new version. |
| |
| [[rn_1.1.0_incompatible_changes]] |
| == Incompatible changes in Kudu 1.1.0 |
| |
| === Client APIs ({cpp}/Java/Python) |
| |
| * The {cpp} client no longer requires the |
| link:https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html[old gcc5 ABI]. |
| Which ABI is actually used depends on the compiler configuration. Some new distros |
| (e.g. Ubuntu 16.04) will use the new ABI. Your application must use the same ABI as is |
| used by the client library; an easy way to guarantee this is to use the same compiler |
| to build both. |
| |
| * The {cpp} client's `KuduSession::CountBufferedOperations()` method is |
| deprecated. Its behavior is inconsistent unless the session runs in the |
| `MANUAL_FLUSH` mode. Instead, to get number of buffered operations, count |
| invocations of the `KuduSession::Apply()` method since last |
| `KuduSession::Flush()` call or, if using asynchronous flushing, since last |
| invocation of the callback passed into `KuduSession::FlushAsync()`. |
| |
| * The Java client's `OperationResponse.getWriteTimestamp` method was renamed to `getWriteTimestampRaw` |
| to emphasize that it doesn't return milliseconds, unlike what its Javadoc indicated. The renamed |
| method was also hidden from the public APIs and should not be used. |
| |
| * The Java client's sync API (`KuduClient`, `KuduSession`, `KuduScanner`) used to throw either |
| a `NonRecoverableException` or a `TimeoutException` for a timeout, and now it's only possible for the |
| client to throw the former. |
| |
| * The Java client's handling of errors in `KuduSession` was modified so that subclasses of |
| `KuduException` are converted into RowErrors instead of being thrown. |
| |
| [[rn_1.0.1]] |
| == Release notes specific to 1.0.1 |
| |
| Apache Kudu 1.0.1 is a bug fix release, with no new features or backwards |
| incompatible changes. |
| |
| [[rn_1.0.1_fixed_issues]] |
| === Fixed Issues |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1681[KUDU-1681] Fixed a bug in |
| the tablet server which could cause a crash when the DNS lookup during master |
| heartbeat failed. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1660[KUDU-1660]: Fixed a bug |
| which would cause the Kudu master and tablet server to fail to start on single |
| CPU systems. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1651[KUDU-1652]: Fixed a bug |
| that would cause the C++ client, tablet server, and Java client to crash or |
| throw an exception when attempting to scan a table with a predicate which |
| simplifies to `IS NOT NULL` on a non-nullable column. For instance, setting a |
| `<= 127` predicate on an `INT8` column could trigger this bug, since the |
| predicate only filters null values. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1651[KUDU-1651]: Fixed a bug |
| that would cause the tablet server to crash when evaluating a scan with |
| predicates over a dictionary encoded column containing an entire block of null |
| values. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1623[KUDU-1623]: Fixed a bug |
| that would cause the tablet server to crash when handling UPSERT operations |
| that only set values for the primary key columns. |
| |
| - link:http://gerrit.cloudera.org:8080/4488[Gerrit #4488] Fixed a bug in the |
| Java client's KuduException class which could cause an unexpected |
| NullPointerException to be thrown when the exception did not have an |
| associated message. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1090[KUDU-1090] Fixed a bug in |
| the memory tracker which could cause a rare crash during tablet server |
| startup. |
| |
| [[rn_1.0.0]] |
| == Release notes specific to 1.0.0 |
| |
| After approximately a year of beta releases, Apache Kudu has reached version 1.0. |
| This version number signifies that the development team feels that Kudu is stable |
| enough for usage in production environments. |
| |
| If you are new to Kudu, check out its list of link:index.html[features and benefits]. |
| |
| [[rn_1.0.0_new_features]] |
| === New features |
| |
| Kudu 1.0.0 delivers a number of new features, bug fixes, and optimizations. |
| |
| - Removal of multiversion concurrency control (MVCC) history is now supported. |
| This is known as tablet history GC. This allows Kudu to reclaim disk space, |
| where previously Kudu would keep a full history of all changes made to a |
| given table since the beginning of time. Previously, the only way to reclaim |
| disk space was to drop a table. |
| + |
| Kudu will still keep historical data, and the amount of history retained is |
| controlled by setting the configuration flag `--tablet_history_max_age_sec`, |
| which defaults to 15 minutes (expressed in seconds). The timestamp |
| represented by the current time minus `tablet_history_max_age_sec` is known |
| as the ancient history mark (AHM). When a compaction or flush occurs, Kudu |
| will remove the history of changes made prior to the ancient history mark. |
| This only affects historical data; currently-visible data will not be |
| removed. A specialized maintenance manager background task to remove existing |
| "cold" historical data that is not in a row affected by the normal compaction |
| process will be added in a future release. |
| |
| - Most of Kudu's command line tools have been consolidated under a new |
| top-level `kudu` tool. This reduces the number of large binaries distributed |
| with Kudu and also includes much-improved help output. |
| |
| - The Kudu Flume Sink now supports processing events containing Avro-encoded |
| records, using the new `AvroKuduOperationsProducer`. |
| |
| - Administrative tools including `kudu cluster ksck` now support running |
| against multi-master Kudu clusters. |
| |
| - The output of the `ksck` tool is now colorized and much easier to read. |
| |
| - The {cpp} client API now supports writing data in `AUTO_FLUSH_BACKGROUND` mode. |
| This can provide higher throughput for ingest workloads. |
| |
| === Optimizations and improvements |
| |
| - The performance of comparison predicates on dictionary-encoded columns has |
| been substantially optimized. Users are encouraged to use dictionary encoding |
| on any string or binary columns with low cardinality, especially if these |
| columns will be filtered with predicates. |
| |
| - The Java client is now able to prune partitions from scanners based on the |
| provided predicates. For example, an equality predicate on a hash-partitioned |
| column will now only access those tablets that could possibly contain matching |
| data. This is expected to improve performance for the Spark integration as well |
| as applications using the Java client API. |
| |
| - The performance of compaction selection in the tablet server has been |
| substantially improved. This can increase the efficiency of the background |
| maintenance threads and improve overall throughput of heavy write workloads. |
| |
| - The policy by which the tablet server retains write-ahead log (WAL) files has |
| been improved so that it takes into account other replicas of the tablet. |
| This should help mitigate the spurious eviction of tablet replicas on machines |
| that temporarily lag behind the other replicas. |
| |
| === Wire protocol compatibility |
| |
| Kudu 1.0.0 maintains client-server wire-compatibility with previous releases. |
| Applications using the Kudu client libraries may be upgraded either |
| before, at the same time, or after the Kudu servers. |
| |
| Kudu 1.0.0 does _not_ maintain server-server wire compatibility with previous |
| releases. Therefore, rolling upgrades between earlier versions of Kudu and |
| Kudu 1.0.0 are not supported. |
| |
| [[rn_1.0.0_incompatible_changes]] |
| === Incompatible changes in Kudu 1.0.0 |
| |
| ==== Command line tools |
| |
| - The `kudu-pbc-dump` tool has been removed. The same functionality is now |
| implemented as `kudu pbc dump`. |
| |
| - The `kudu-ksck` tool has been removed. The same functionality is now |
| implemented as `kudu cluster ksck`. |
| |
| - The `cfile-dump` tool has been removed. The same functionality is now |
| implemented as `kudu fs cfile dump`. |
| |
| - The `log-dump` tool has been removed. The same functionality is now |
| implemented as `kudu wal dump` and `kudu local_replica dump wals`. |
| |
| - The `kudu-admin` tool has been removed. The same functionality is now |
| implemented within `kudu table` and `kudu tablet`. |
| |
| - The `kudu-fs_dump` tool has been removed. The same functionality is now |
| implemented as `kudu fs dump`. |
| |
| - The `kudu-ts-cli` tool has been removed. The same functionality is now |
| implemented within `kudu master`, `kudu remote_replica`, and `kudu tserver`. |
| |
| - The `kudu-fs_list` tool has been removed and some similar useful |
| functionality has been moved under 'kudu local_replica'. |
| |
| ==== Configuration flags |
| |
| - Some configuration flags are now marked as 'unsafe' and 'experimental'. Such flags |
| are disallowed by default. Users may access these flags by enabling the additional |
| flags `--unlock_unsafe_flags` and `--unlock_experimental_flags`. Usage of such flags |
| is not recommended, as the flags may be removed or modified with no deprecation period |
| and without notice in future Kudu releases. |
| |
| ==== Client APIs ({cpp}/Java/Python) |
| |
| - The `TIMESTAMP` column type has been renamed to `UNIXTIME_MICROS` in order to |
| reduce confusion between Kudu's timestamp support and the timestamps supported |
| by other systems such as Apache Hive and Apache Impala (incubating). Existing |
| tables will automatically be updated to use the new name for the type. |
| + |
| Clients upgrading to the new client libraries must move to the new name for |
| the type. Clients using old client libraries will continue to operate using |
| the old type name, even when connected to clusters that have been |
| upgraded. Similarly, if clients are upgraded before servers, existing |
| timestamp columns will be available using the new type name. |
| |
| |
| - `KuduSession` methods in the {cpp} library are no longer advertised as thread-safe |
| to have one set of semantics for both {cpp} and Java Kudu client libraries. |
| |
| - The `KuduScanToken::TabletServers` method in the {cpp} library has been removed. |
| The same information can now be found in the KuduScanToken::tablet method. |
| |
| ==== Apache Flume Integration |
| |
| - The `KuduEventProducer` interface used to process Flume events into Kudu operations |
| for the Kudu Flume Sink has changed, and has been renamed `KuduOperationsProducer`. |
| The existing `KuduEventProducer`s have been updated for the new interface, and have |
| been renamed similarly. |
| |
| [[rn_0.10.0]] |
| == Release notes specific to 0.10.0 |
| |
| Kudu 0.10.0 delivers a number of new features, bug fixes, and optimizations, |
| detailed below. |
| |
| Kudu 0.10.0 maintains wire-compatibility with previous releases, meaning |
| that applications using the Kudu client libraries may be upgraded either |
| before, at the same time, or after the Kudu servers. However, if you begin |
| using new features of Kudu 0.10.0 such as manually range-partitioned tables, |
| you must first upgrade all clients to this release. |
| |
| This release does not maintain full Java API or ABI compatibility with |
| Kudu 0.9.x due to a package rename and some other small changes. See below for details. |
| |
| See also +++<a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20KUDU%20AND%20status%20%3D%20Resolved |
| %20AND%20fixVersion%20%3D%200.10.0">JIRAs resolved |
| for Kudu 0.10.0</a>+++ and +++<a href="https://github.com/apache/kudu/compare/0.9.1...0.10.0">Git |
| changes between 0.9.1 and 0.10.0</a>+++. |
| |
| [[rn_0.10.0_incompatible_changes]] |
| === Incompatible changes and deprecated APIs in 0.10.0 |
| |
| - link:http://gerrit.cloudera.org:8080/3737[Gerrit #3737] The Java client has been repackaged |
| under `org.apache.kudu` instead of `org.kududb`. Import statements for Kudu classes must |
| be modified in order to compile against 0.10.0. Wire compatibility is maintained. |
| |
| - link:https://gerrit.cloudera.org/#/c/3055/[Gerrit #3055] The Java client's |
| synchronous API methods now throw `KuduException` instead of `Exception`. |
| Existing code that catches `Exception` should still compile, but introspection of an |
| exception's message may be impacted. This change was made to allow thrown exceptions to be |
| queried more easily using `KuduException.getStatus` and calling one of `Status`'s methods. |
| For example, an operation that tries to delete a table that doesn't exist would return a |
| `Status` that returns true when queried on `isNotFound()`. |
| |
| - The Java client's `KuduTable.getTabletsLocations` set of methods is now |
| deprecated. Additionally, they now take an exclusive end partition key instead |
| of an inclusive key. Applications are encouraged to use the scan tokens API |
| instead of these methods in the future. |
| |
| - The C++ API for specifying split points on range-partitioned tables has been improved |
| to make it easier for callers to properly manage the ownership of the provided rows. |
| + |
| The `TableCreator::split_rows` API took a `vector<const KuduPartialRow*>`, which |
| made it very difficult for the calling application to do proper error handling with |
| cleanup when setting the fields of the `KuduPartialRow`. This API has been now been |
| deprecated and replaced by a new method `TableCreator::add_range_split` which allows |
| easier use of smart pointers for safe memory management. |
| |
| - The Java client's internal buffering has been reworked. Previously, the number of |
| buffered write operations was constrained on a per-tablet-server basis. Now, the configured |
| maximum buffer size constrains the total number of buffered operations across all |
| tablet servers in the cluster. This provides a more consistent bound on the memory |
| usage of the client regardless of the size of the cluster to which it is writing. |
| + |
| This change can negatively affect the write performance of Java clients which rely on |
| buffered writes. Consider using the `setMutationBufferSpace` API to increase a |
| session's maximum buffer size if write performance seems to be degraded after upgrading |
| to Kudu 0.10.0. |
| |
| - The "remote bootstrap" process used to copy a tablet replica from one host to |
| another has been renamed to "Tablet Copy". This resulted in the renaming of |
| several RPC metrics. Any users previously explicitly fetching or monitoring metrics |
| related to Remote Bootstrap should update their scripts to reflect the new names. |
| |
| - The SparkSQL datasource for Kudu no longer supports mode `Overwrite`. Users should |
| use the new `KuduContext.upsertRows` method instead. Additionally, inserts using the |
| datasource are now upserts by default. The older behavior can be restored by setting |
| the `operation` parameter to `insert`. |
| |
| [[rn_0.10.0_new_features]] |
| === New features |
| |
| - Users may now manually manage the partitioning of a range-partitioned table. |
| When a table is created, the user may specify a set of range partitions that |
| do not cover the entire available key space. A user may add or drop range |
| partitions to existing tables. |
| + |
| This feature can be particularly helpful with time series workloads in which |
| new partitions can be created on an hourly or daily basis. Old partitions |
| may be efficiently dropped if the application does not need to retain historical |
| data past a certain point. |
| + |
| This feature is considered experimental for the 0.10 release. More details of |
| the new feature can be found in the accompanying |
| link:https://kudu.apache.org/2016/08/23/new-range-partitioning-features.html[blog post]. |
| |
| - Support for running Kudu clusters with multiple masters has been stabilized. |
| Users may start a cluster with three or five masters to provide fault tolerance |
| despite a failure of one or two masters, respectively. |
| + |
| Note that certain tools (e.g. `ksck`) are still lacking complete support for |
| multiple masters. These deficiencies will be addressed in a following release. |
| |
| - Kudu now supports the ability to reserve a certain amount of free disk space |
| in each of its configured data directories. If a directory's free disk space |
| drops to less than the configured minimum, Kudu will stop writing to that |
| directory until space becomes available. If no space is available in any |
| configured directory, Kudu will abort. |
| + |
| This feature may be configured using the `fs_data_dirs_reserved_bytes` and |
| `fs_wal_dir_reserved_bytes` flags. |
| |
| - The Spark integration's `KuduContext` now supports four new methods for writing to |
| Kudu tables: `insertRows`, `upsertRows`, `updateRows`, and `deleteRows`. These are |
| now the preferred way to write to Kudu tables from Spark. |
| |
| [[rn_0.10.0_improvements]] |
| === Improvements and optimizations |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1516[KUDU-1516] The `kudu-ksck` tool |
| has been improved and now detects problems such as when a tablet does not have |
| a majority of replicas on live tablet servers, or if those replicas aren’t in a |
| good state. Users who currently depend on the tool to detect inconsistencies may now see |
| failures when before they wouldn't see any. |
| |
| - link:https://gerrit.cloudera.org:8080/3477[Gerrit #3477] The way operations are buffered in |
| the Java client has been reworked. Previously, the session's buffer size was set per tablet, meaning that a buffer |
| size of 1,000 for 10 tablets being written to allowed for 10,000 operations to be buffered at the |
| same time. With this change, all the tablets share one buffer, so users might need to set a |
| bigger buffer size in order to reach the same level of performance as before. |
| |
| - link:https://gerrit.cloudera.org/#/c/3674/[Gerrit #3674] Added LESS and GREATER options for |
| column predicates. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1444[KUDU-1444] added support for passing |
| back basic per-scan metrics (e.g cache hit rate) from the server to the C++ client. See the |
| `KuduScanner::GetResourceMetrics()` API for detailed usage. This feature will be supported |
| in the Java client API in a future release. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1446[KUDU-1446] improved the order in |
| which the tablet server evaluates predicates, so that predicates on smaller columns |
| are evaluated first. This may improve performance on queries which apply predicates |
| on multiple columns of different sizes. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1398[KUDU-1398] improved the storage |
| efficiency of Kudu's internal primary key indexes. This optimization should decrease space |
| usage and improve random access performance, particularly for workloads with lengthy |
| primary keys. |
| |
| [[rn_0.10.0_fixed_issues]] |
| === Fixed Issues |
| |
| - link:https://gerrit.cloudera.org/#/c/3541/[Gerrit #3541] Fixed a problem in the Java client |
| whereby an RPC could be dropped when a connection to a tablet server or master was forcefully |
| closed on the server-side while RPCs to that server were in the process of being encoded. |
| The effect was that the RPC would not be sent, and users of the synchronous API would receive |
| a `TimeoutException`. Several other Java client bugs which could cause similar spurious timeouts |
| were also fixed in this release. |
| |
| - link:https://gerrit.cloudera.org/#/c/3724/[Gerrit #3724] Fixed a problem in the Java client |
| whereby an RPC could be dropped when a socket timeout was fired while that RPC was being sent to |
| a tablet server or master. This would manifest itself in the same way |
| link:https://gerrit.cloudera.org/#/c/3541/[Gerrit #3541]. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1538[KUDU-1538] fixed a bug in which recycled |
| block identifiers could cause the tablet server to lose data. Following this bug fix, block |
| identifiers will no longer be reused. |
| |
| [[rn_0.10.0_changes]] |
| === Other noteworthy changes |
| |
| - This is the first release of Apache Kudu as a top-level (non-incubating) |
| project! |
| |
| - The default false positive rate for Bloom filters has been changed |
| from 1% to 0.01%. This will increase the space consumption of Bloom |
| filters by a factor of two (from approximately 10 bits per row to |
| approximately 20 bits per row). This is expected to substantially |
| improve the performance of random-write workloads at the cost of an |
| incremental increase in disk space usage. |
| |
| - The Kudu C++ client library now has Doxygen-based |
| link:http://kudu.apache.org/cpp-client-api/[API documentation] |
| available online. |
| |
| - Kudu now |
| link:http://kudu.apache.org/2016/06/17/raft-consensus-single-node.html[ |
| uses the Raft consensus algorithm even for unreplicated tables]. |
| This change simplifies code and will also allow administrators to enable |
| replication on a previously-unreplicated table. This change is internal and |
| should not be visible to users. |
| |
| [[rn_0.9.1]] |
| == Release notes specific to 0.9.1 |
| |
| Kudu 0.9.1 delivers incremental bug fixes over Kudu 0.9.0. It is fully compatible with |
| Kudu 0.9.0. |
| |
| See also +++<a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20KUDU%20AND%20status%20%3D%20Resolved |
| %20AND%20fixVersion%20%3D%200.9.1">JIRAs resolved |
| for Kudu 0.9.1</a>+++ and +++<a href="https://github.com/apache/kudu/compare/0.9.0...0.9.1">Git |
| changes between 0.9.0 and 0.9.1</a>+++. |
| |
| [[rn_0.9.1_fixed_issues]] |
| === Fixed Issues |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1469[KUDU-1469] fixed a bug in |
| our Raft consensus implementation that could cause a tablet to stop making progress after a leader |
| election. |
| |
| - link:https://gerrit.cloudera.org/#/c/3456/[Gerrit #3456] fixed a bug in which |
| servers under high load could store metric information in incorrect memory |
| locations, causing crashes or data corruption. |
| |
| - link:https://gerrit.cloudera.org/#/c/3457/[Gerrit #3457] fixed a bug in which |
| errors from the Java client would carry an incorrect error message. |
| |
| - Several other small bug fixes were backported to improve stability. |
| |
| [[rn_0.9.0]] |
| == Release notes specific to 0.9.0 |
| |
| Kudu 0.9.0 delivers incremental features, improvements, and bug fixes over the previous versions. |
| |
| See also +++<a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20KUDU%20AND%20status%20%3D%20Resolved |
| %20AND%20fixVersion%20%3D%200.9.0">JIRAs resolved |
| for Kudu 0.9.0</a>+++ and +++<a href="https://github.com/apache/kudu/compare/0.8.0...0.9.0">Git |
| changes between 0.8.0 and 0.9.0</a>+++. |
| |
| [[rn_0.9.0_incompatible_changes]] |
| === Incompatible changes |
| |
| - The `KuduTableInputFormat` command has changed the way in which it handles |
| scan predicates, including how it serializes predicates to the job configuration |
| object. The new configuration key is `kudu.mapreduce.encoded.predicate`. Clients |
| using the `TableInputFormatConfigurator` are not affected. |
| |
| - The `kudu-spark` sub-project has been renamed to follow naming conventions for |
| Scala. The new name is `kudu-spark_2.10`. |
| |
| - Default table partitioning has been removed. All tables must now be created |
| with explicit partitioning. Existing tables are unaffected. See the |
| link:schema_design.html#no_default_partitioning[schema design guide] for more |
| details. |
| |
| [[rn_0.9.0_new_features]] |
| === New features |
| - link:https://issues.apache.org/jira/browse/KUDU-1002[KUDU-1002] Added support for |
| `UPSERT` operations, whereby a row is inserted if it does not already exist, but |
| updated if it does. Support for `UPSERT` is included in Java, C++, and Python APIs, |
| but not in Impala. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1306[KUDU-1306] Scan token API |
| for creating partition-aware scan descriptors. This API simplifies executing |
| parallel scans for clients and query engines. |
| |
| - link:http://gerrit.cloudera.org:8080/#/c/2848/[Gerrit 2848] Added a kudu datasource |
| for Spark. This datasource uses the Kudu client directly instead of |
| using the MapReduce API. Predicate pushdowns for `spark-sql` and Spark filters are |
| included, as well as parallel retrieval for multiple tablets and column projections. |
| See an example of link:developing.html#_kudu_integration_with_spark[Kudu integration with Spark]. |
| |
| - link:http://gerrit.cloudera.org:8080/#/c/2992/[Gerrit 2992] Added the ability |
| to update and insert from Spark using a Kudu datasource. |
| |
| [[rn_0.9.0_improvements]] |
| === Improvements |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1415[KUDU-1415] Added statistics in the Java |
| client such as the number of bytes written and the number of operations applied. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1451[KUDU-1451] Improved tablet server restart |
| time when the tablet server needs to clean up of a lot previously deleted tablets. Tablets are |
| now cleaned up after they are deleted. |
| |
| [[rn_0.9.0_fixed_issues]] |
| === Fixed Issues |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-678[KUDU-678] Fixed a leak that happened during |
| DiskRowSet compactions where tiny blocks were still written to disk even if there were no REDO |
| records. With the default block manager, it usually resulted in block containers with thousands |
| of tiny blocks. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1437[KUDU-1437] Fixed a data corruption issue |
| that occured after compacting sequences of negative INT32 values in a column that |
| was configured with RLE encoding. |
| |
| [[rn_0.9.0_changes]] |
| === Other noteworthy changes |
| |
| All Kudu clients have longer default timeout values, as listed below. |
| |
| .Java |
| - The default operation timeout and the default admin operation timeout |
| are now set to 30 seconds instead of 10. |
| - The default socket read timeout is now 10 seconds instead of 5. |
| |
| .C++ |
| - The default admin timeout is now 30 seconds instead of 10. |
| - The default RPC timeout is now 10 seconds instead of 5. |
| - The default scan timeout is now 30 seconds instead of 15. |
| |
| - Some default settings related to I/O behavior during flushes and compactions have been changed: |
| The default for `flush_threshold_mb` has been increased from 64MB to 1000MB. The default |
| `cfile_do_on_finish` has been changed from `close` to `flush`. |
| link:http://getkudu.io/2016/04/26/ycsb.html[Experiments using YCSB] indicate that these |
| values will provide better throughput for write-heavy applications on typical server hardware. |
| |
| |
| [[rn_0.8.0]] |
| == Release notes specific to 0.8.0 |
| |
| Kudu 0.8.0 delivers incremental features, improvements, and bug fixes over the previous versions. |
| |
| See also +++<a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20KUDU%20AND%20status%20%3D%20Resolved |
| %20AND%20fixVersion%20%3D%200.8.0">JIRAs resolved |
| for Kudu 0.8.0</a>+++ and +++<a href="https://github.com/apache/kudu/compare/0.7.1...0.8.0">Git |
| changes between 0.7.1 and 0.8.0</a>+++. |
| |
| To upgrade to Kudu 0.8.0, see link:installation.html#upgrade[Upgrade from 0.7.1 to 0.8.0]. |
| |
| [[rn_0.8.0_incompatible_changes]] |
| === Incompatible changes |
| |
| - 0.8.0 clients are not fully compatible with servers running Kudu 0.7.1 or lower. |
| In particular, scans that specify column predicates will fail. To work around this |
| issue, upgrade all Kudu servers before upgrading clients. |
| |
| [[rn_0.8.0_new_features]] |
| === New features |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-431[KUDU-431] A simple Flume |
| sink has been implemented. |
| |
| [[rn_0.8.0_improvements]] |
| === Improvements |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-839[KUDU-839] Java RowError now uses an enum error code. |
| |
| - link:http://gerrit.cloudera.org:8080/#/c/2138/[Gerrit 2138] The handling of |
| column predicates has been re-implemented in the server and clients. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1379[KUDU-1379] Partition pruning |
| has been implemented for C++ clients (but not yet for the Java client). This feature |
| allows you to avoid reading a tablet if you know it does not serve the row keys you are querying. |
| |
| - link:http://gerrit.cloudera.org:8080/#/c/2641[Gerrit 2641] Kudu now uses |
| `earliest-deadline-first` RPC scheduling and rejection. This changes the behavior |
| of the RPC service queue to prevent unfairness when processing a backlog of RPC |
| threads and to increase the likelihood that an RPC will be processed before it |
| can time out. |
| |
| |
| [[rn_0.8.0_fixed_issues]] |
| === Fixed Issues |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1337[KUDU-1337] Tablets from tables |
| that were deleted might be unnecessarily re-bootstrapped when the leader gets the |
| notification to delete itself after the replicas do. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-969[KUDU-969] If a tablet server |
| shuts down while compacting a rowset and receiving updates for it, it might immediately |
| crash upon restart while bootstrapping that rowset's tablet. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1354[KUDU-1354] Due to a bug in Kudu's |
| MVCC implementation where row locks were released before the MVCC commit happened, |
| flushed data would include out-of-order transactions, triggering a crash on the |
| next compaction. |
| |
| - link:https://issues.apache.org/jira/browse/KUDU-1322[KUDU-1322] The C++ client |
| now retries write operations if the tablet it is trying to reach has already been |
| deleted. |
| |
| - link:http://gerrit.cloudera.org:8080/#/c/2571/[Gerrit 2571] Due to a bug in the |
| Java client, users were unable to close the `kudu-spark` shell because of |
| lingering non-daemon threads. |
| |
| [[rn_0.8.0_changes]] |
| === Other noteworthy changes |
| |
| - link:http://gerrit.cloudera.org:8080/#/c/2239/[Gerrit 2239] The concept of "feature flags" |
| was introduced in order to manage compatibility between different |
| Kudu versions. One case where this is helpful is if a newer client attempts to use |
| a feature unsupported by the currently-running tablet server. Rather than receiving |
| a cryptic error, the user gets an error message that is easier to interpret. |
| This is an internal change for Kudu system developers and requires no action by |
| users of the clients or API. |
| |
| [[rn_0.7.1]] |
| == Release notes specific to 0.7.1 |
| |
| Kudu 0.7.1 is a bug fix release for 0.7.0. |
| |
| [[rn_0.7.1_fixed_issues]] |
| |
| === Fixed Issues |
| |
| - https://issues.apache.org/jira/browse/KUDU-1325[KUDU-1325] fixes a tablet server crash that could |
| occur during table deletion. In some cases, while a table was being deleted, other replicas would |
| attempt to re-replicate tablets to servers that had already processed the deletion. This could |
| trigger a race condition that caused a crash. |
| |
| - https://issues.apache.org/jira/browse/KUDU-1341[KUDU-1341] fixes a potential data corruption and |
| crash that could happen shortly after tablet server restarts in workloads that repeatedly delete |
| and re-insert rows with the same primary key. In most cases, this corruption affected only a single |
| replica and could be repaired by re-replicating from another. |
| |
| - https://issues.apache.org/jira/browse/KUDU-1343[KUDU-1343] fixes a bug in the Java client that |
| occurs when a scanner has to scan multiple batches from one tablet and then start scanning from |
| another. In particular, this would affect any scans using the Java client that read large numbers |
| of rows from multi-tablet tables. |
| |
| - https://issues.apache.org/jira/browse/KUDU-1345[KUDU-1345] fixes a bug where in some cases the |
| hybrid clock could jump backwards, resulting in a crash followed by an inability to |
| restart the affected tablet server. |
| |
| - https://issues.apache.org/jira/browse/KUDU-1360[KUDU-1360] fixes a bug in the kudu-spark module |
| which prevented reading rows with `NULL` values. |
| |
| [[rn_0.7.0]] |
| == Release notes specific to 0.7.0 |
| |
| Kudu 0.7.0 is the first release done as part of the Apache Incubator and includes a number |
| of changes, new features, improvements, and fixes. |
| |
| See also +++<a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20KUDU%20AND%20status%20in%20 |
| (Resolved)%20AND%20fixVersion%20%3D%200.7.0%20ORDER%20BY%20key%20ASC">JIRAs resolved |
| for Kudu 0.7.0</a>+++ and +++<a href="https://github.com/apache/kudu/compare/branch-0.6.0...branch-0.7.0">Git |
| changes between 0.6.0 and 0.7.0</a>+++. |
| |
| The upgrade instructions can be found at link:installation.html#upgrade[Upgrade from 0.6.0 to 0.7.0]. |
| |
| [[rn_0.7.0_incompatible_changes]] |
| === Incompatible changes |
| |
| - The C++ client includes a new API, `KuduScanBatch`, which performs better when a |
| large number of small rows are returned in a batch. The old API of `vector<KuduRowResult>` |
| is deprecated. |
| + |
| NOTE: This change is API-compatible but *not* ABI-compatible. |
| |
| - The default replication factor has been changed from 1 to 3. Existing tables will |
| continue to use the replication factor they were created with. Applications that create |
| tables may not work properly if they assume a replication factor of 1 and fewer than |
| 3 replicas are available. To use the previous default replication factor, start the |
| master with the configuration flag `--default_num_replicas=1`. |
| |
| - The Python client has been completely rewritten, with a focus on improving code |
| quality and testing. The read path (scanners) has been improved by adding many of |
| the features already supported by the C++ and Java clients. The Python client is no |
| longer considered experimental. |
| |
| [[rn_0.7.0_new_features]] |
| === New features |
| |
| - With the goal of Spark integration in mind, a new `kuduRDD` API has been added, |
| which wraps `newAPIHadoopRDD` and includes a default source for Spark SQL. |
| |
| [[rn_0.7.0_improvements]] |
| === Improvements |
| |
| - The Java client includes new methods `countPendingErrors()` and |
| `getPendingErrors()` on `KuduSession`. These methods allow you to count and |
| retrieve outstanding row errors when configuring sessions with `AUTO_FLUSH_BACKGROUND`. |
| |
| - New server-level metrics allow you to monitor CPU usage and context switching. |
| |
| - Kudu now builds on RHEL 7, CentOS 7, and SLES 12. Extra instructions are included |
| for SLES 12. |
| |
| |
| [[rn_0.7.0_fixed_issues]] |
| === Fixed Issues |
| |
| - https://issues.apache.org/jira/browse/KUDU-1288[KUDU-1288] fixes a severe file descriptor |
| leak, which could previously only be resolved by restarting the tablet server. |
| |
| - https://issues.apache.org/jira/browse/KUDU-1250[KUDU-1250] fixes a hang in the Java |
| client when processing an in-flight batch and the previous batch encountered an error. |
| |
| [[rn_0.7.0_changes]] |
| === Other noteworthy changes |
| |
| - The file block manager's performance was improved, but it is still not recommended for |
| real-world use. |
| |
| - The master now attempts to spread tablets more evenly across the cluster during |
| table creation. This has no impact on existing tables, but will improve the speed |
| at which under-replicated tablets are re-replicated after a tablet server failure. |
| |
| - All licensing documents have been modified to adhere to ASF guidelines. |
| |
| - Kudu now requires an out-of-tree build directory. Review the build instructions |
| for additional information. |
| |
| - The `C++` client library is now explicitly built against the |
| link:https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html[old gcc5 ABI]. |
| If you use gcc5 to build a Kudu application, your application must use the old ABI |
| as well. This is typically achieved by defining the `_GLIBCXX_USE_CXX11_ABI` macro |
| at compile-time when building your application. For more information, see the |
| previous link and link:http://developerblog.redhat.com/2015/02/05/gcc5-and-the-c11-abi/. |
| |
| - The Python client is no longer considered experimental. |
| |
| |
| [[rn_0.6.0]] |
| == Release notes specific to 0.6.0 |
| |
| The 0.6.0 release contains incremental improvements and bug fixes. The most notable |
| changes are: |
| |
| - The Java client's CreateTableBuilder and AlterTableBuilder classes have been renamed |
| to CreateTableOptions and AlterTableOptions. Their methods now also return `this` objects, |
| allowing them to be used as builders. |
| - The Java client's AbstractKuduScannerBuilder#maxNumBytes() setter is now called |
| batchSizeBytes as is the corresponding property in AsyncKuduScanner. This makes it |
| consistent with the C++ client. |
| - The "kudu-admin" tool can now list and delete tables via its new subcommands |
| "list_tables" and "delete_table <table_name>". |
| - OSX is now supported for single-host development. Please consult its specific installation |
| instructions in link:installation.html#osx_from_source[OS X]. |
| |
| |
| [[rn_0.5.0]] |
| == Release Notes Specific to 0.5.0 |
| |
| Kudu 0.5.0 was the first public release. As such, no improvements or changes were |
| noted in its release notes. |