CHANGES.txt

Tajo Change Log

Release 0.8.0 - unreleased

  NEW FEATURES

    TAJO-460: CTAS statement should support partitioned table.
    (Min Zhou via hyunsik)

    TAJO-381: Implement find_in_set function. (Jae Young Lee via hyunsik)

    TAJO-439: Time literal support. (DaeMyung Kang via jihoon)

    TAJO-437: Timestamp literal support. (hyunsik)

    TAJO-382: Implement encode/decode functions. (Seungun Choe via jihoon)

    TAJO-436: Implement ceiling(FLOAT8) function. (DaeMyung Kang via hyunsik)

    TAJO-338 : Add Query Optimization Part for Column-Partitioned Tables.
    (hyunsik)

    TAJO-333: Add metric system to Tajo. (hyoungjunkim via jihoon)

    TAJO-413: Implement pi function. (DaeMyung Kang via jihoon)

    TAJO-61: Implement Time Datum Type. (DaeMyung Kang via hyunsik)

    TAJO-60: Implement Date Datum Type. (hyunsik)

    TAJO-400: Implement pow(float8, float8) function. (DaeMyung Kang via jaehwa)

    TAJO-384: to_bin(). (Jae Young Lee via jaehwa)
   
    TAJO-62: Implement Timestamp Datum type. (hyunsik)

    TAJO-307: Implement chr(int) function. (DaeMyung Kang via hyunsik)

    TAJO-365: Implement degrees/radians function. (DaeMyung Kang via hyunsik)

    TAJO-368: Implement quote_ident function. (Seungun Choe via hyunsik)

    TAJO-392: Implement cbrt function. (DaeMyung Kang via hyunsik)

    TAJO-394: Implement abs function. (DaeMyung Kang via hyunsik)

    TAJO-395: Implement exp function. (DaeMyung Kang via hyunsik)

    TAJO-396: Implement sqrt function. (DaeMyung Kang via hyunsik)

    TAJO-397: Implement sign function. (DaeMyung Kang via hyunsik)

    TAJO-343: Implement locate function. (KyoungBok Lee via hyunsik)

    TAJO-364: Implement mod/div function. (DaeMyung Kang via jaehwa)

    TAJO-361: Implement rpad function. (Seungun Choe via jaehwa)

    TAJO-359: Implement lpad function. (Seungun Choe via jaehwa)

    TAJO-306: Implement ascii(string) function. (SeongHwa Ahn via jaehwa)

    TAJO-354: Fix invalid type to valid type for udfs(bit_length/char_length).
    (DaeMyung Kang via hyunsik)

    TAJO-366: Implement trigonometric functions. (Jae Young Lee via jihoon)

    TAJO-358: Implement initcap(string) function. (Seungun Choe via hyunsik)

    TAJO-355: Implement repeat(text,int) function. (DaeMyung Kang via jaehwa)

    TAJO-339: Implement sin( x ) - returns the sine of x (x is in radians). 
    (Jae Young Lee via jaehwa)

    TAJO-348: Implement octet_length(text). (DaeMyung Kang via jaehwa)

    TAJO-357: Fix invalid filename TestMethFunction to TestMathFUnction.
    (DaeMyung Kang via hyunsik)

    TAJO-352: Implement right/left(text, size) function. 
    (DaeMyung Kang via hyunsik)

    TAJO-346: Implement hex function. (DaeMyung Kang via hyunsik)

    TAJO-349: Implement md5(text). (DaeMyung Kang via hyunsik)

    TAJO-351: Implement reverse(text). (DaeMyung Kang via hyunsik)

    TAJO-342: Implement strpos(string, substring) function. 
    (hyoungjunkim via hyunsik)

    TAJO-350: Implement round, floor, ceil. (hyoungjunkim via hyunsik)

    TAJO-207: Implement bit_length(string) function. (DaeMyung Kang via jihoon)

    TAJO-341: Implement substr function. (hyoungjunkim via hyunsik)

    TAJO-308: Implement length(string) function. (hyoungjunkim via hyunsik)

    TAJO-200: RCFile compatible to apache hive. (jinho)

    TAJO-176: Implement Tajo JDBC Driver. (Keuntae Park via jihoon)

    TAJO-16: Enable Tajo catalog to access Hive metastore. (jaehwa)

    TAJO-285: Add CREATE TABLE... BY PARTITION statement to parser. (hyunsik)

    TAJO-267: Implement equals() and deepEquals() functions at LogicalNode.
    (jihoon)

  IMPROVEMENTS

    TAJO-432: Add shuffle phase for column-partitioned table store. (Min Zhou via jihoon)

    TAJO-135: Bump up hadoop to 2.2.0. (jihoon)

    TAJO-435: Improve intermediate file. (jinho)

    TAJO-424: Make serializer/deserializer configurable in CSVFile. (jinho)

    TAJO-419: Add missing visitor methods of AlgebraVisitor and
    BaseAlgebraVisitor. (hyunsik)

    TAJO-421: Improve split for compression file. (jinho)

    TAJO-327: Add testcase to verify TAJO-16. (jaehwa)

    TAJO-405: Improve HCatalogStore to support partitioned table. (jaehwa)

    TAJO-336: Separate catalog stores into separate modules. (jaehwa)

    TAJO-329: Implement physical operator to store in column-partitioned table.
    (jaehwa)

    TAJO-391: Change the default type of real values from FLOAT4 to FLOAT8 
    when parsing the user queries. (DaeMyung Kang via jihoon)

    TAJO-304: drop table command should not remove data files in default.
    (hyunsik)

    TAJO-316: Improve GreedyHeuristicJoinOrderAlgorithm to deal with
    non-commutative joins. (hyunsik)

    TAJO-371: Increase the default value of worker memory. (jihoon)

    TAJO-284: Add table partitioning entry to Catalog. (jaehwa)

    TAJO-317: Improve TajoResourceManager to support more elaborate resource 
    management. (Keuntae Park via jihoon)

    TAJO-314: Make TaskScheduler be pluggable. (jihoon)

    TAJO-325: QueryState.NEW and QueryState.INIT should be combined into one
    state. (Min Zhou via hyunsik)

    TAJO-313: Support deprecated variables in CatalogConstants. (jaehwa)

    TAJO-310: Make the DataLocation class as a separate class and move it to 
    the tajo-core-storage package. (jihoon)

    TAJO-311: Improve Hive dependency. (jaehwa)

    TAJO-309: Remove unused fields in FileFragment. (jihoon)

    TAJO-297: Rename JDBC variables in CatalogConstants to be more generic. 
    (jaehwa)

    TAJO-287: Improve Fragment to be more generic. (hyunsik)

    TAJO-274: Maintaining connectivity to Tajo master regardless of the restart
    of the Tajo master. (Keuntae Park via hyunsik)

    TAJO-287: Refactor TableDesc, TableMeta, and Fragment. (hyunsik)

    TAJO-275: Separating QueryMaster and TaskRunner roles in worker.
    (Keuntae Park via jihoon)

  BUG FIXES

    TAJO-455: Throw PlanningException when Creating table with Partition
    exception COLUMN. (DaeMyung Kang via hyunsik)

    TAJO-452: Timstamp literal with fractional seconds results in 
    java.lang.ArrayIndexOutOfBoundsException. (Keuntae Park via hyunsik)

    TAJO-454: pass invalid argument to DateTime constructor in LogicalPlanner.
    (DaeMyung Kang via hyunsik)

    TAJO-451: Update documentation and version constant for Tajo 0.8.
    (hyunsik)

    TAJO-448: Timestamp should be based on unixtime. (hyunsik)
    
    TAJO-450: Incorrect inet4datum comparison. (jinho)

    TAJO-182: Comparison of primitive values including null value should
    return NULL. (hyunsik)

    TAJO-431: HCatalogStore can't write any data using INSERT OVERWRITE clause.
    (jaehwa)

    TAJO-442: Cast operator with nested functions causes NPE. (hyunsik)

    TAJO-426: HCatalogStore created partitions automatically. (jaehwa)

    TAJO-418: sort operator after inline views consisting of unions can cause 
    an incorrect distributed plan. (hyunsik)

    TAJO-417: TestSQLExpression.testCastFromTable causes unit test failure.
    (hyunsik)

    TAJO-415: Some complex queries causes NPE and unlimited recursions.
    (hyunsik)

    TAJO-414: Fix bug of bit operations in decode() method of DateDatum class.
    (Keuntae Park via jihoon)

    TAJO-407: PostgreSQL-style cast should be higher operator priority.
    (hyunsik)

    TAJO-411: Fix Bug: createFromInt8's DATE type should be TIMESTAMP.
    (DaeMyung Kang via jihoon)

    TAJO-390: Queries on history are expired ealier than a given expiry time.
    (hyoungjunkim via hyunsik)

    TAJO-410: A query with a combination of general and distinct aggregation
    functions fails. (hyunsik)

    TAJO-404: Tajo does not recognize boolean literal. (hyunsik)

    TAJO-344: Tajo cannot recognize negative numeric expressions. (hyunsik)

    TAJO-393: Unit tests must use test-data directory. (hyunsik)

    TAJO-388: limit clause does not work properly. (hyunsik)

    TAJO-389: The LazyTuple does not work when number format exception occurs 
    in text deserializer. (jinho)

    TAJO-387: Query is hanging when errors occurs in Query or SubQuery class.
    (hyunsik)

    TAJO-268: Temporal files should be removed after query is finished. (jinho)

    TAJO-292: Too many intermediate partition files. (jinho)

    TAJO-375: TajoClient can't get result data when different os user
    (hyoungjunkim via hyunsik)

    TAJO-347: Fix bug when to call function with insensitive function name.
    (DaeMyung Kang via hyunsik)

    TAJO-345: MergeScanner should support projectable storages. (jihoon)

    TAJO-290: TajoDataType.Type.NULL should be NULL_TYPE. (DaeMyung Kang via 
    jinho)

    TAJO-332: Invalid row count of CSVScanner. (jinho)

    TAJO-326: In ExecutionBlock, isRoot() and isLeafBlock() return invalid 
    values. (jihoon)

    TAJO-296: Late registration of Tajo workers. (hyoungjunkim via hyunsik)

    TAJO-321: Invalid split file of compressed text file. (jinho)

    TAJO-295: ConcurrentModificationException in TaskScheduler. (jinho)

    TAJO-293: querymasters directory not found in single node setup. (hyunsik)

  TASKS

    TAJO-322: Documentation by version. (hyunsik)

    TAJO-320: Visualize Tajo statemachine. (Min Zhou via hyunsik)

    TAJO-319: Update homepage and bump tajo version to 0.8. (hyunsik)


Release 0.2.0 - released

  NEW FEATURES

    TAJO-260: Add between predicate. (hyunsik)

    TAJO-208: Implement char_length(string) function. (hyunsik)

    TAJO-99: Design the interface of join enumeration algorithm. (hyunsik)

    TAJO-179: Support MySQL CatalogStore. (jinho)

    TAJO-147: Implement trim(text), ltrim(text), and rtrim(text) function.
    (hyunsik)

    TAJO-148: Implement UPPER(text), LOWER(text) function. (hyunsik)

    TAJO-220: Implement catalog dump feature. (hyunsik)

    TAJO-216: Improve FilterPushDownRule and Implement physical operators 
    for outer join. (camelia_c via hyunsik)

    TAJO-211: Implement regexp_replace function. (hyunsik)

    TAJO-212: Implement type cast expresion. (hyunsik)

    TAJO-206: Implement String concatenation operator (||). (hyunsik)

    TAJO-213: NULL characters in meta of csv table should be supported.
    (jinho)    

    TAJO-185: Implement split_part function. (hyunsik)

    TAJO-193: Add string pattern matching operators. (hyunsik)

    TAJO-101: HiveQL converter. (jaehwa)

    TAJO-144: Implement INSERT OVERWRITE clause. (hyunsik)

    TAJO-139: TAJO-139: In predicate support. (hyunsik)

    TAJO-134: Support for compression/decompression of CSVFile. (jinho)

    TAJO-59: Implement Char Datum Type. (jihoon)
   
    TAJO-96: Design and implement rewrite rule interface and the rewrite rule 
    engine. (hyunsik)

    TAJO-19: CREATE TABLE without external data should be supported. (jinho)

    TAJO-57: Recognize Parser and Catalog Standard SQL data types. (hyunsik)

    TAJO-33: Implement a basic query progress indicator. (hyunsik)

  IMPROVEMENTS

    TAJO-270: Boolean datum compatible to apache hive. (jinho)

    TAJO-261: Rearrange default port numbers and config names. (hyunsik)

    TAJO-236: Implement LogicalPlanVerifier to check if a logical plan is 
    valid. (hyunsik)

    TAJO-254: Refactor package names and class names to have proper names.
    (add a missed changelog)

    TAJO-255: Cleanup exceptions of engine. (hyunsik)

    TAJO-253: Clean up tajo-default.xml in test resource. (hyunsik)

    TAJO-239: Improving web UI. (Keuntae Park via hyunsik)

    TAJO-232: Rename join operators and add other join operators to
    PhysicalPlanner. (hyunsik)

    TAJO-229: Implement JoinGraph to represent a graph of relation joins.
    (hyunsik)

    TAJO-223: Maximize disk read bandwidth utilization of StorageManagerV2 by
    moving Tuple creation role to next(). (Keuntae Park via hyunsik)

    TAJO-199: All relations in catalog must have data volume size. (hyunsik)

    TAJO-224: Rearrange DataType enumeration and Refactor type systems.
    (hyunsik)

    TAJO-214: System should inquire finished query history after execution 
    in web. (SeongHwa Ahn via hyunsik)

    TAJO-145: count(distinct column) should be supported. (hyunsik)

    TAJO-197: Implement Enforcer that forces physical planner to choose
    specified algorithms. (hyunsik)

    TAJO-194: LogicalNode should have an identifier to distinguish each
    logical node instance. (hyunsik)

    TAJO-183: Creating too many TableMetaProto objects might lead a potential 
    memory leak. (jihoon)

    TAJO-184: Refactor GlobalPlanner and global plan data structure. (hyunsik)

    TAJO-119: Refactor and Improve Datum. (jinho)

    TAJO-178: Implements StorageManager for scanning asynchronously. 
    (hyoungjunkim via hyunsik)

    TAJO-167: Implement GeneralSetFunctionExpr to represent all SQL standard
    set functions. (hyunsik)

    TAJO-149: Eliminate QueryConf and its file write. (hyunsik)

    TAJO-143: Implement hash semi-join operator. (hyunsik)

    TAJO-142: Implement hash anti-join operator. (hyunsik)

    TAJO-94: Remove duplicate proto files. (hyunsik)

    TAJO-141: Set on demand as the default cluster mode. (hyunsik)

    TAJO-136: Broadcast join should be enabled. (hyunsik)

    TAJO-127: Implement Tajo Resource Manager. (hyoungjunkim via hyunsik)

    TAJO-84: Task scheduling with considering disk load balance. (jinho)

    TAJO-123: Clean up the logical plan's json format. (hyunsik)

    TAJO-129: Enable the constructor of NettyServerBase to take a service
    name. (hyunsik)

    TAJO-91: Launch QueryMaster on NodeManager per query. 
    (hyoungjunkim via hyunsik)

    TAJO-100: Port the parse error handling to the new parser. (jinho)

    TAJO-121: Add LogicalPlanVisitor and Refactor LogicalOptimizer to use the
    visitor. (hyunsik)

    TAJO-118: Refactor and Improve text file Scanner. (jinho)

    TAJO-95: Eliminate the lazy copy approach from the classes wrapping
    protobuf-generated classes. (hyunsik)

    TAJO-102: Add AlgebraVisitor and Refactor LogicalPlanner to use the visitor.
    (hyunsik)

    TAJO-87: Integration of tajo algebra module and SQL parser. (hyunsik)

    TAJO-70: Refactor GlobalEngine to handle DDL statements. (hyunsik)

    TAJO-86: Improved error messages in query analyzer. (jinho)

    TAJO-85: Add an method of create(DataType, byte[]) to DatumFactory. (jihoon)

    TAJO-79: Refactor and Improve TajoCli. (hyunsik)

    TAJO-77: Rename TabletProto as FragmentProto in CatalogProtos.proto. 
    (jihoon)

    TAJO-22: The package prefix should be org.apache.tajo. (DaeMyung Kang via 
    hyunsik)

    TAJO-75: Separate SQL.g into SQLParser.g and SQLLexer.g. (hyunsik)

    TAJO-74: Refactor parser rules of SQL.g. (hyunsik)

    TAJO-7: Enabling TajoCli to take multiple lines as one statement. (jinho)

    TAJO-72: Rename NQL.g to SQL.g. (hyunsik)

    TAJO-71: Case-insensitive parser. (hyunsik)

    TAJO-40: Adopt YarnClient to GlobalEngine. (jihoon)

    TAJO-69: Add .reviewboardrc file to the ignore lists of Rat and .gitignore. 
    (hyunsik)

    TAJO-66: Support compiling and running Tajo with Java7 (Tejas Patil via 
    hsaputra)

    TAJO-51: Parallel Container Launch of TaskRunnerLauncherImpl. (hyunsik)
    
    TAJO-39 Remove the unused package tajo.engine.plan.global and all files 
    inside the directory. (hsaputra)

    TAJO-37: Remove obsolete classes WorkerEventDispatcher, WorkerEvent and 
    WorkerEventType. (sunny.1324 via hyunsik)

    TAJO-50: Cleanup SubQuery. (hyunsik)

    TAJO-44: Adopt AMRMClient to RMContainerAllocator, RMCommunicator. (hyunsik)
    
    TAJO-42: Divide SubQuery into FSM and execution block parts. (hyunsik)

    TAJO-32: Cleanup TaskRunner. (hyunsik)

    TAJO-27: Modify the document links to point the wiki's ones. (hyunsik)
    
    TAJO-17: Improve the examples for jvm version and auxiliary service in 
    getting_started.apt. (hyunsik)

    TAJO-10: Modify git ignore to include Apache derby log file. (hsaputra)

    TAJO-12: Add information in README on how to subscribe to mailing 
    lists (mattmann, hyunsik)

    TAJO-11: Update file README to reflect Apache incubation. (hyunsik)

    TAJO-4: Update the project site. (hyunsik)

    TAJO-2: remove all @author tags and update license header. (hyunsik)

  BUG FIXES

    TAJO-281: 'mvn package -Pdist' generates duplicate Tajo jar files.
    (hyunsik)

    TAJO-277: Infinite loop occurs when a table is empty. (jinho)

    TAJO-273: NotEval incurs NPE with boolean column. (hyunsik)

    TAJO-272: boolean test does not work correctly. (hyunsik)

    TAJO-264: Remove hard-coded 'standby' mode. (jinho)

    TAJO-262: Integration test hang on jenkins. (jinho)

    TAJO-257: Unit tests occassionally fail. (hyunsik)

    TAJO-169: the default TAJO_WORKER_STANDBY_MODE in tajo-env.sh is wrong. 
    (hyunsik)

    TAJO-231: In StorageManger v2, TestCSVCompression incurs unexpected end of 
    stream occasionally. (hyunsik)

    TAJO-250: QueryMaster must send the query finish heartbeat. (hyunsik)

    TAJO-245: org.apache.tajo.algebra.FunctionExpr cannot be cast to 
    org.apache.tajo.algebra.ColumnReferenceExpr. (hyunsik)

    TAJO-243: The 'tajo_dump' does not include options of table. (jinho)

    TAJO-244: assignToNonLeafTasks causes NPE occurs occasionally. (hyunsik)

    TAJO-242: Enable omitted broadcast join feature after TAJO-184. (hyunsik)

    TAJO-240: Reformat HiveConverter source code to match adopted conventions 
    (jaehwa)   

    TAJO-241: Does not reconnect the meta store. (jinho)

    TAJO-226: Can't use group by column alias. (hyunsik)

    TAJO-203: Field delimiter does not work correctly. (jinho)

    TAJO-205: Repartitioner occasionally chooses a partition number as one.
    (hyunsik)

    TAJO-191: INSERT OVERWRITE INTO statement should follow the table meta in 
    catalog. (jinho)

    TAJO-198: The split function does not work properly. (jinho)

    TAJO-187: The exception 'Some targets cannot be evaluated' is caused in 
    nested joins. (hyunsik)

    TAJO-46: The "having" clause does not work properly. (hyunsik)

    TAJO-186: Improve column resolving method. (hyunsik)

    TAJO-168: infinite loop occurs when QueryMaster is stopping. (jinho)

    TAJO-180: Better error messages for 
    StorageManager.listStatus$InvalidInputException. (hyunsik)

    TAJO-177: Call by reference bug in MergeJoinExec. (jihoon)

    TAJO-171: BitArray always returned false. (jinho)

    TAJO-172: TajoWorker allocate wrong container slots.(jinho)

    TAJO-174: SQL error handler sometimes occurred NPE. (jinho)

    TAJO-170: mvn compile error when building tar file. (jaehwa)

    TAJO-154: Query calculates wrong progress in multiple step queries.
    (hyunsik)

    TAJO-152: The tajo local directory path is mismatched in PullServer. 
    (jihoon)

    TAJO-80: Remove codes of legacy data paths. (jihoon)
 
    TAJO-159: QueryMaster cannot connect the remote CatalogServer. (hyunsik)

    TAJO-158: Can't allocate worker when single SubQuery requests more than
    cluster capacity. (hyoungjunkim via hyunsik)

    TAJO-157: The CSVScanner.isSplittable() function does not work properly.
    (jinho)

    TAJO-156: BufferUnderflowException occurs during processing the 
    intermediate file. (jinho)

    TAJO-151: Multiple union queries within a table subquery causes NPE.
    (hyunsik)

    TAJO-137: Unreleased resources and wrong allocation requests in
    TajoWorkerResourceManager. (hyoungjunkim via hyunsik)

    TAJO-130: Same queryConf file conflicts. (jinho)

    TAJO-82: NullPointerException occurs when Schema is converted as an array 
    of columns. (jihoon)

    TAJO-65: Invalid pom.xml configurations for source directories. 
    (Takuya Ueshin via hyunsik)

    TAJO-48: BufferUnderflowException occurs during the initialization of 
    RowFile. (jihoon)

    TAJO-58: Remove obsolete methods in GlobalPlanner. (hyunsik)

    TAJO-54: SubQuery::allocateContainers() may ask 0 containers. (hyunsik)

    TAJO-41: Storage handler name row in storage-default.xml must be rowfile. 
    (hsaputra)

    TAJO-47: RowFile has the duplicated initialization problem and unflipped 
    ByteBuffer problem. (jihoon)

    TAJO-38: Update class comment in TaskAttemptContext from Korean to English 
    (hsaputra)

    TAJO-15: The Integration test is getting hanged on Mac OS X. (hyunsik)

    TAJO-8: MapReduce's ShuffleHandler and Tajo's PullServerAuxService conflict
    each other. (jhkim)

    TAJO-13: Update the groupId property in the pom.xml to use ASF org instead
    of kr.ac.korea.dbserver (hsaputra)

    TAJO-1: RCFileWrapper always reads whole columns regardless of the target 
    schema. (jihoonson via hyunsik)

    TAJO-6: Rename tajo.engine.function.builtin.NewSumInt to SumInt. (rsumbaly)

    TAJO-21: CREATE EXTERNAL TABLE should support a file path. (JaeHwa Jung)

  TASKS

    TAJO-288: Correct NOTICE file and LICENSE.txt. (hyunsik)

    TAJO-271: Add MIT license to NOTICE.txt and LICENSE.txt for jquery and 
    jsPlumb. (hyunsik)

    TAJO-265: Update installation guide and other documentation for 0.2
    release. (hyunsik)

    TAJO-252: Add DISCLAIMER file. (hyunsik)

    TAJO-251: Rename the legacy name *.tql to *.sql. (hyunsik)