| Apex Changelog |
| ======================================================================================================================== |
| |
| Version 3.1.1 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Improvement |
| |
| * [APEX-122] - Fail release build when there are dependencies on snapshot version |
| |
| ### Task |
| |
| * [APEX-81] - Update netlet dependency to release version |
| |
| ### Bug |
| |
| * [APEX-89] - Javascript error when launching an app under CDH 5.2 (hadoop 2.5.0) |
| * [APEX-101] - Negative Memory reported for Application Master |
| * [APEX-121] - Making sure that state is transferred to client for Statefull Codec |
| * [APEX-126] - handleIdleTime Called Outside Of The Space Between beginWindow and endWindow |
| * [APEX-127] - Tags of overridden setter/getter methods are lost if there are no comments |
| * [APEX-149] - In secure mode non-HA setup STRAM web service calls are failing |
| * [APEX-170] - StringCodec registered with BeanUtils is throwing null pointer |
| |
| |
| Version 3.1.0 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Improvement |
| * [APEX-12] - Fix Base Operator To Not Show Name Property In App Builder |
| |
| ### Bug |
| * [APEX-35] - Test exceptions due to missing directory in saveMetaInfo |
| * [APEX-36] - FSStorageAgent to account for HDFS lease when writing checkpoint files |
| * [APEX-37] - Container and operator json line file in StreamingContainerManager should not be appended from previous app attempt |
| * [APEX-43] - SchemaSupport: TUPLE_CLASS attribute should use Class2String StringCodec |
| * [APEX-56] - Controlled plan modification on operator shutdown |
| |
| |
| |
| Version 3.0.0 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| * Add jersey client jar for app execution |
| * Must catch NoClassDefFoundError when processing operator classes in jar, previously catching Throwable was changed to catching Exception |
| * Do not catch throwable in DTCli and exit immediately when worker thread throws an error |
| * Depend on published netlet version |
| * Catch throwable when executing command because launching an app can throw java.lang.VerifyError: bad type on operand stack |
| * Removed runOnlyOnce when generating dt-git.properties and generate even when not using release profile |
| * Undeploy heartbeat requests are not processes if container is idle |
| * Fix potential NPE |
| * Comment hide the actually type for string types(URI, URL, Class etc) and add 2 missing wrapper types |
| * Fixed typo in webservice url for get-physical-plan command |
| * Resolve deleting checkpoint in different thread |
| * Removed duplicate code and added unit test for json stream codec |
| * APEX-11 #resolve added checkpoint metric |
| * Have default connect and read timeouts because the jersey defaults are infinity and that blocks threads and those threads can't be interrupted either |
| * Removed invalid app data push annotation |
| * Use FileContext instead of FileSystem to write the meta json file |
| * Comment added required memory and required vcores in the appinfo |
| * Comment filter abstract types from getAssignableClasses call and rename initializable to instantiable |
| * Deploy all artifacts by default. |
| * Comment fix the bug in trimming the graph |
| * HA support for stram webservice filter. |
| * Removed dependencies in filter to hadoop classes with private audience as their interface has changed from Hadoop 2.2 to 2.6 |
| * Related doc updates |
| * Comment Prune the external dependencies from type graph and break the type circle |
| * Fixing class not found issue due to missing dt-common jar. |
| * Resolve removed old alert support in stram |
| * Use tokens from Credentials as UserGroupInformation.getTokens() returns HDFS Namenode hidden tokens that should not be passed to containers. |
| * Support for RM delegation token renewal in secure HA environments |
| * Resolve fixed bug when custom metric keys are not stored correctly in meta file |
| * Comment Use apache xbean shaded asm to resolve jdk 8 class and avoid conflict with old asm version that hadoop depends on |
| * APEX-5 #resolve Set APPLICATION_PATH for unit tests that launch an app |
| * Resolve Added dependency on dt-engine in test scope of archetype |
| * Corrected the place to get pom.properties for version info |
| * Token authentication support for buffer server |
| * Adding default aggregator for primitive customMetrics |
| * Netlet has a few fixed related to non firing OP_CONNECT call, so test against those. |
| * Print stack trace when exception occurs |
| * Commented out app master mb in properties.xml |
| * Ability to extract javadocs as xml |
| * Moved PAYLOAD_VALUE out of default case. Count number of skipped payload tuples. |
| * Resolve fixing ASM signature visitor to add upper bound to Object for unbounded wild card entries |
| * Removing unused imports |
| * Addressing review comments |
| * Reverting code format change |
| * Resolve Added END_STREAM to be distributed to physical nodes. Introduced debug message for default case. |
| * Comment removed the coding convention |
| * Resolve Changed reading of portTypeInfo, port annotations in operator discoverer to read from ASM |
| * Added test case for operator discoverer |
| * Updated Type Discovery tests to getPortInfo via ASM instead of reflection API |
| * Addressing review comments |
| * Removed condition for public, final, transient ports when generating typeGraph using ASM |
| * Added a class for storing input, output port annotations |
| * Added port type info handling for ports other than DefaultInputPort and DefaultOutputPort types |
| * Updated a test case for the same |
| * Adding annotations to list.. Missed in previous commit |
| * Added rescan of typegraph to capture operator ports |
| * Changing ASMUtil utility methods to public |
| * Addressing review comments. |
| * Correcting jar entry name |
| * Updated a test after merge |
| * Removed system.out print from test cases and added debug print on failure |
| * Removing temp file |
| * Reversed the order of setting jvm_options for app master |
| * Resolve Added validation for root operator should be input operator in logical plan validation |
| * Fixing formatting and added lincense header |
| * Removed extra line spaces |
| * Correcting output port in dt-site.xml stream connection |
| * Resolve Added validation for root operator should be input operator in logical plan validation |
| * Fixing formatting and added lincense header |
| * Removed extra line spaces |
| * Correcting output port in dt-site.xml stream connection |
| * Removing white spaces |
| * Renaming the properties to reflect the code change |
| * Comment add the missing open tag <p> |
| * Removed unnecessary depedency |
| * Resolve Remove unnecessary unpack in app package archetype |
| * Resolve support java.lang.Class and resolve uiType to special types |
| * Resolve update dependency to dt-common in archetype |
| * Resolve added the container jvm options for app master |
| * Resolve attach apa file as an artifact |
| * Resolve test app package should depend on dt-common instead of dt-engine |
| * Removed additional license header |
| * Removed incorrectly left in license headers due to incorrect formatting |
| * Put back revision info. |
| * Added a TODO note for handling of stram delegation tokens for future |
| * Comment Add dag attributes to LogicalPlanSerializer |
| * Using resolveType method in OperatorDiscoverer to describe an attribute completely |
| * Switch to Java7 and update compiler plugin. |
| * Remove invalid module references. |
| * Separated out HA token creation from non-HA case as it involves special handling that is subject to change if Hadoop's internals change. |
| * Resolve - Add all resource managers to RM delegation token service. |
| * Fix CLI script. |
| * Stram directory is moved to engine |
| * Fix depenency plugin version (2.3 shipped with maven 3.2.5 pulls dependencies from test scope) |
| * Setup the pom files for OS with optimized dependencies |
| * Changed the license header to Apache 2.0 license. |
| |
| |
| Version 2.1.0 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-3732] - Get User Info APIs creates users |
| * [SPOI-3816] - Ingestion UI: Node server should store pipelines on hdfs rather than local filesystem |
| * [SPOI-3820] - Ingestion UI: Rest calls to gateway from Node server fails |
| * [SPOI-3845] - BlockSynchronizer sometimes misbehaves when BlockReader is killed. |
| * [SPOI-3862] - Ingestion UI - left/right margins have no width |
| * [SPOI-3876] - Review the changes done to DimensionsComputation for HadoopWorld Demo |
| * [SPOI-3940] - Demo app package version stuck at v1.0-SNAPSHOT |
| * [SPOI-4029] - PiJavaScript demo pi calc operator fails to re deploy |
| * [SPOI-4067] - Launch of ingestionApp using DTCP is failing |
| * [SPOI-4070] - Stray directory /opt/datatorrent/current/datatorrent is created after installation. |
| * [SPOI-4077] - Ingestion UI: Create option to scan the directory recursively |
| * [SPOI-4079] - Gateway does not retain listen address specified during installation |
| * [SPOI-4080] - Gateway guard hides errors with restarts |
| * [SPOI-4081] - Ingestion UI: pipeline table doesn't auto refreshes |
| * [SPOI-4085] - UI: wrong application selection after sorting in pipeline instances table |
| * [SPOI-4087] - Relaunch functionality keeps relaunching multiple apps |
| * [SPOI-4094] - Ingestion app tests failing and empty directory getting created under target that are not removed |
| * [SPOI-4128] - Need to explain difference between properties & state holding data structure |
| * [SPOI-4175] - Remove directory prop from base block reader and ftp block reader from ingestion |
| * [SPOI-4229] - committed function not getting invoked in local mode |
| * [SPOI-4352] - App Builder uiType not present for several classes |
| * [SPOI-4362] - Remove dependencies on DirectoryScanner from OperatorDiscoverer when used in stram |
| * [SPOI-4691] - DT console shows negative memory size for killed application |
| * [SPOI-4702] - IndexOutOfBoundsException in Stram due to counters in AbstractBlockReader |
| |
| |
| ### Improvement |
| * [SPOI-3592] - Clean up appInstance widget definitions |
| * [SPOI-3783] - Remove gateway port automatic re-selection on startup feature |
| * [SPOI-3932] - demos ui references web services v1 |
| * [SPOI-3944] - Ingestion UI : Display pipelines progress |
| * [SPOI-3945] - UI: Add multiple input support |
| * [SPOI-3946] - Create an auto scaling scheme for the Block Reader and Writer |
| * [SPOI-4003] - Ingestion app: add a feature to re-try failed blocks |
| * [SPOI-4026] - Aggregated counters are not published via web-socket in the logical operators topic |
| * [SPOI-4048] - Block application launch during critical system issues |
| * [SPOI-4078] - Gateway address argument validation during installation and launch |
| * [SPOI-4551] - Enhance Gateway API to return AppIDs for a given application name |
| * [SPOI-4578] - Allow supporting archive jars in configuration definition for app packages |
| * [SPOI-4728] - Support -originalAppId when launching apps through the DT Gateway API |
| * [SPOI-4751] - Upgrade sniffer maven plugin to the latest version |
| |
| |
| ### New Feature |
| * [SPOI-3028] - [HDHT] Hive Interoperability |
| * [SPOI-3029] - [HDHT] Export to ORC Format |
| * [SPOI-4050] - AppData UI Dashboard page |
| * [SPOI-4093] - Ingestion UI: Control for triggering scan of files |
| * [SPOI-4636] - Added $? feature in dtcli and use it in CLIProxy |
| * [SPOI-4718] - CLI commands to support variable arguments |
| |
| |
| ### Task |
| * [SPOI-3242] - Rethrow the exceptions being caught in catch block of operators. |
| * [SPOI-3502] - Create the landing page for the UI of ingestion application |
| * [SPOI-3504] - [Ingestion UI] Create views for creating a pipeline and its description |
| * [SPOI-3762] - Provide installation version option for installer |
| * [SPOI-3763] - Update document export tokens |
| * [SPOI-3870] - Implement transformation functions for rainier poc1 |
| * [SPOI-3871] - Filter records based field values for rainier poc1 |
| * [SPOI-3872] - Generate audit and bad record logs for rainier poc1 |
| * [SPOI-3970] - Ingestion UI : Use the console package |
| * [SPOI-3981] - Remove unused imports |
| * [SPOI-4040] - Add an operator that keeps track of failed files |
| * [SPOI-4122] - [UI] : Display list of skipped files on UI when overwrite flag is false |
| * [SPOI-4172] - After effects of removing threshold property from BlockReader in ReaderWriter partitioner/stats-listener |
| * [SPOI-4369] - [AppData][AppDataTracker] Deserializer operator |
| * [SPOI-4385] - Create metrics aggregators - sum, min, max, count |
| * [SPOI-4544] - Custom metrics can be cumulative or per window which can be statically declared by the operator developer |
| * [SPOI-4653] - Create Aggregators registry in AppDataTracker |
| * [SPOI-3767] - Auto scaling of BlockReader using partitioning |
| * [SPOI-4748] - Add data query to custom metrics store |
| |
| |
| ### Bug |
| * [MLHR-1614] - AbstractFSWriter in append mode is not fault-tolerant |
| * [MLHR-1620] - Remove close file from AbstractFSWriter |
| * [MLHR-1637] - Cleanup skipping of records from setup of FSDirectoryInputOperator |
| * [MLHR-1643] - FileSplitter recovery fails |
| * [MLHR-1644] - Add Mock Server Libraries in unit tests of database/key value store operators |
| * [MLHR-1653] - Remove JavaScriptOperatorBenchMark form library |
| * [MLHR-1656] - AbstractFileOutputOperator LeaseExpired exception when cache reaches its threshold |
| * [MLHR-1668] - Stateless Partitioner in case of parallel partition ignores parallel partition count (except the first time define partitions is called) |
| * [MLHR-1687] - AbstractBlockReader threshold breaks the idempotency |
| * [MLHR-1708] - Duplicate data read from kafka if kafka partitions are less than DT partitions |
| * [MLHR-1712] - The directory under which the idempotent state is stored should be relative to the app directory so that the state is copied on relaunch |
| * [MLHR-1723] - FTPStringInputOperatorTest fails on Windows OS |
| |
| |
| ### Improvement |
| * [MLHR-1547] - Integration of Idempotent storage manager to Directory scanner |
| * [MLHR-1621] - Parititon Couchbase Output Operator |
| * [MLHR-1632] - Add couchbase mock to couchbase tests. |
| * [MLHR-1634] - Improve the BlockReader partitioning scheme to accommodate ingestion rate |
| * [MLHR-1641] - Improve the BlockReader |
| * [MLHR-1661] - Ability to override the stream-codec of input port in AbstractFileOutputOperator |
| * [MLHR-1684] - Improve file splitter not to emit all files in one shot and hold scanned file names in memory |
| * [MLHR-1694] - BasicCounters improvements |
| |
| ### New Feature |
| * [MLHR-1355] - Supports secure hadoop cluster in the installer |
| * [MLHR-1497] - Operators for ElasticSearch |
| * [MLHR-1578] - UI Auth: UI to show different tabs and/or buttons for different user permissions |
| |
| |
| Version 2.0.1 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-4379] - Operator removed from physical plan due to invalid SHUTDOWN status |
| * [SPOI-4381] - Queue size missing on physical operator page |
| * [SPOI-4382] - queueSize port metric reports bogus values |
| * [SPOI-4384] - Recovery fails due to corrupted checkpoints. |
| * [SPOI-4437] - DTCli not recognizing $HOME variable set |
| |
| ### Improvement |
| * [SPOI-4728] - Support -originalAppId when launching apps through the DT Gateway API |
| * [SPOI-4390] - Replace 'start time' on console to 'Up time' |
| |
| |
| Version 2.0.0 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-4046] - Gateway /containers?states={state} call returns erroneous state information |
| * [SPOI-4057] - License can not be upgraded to Evaluation / Production License |
| * [SPOI-4037] - CONTAINERS_MAX_COUNT in sandbox prevents demos launch from dtcli |
| * [SPOI-4008] - HDHT Error while flushing write cache |
| * [SPOI-3837] - Partitioner interface semantics broken in 2.0.0 physical plan implementation] |
| * [SPOI-3901] - twitter demo app package launch throws NoClassDefFoundError |
| * [SPOI-3900] - Gateway gets 400 error when phone home |
| * [SPOI-3922] - dtcli does not show output port attributes. |
| * [SPOI-3934] - DT CLI Needs to validate user inputs |
| * [SPOI-3054] - # Of Tuples Produced != Number Of Tuples Output By Unifier When Repartitioning |
| * [SPOI-3210] - HDHT DTFile reader bug |
| * [SPOI-3340] - Settings in ~/.dt/dt-site.xml don't override app package defaults |
| * [SPOI-3349] - Can we document how to enable password security in gateway |
| * [SPOI-3365] - Launch App Package does not honor -local option |
| * [SPOI-3369] - Implement the pam authentication as a hadoop authentication handler |
| * [SPOI-3396] - launch command should be able to specify local conf file when launching app package |
| * [SPOI-3397] - DT Gateway doesn't start on Mapr 4.0.1 |
| * [SPOI-3402] - Checking the UID for Dtadmin |
| * [SPOI-3419] - KafkaAdsDimensionsDemo demo available with installer fails |
| * [SPOI-3420] - Installation issues on HDP 2.2 |
| * [SPOI-3469] - gateway, dtcli fail to start in development mode |
| * [SPOI-3503] - AppPackage tests produce 100+GB files |
| * [SPOI-3522] - MaxEventsPerSecond for Flume Ingestor should adjust for partitioned instances |
| * [SPOI-3528] - Installer does not show relevant error if nonexistent env file passed as argument and finishes silently with defaults |
| * [SPOI-3548] - browser inconsistencies with table |
| * [SPOI-3550] - Confusing Breadcrumb navigation on Application Page |
| * [SPOI-3555] - Unifier names should not be links to nonexistent pages |
| * [SPOI-3594] - Systems Diagnostic produces conflicting results for Hadoop installation |
| * [SPOI-3595] - Correctly Implement the clone method in platform. |
| * [SPOI-3608] - User Profile and User Management should not show when auth is disabled |
| * [SPOI-3613] - RBAC Configuration screen is completely missing after recent changes |
| * [SPOI-3628] - Pubsub websocket auth not working with kerberos frontend authentication |
| * [SPOI-3638] - Cant assign roles to user in console |
| * [SPOI-3644] - Develop Tab Can not be seen on naviagation bar |
| * [SPOI-3646] - Error creating new config xml file in package |
| * [SPOI-3647] - MR Operator demo throws null pointer exception |
| * [SPOI-3648] - Cant add admin permissions to existing roles |
| * [SPOI-3654] - In passwd policy user can not change own password using ui console |
| * [SPOI-3657] - Can't Launch Apps Using ui console if user is not the same as dtgateway user |
| * [SPOI-3663] - modify certification scripts and gateway proxy to upload demo jars from repackaged malhar demos |
| * [SPOI-3665] - DataTorrent logo link broken in console |
| * [SPOI-3670] - Update DTCliTest to create app package like AppPackageTest.java |
| * [SPOI-3672] - Uploaded license not used when launching app |
| * [SPOI-3674] - License info not updated after license upload until page reload |
| * [SPOI-3675] - Display Error in Sandbox. License Manager Error:null |
| * [SPOI-3682] - doubleclick select active kills containers |
| * [SPOI-3684] - Gateway deletes JSON app silently (no error) on PUT |
| * [SPOI-3685] - Error message in launch app modal is not red |
| * [SPOI-3686] - Launch properties do not get sent with the launch request |
| * [SPOI-3689] - AppPackages APIs give 404 not found messages |
| * [SPOI-3690] - APIs falsely report role assignments |
| * [SPOI-3691] - API /ws/v1/config/hadoopInstallDirectory does not work |
| * [SPOI-3708] - Gateway shows License error after license agent gets back to normal |
| * [SPOI-3710] - Sandbox packaging invalid due to HDFS dependency |
| * [SPOI-3712] - Exclamation point in DT cli tries to expand "event" |
| * [SPOI-3713] - When CLI has a problem running a CLIProxy command, there is out of memory error from the gateway |
| * [SPOI-3714] - Launch button from other screens should bring up launch modal |
| * [SPOI-3716] - locality performance benchmarks operators failing with OOM exception |
| * [SPOI-3717] - Insufficient license memory for Node Local Performance benckmark application |
| * [SPOI-3718] - change high availability certification benchmarking to compare number of active containers instead of total number of containers |
| * [SPOI-3720] - License manager not found returned from Gateway even though it returns valid license including agent info |
| * [SPOI-3721] - Changes to XMLConfig to incorporate changed properties |
| * [SPOI-3725] - NullPointerException in Certification-initDemo for PerformanceBenchmarkForFixedNumberOfTuples |
| * [SPOI-3729] - Thread local validation failure for multiple ports between two operators |
| * [SPOI-3732] - Get User Info APIs creates users |
| * [SPOI-3733] - Can't Launch Apps Using UI or gateway APIs in kerberos environment |
| * [SPOI-3734] - Remove macro from cli for application or add relevant application |
| * [SPOI-3754] - Gateway can only restore roles once |
| |
| ### Improvement |
| * [SPOI-2036] - Ensure that definePartitions and partitioned are called for parallel partitioned operators |
| * [SPOI-2955] - Provide tool-tip capability to RTS UI |
| * [SPOI-3194] - Support PAM authentication mechanism as an agent for LDAP |
| * [SPOI-3304] - Reorganize code to ensure easy consumability |
| * [SPOI-3345] - Delete OperatorContext.PartitionTPSMax and .PartitionTPSMin |
| * [SPOI-3346] - Delete OperatorContext.InitialPartitionCount |
| * [SPOI-3366] - Update Eval license for RTS 2.0 |
| * [SPOI-3401] - Switch installer, gateway, console from using HADOOP_PREFIX to hadoop executable |
| * [SPOI-3404] - Provide support for viewing/collecting YARN logs across the cluster |
| * [SPOI-3415] - Remove defaults from dt-site.xml and add version |
| * [SPOI-3418] - Remove dependency on /etc/datatorrent |
| * [SPOI-3447] - Partitioning of couchbase input operator |
| * [SPOI-3525] - Messages shown in installer and uninstaller are longer than 80 characters |
| * [SPOI-3554] - Log-Level Setter for angular |
| * [SPOI-3626] - appPackageDisplayName in /ws/v1/appPackages |
| * [SPOI-3667] - Confirmation box to shutdown or kill apps should have "warm" colors |
| * [SPOI-3668] - DTGateway logs are flooded with permission WARN logs |
| * [SPOI-3694] - Populate launch properties with required properties of app |
| * [SPOI-3697] - Add content-disposition so that app package download has a reasonable file name when downloaded from browser |
| * [SPOI-3698] - Support URI codec out the box so that UI can set URI operator properties |
| * [SPOI-4066] - Portable environment settings across releases |
| |
| ### New Feature |
| * [SPOI-3393] - Track recordings by a generated ID rather than startTime |
| * [SPOI-3422] - Restore default roles with backend |
| * [SPOI-3683] - Add download button to app package |
| * [SPOI-3753] - "Reset to Defaults" for user roles |
| |
| ### Task |
| * [SPOI-3927] - Support per application configuration defaults and required properties |
| * [SPOI-2809] - Implement password authentication as a hadoop authentication type |
| * [SPOI-3115] - Update document with password authentication once UI supports user mgmt |
| * [SPOI-3239] - HDHT store removal support |
| * [SPOI-3351] - Abstract the reconciler used in Guava and Rainier to malhar |
| * [SPOI-3368] - Configuration handling on major version upgrade |
| * [SPOI-3371] - CLI to doAs the user specified by HADOOP_USER_NAME if security is enabled |
| * [SPOI-3372] - Gateway REST service to only return rows the user is authorized to view |
| * [SPOI-3373] - Gateway Websocket service to only publish to subscribers with info individual subscribers are authorized to view |
| * [SPOI-3374] - Gateway to return 403 Forbidden for resources the user is not authorized to view or operations the user is not authorized to perform |
| * [SPOI-3375] - Gateway support for permission based authorization |
| * [SPOI-3376] - Gateway REST call to show what permissions the current user has |
| * [SPOI-3377] - Gateway REST call to support role and permission management |
| * [SPOI-3378] - HTTP Header auth support in Gateway |
| * [SPOI-3381] - Implement Kerberos group to role mapping |
| * [SPOI-3385] - Gateway ACL support at the application level |
| * [SPOI-3390] - LDAP/AD support in Gateway |
| * [SPOI-3405] - Implement Kerberos security context and security context filter in Gateway |
| * [SPOI-3443] - Ingestion Application repo and package |
| * [SPOI-3472] - Installer testing |
| * [SPOI-3485] - Console repository relocation |
| * [SPOI-3486] - Create HDHT section in Application Developer Guide |
| * [SPOI-3517] - Document JVM_OPTIONS and QUEUE_NAME features in user docs |
| * [SPOI-3519] - Add PAM auth user docs |
| * [SPOI-3520] - Update RBAC user docs |
| * [SPOI-3521] - Console user docs updates |
| * [SPOI-3536] - Create hdfs output operator to write table files |
| * [SPOI-3537] - write hdfs writer for small files to copy to vertica |
| * [SPOI-3538] - Combine small file writer with vertica writer |
| * [SPOI-3540] - Sandbox updates to support app package launches from console |
| * [SPOI-3593] - Clean up import functionality for demo apps |
| * [SPOI-3610] - Implement RBAC features for App Packages |
| * [SPOI-3619] - Update console for the new app package REST API |
| * [SPOI-3627] - Convert our REST API from v1 to v2 as we have made some backward incompatible changes in the REST API |
| * [SPOI-3631] - Test HDHT recovery |
| * [SPOI-3632] - Upgrade and test demos on 2.0.0 |
| * [SPOI-3633] - Set certain sensitive permissions "admin" role only |
| * [SPOI-3634] - Allow default app instance and app package permissions for each user |
| * [SPOI-3635] - Remove INITITAL_PARTITION_COUNT from demo app pkg properties |
| * [SPOI-3636] - Upgrade and test demos on 2.0.0 |
| * [SPOI-3650] - Test demos on sandbox on dt 2.0.0 |
| * [SPOI-3651] - benchmarks for dt 2.0.0 |
| * [SPOI-3652] - list of current benchmarks |
| * [SPOI-3653] - Document property changes needed run demos on 2.0.0 |
| * [SPOI-3661] - Update security section in operations guide |
| * [SPOI-3662] - Operations Guide: Update application configuration section |
| * [SPOI-3669] - Remove gateway jars api usage from benchmarking and use app pkg api instead |
| * [SPOI-3695] - Documentation on System Alerts in the Gateway |
| * [SPOI-3700] - run app memory usage benchmarks for 2.0 |
| * [SPOI-3701] - run performance benchmarks for 2.0 |
| * [SPOI-3702] - run high availability benchmarks for 2.0 |
| * [SPOI-3703] - Run Fs output operator benchmark |
| * [SPOI-3704] - run performance across tuple size benchmarks for 2.0 |
| * [SPOI-3705] - Change jars api usage to apps package api usage in certification |
| * [SPOI-3706] - Convert Malhar benchmark module to app package |
| * [SPOI-3730] - Getting Started Guide updates |
| |
| ### Bug |
| * [MLHR-1237] - Scrolling log viewer with mouse does not trigger getting more log content |
| * [MLHR-1242] - "last heartbeat" in killed container list shows date in 1969 |
| * [MLHR-1443] - Attempt reconnect if web socket connection is dropped |
| * [MLHR-1523] - Physical operator ids sorted lexicographically |
| * [MLHR-1552] - Port JMS Input operator changes to library. |
| * [MLHR-1553] - shutdown and kill cmds still visible when ended apps are selected |
| * [MLHR-1588] - New installation does not walk through welcome steps |
| * [MLHR-1590] - App Instance dashboard cannot find Logical DAG widget for starting app |
| * [MLHR-1591] - pending undeploy does not have an icon or color associated |
| * [MLHR-1592] - webSocket does not reconnect if user logs out then logs back in |
| * [MLHR-1593] - Installer should be ok with 404 errors when fetching HadoopLocation and dfsRootDirectory |
| * [MLHR-1594] - DFS permissions error during install does not provide resolution steps |
| * [MLHR-1595] - Installer asks for login even if auth is disabled |
| * [MLHR-1606] - Change EDIT_AND_KILL_OTHERS_APPS to MANAGE_OTHERS_APPS |
| * [MLHR-1609] - Physical Operators have an invalid heartbeat when PENDING_DEPLOY |
| * [MLHR-1611] - Do not try to put or edit admin role in auth management page |
| * [MLHR-1612] - New containers from websocket do not get jvm name |
| * [MLHR-1615] - Feedback for retrieving ended apps |
| * [MLHR-1619] - When launching app package, allow launch time parameters to be specified |
| * [MLHR-1622] - YahooFinanceApplication throws null pointer exception on 2.0 |
| * [MLHR-1627] - Make twitter credentials into requiredProperties in app package |
| * [MLHR-1631] - NullPointerException in launching HDFSBenchmarking App:TupleSize property missing in appResponse |
| |
| ### Improvement |
| * [MLHR-1190] - Infinite scroll mechanism in stram events widget |
| * [MLHR-1234] - Add version information to system diagnostics screen |
| * [MLHR-1555] - Remove dashboard component from main operations page |
| * [MLHR-1571] - Height of app list should be determined by available space |
| * [MLHR-1596] - Extract two-way infinite scroll behavior into directive |
| * [MLHR-1597] - Fix FTP Input operator |
| * [MLHR-1602] - Partitioning of couchbase input operator |
| |
| ### New Feature |
| * [MLHR-1228] - create profile storage object |
| * [MLHR-1261] - AngularJS Migration - Tuple Viewer |
| * [MLHR-1262] - AngularJS Migration - Tuple Recording |
| * [MLHR-1354] - Auth management support in the UI |
| |
| ### Story |
| * [MLHR-975] - Migration from Backbone to Angular |
| |
| ### Task |
| * [MLHR-1292] - Recording View |
| * [MLHR-1331] - Update the tuple recording topic |
| * [MLHR-1586] - Remove DirectoryScanInputOperator from library |
| * [MLHR-1601] - Re-arrange logstream app in dt application package format |
| * [MLHR-1608] - Create a Block reader which emits Slice and doesn't read ahead of a block boundary |
| * [MLHR-1617] - Sensitive permission assignment need dialog box warning with consequences |
| * [MLHR-1618] - Update console for the new app package REST API |
| * [MLHR-1623] - Create synchronizer for asynchronously processing streaming data for committed windows |
| * [MLHR-1626] - Write unit tests for AbstractSynchronizer |
| |
| ### Bug |
| * [SPOI-2946] - Provide a tool that generates a license report based on license audit logs |
| * [SPOI-2974] - Better error handling when license is expired |
| * [SPOI-3147] - Issues with dynamic partitioning of Block readers |
| * [SPOI-3192] - Send proxy user when making web service calls from gateway |
| * [SPOI-3267] - HDHT Operator crash when launching application against existing store |
| * [SPOI-3271] - Change port stats propogation test to check for cumulative buffer server bytes |
| * [SPOI-3286] - HDHT WAL recovery crashing with NegativeArraySizeException |
| * [SPOI-3287] - Research the types of data that can be stored in hive orc files |
| * [SPOI-3288] - NullPointerException in StreamingContainerManager-> fillLogicalOperatorInfo |
| * [SPOI-3289] - App Packages launch not reading required properties from dt-site.xml |
| * [SPOI-3293] - Incorrect application name when launching JSON app. |
| * [SPOI-3294] - Configuration setting from property.xml not effective for JSON app |
| * [SPOI-3296] - npm install fails for malhar-ui-console |
| * [SPOI-3297] - Remove keys from metric list in app dashboard widget. |
| * [SPOI-3300] - Two operators named "Dimension Computation" |
| * [SPOI-3305] - Create Sales Demo JSON Input Generator |
| * [SPOI-3306] - Change Event Schema to default to Sales Schema |
| * [SPOI-3309] - DT flume sink not draining under some circumstances |
| * [SPOI-3313] - Setting attributes with JSON and properties app needs cleanup |
| * [SPOI-3319] - NPE in FSStatsRecorder |
| * [SPOI-3320] - The platform should use user folder in hdfs for storage |
| * [SPOI-3323] - Gateway resource leak when RM is not running and/or during network problem |
| * [SPOI-3325] - HDHT DTFile clean cache when reader close |
| * [SPOI-3338] - Application level operator properties do not override "global" operator properties |
| * [SPOI-3343] - Gateway was stuck |
| |
| ### Improvement |
| * [SPOI-2918] - Document Counters |
| * [SPOI-3188] - Ability to interpret an input operator as a regular operator |
| * [SPOI-3312] - OperatorAnnotation that enforces checkpointing to happen at application window boundary |
| * [SPOI-3317] - Add sales generator tuples per window randomization controls |
| * [SPOI-3318] - Add more dimensions and aggregates to Sales demo |
| * [SPOI-3321] - Improve data variation between categories, regions, and discounts |
| * [SPOI-3329] - Implement simple variable substitution in properties files |
| |
| ### New Feature |
| * [SPOI-2014] - Support for doc link for an operator |
| * [SPOI-2931] - Make sure app packages work in sandbox |
| * [SPOI-2989] - HDHT - Recovery |
| |
| ### Task |
| * [SPOI-2663] - Certify datatorrent on a CDH5 secure cluster |
| * [SPOI-2702] - HDS - Generic time series query |
| * [SPOI-2843] - Tracker ticket for work related to app package in Malhar |
| * [SPOI-2945] - Allow application to start even if license manager is not running for production license file |
| * [SPOI-2950] - Generate monthly alerts when the memory used crosses the licensed memory |
| * [SPOI-3022] - Run jpa and jdbc tests on vertica installation in cluster |
| * [SPOI-3098] - Scrub all components that make it to the release list of jiras |
| * [SPOI-3119] - Generate audit logs in the applications as well |
| * [SPOI-3153] - Certify datatorrent on HDP secure cluster |
| * [SPOI-3156] - Tool to generate license report from a single Hadoop grid |
| * [SPOI-3158] - Webservice for license report |
| * [SPOI-3160] - Uptime debugging for license mgr |
| * [SPOI-3161] - license mgr should write memory reported by each app in the logs |
| * [SPOI-3184] - Test case to reproduce locality issue in CDH |
| * [SPOI-3229] - New app will launch for production license even if no memory is available in license |
| * [SPOI-3261] - Kafka one to many dynamic partitioning pilot |
| * [SPOI-3269] - Support contact update |
| * [SPOI-3278] - Research elastic search functionality to build operators |
| * [SPOI-3280] - Create a model parser that loads the model mapping file and generates sql mapping |
| * [SPOI-3282] - Create a vertica output operator |
| * [SPOI-3283] - Start License Manager as part of Installation |
| * [SPOI-3284] - Watch dog process for License Manager in Gateway |
| * [SPOI-3299] - O15 demo: Dictionary for keys |
| * [SPOI-3307] - Add Enrichment operator data file |
| * [SPOI-3315] - O15 demo: Update widget configuration for sales schema |
| * [SPOI-3335] - Kafka offset manager |
| * [SPOI-3339] - Move FSStorageAgent to Malhar |
| * [SPOI-3352] - Process OperatorCommand returned by stats listener |
| * [SPOI-3411] - Create a feed processor operator that converts input event to table rows |
| * [SPOI-3412] - Create gzip input operator |
| * [SPOI-3413] - Create pluggable functions for each column based given function name and parameters |
| * [SPOI-3414] - Create Table controller to handle value generation for each column including user defined functions |
| * [SPOI-3561] - Recording View: links to start and stop recording |
| * [SPOI-3562] - Recording View: page module |
| * [SPOI-3565] - Container Page (angular) |
| |
| ### Bug |
| * [MLHR-1267] - AngularJS Migration - Dashboard Layout Lock |
| * [MLHR-1352] - Custom directive for breadcrumbs |
| * [MLHR-1489] - Create a hive output operator that can write to hive orc files |
| * [MLHR-1500] - Cleanup of the junit.framework.Assert |
| * [MLHR-1505] - Stram event error closes when clicking on stack trace |
| * [MLHR-1506] - Stram event collection removes events |
| * [MLHR-1512] - UI Console - Dashboard Reset |
| * [MLHR-1532] - Table says "loading" when active filter result is 0 rows |
| * [MLHR-1541] - Add orc file output to adsdimension demo |
| * [MLHR-1542] - App Data Framework - WebSocket Support |
| * [MLHR-1585] - Installer fails with auth enabled |
| |
| ### Improvement |
| * [MLHR-1256] - Create a new landing dashboard |
| * [MLHR-1432] - Add confirm for deleting an app package |
| * [MLHR-1451] - Loading feedback for malhar-angular-table |
| * [MLHR-1452] - Loading feedback for app packages and package apps |
| * [MLHR-1454] - Performance Tuning for malhar-angular-table |
| * [MLHR-1499] - Idempotent State Manager |
| * [MLHR-1531] - App overview does not show started time |
| * [MLHR-1540] - Remove stram events from physical view |
| * [MLHR-1543] - Switch to gulp in malhar-angular-table |
| * [MLHR-1549] - Short operator properties should be shown in line |
| |
| ### New Feature |
| * [MLHR-1282] - AngularJS Migration - App Instance Page - Alerts Recordings View |
| * [MLHR-1301] - AngularJS Migration - Physical Operator Page - Recordings |
| * [MLHR-1317] - AngularJS Migration - App Instance Page - Logical View - Log Levels |
| * [MLHR-1362] - AngularJS Migration - App Instance Page - Stram Events Widget Resize |
| * [MLHR-1370] - AngularJS Migration - Container View - Chart |
| * [MLHR-1428] - Dashboard Component - Option to Disable Vertical Resize |
| * [MLHR-1462] - Socket.IO Kafka Communication Protocol |
| * [MLHR-1463] - UI Console - Client Settings |
| * [MLHR-1464] - Kafka App Data - Server Socket.IO Node.js |
| * [MLHR-1465] - Kafka Socket.IO Service - Latency |
| * [MLHR-1466] - Kafka Server - Query Cache |
| * [MLHR-1470] - Dashboard Builder Integration |
| * [MLHR-1486] - UI Console - Deploy Scripts |
| * [MLHR-1488] - App Package Dag Viewer - Handle Case when DAG is empty |
| * [MLHR-1491] - App Data UI - Runtime Configuration for Default Dashboard/Widgets/Queries |
| * [MLHR-1495] - Kafka Debug Widget - Kafka Producer/Consumer Topics |
| * [MLHR-1498] - Provide Ability to Reset Dashboard Configuration |
| * [MLHR-1502] - App Data Server - Fetch Latest Kafka Offset |
| * [MLHR-1514] - App Data Server - EventEmitter |
| * [MLHR-1515] - App Data Server - UML Diagrams |
| * [MLHR-1520] - App Data UI - Table Widget |
| * [MLHR-1521] - App Data UI - WebSocket Support |
| * [MLHR-1530] - App Data UI - Table Widget |
| |
| ### Story |
| * [MLHR-1295] - AngularJS Migration - Logical Operator Page |
| * [MLHR-1296] - AngularJS Migration - Physical Operator Page |
| |
| ### Task |
| * [MLHR-1270] - Container Page (angular) |
| * [MLHR-1272] - Container Page: memory gauge |
| * [MLHR-1274] - Container Page: metric chart |
| * [MLHR-1281] - Stream View |
| * [MLHR-1286] - Stream View: sources table |
| * [MLHR-1287] - Stream View: sinks |
| * [MLHR-1288] - Port View |
| * [MLHR-1290] - Port View: overview |
| * [MLHR-1291] - Port View: chart |
| * [MLHR-1293] - Recording View: page module |
| * [MLHR-1294] - Recording View: links to start and stop recording |
| * [MLHR-1307] - Installation Wizard |
| * [MLHR-1310] - Info Menu Links |
| * [MLHR-1312] - Lock Layout (malhar-angular-dashboard) |
| * [MLHR-1453] - System Diagnostics Page |
| * [MLHR-1460] - Installation Wizard: welcome page |
| * [MLHR-1461] - Installation Wizard: hadoop config page |
| * [MLHR-1467] - Installation Wizard: license screen |
| * [MLHR-1482] - Installation Wizard: summary screen |
| * [MLHR-1483] - Installation Wizard: license upload |
| * [MLHR-1496] - Create an operator for Solr |
| * [MLHR-1516] - Annotate new FS output operators |
| * [MLHR-1518] - WebSocket support for dimension demo |
| * [MLHR-1574] - UI Auth: support for current password auth |
| * [MLHR-1575] - UI Auth: Managing role and permissions |
| |
| ### Sub-task |
| * [SPOI-3123] - Add troubleshooting section to Operations and Installation Guide |
| * [SPOI-3124] - Check for vmem/pmem ratio during System Diagnostics test |
| * [SPOI-3129] - Integrate Kafka query protocol for AdsDimension demo. |
| * [SPOI-3154] - Setup AdsDimension demo on cluster |
| * [SPOI-3226] - Create JSON AdInfo Generator for App Builder Demo |
| * [SPOI-3227] - Crate JSON to Map converter for App Builder Demo |
| * [SPOI-3228] - Create shared dimensions computation schema for App Builder Demo |
| |
| ### Bug |
| * [SPOI-1667] - Drop of Performance for Machine demo |
| * [SPOI-2742] - Core dumps on EMR |
| * [SPOI-2942] - Disable physical plan locking when there is no available license memory |
| * [SPOI-2947] - Provide a tool that distributes memory and generates licenses that can be deployed on multiple clusters |
| * [SPOI-3018] - Investigate raw performance of vertica jdbc |
| * [SPOI-3024] - Adding Virtual Mem to Physical Memory setting in Readme file |
| * [SPOI-3025] - Can't see the container logs from DT Console in Pivotal installation |
| * [SPOI-3053] - Continual Repartitioning Of Operator Causes Out Of Memory Exception |
| * [SPOI-3081] - Core StatsTest.testPortStatsPropagation Failing |
| * [SPOI-3107] - Failed to load Logical Plan |
| * [SPOI-3132] - User Interface Guide URL specified in current Sandbox gives 404 |
| * [SPOI-3181] - dtgateway script: $PATH is being appended to repeatedly when gateway has trouble starting |
| * [SPOI-3187] - HDHT query results partition duplication |
| * [SPOI-3190] - HDHT AdsDimension demo last bar should grow |
| * [SPOI-3195] - Operator class exception when loading app package |
| * [SPOI-3196] - CLI returns Perm Gen Space errors when there are too many classes to inspect in app package |
| * [SPOI-3197] - Provide meta information for operators that can be used by DAG builder |
| * [SPOI-3198] - App Package archetype is erroneously including app package manifest in the low level jar file |
| * [SPOI-3213] - Newer maven versions (>3.1) do not work with copy-maven-plugin in app package archetype |
| * [SPOI-3215] - Duplicate streams in OperatorDeploymentInfo with default partition/parallel partitioning |
| * [SPOI-3232] - Mobile demo not dynamically partitioning |
| * [SPOI-3262] - Issue with serialization of xml cartesian product operator |
| * [SPOI-3319] - NPE in FSStatsRecorder |
| |
| ### Improvement |
| * [SPOI-1967] - Accept Password for Private Key |
| * [SPOI-2997] - License file content updates |
| * [SPOI-3083] - Ability to save DAG as a DT App Package |
| * [SPOI-3084] - Operator Library Browser |
| * [SPOI-3087] - Properties Section |
| * [SPOI-3088] - Attributes Section |
| * [SPOI-3089] - Operators needed for demo |
| * [SPOI-3102] - FileSplitter needs to be idempotent |
| * [SPOI-3136] - Add a check to see if stram is connected to THIS gateway |
| |
| ### New Feature |
| * [SPOI-3142] - Ability to take Logical Plan Serialization JSON to construct a DAG and launch it |
| * [SPOI-3148] - App Package REST API needs to allow PUT of json logical plan for adding/replacing application in app package |
| * [SPOI-3149] - Ability to launch a json-specified app within app package |
| * [SPOI-3185] - Import demo app packages from dtgateway fs |
| * [SPOI-3186] - Group app packages under demos folder in release |
| * [SPOI-3189] - Add display name and description to app package's manifest |
| * [SPOI-3199] - XML Javadoc |
| * [SPOI-3266] - Provide Ability to Allow Cross Origin Access |
| |
| ### Task |
| * [SPOI-3039] - Print data upon license mgr app start |
| * [SPOI-3061] - Add port tuple type in logical plan serialization |
| * [SPOI-3067] - Remove time-bombed public/private key pair from the code |
| * [SPOI-3068] - Make sure the release process (sandbox, installer binary) is okay with the time bound key pair removed from the code |
| * [SPOI-3069] - Remove the entire request blob process and generate license straight from customer info and license info |
| * [SPOI-3070] - Device ID change in license |
| * [SPOI-3071] - Generate License web tool to prompt for password for the private key |
| * [SPOI-3126] - O15 - Generic Dimension Computation/Store Operator |
| * [SPOI-3128] - HDHT - AdsDimension app integration for internal demo |
| * [SPOI-3130] - HDHT - Test files with varying block sizes |
| * [SPOI-3134] - Add a organization field in license file |
| * [SPOI-3162] - Kafka ingestion demo |
| * [SPOI-3164] - Integrate Kafka Query frontend into demos |
| * [SPOI-3165] - Ability to return JSON app as-is from app package through REST |
| * [SPOI-3166] - Ability to delete JSON apps from app package |
| * [SPOI-3167] - Ability to save incomplete json-based app to an app package |
| * [SPOI-3179] - Investigate how IDEs parse javadoc |
| * [SPOI-3182] - Complete dag annotations spec |
| * [SPOI-3193] - Investigate pluggable authorization mechanisms supported by Hadoop |
| * [SPOI-3200] - Generate resource file containing javadoc comments and custom tags |
| * [SPOI-3201] - Configure default javadoc doclet in pom to include custom tags |
| * [SPOI-3202] - Transform xml javadoc resources file to contain only comments and tags |
| * [SPOI-3203] - Add javadoc resource file to the class jar at maven build |
| * [SPOI-3206] - Add field and method comments to transformed xml javadoc |
| * [SPOI-3217] - Limit events DTFlumeSink pumps into the dag |
| * [SPOI-3219] - Help diagnose launch issue |
| * [SPOI-3225] - Check and fix machine and twitter hash tag demos |
| * [SPOI-3236] - displayName should be available from appPackage/applications call |
| |
| ### Bug |
| * [MLHR-1347] - Create a block reader in library which is capable of dynamic partitioning itself |
| * [MLHR-1366] - HdfsBucketStore is not completely covered by unit test cases |
| * [MLHR-1367] - Add Counter Aggregators For Monitoring AbstractThroughputFSDirectoryInputOperator |
| * [MLHR-1373] - Container log download link |
| * [MLHR-1404] - App Package File Upload |
| * [MLHR-1407] - App List Kill App Selection Issue |
| * [MLHR-1410] - Add a "remove" option to the "set/subscribe" method of BaseCollection |
| * [MLHR-1412] - App Package archetype is erroneously including app package manifest in the low level jar file |
| * [MLHR-1419] - Xml cartesian product operator has kryo serialization errors in some cases |
| * [MLHR-1422] - UI Physical tab-> sorting by operator id is by lexical order not numerical |
| * [MLHR-1423] - app instance page does not update when state goes ACCEPTED => RUNNING |
| * [MLHR-1426] - favicon not being handled in gulp build |
| * [MLHR-1436] - Uptime shows -1 day even when the app is runing for some time |
| * [MLHR-1438] - AdsDimensionsWithHDSDemo - Expose Aggregation in Operator Properties |
| * [MLHR-1442] - Move the recipient property for the machine data to config |
| * [MLHR-1557] - subscribe handler on app instance alters original ws message |
| * [MLHR-1558] - Palette in the Apps List does not update after one app gets killed |
| |
| ### Improvement |
| * [MLHR-1427] - Make app name link to instance page as well in AppsList |
| |
| ### New Feature |
| * [MLHR-1388] - Base DAG Viewer |
| * [MLHR-1389] - App Package List Page |
| * [MLHR-1390] - Applications/Package Info Page |
| * [MLHR-1391] - App Package REST API Integration |
| * [MLHR-1392] - App Package CRUD |
| * [MLHR-1406] - UI Support of app package import |
| * [MLHR-1408] - Allow Shutting Down/Killing Multiple Applications |
| * [MLHR-1424] - UI Grid 3 Integration - Fonts |
| * [MLHR-1429] - Dashboard Component - Configurable Widget |
| * [MLHR-1430] - Kafka UI Demo |
| * [MLHR-1431] - Visual Data Demo Update |
| |
| ### Story |
| * [MLHR-1241] - App Package support in UI |
| * [MLHR-1357] - UI Dashboard Front Page Design |
| |
| ### Task |
| * [MLHR-1079] - Normalize timestamps for info pages |
| * [MLHR-1104] - Design App Package and Upload/Launch Feature |
| * [MLHR-1105] - Polish simplified (current) app data alert feature |
| * [MLHR-1207] - Convert demo apps to app packages |
| * [MLHR-1309] - License Info |
| * [MLHR-1363] - Create file splitter that breaks file into blocks and emits block metadata and a block reader |
| * [MLHR-1415] - Clean-up Hdfs Output operators in library and incorporate the features of fault-tolerant Writer |
| * [MLHR-1503] - Remove deprecated jdbc package from contrib |
| |
| |
| |
| Version 1.0.4 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-2346] - ExactlyOnceTest#testLinearInputOperatorRecovery hangs |
| * [SPOI-2511] - ResourceManager HA support |
| * [SPOI-2625] - uninstall.sh should print message that reminds users of running DT applications |
| * [SPOI-2922] - Counters Aggregator gets lost when an operator is parallel partitioned - test fix |
| * [SPOI-2939] - AppBundles test resource refers to fixed version |
| * [SPOI-2967] - Mobile demo dies after couple of days |
| * [SPOI-2979] - App Bundle unit test should load properties.xml |
| * [SPOI-2991] - [kafka-yarn] Archive resource to use less space in HDFS |
| * [SPOI-2993] - appBundle with conflicting names in config and ApplicationAnnotation can't launch in CLI with either name |
| * [SPOI-2995] - Generate MANIFEST.MF automatically instead of requiring user to change it |
| * [SPOI-2999] - Investigate application integration to Vertica |
| * [SPOI-3012] - launch application throws NPE when yarn.application.classpath is not defined |
| * [SPOI-3015] - DTGateway log grows indefinitely |
| * [SPOI-3019] - Investigate how many objects per second (with a single field) using JPA can be written to vertica |
| * [SPOI-3027] - Sandbox GATEWAY_CONNECT_ADDRESS change to support VMWare |
| * [SPOI-3055] - NullPointerException When Repartitioning Too Frequently |
| * [SPOI-3074] - NullPointerException in AppMaster |
| * [SPOI-3079] - NPE while launching application |
| * [SPOI-3080] - Content-disposition: attachment for container logs |
| * [SPOI-3091] - Null Pointer Exception/Internal Server Error When Getting Operator Stats |
| * [SPOI-3092] - DefaultUnifier of a port in an operator does not function correctly when the port has more than one sinks |
| |
| ### Improvement |
| * [SPOI-2821] - Add bucket processed time |
| * [SPOI-2891] - DTGateway default log setting too verbose |
| * [SPOI-2938] - Document AppBundles development and deployment |
| * [SPOI-3044] - gateway should show the full stack trace of the origin upon error when proxying |
| |
| ### New Feature |
| * [SPOI-2720] - API Call for determining if an application's stram can connect to the gateway |
| * [SPOI-2981] - HDS - File Format |
| * [SPOI-2982] - HDS - Writing Data Files |
| * [SPOI-2983] - HDS - Bucket Meta Data |
| * [SPOI-2984] - HDS - WAL |
| * [SPOI-2986] - HDS - Bucket Management |
| * [SPOI-3031] - Add ability to specify number of recording windows |
| * [SPOI-3032] - Delete request for recording |
| * [SPOI-3033] - Provide byte offset for each line when grepping container log |
| * [SPOI-3049] - Ingestion streamlet design for first cut |
| |
| ### Task |
| * [SPOI-1734] - Licensing Agent Logging for Audit |
| * [SPOI-2810] - Update document with details on password authentication |
| * [SPOI-2906] - Write documentation on App Bundle |
| * [SPOI-2908] - Prototype kafka-on-yarn |
| * [SPOI-2934] - Add application id to the tuple record topic |
| * [SPOI-2962] - Create operator to write events in bucket file and write index files for keys |
| * [SPOI-2963] - Change query processor operator to fetch events based on indices |
| * [SPOI-2996] - Support launch-app-bundle command without the appname argument |
| * [SPOI-3006] - Create a LATLON to MGRS converter utility |
| * [SPOI-3009] - Create a utility to calculate MD5 hash |
| * [SPOI-3010] - Create a utility for Blowfish |
| * [SPOI-3014] - Update sandbox release docs |
| * [SPOI-3026] - Change the name app bundle to app package and the extension zip to jar |
| * [SPOI-3036] - Review issue with ingestion |
| * [SPOI-3050] - A first cut at design for ingestion streamlet |
| * [SPOI-3051] - License work |
| * [SPOI-3108] - Add REST API to get the entire hadoop configuration |
| * [SPOI-3121] - HDS - File Format - Performance testing TFile / DTFile |
| * [SPOI-2985] - HDS - File Format - Performance testing - HFile |
| * [SPOI-2987] - HDS - File Format - Performance testing - MapFile |
| * [SPOI-3122] - HDS - File Format - HFile implementation |
| * [SPOI-3017] - HDS - File Format - Common Interface |
| * [SPOI-3100] - Create the SinglePointCalculator |
| * [SPOI-3103] - Create Synchronizer Operator |
| * [SPOI-3104] - Create Persister to store the data |
| |
| |
| ### Bug |
| * [MLHR-1223] - JDBC Store doesn't support connection properties |
| * [MLHR-1253] - documenting keyhashvalpair |
| * [MLHR-1255] - db api in malhar lib has KeyValueStore interface which is similar to db/cache/Store api |
| * [MLHR-1260] - AngularJS Migrations - Metrics Grouping |
| * [MLHR-1268] - AbstractBatchTransactionableStoreOutputOperator add to library db is same as public abstract class AbstractAggregateTransactionableStoreOutputOperator |
| * [MLHR-1269] - Mobile demo dies after couple of days |
| * [MLHR-1319] - AngularJS Migration - WebSocket Service |
| * [MLHR-1320] - AngularJS Migration - Page Visibility API |
| * [MLHR-1323] - stram events links broken (angular) |
| * [MLHR-1325] - Wrong "track by" in physical operators list |
| * [MLHR-1328] - "inspect" button in appslist broken |
| * [MLHR-1335] - dtPageHref directive does not update href of element on changes |
| * [MLHR-1336] - Container shorthand directive not watching value changes |
| * [MLHR-1342] - Allow DAG widgets to be vertically resized (using jquery resizable) |
| * [MLHR-1344] - AngularJS Migration - Metrics Chart |
| * [MLHR-1345] - AngularJS Migration - Physical Operator Page - Chart |
| * [MLHR-1346] - Test for AbstractFSDirectoryInputOperator fails sporadically |
| * [MLHR-1359] - AngularJS Migration - Dashboard Widgets Vertical Resize |
| * [MLHR-1364] - BucketManager while reading values doesn't ensure that latest value for a key is read |
| * [MLHR-1368] - AngularJS Migration - App Instance Page - Overview Grouping |
| * [MLHR-1374] - Remove all occurrences of dag.setAttribute(DAG.APPLICATION_NAME, "xxx") in demos code |
| |
| |
| ### Improvement |
| * [MLHR-1263] - Migrate JDBC non-transaction output operator to new db api and move it to lib |
| * [MLHR-1322] - Changing Machine Data Operator and Stream Names |
| * [MLHR-1324] - appState directive should be renamed to something more generic |
| * [MLHR-1327] - Assume lodash is global throughout console |
| * [MLHR-1343] - Use userStorage to store height of stram events |
| * [MLHR-1351] - Allow transformResponse to set a fetchError |
| * [MLHR-1353] - Centralize location of breadcrumbs |
| * [MLHR-1360] - Put dashboards on the left |
| |
| ### New Feature |
| * [MLHR-419] - DAG Widget with AngularJS |
| * [MLHR-1220] - Send the "unsubscribe" webSocket message for topic when no more listeners |
| * [MLHR-1239] - AngularJS Migration - Physical DAG View |
| * [MLHR-1240] - AngularJS Migration - Logical DAG View |
| * [MLHR-1243] - Angular JS Migration - Current/Recover Window Id Metrics |
| * [MLHR-1245] - AngularJS Migration - Partitions/Container Metrics |
| * [MLHR-1246] - AngularJS Migrations - DAG View Zoom |
| * [MLHR-1247] - AngularJS Migration - Physical View |
| * [MLHR-1248] - AngularJS Migration - Container View |
| * [MLHR-1249] - AngularJS Migration - Landing Page High-Level Metrics |
| * [MLHR-1250] - AngularJS Migration - Charts |
| * [MLHR-1251] - AngularJS Migration - Gauge Widget |
| * [MLHR-1252] - AngularJS Migration - DAG View Base Renderer |
| * [MLHR-1278] - AngularJS Migration - App Instance Page - Logical View |
| * [MLHR-1279] - AngularJS Migration - App Instance Page - Physical View |
| * [MLHR-1280] - AngularJS Migration - App Instance Page - Physical DAG View |
| * [MLHR-1283] - AngularJS Migration - App Instance Page - Metric View |
| * [MLHR-1298] - AngularJS Migration - Physical Operator Page - Overview |
| * [MLHR-1299] - AngularJS Migration - Physical Operator Page - Port List |
| * [MLHR-1300] - AngularJS Migration - Physical Operator Page - Properties |
| * [MLHR-1303] - AngularJS Migration - Logical Operator Page - Overview |
| * [MLHR-1305] - AngularJS Migration - Logical Operator Page - Partitions |
| * [MLHR-1306] - AngularJS Migration - Logical Operator Page - Chart |
| * [MLHR-1308] - AngularJS Migration - Logical Operator Page - Properties |
| * [MLHR-1313] - AngularJS Migration - App Instance Page - Physical View - Container List |
| * [MLHR-1314] - AngularJS Migration - App Instance Page - Physical View - Operators |
| * [MLHR-1315] - AngularJS Migration - App Instance Page - Logical View - Chart |
| * [MLHR-1316] - AngularJS Migration - App Instance Page - Logical View - Stream List |
| * [MLHR-1321] - AngularJS Migration - Distribution Build with Gulp |
| * [MLHR-1334] - Front-End Build with Gulp - WebSocket Proxy |
| * [MLHR-1337] - Front-End Build with Gulp - JS/CSS Revisions |
| * [MLHR-1338] - Front-End Build with Gulp - Angular Templates Injection |
| * [MLHR-1341] - create userStorage service for user settings |
| * [MLHR-1358] - Breadcrumbs to jump between collection elements |
| * [MLHR-1361] - Logical/Physical DAG - Stream Locality Legend |
| * [MLHR-1365] - AngularJS Migration - Widget Settings Modal - Height Management |
| |
| ### Task |
| * [MLHR-1208] - Convert contrib apps to app bundles |
| * [MLHR-1244] - Update tuple recorder topic to use "applications.<appid>.tupleRecorder.<startTime>" |
| * [MLHR-1254] - Create a test case for UniqueValueCountAppender |
| * [MLHR-1264] - Start Logical Operator Page |
| * [MLHR-1265] - Start Physical Operator Page |
| * [MLHR-1266] - Start Container Page View |
| * [MLHR-1271] - Container Page: overview |
| * [MLHR-1273] - Container Page: operator list |
| * [MLHR-1275] - Container Page: log viewer |
| * [MLHR-1276] - Container Page: page module |
| * [MLHR-1284] - Stream View: overview |
| * [MLHR-1285] - Stream View: page module |
| * [MLHR-1289] - Port View: page module |
| * [MLHR-1302] - Operations Landing Page: memory gauge |
| * [MLHR-1311] - Vertical Resize (malhar-angular-dashboard) |
| * [MLHR-1318] - Move webSocket service to malhar-angular-widgets |
| * [MLHR-1326] - Create .jshintrc for IDE/editor, grunt tasks |
| * [MLHR-1330] - Document gulp usage in README |
| * [MLHR-1333] - add "gulp coverage" task |
| * [MLHR-1339] - Need a dynamic partitioner for File processing operator where the count of the operator instances is controlled by the backlog present |
| * [MLHR-1348] - Change app bundle to app package on all demos |
| * [MLHR-1349] - Link tables to userStorage service |
| * [MLHR-1372] - Adding the tool-tip capability to console |
| * [MLHR-1376] - Add counters and counters aggregator to mobile demo |
| |
| |
| |
| Version 1.0.3 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-2620] - S3 reader error |
| * [SPOI-2673] - Move the Kafka Exactly Once Producer to Malhar Library |
| * [SPOI-2775] - Log file being processed on seperate port to monitor progress by input operators. |
| * [SPOI-2813] - Send top gateways in descending order |
| * [SPOI-2814] - Change field name of input event in CDREvent |
| * [SPOI-2815] - Show top 3 gateways using a widget |
| * [SPOI-2816] - No query poll in alert information view |
| * [SPOI-2817] - For non-finalized buckets threshold flags should not be set |
| * [SPOI-2819] - Send data continuously to the application |
| * [SPOI-2820] - Provide more alerts in the UI |
| * [SPOI-2837] - Operators crashing sporadically when alert information requested from frontend |
| * [SPOI-2838] - Clear out the cdr event files and aggregates are cleared |
| * [SPOI-2844] - Machine data demo waits long before starting |
| * [SPOI-2845] - Recovery is failing in cdr events output operator after upstream operator redeploys |
| * [SPOI-2854] - Buckets are not being finalized correctly when there is a bucket gap in input data |
| * [SPOI-2861] - GATEWAY_CONNECT_ADDRESS not set during install |
| * [SPOI-2866] - Change lablel of top gateways to top 3 gateways |
| * [SPOI-2867] - Change null to empty value in the UI for alert event information |
| * [SPOI-2869] - Need a cleanup script to delete old files from hdfs |
| * [SPOI-2882] - Backup the allatory log file for flume integration build as well |
| * [SPOI-2883] - NPE in bufferserver |
| * [SPOI-2892] - DTGateway WS fails to get YARN logs |
| * [SPOI-2894] - Investigate bucket manager for cdr event storage |
| * [SPOI-2900] - First few events for an application do not get published |
| * [SPOI-2902] - Documentation for sandbox missing images |
| * [SPOI-2910] - PartitioningTest.testDynamicDefaultPartitioning sometimes fails and sometimes succeeds |
| * [SPOI-2913] - Events API returns no events when limit and offset not supplied |
| * [SPOI-2919] - Change cleanup to cleanup folders older than an hour |
| * [SPOI-2920] - make stats recorder asynchronous so that it won't block stram |
| * [SPOI-2921] - Test Node Locality feature on HDP |
| * [SPOI-2932] - Historical containers don't show up on gateway if there is a second attempt on application |
| * [SPOI-2936] - Historical data store high level design |
| * [SPOI-2937] - Gateway throwing exception while launching app |
| |
| ### Improvement |
| * [SPOI-2811] - Show input record in alert information dialog |
| * [SPOI-2812] - Show top gateways for all defect aggregates and possibly all aggregates |
| * [SPOI-2881] - Show actual memory usage in dashboard |
| * [SPOI-2891] - DTGateway default log setting too verbose |
| * [SPOI-2911] - Create a Bean2String codec |
| |
| ### New Feature |
| * [SPOI-2228] - Gateway to load and to allow manipulation of app bundles |
| * [SPOI-2753] - Front End POC - Aggregates Table |
| * [SPOI-2757] - Front End POC - Kafka Operations |
| * [SPOI-2758] - Front End POC - Historical Data Navigation |
| * [SPOI-2808] - Support YARN log aggregation for dead container log retrieval in the gateway |
| * [SPOI-2826] - Maven archetype for assembling app bundle |
| * [SPOI-2827] - CLI to load app bundles |
| * [SPOI-2828] - Supports launching apps in app bundle in CLI |
| * [SPOI-2829] - Gateway to allow uploading and changing configuration of existing App Bundles |
| * [SPOI-2849] - Demonstrate Datatorrent RTS as front end ingestion for Spark. |
| * [SPOI-2879] - Kafka Request/Response Debug Collapsible Panel |
| * [SPOI-2896] - Unique identifier for stram events |
| |
| ### Story |
| * [SPOI-2835] - Streamlet Design - 1.0.3 |
| * [SPOI-2836] - Kafka integration for 1.0.3 |
| * [SPOI-2851] - Platform Excellence - 1.0.3 |
| |
| ### Task |
| * [SPOI-2212] - Determine logistics of application bundles |
| * [SPOI-2521] - Add counters for checkpointo operations |
| * [SPOI-2546] - Pi demo does not seem to work in 512MB |
| * [SPOI-2547] - Twitter demo needs 1024MB |
| * [SPOI-2684] - Automated distro certification - collect minimum requirements |
| * [SPOI-2786] - Include setters and getters for output fields in the event |
| * [SPOI-2795] - Add top 3 gateways to alert aggregate csv in HDFS |
| * [SPOI-2802] - Dynamic partitioning for S3 input operator. |
| * [SPOI-2822] - Add operator to write cdr events to hdfs |
| * [SPOI-2823] - Fetch events from hdfs for alert events query response |
| * [SPOI-2846] - Partition Aggregations operator |
| * [SPOI-2859] - Separate out query processing from aggregations operator |
| * [SPOI-2870] - Application template classpath does not include malhar libraries |
| * [SPOI-2874] - Add Port Queue Capacity usage as part of the operator stats |
| * [SPOI-2878] - Move the gateway randomization to generator |
| * [SPOI-2890] - Update support contacts to malhar-users group |
| * [SPOI-2914] - Configure Store operator to join logger and tracker stream. |
| |
| ### Sub-task |
| * [SPOI-2747] - Total emitted window of an operator keeps on increasing even when it has stopped emitting |
| * [SPOI-2850] - Sort out the dependencies for dt-flume so we do not include the jars which are already part of dt dist and hadoop dist |
| * [SPOI-2873] - Deprecate ShipContainingJars. |
| * [SPOI-2895] - launch -license does not work |
| * [SPOI-2907] - Counters appear in physical plan only when a Counters aggregator or a stats listener is set on the operator |
| |
| |
| ### Bug |
| * [MLHR-1176] - Update demo section of README.md |
| * [MLHR-1219] - Non-chrome browsers cannot parse date format from date picker |
| * [MLHR-1227] - AbstractKafkaOutputOperator fails with java.lang.NoClassDefFoundError: com/yammer/metrics/Metrics |
| * [MLHR-1235] - close button on gateway info modal does not close the modal |
| |
| ### Improvement |
| * [MLHR-1213] - Use trackBy for malhar-angular-table |
| * [MLHR-1216] - DB key value store manager enhancement |
| * [MLHR-1218] - Clicking on an open stram log event should close it in UI |
| |
| ### New Feature |
| * [MLHR-1212] - Make scrollbar draggable on malhar-angular-table |
| * [MLHR-1225] - Logical Operator Table |
| * [MLHR-1226] - Stram Event widget |
| * [MLHR-1229] - Provide better widget options dialog |
| * [MLHR-1231] - Custom template/controller for widget options |
| |
| ### Task |
| * [MLHR-1205] - Restructure directory structure in angular console |
| * [MLHR-1206] - Change malhar pom to use provided scope to prevent DT and hadoop jars from being sucked into the runtime classpath |
| * [MLHR-1210] - Create script to automatically add scripts to index.html |
| * [MLHR-1224] - Kafka enabled AdsDimensions demo |
| |
| |
| |
| Version 1.0.2 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-431] - Fix mergeSort operator to actually do mergeSort |
| * [SPOI-2499] - Provide license delegation tokens in secure environment |
| * [SPOI-2500] - Provide new delegation token before an old one expires in secure enviroment |
| * [SPOI-2501] - dt.log not written under CDH5 GA |
| * [SPOI-2506] - [MapR]FileSystem.mkdirs() doesn't work for existing folders in MapRFileSystem |
| * [SPOI-2605] - DT install as root cannot use /user/chetan/dt0528 as DFS location and not printing out detailed error |
| * [SPOI-2608] - LicensingAppMaster's name should not be obfuscated |
| * [SPOI-2617] - Memory usage counted by license agent and not released after app is killed |
| * [SPOI-2619] - Front End Server - fill missing slots for time series |
| * [SPOI-2630] - Not able to launch apps in cloudera cluster |
| * [SPOI-2634] - Uninstall is removing entire datatorrent folder as opposed to the release folder only |
| * [SPOI-2639] - dtcli allows apps to be launched before configuration is finished |
| * [SPOI-2668] - Parent jira for Kafka work for 1.0.2 |
| * [SPOI-2678] - Put Fraud demo into the demo UI |
| * [SPOI-2679] - Front End Server - Query ID as Query JSON |
| * [SPOI-2683] - Appmaster Logs are not shown in the Console |
| * [SPOI-2689] - set-operator-property produces NPE on stram |
| * [SPOI-2699] - Mobile demo app mis-behaving |
| * [SPOI-2703] - Gateway fails to start in devel mode with hadoop 2.3.0 |
| * [SPOI-2705] - Demo UI instructions missing from install README |
| * [SPOI-2706] - Logical plan change fails w/ obfuscated build |
| * [SPOI-2709] - Partition events should not be registered if same no. partitions result |
| * [SPOI-2710] - Containers running the unifier are not released |
| * [SPOI-2723] - Design and develop the File Ingestion app |
| * [SPOI-2737] - Gateway installation replaces dt-site.xml with invalid version |
| * [SPOI-2744] - ZIP of version's docs not being made available for download |
| * [SPOI-2748] - dtdemos service fails to stop in sandbox |
| * [SPOI-2749] - docs distribution files at root path |
| * [SPOI-2759] - Installation instructions for user setup |
| * [SPOI-2760] - Get container log content fails with 500 |
| * [SPOI-2761] - Remove dtadmin reference from install wizzard |
| * [SPOI-2762] - EMR configuration issues |
| * [SPOI-2779] - dtcli failed to read the license file which was uploaded through UI |
| * [SPOI-2798] - Request for log content outside of range results in 500, long request time |
| * [SPOI-2839] - set-operator-property broken in master |
| |
| ### Improvement |
| * [SPOI-2602] - Containers published via websocket should only contain live containers and recently finished containers |
| * [SPOI-2610] - Parent: Automatic unobfuscation tool |
| * [SPOI-2613] - Create web tool to run unobfuscator |
| * [SPOI-2614] - Automate transfer of allatori-log.xml to web server |
| * [SPOI-2712] - Add window_width to app info REST call |
| * [SPOI-2736] - Merge /systemAlerts/alerts and /systemAlerts/inAlerts API calls |
| * [SPOI-2751] - Include sandbox README.html with docs.zip and on website |
| |
| ### New Feature |
| * [SPOI-950] - Specify memory requirements on per operator basis (duplicate) |
| * [SPOI-2220] - Gateway App Bundle API spec |
| * [SPOI-2339] - Enable container size for each operator(s) |
| * [SPOI-2515] - Ability to dynamically change the logger level for any instantiated loggers within the application |
| * [SPOI-2523] - System alerts for application state and metrics |
| * [SPOI-2525] - Add operator serialization check in local mode |
| * [SPOI-2528] - Expose "counters" in REST API |
| * [SPOI-2575] - Kafka Pub/Sub Protocol |
| * [SPOI-2588] - Record physical counters per window |
| * [SPOI-2589] - Expose counters logical and physical through web services/ web socket |
| * [SPOI-2606] - Initializing loggers with levels specified in the configuration |
| * [SPOI-2627] - Front End Server - "Countdown" usage |
| * [SPOI-2629] - Front End Server - Support of Multiple Partitions |
| * [SPOI-2631] - Front End Server - Cache Expiration Strategy |
| * [SPOI-2632] - Front End Server - Kafka Reconnect |
| * [SPOI-2633] - Front End Server - Page Performance (Visibility API) |
| * [SPOI-2638] - Front End Server - Kafka Errors Handling |
| * [SPOI-2645] - Front End Server - Kafka SimpleConsumer |
| * [SPOI-2652] - Develop and Stage Hadoop Summit Landing Page |
| * [SPOI-2696] - Front End Server - Top N Metrics |
| * [SPOI-2697] - Front End Server - Dynamic Publisher and Site |
| * [SPOI-2698] - Front End Server - Kafka Troubleshooting |
| * [SPOI-2752] - Front End Server - LRU Cache |
| * [SPOI-2756] - Front End POC - Alert Modal |
| * [SPOI-2804] - Create a hbase operator that uses a config to map incoming csv tuples to hbase table data and saves them in hbase |
| |
| ### Story |
| * [SPOI-2455] - HDFS storage layer for 1.0.2 |
| * [SPOI-2565] - Front End Server - Define cache policy for push data in Node.js |
| * [SPOI-2571] - Parent jira for Hadoop Summit work |
| * [SPOI-2590] - Parent: GA Testing |
| * [SPOI-2611] - Parent jira for facilitating trouble shooting and debugging in 1.0.2 |
| * [SPOI-2615] - Parent jira for security work in 1.0.2 |
| * [SPOI-2662] - Parent jira for licensing security for 1.0.2 |
| * [SPOI-2664] - HDFS Storage, Ingestion, Access application data and file storage for 1.0.2 |
| * [SPOI-2667] - Automate distro certification and make it part of CI for 1.0.2 |
| * [SPOI-2781] - Determine which fields are present in output that are not present in input |
| |
| ### Task |
| * [SPOI-2341] - Update on authentication and authorization in Yarn-open source |
| * [SPOI-2434] - Short document on deduper checkpointing scheme |
| * [SPOI-2440] - Kafka support in POC for 1.0.2 |
| * [SPOI-2509] - Investigate how to use Kafka to replace the pub/sub mechanism in Gateway |
| * [SPOI-2532] - Join Operator |
| * [SPOI-2534] - Add support of getting container info of dead applications and dead containers |
| * [SPOI-2535] - Add support for retrieving Aggregated Counters from Response processStats of StatsListener |
| * [SPOI-2542] - Get user to look at instructions before .bin file download |
| * [SPOI-2544] - Twitter demo "feedMultiplier" should be RW and test to ensure that it can be changed in runtime |
| * [SPOI-2548] - Memory gauge always shows 100% |
| * [SPOI-2549] - Total memory on right corner needs to be discussed |
| * [SPOI-2552] - Need to list certified Distros on site |
| * [SPOI-2558] - Help set up Kafka operations |
| * [SPOI-2568] - Making Kafka Producer Exactly Once |
| * [SPOI-2569] - Add the Rewind Feature |
| * [SPOI-2573] - Integrate the Scoring of the quality logs into the DAG |
| * [SPOI-2574] - Add counter calculation in hdfs operator |
| * [SPOI-2584] - DT counters as Key,Val (String,Number) |
| * [SPOI-2585] - Troubleshooting work for reporting min, max, ave, ... on resources |
| * [SPOI-2591] - GA Testing: CDH5 end-to-end |
| * [SPOI-2592] - GA Testing: HDP2 end-to-end |
| * [SPOI-2593] - GA Testing: MapR end-to-end |
| * [SPOI-2594] - GA Testing: UI end-to-end |
| * [SPOI-2595] - GA Testing: dtcli / gateway end-to-end |
| * [SPOI-2596] - GA Testing: Apache Hadoop end-to-end |
| * [SPOI-2597] - GA Testing: High Availability / Recovery |
| * [SPOI-2598] - GA Testing: Application configuration and launch |
| * [SPOI-2599] - GA Testing: Sandbox functionality |
| * [SPOI-2600] - GA Testing: Sandbox UX |
| * [SPOI-2621] - Top 10 support |
| * [SPOI-2622] - Partitioning of DimensionStore operator |
| * [SPOI-2624] - Front End Server - Architecture |
| * [SPOI-2626] - Setting up of number of partitions for Kafka Producer |
| * [SPOI-2636] - Twitter demo app with hashtag top 10 |
| * [SPOI-2640] - Setup UI server for HDP grid for Hadoop Summit |
| * [SPOI-2641] - Setup Ambari for Hadoop Summit Demo |
| * [SPOI-2642] - Run Twitter demo on HDP cluster for Hadoop Summit |
| * [SPOI-2643] - Setup Mobile demo for HDP demo at Hadoop Summit |
| * [SPOI-2644] - Set up machine data demo on HDP cluster for Hadoop Summit |
| * [SPOI-2646] - Setup Ads demo on HDP cluster for Hadoop Summit |
| * [SPOI-2647] - Add the new Twitter HashTag top 10 demo to Frontend server for Summit |
| * [SPOI-2650] - Change default license memory settings |
| * [SPOI-2651] - Milestone 1 update |
| * [SPOI-2670] - Launch twitter hashtags on CDH and HDP clusters |
| * [SPOI-2671] - Update CDH cluster DT UI |
| * [SPOI-2672] - Move the HDFSOutputOperator to Malhar Library |
| * [SPOI-2676] - Get events, tuple records from kafka |
| * [SPOI-2680] - Make more managable kafka cluster |
| * [SPOI-2682] - Customer Demos Support |
| * [SPOI-2685] - Automated distro certification - virtualization review |
| * [SPOI-2686] - Automated distro certification - provisioning review |
| * [SPOI-2687] - Certify DT on Pivotal HD |
| * [SPOI-2688] - Make twitter HashTags links in demo |
| * [SPOI-2690] - Upgrade security sections in guides |
| * [SPOI-2700] - Test DimensionStore operator checkpointing and recovery |
| * [SPOI-2714] - Document stram event types |
| * [SPOI-2732] - Create a multi level key map with multiple keys |
| * [SPOI-2733] - Create a CSV lookup class to load CSV and lookup values based on row keys |
| * [SPOI-2735] - Ensure all dimemsions are implemented and covered by test |
| * [SPOI-2763] - Create a csv parser framework that can handle the different csv file formats |
| * [SPOI-2764] - Create a template application |
| * [SPOI-2765] - Create the template configuration file |
| * [SPOI-2766] - Create kafka input for the application |
| * [SPOI-2767] - Add a directory scan operator that progressively scans for new folders and files for input data to the application |
| * [SPOI-2768] - Add a kafka upload script to upload data to the application |
| * [SPOI-2769] - Create an upload script that uploads application files to hdfs |
| * [SPOI-2773] - Add a property to clear the aggregates in the AggregationsOperator |
| * [SPOI-2785] - Include the input event string in the CDR event |
| * [SPOI-2789] - Add query support for getting events for alerts |
| * [SPOI-2790] - Scale the input events into millions |
| * [SPOI-2792] - Send top 3 gateway list in alert event details |
| * [SPOI-2793] - Add the capability to finalize a bucket |
| * [SPOI-2794] - Add ability to clear out old results when replaying data |
| * [SPOI-2797] - Add query support to send the latest five minute buckets and aggregates |
| * [SPOI-2800] - Install a kafka web-console in our cluster |
| * [SPOI-2805] - Investigate yarn log aggregation to see how this fits into our container logs API |
| |
| ### Sub-task |
| * [SPOI-2725] - Create skeleton App |
| * [SPOI-2728] - Convert the Skeleton Operators into Real Operators - Part I |
| * [SPOI-2739] - File input operator |
| * [SPOI-2740] - DAG level unit test |
| * [SPOI-2776] - Test and Certify RPM packaging for Cloudera |
| * [SPOI-2777] - Gateway throws a bunch of exceptions when trying to install using rpm |
| |
| |
| ### Bug |
| * [MLHR-724] - RabbitMQ test timing issue |
| * [MLHR-1128] - Breadcrumbs should say physical operator, not operator on port page |
| * [MLHR-1129] - DAG should not display underneath controls |
| * [MLHR-1137] - Memory gauge always shows 100% |
| * [MLHR-1142] - Front End Server - Kafka Reconnect |
| * [MLHR-1143] - KafkaOutputOperator is not configurable for Kafka Producer properties |
| * [MLHR-1144] - Container Actions - use wording that's not confusing |
| * [MLHR-1157] - Parent JIRA for Front End Server |
| * [MLHR-1159] - Stop Recording function does not give visual indication of stopped recording unless refreshed |
| * [MLHR-1169] - Update readme for console repo |
| * [MLHR-1170] - Appmaster Logs are not shown in the Console |
| * [MLHR-1185] - Stram Event widget should check physical plan for existence of operator |
| * [MLHR-1186] - some numbers in logical dag hidden behind control |
| * [MLHR-1189] - AbstractHdfsTupleFileOutputOperator output port is not optional |
| * [MLHR-1192] - Parametrize the demos |
| * [MLHR-1195] - App master container is not showing proper metrics |
| * [MLHR-1202] - failureCount missing for physical operators |
| |
| ### Improvement |
| * [MLHR-1132] - Improvements to errors in the stram events widget |
| * [MLHR-1146] - Only get active containers for initial container list |
| * [MLHR-1167] - UI support for searching log levels of classes |
| * [MLHR-1168] - Validate input for log level widget |
| * [MLHR-1194] - Align app counts with labels in cluster overview widget |
| * [MLHR-1209] - Support JMS providers other than ActiveMQ |
| |
| ### New Feature |
| * [MLHR-921] - Installer - List of Errors/Codes |
| * [MLHR-1111] - Container Log Widget, using new REST call |
| * [MLHR-1121] - Front End Server - Kafka Query JSON |
| * [MLHR-1122] - Front End Server - Kafka Keep Alive |
| * [MLHR-1123] - Front End Server - Kafka Parameterized Queries |
| * [MLHR-1124] - Front End Server - Kafka Response Debug/Format |
| * [MLHR-1125] - Front End Server - Kafka Pub/Sub Protocol for Widgets |
| * [MLHR-1130] - Widget to change the log levels of classes dynamically |
| * [MLHR-1133] - Put "float" control in widget config popup |
| * [MLHR-1134] - Front End Server - Editable JSON request |
| * [MLHR-1135] - Front End Server - Setup Instructions |
| * [MLHR-1136] - Front End Server - Packaging/Distribution |
| * [MLHR-1147] - Front End Server - Selecting Metrics |
| * [MLHR-1149] - Front End Server - Use Latest Kafka Offset |
| * [MLHR-1151] - Front End Server - Data Validation |
| * [MLHR-1152] - Front End Server - Page Performance (Visibility API) |
| * [MLHR-1175] - Widget Library - Select Widget |
| * [MLHR-1179] - AngularJS Migration - Application Instance Page |
| * [MLHR-1181] - AngularJS Migration - Application Overview Widget |
| * [MLHR-1182] - Angular JS Migration - App Structure |
| * [MLHR-1187] - UI for adding system alerts |
| * [MLHR-1193] - Create Basic Counters and its aggregator in library |
| |
| ### Story |
| * [MLHR-1178] - AngularJS Migration for 1.0.2 |
| |
| ### Task |
| * [MLHR-960] - Create a Tableau output adapter |
| * [MLHR-1090] - Create first resource objects (models) for angular console |
| * [MLHR-1106] - Improve ended app page |
| * [MLHR-1139] - Add HTTP Get Operator |
| * [MLHR-1140] - Add HDFS output operator that writes to file names specified in tuple |
| * [MLHR-1160] - Create HDFS Output Operator to write with exactly once semantics |
| * [MLHR-1162] - Implement app list with angular |
| * [MLHR-1164] - Wireframe for UI DAG builder |
| * [MLHR-1171] - Make twitter hashTab a link in twitter demo |
| * [MLHR-1172] - Create a Cassandra read operator |
| * [MLHR-1173] - Create a Cassandra write operator |
| |
| ### Sub-task |
| * [MLHR-1163] - Wireframe for DAG builder |
| |
| |
| |
| |
| Version 1.0.1 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-1404] - Create seperate environment for node0 for demos |
| * [SPOI-1952] - Operation and Install guide review |
| * [SPOI-2203] - Configure adsDimesions and machine data to use minimum memory |
| * [SPOI-2340] - License Agent is persisting state in a wrong location |
| * [SPOI-2397] - Monitoring and fixing the bugs in Customer demos |
| * [SPOI-2408] - Verify DT Platform on HDP 2.1 |
| * [SPOI-2410] - Master script to restart hadoop on the cluster |
| * [SPOI-2411] - Not able to starte license with 0.9.5 release |
| * [SPOI-2420] - Core doesn't compile with hadoop 2.4 |
| * [SPOI-2448] - Test xml ingestion demo on a cluster |
| * [SPOI-2449] - Test golden gate demo on a cluster |
| * [SPOI-2450] - Test twitter sentiment demo on a cluster |
| * [SPOI-2474] - Containers not provisioned diagnostics error |
| * [SPOI-2477] - LicenceAgent client fails with NPE |
| * [SPOI-2489] - Operator downstream of EXACTLY_ONCE operator not checkpointing |
| * [SPOI-2490] - HDFS |
| * [SPOI-2491] - Testing the HDFS drain for Flume Sink |
| * [SPOI-2493] - Add HDFS operator updates into Malhar |
| * [SPOI-2496] - viewdag does not work any more |
| * [SPOI-2503] - Provide a rewind feature to replay tuples from a previous time |
| * [SPOI-2505] - Sandbox demos need customization |
| * [SPOI-2512] - Apps fail to start due to serialization errors on CDH5 |
| * [SPOI-2533] - Fraud app needs debugging |
| * [SPOI-2543] - Applications not reporting the license information to License Agent if LA is restarted |
| * [SPOI-2550] - License Agent Dying |
| * [SPOI-2554] - Third party libraries which are required for demos are missing from /opt/datatorrent/releases/version/lib/ |
| * [SPOI-2556] - Add demo guides to documentation index |
| |
| |
| ### Improvement |
| * [SPOI-2210] - Default license generation should be part of build process |
| * [SPOI-2468] - README for certifications and benchmarks |
| * [SPOI-2482] - DTCli should handle ^C more gracefully |
| * [SPOI-2494] - Add HTTP Get operator to Malhar |
| * [SPOI-2555] - Include README, LICENSE, and CHANGELOG in docs |
| |
| ### New Feature |
| * [SPOI-406] - Document and test download to work with Hortonworks HDP 2.0 |
| * [SPOI-408] - Certify with MapR (Hadoop 2.3) release in 1.0.1 |
| * [SPOI-1146] - API for retrieving raw Container Logs |
| * [SPOI-1834] - Pub-Sub/REST mechanisms for exceptions thrown by an application |
| * [SPOI-2080] - Application to certify installation |
| * [SPOI-2253] - Username/password feature in the installer |
| * [SPOI-2381] - dt-site.xml logistics |
| * [SPOI-2387] - Aggregate Logical Operators on Gateway |
| * [SPOI-2402] - Update dt-site.xml with container memory less than 512MB after getting the app-memory-automation results for all the apps. |
| * [SPOI-2460] - dt-site.xml new logistics implementation |
| * [SPOI-2470] - Create a certification property file for Distro/Install Certification |
| * [SPOI-2486] - Workaround for "FileSyste.getScheme is not overrided by MapRFileSystem" |
| * [SPOI-2519] - add grep feature when getting container log |
| * [SPOI-2526] - Add REST call to return info of historical containers |
| |
| ### Task |
| * [SPOI-62] - Buffer Server logging |
| * [SPOI-1206] - Review Licensing of Third Party Libraries |
| * [SPOI-1745] - Document Demo Application Setup |
| * [SPOI-1784] - License agent app to have webservice |
| * [SPOI-2199] - Readme file for AWS installation |
| * [SPOI-2223] - Test if all the applications with reduced container memory sizes run in sandbox 0.9.4. |
| * [SPOI-2348] - Enable checkpoint for aggregator operator |
| * [SPOI-2354] - Certify demos on 0.9.4 sandbox |
| * [SPOI-2370] - Fully automate Hadoop cluster restart |
| * [SPOI-2379] - Certify DT with Hadoop 2.3/CDH |
| * [SPOI-2392] - Add more basic demos to sandbox launcher |
| * [SPOI-2393] - Certify Amazon EMR Hadoop 2 version |
| * [SPOI-2414] - Stop the nightly benchmarking jenkins jobs |
| * [SPOI-2415] - remove the "page has not loaded since 60 second" message at top |
| * [SPOI-2422] - Add HDFS cache limit on FlumeAgent operator |
| * [SPOI-2424] - Log metrics across all poc operators to a file |
| * [SPOI-2425] - Compare HDFS with customer site |
| * [SPOI-2426] - Benchmark HDFS drain rate by agent operator |
| * [SPOI-2427] - Enable dedupper to be run with checkpointing turned OFF |
| * [SPOI-2428] - Evaluate bigger bucket size for checkpointing for deduper |
| * [SPOI-2430] - Number validation for poc |
| * [SPOI-2432] - Compute the checkpoint I/O for dedupper |
| * [SPOI-2439] - Compute deduper HDFS I/O for current checkpointing scheme |
| * [SPOI-2441] - Estimate the feed time for HDFS agent replay |
| * [SPOI-2445] - Certify Cloudera CDH 5.0 GA |
| * [SPOI-2451] - Gateway Load Test |
| * [SPOI-2457] - Widget Library |
| * [SPOI-2458] - Generate certification report for HDP 2.1 |
| * [SPOI-2466] - Monitoring and notifications for cluster |
| * [SPOI-2475] - Document process to certify demo re-start |
| * [SPOI-2476] - Document process for grid restart |
| * [SPOI-2483] - Node.js Kafka Client POC |
| * [SPOI-2513] - Create demos ui service wrapper compatible with CentOS |
| * [SPOI-2517] - Configure customer demos ui for as a service |
| * [SPOI-2531] - HDFSOutputOperator write with exactly once semantics |
| * [SPOI-2535] - Add support for retrieving Aggregated Counters from Response processStats of StatsListener |
| * [SPOI-2537] - Need to list previous versions on datatorrent site |
| * [SPOI-2540] - Fix URL for AWS EMR document |
| * [SPOI-2551] - Define a process on backend to ensure archive of previous downloads works for customers |
| * [SPOI-2553] - Holding ticket for Malhar maintenance work |
| |
| |
| ### Sub-task |
| * [MLHR-774] - Terms of service license |
| * [MLHR-976] - Implement table module for angular |
| |
| |
| ### Bug |
| * [MLHR-998] - Compilation error while using UniqueValueCount operator. |
| * [MLHR-1054] - Update topic and rest calls for Logical/Physical Split |
| * [MLHR-1056] - Rename demo apps to include Demo suffix |
| * [MLHR-1089] - Error handling for license page |
| * [MLHR-1093] - AbstractKafkaInputOperator is not committing the txn to Kafka in commit call back |
| * [MLHR-1096] - Get last N events when tailing stram events |
| * [MLHR-1108] - Logical DAG resizing |
| * [MLHR-1113] - AdsDimension demo fails to launch |
| * [MLHR-1114] - FraudDetect demo fails to launch |
| * [MLHR-1115] - MachineData demo fails to launch |
| * [MLHR-1127] - HdfsTextFileInputOperator Fails during checkpointing |
| * [MLHR-1138] - Remove unused CPU columns in logical operator table |
| |
| |
| ### Improvement |
| * [MLHR-1092] - Add environment variable override to rename demo apps |
| * [MLHR-1098] - Improved code coverage for malhar-angular-dashboard |
| * [MLHR-1099] - Extend dataModelOptions with non-serializable defaults |
| * [MLHR-1100] - Improved widget directive in malhar-angular-dashboard |
| * [MLHR-1116] - Use negative offset to get last N events for stram events widget |
| * [MLHR-1119] - Add a confirm box when killing app master container |
| * [MLHR-1120] - Save range selection between page loads for stram events |
| |
| ### New Feature |
| * [MLHR-783] - DAG should adjust in height as the widget height changes |
| * [MLHR-784] - WidgetOutputOperator - Schema Update |
| * [MLHR-984] - STRAM decisions widget |
| * [MLHR-1070] - Web Demos - Upgrade to AngularJS 1.2.16 |
| * [MLHR-1076] - Widget Library - WebSocket |
| * [MLHR-1077] - Widget Library - Data Models |
| * [MLHR-1087] - Dashboard Component - AngularJS Scope Documentation |
| * [MLHR-1088] - Dashboard Web App - Dependencies Documentation |
| * [MLHR-1095] - Node.js Kafka Integration |
| * [MLHR-1097] - Implement storage on malhar-angular-dashboard |
| * [MLHR-1118] - Add storage hash for clearing out invalid dashboards |
| |
| ### Story |
| * [MLHR-719] - Time Series Forecasting |
| * [MLHR-1060] - Widget Library |
| * [MLHR-1061] - Dashboard Component |
| * [MLHR-1082] - Parent jira for Real Time ETL Application |
| |
| ### Task |
| * [MLHR-700] - Develop operator for calculating Coefficient of Determination (RSquare) |
| * [MLHR-841] - Demo guide for machine data application |
| * [MLHR-850] - Demo guide for Ads Dimension demo |
| * [MLHR-852] - Demo guide for Twitter (rolling topwards) demo application |
| * [MLHR-874] - Demo guide for charting demo - Yahoo finance |
| * [MLHR-896] - widget output operator - widget type should not be decided in backend |
| * [MLHR-950] - Create a demo for distributed distinct |
| * [MLHR-953] - Set up "upgrade path" for CustomerApplications |
| * [MLHR-983] - UI to expose the "events" API of the gateway |
| * [MLHR-1042] - ETL: Consolidate output operators properties |
| * [MLHR-1063] - Create contributing section on the main page of malhar repo |
| * [MLHR-1064] - Create contributing secion on main page of UI repos |
| * [MLHR-1065] - create line chart operator |
| * [MLHR-1075] - add .travis.yml file to ui-console |
| * [MLHR-1078] - Update license header script, keep year updated |
| * [MLHR-1080] - create real time chart output operator |
| * [MLHR-1083] - Consolidate realtime output operator properties |
| * [MLHR-1084] - Consolidate aggregate operator properties |
| * [MLHR-1085] - Consolidate linechart operator properties |
| * [MLHR-1104] - Design App Bundle and Upload/Launch Feature |
| |
| |
| |
| |
| |
| Version 1.0.0 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-2086] - creation of logs directory under .dt folder of user |
| * [SPOI-2497] - Remove hard-coded "hdfs" scheme |
| * [SPOI-2501] - dt.log not written under CDH5 GA |
| * [SPOI-2577] - After installation, I cannot launch license agent |
| * [SPOI-2604] - The example DFS directory should be changed from /home/... to /user/... |
| * [SPOI-2607] - Install page should only list error (not warnings) issues |
| * [SPOI-2609] - When yarn.scheduler.minimum-allocation-mb is set to 256, PiDemo fails with NPE |
| |
| ### Improvement |
| * [SPOI-2522] - Replace InputPort.getStreamCodec method with an attribute on InputPort called StreamCodec |
| * [SPOI-2576] - Change local install from .datatorrent to datatorrent directory |
| * [SPOI-2603] - installer complains about gateway has trouble starting but it actually runs OK |
| * [SPOI-2637] - Improve the Getting Started Guide |
| |
| ### Task |
| * [SPOI-2520] - Change the name of the CustomStats to Counters |
| * [SPOI-2578] - Check dir before mkdir call (MapR requirement) |
| * [SPOI-2579] - Take out launch from gateway webservice spec document |
| * [SPOI-2649] - Update software license |
| |
| ### Bug |
| * [MLHR-1101] - Install errors formatting |
| * [MLHR-1103] - Change installer instructions for DFS directory path |
| |
| |
| Version 0.9.5.1 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-2554] - Third party libraries which are required for demos are missing from /opt/datatorrent/releases/version/lib/ |
| * [MLHR-1113] - AdsDimension demo fails to launch |
| * [MLHR-1114] - FraudDetect demo fails to launch |
| * [MLHR-1115] - MachineData demo fails to launch |
| |
| |
| Version 0.9.5 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Sub-task |
| * [SPOI-1692] - Certify fault tolerance for multiple containers failure |
| * [SPOI-1693] - Develop automated test scenario for StrAM failure to certify one of high availability aspects. |
| * [SPOI-2008] - Setup virtual cluster for high availability certification |
| * [SPOI-2009] - WordCount app to be used in certification of high availability |
| * [SPOI-2127] - Error/Warning message not clear - nightly build installer (datatorrent-dist-0.9.4-SNAPSHOT.bin) if executed as non-root user. |
| * [SPOI-2128] - Nightly build installer needs to check if port 9090 is available before attempting to run dtgateway service. |
| * [SPOI-2129] - Error/Warning message not clear while updating the Hadoop configuration through Installer-Web-based UI |
| * [SPOI-2247] - Gateway user authentication REST API spec |
| * [SPOI-2248] - Implement user authentication and basic authorization in gateway |
| * [SPOI-2255] - Password protect the web socket |
| * [SPOI-2257] - Support HTTPS in gateway |
| * [SPOI-2277] - Support HTTPS and auth in pubsub connection from stram to Gateway |
| * [SPOI-2291] - Copy GW configurations to HDFS on every write |
| |
| ### Technical task |
| * [SPOI-2005] - Verify and add Machine demo to app memory usage automation |
| |
| ### Bug |
| * [SPOI-1801] - Mobile app has two logical input gen - Need to handle containers on servers with two diff clocks |
| * [SPOI-1890] - Application behavior when resources are not available |
| * [SPOI-1954] - Incorrect Processed Stats |
| * [SPOI-1995] - Sandbox instructions are outdated |
| * [SPOI-1998] - Killing of node manager on node running License AM |
| * [SPOI-2067] - Move apps from contrib to demos |
| * [SPOI-2152] - Stabillity of mobile demo |
| * [SPOI-2194] - Gateway start script prints out repetitive message |
| * [SPOI-2205] - Make generate license tool work with web tool to generate 1TB license in the new flow |
| * [SPOI-2207] - Deduper showed high latency and crashed in staging environment. |
| * [SPOI-2221] - Analyze FileSystem.get calls and if necessary replace them with FileSystem.newInstance |
| * [SPOI-2278] - Relative hdfs path should be created under application directory |
| * [SPOI-2283] - Do not make assumption about FSStorageAgent being the StorageAgent or the default configuration for it being the configuration the user set |
| * [SPOI-2284] - User 'dtadmin' should be added to the group having access to hadoop services and hdfs |
| * [SPOI-2285] - uninstall.sh does not uninstall the platform |
| * [SPOI-2288] - NullPointerException in buffer server |
| * [SPOI-2289] - FSStorageAgent store throws java.nio.channels.ClosedChannelException |
| * [SPOI-2295] - Monitoring and fixing the bugs in Customer demos |
| * [SPOI-2297] - Mobile Demo going to unstable state due to out of order tuple sequence error |
| * [SPOI-2298] - Events returned from REST API has funky keys |
| * [SPOI-2302] - Normalize WebSocket & REST format for Stram Events |
| * [SPOI-2314] - Fix application names |
| * [SPOI-2315] - node1 application launch-xxx macros not working after upgrade |
| * [SPOI-2342] - DTCLI ignoring annotated Application Names |
| * [SPOI-2344] - dtcli fails to load dt-env.sh |
| * [SPOI-2349] - Update default license to 4 months |
| * [SPOI-2357] - Unable to launch application using launch alias, if the application names shown in the dtcli list are picked from dt-site.xml. |
| * [SPOI-2358] - Installer bin extrac does not preserve permissions |
| * [SPOI-2359] - Gateway fails to restart due to missing JAVA_HOME |
| * [SPOI-2361] - After global install local .dt directory may be missing when executing dtcli |
| * [SPOI-2364] - Catastrophic Error- tuples out of sequence in Generic Node |
| * [SPOI-2365] - Hadoop Shell throws exception for dt.attr.APPLICATION_PATH |
| * [SPOI-2371] - DTCLI updating the dt-site.xml |
| * [SPOI-2373] - Sandbox dt-site.xml is missing configurations related to Map reduce applications. |
| * [SPOI-2374] - The apps in sandbox use 1GB as the container size and not 512MB as specified under misc/sandbox/conf/dt/dt-site.xml |
| * [SPOI-2375] - configuration from dt-site.xml in user's directory is not picked by dtcli |
| * [SPOI-2376] - Gateway classpath references invalid directory |
| * [SPOI-2380] - get-physical-operator-properties wrong help text |
| * [SPOI-2389] - DataTorrent discovers override dt-site.xml in CLASSPATH |
| * [SPOI-2390] - DTGateway is killed after terminal is closed |
| * [SPOI-2391] - dtdemos service does not stop on sandbox |
| * [SPOI-2395] - Missing FraudDetection/AdsDeminsions/MachineData demos from sandbox. |
| * [SPOI-2396] - gateway not started after stopping and starting hadoop/datatorrent services in sandbox. |
| * [SPOI-2398] - Extra directory created by gateway in user home |
| * [SPOI-2400] - An operator exits normally when there is OutOfMemory error |
| * [SPOI-2401] - The AppMaster finishes after retrying 6 times to deploy input operator |
| * [SPOI-2412] - App restart HDFS file permission issue on CDH cluster |
| * [SPOI-2417] - DTGateway dt-site.xml backup location |
| * [SPOI-2419] - Ensure principal name propagates to containers in non-secure mode |
| * [SPOI-2435] - Latest changes to latency calculation break tests. |
| |
| |
| ### Improvement |
| * [SPOI-1376] - Have app specific configuration to stram-site.xml in sandbox |
| * [SPOI-1855] - Ability to continue/shutdown application |
| * [SPOI-1986] - In Logical View do not include the unifier stats |
| * [SPOI-2001] - Update Release Process document (GDrive) |
| * [SPOI-2222] - Constraint format in license file to be in JSON |
| * [SPOI-2287] - Allow specifying a complete list of dependencies while deploying the application |
| * [SPOI-2335] - Make BaseOperator Java Serializable |
| * [SPOI-2367] - Create service wrapper for demos server |
| * [SPOI-2385] - Improve the performance of HDFSStorage for POC |
| * [SPOI-2404] - Reset the Application Failure Count upon successful recovery |
| * [SPOI-2405] - Add more params to /ws/v1/applications/{appid}/events call |
| * [SPOI-2407] - Need ability to supply the class name representing the property |
| |
| ### New Feature |
| * [SPOI-91] - Design and implement a socket adapter operator |
| * [SPOI-2007] - Automation to certify High Availability |
| * [SPOI-2012] - Run the app-memory-usage with lower container sizes starting from 128M to 512M when the grid is restarted. |
| * [SPOI-2249] - Simple user authentication / authorization support in the UI |
| * [SPOI-2355] - Add REST call to make existing license file the current license |
| * [SPOI-2386] - REST call to retrieve the datatorrent conf directory |
| * [SPOI-2399] - Starter Web Application |
| |
| ### Story |
| * [SPOI-2215] - Password protect the dashboard |
| |
| ### Task |
| * [SPOI-746] - DT Phome home customer side work |
| * [SPOI-1642] - Certify support for node failure |
| * [SPOI-2045] - Discuss license cleanup/improvements |
| * [SPOI-2099] - Show mandatory fields with '*' in the installer |
| * [SPOI-2101] - Say "Terms of Service" on the text |
| * [SPOI-2103] - Show error message on the side |
| * [SPOI-2117] - An error during uninstall |
| * [SPOI-2118] - 1 yr license mail not getting mailed |
| * [SPOI-2145] - Set up separate access to jars for big customers |
| * [SPOI-2178] - Filters HashMap from HDFSLoader |
| * [SPOI-2190] - Create web page (navigator and table view) for 5 dimensions and 2 computations |
| * [SPOI-2201] - Create a upgrade license page |
| * [SPOI-2202] - Upgrade Machine Data to latest code |
| * [SPOI-2211] - Getting too many deprecated warnings in demos |
| * [SPOI-2213] - Be able to diagnose Stram decisions |
| * [SPOI-2214] - Ability to access Stram decisions for failed/finished/killed app |
| * [SPOI-2216] - Upgrade Twitter Demo to latest Code |
| * [SPOI-2217] - Upgrade Fraud Detection Demo to latest Code |
| * [SPOI-2218] - Upgrade Ads Dimension Demo to latest Code |
| * [SPOI-2219] - Upgrade Mobile Locator Demo to latest Code |
| * [SPOI-2224] - Fraud demo issues |
| * [SPOI-2225] - Issues with Machine Data Demo |
| * [SPOI-2226] - Issues with Ads demo |
| * [SPOI-2233] - Remove max num container parameter that is included by default |
| * [SPOI-2234] - Give warning when operators are automatically inlined |
| * [SPOI-2237] - CPU column on logical view |
| * [SPOI-2243] - Use hammerdb to generate load against Oracle |
| * [SPOI-2256] - Document certification suite used to certify datatorrent platform |
| * [SPOI-2286] - Implement Goldengate java handler to capture db change and send to kafka |
| * [SPOI-2290] - Hide configuration and launch app for GA |
| * [SPOI-2292] - Add to README: not recommended to launch more than one gateway |
| * [SPOI-2293] - Manage license file on HDFS |
| * [SPOI-2303] - Make DFS error keys more fine grained |
| * [SPOI-2309] - Use connection to RM to guess the cluster accessible IP address for GATEWAY_CONNECT_ADDRESS |
| * [SPOI-2310] - Allow different filesystem from fs.defaultFS for dt.dfsRootDirectory |
| * [SPOI-2338] - The "#" in the boxes should not change |
| * [SPOI-2351] - Bundle the default passwd file (with admin/admin) in the installer |
| * [SPOI-2362] - Create service wrappers for Apache Hadoop on Sandbox |
| * [SPOI-2363] - Update dtdemo sandbox script to work with Hadoop/DTGateway service wrappers |
| * [SPOI-2394] - Install wizard shows up again after gateway restart |
| * [SPOI-2416] - Separate malhar and malhar-ui-console |
| * [SPOI-2418] - Create an ability for the DTFlumeSink to backoff if it suspects that the DAG is not processing the data in healthy fashion. |
| * [SPOI-2421] - Update end-user documentation |
| |
| |
| ### Sub-task |
| * [MLHR-726] - Time Series Forecasting Operator using Holt's Linear Trend Model |
| * [MLHR-933] - Time Series Forecasting using Holt-Winters' Seasonal Method |
| * [MLHR-965] - create non transactional output operator |
| * [MLHR-966] - create transactional output operator |
| * [MLHR-967] - create a data store writer implementation to use in real time etl app |
| * [MLHR-1036] - Create incremental model for Holt's Linear Trend Forecasting Algorithm |
| * [MLHR-1040] - Develop incremental model creation for Holt-Winters' Multiplicative Method Time-Series Forecasting |
| |
| |
| ### Bug |
| * [MLHR-778] - Logical Operators list has no palette |
| * [MLHR-857] - Machinedata demo stop working with new Redis operator |
| * [MLHR-963] - create output operator to add historical data to supplied datastore |
| * [MLHR-964] - create interface for data stores that can be used by output operator to persist historical data |
| * [MLHR-968] - Remove test related code from AbstractTransactionableStoreOutputOperator |
| * [MLHR-969] - Create a manageable version of our demo docs to be included with our installation |
| * [MLHR-978] - create resource leak for the hdfs input/output operators |
| * [MLHR-979] - AdsDimension: Redis operator shows high latency |
| * [MLHR-986] - Page Not Found errors from the web-site |
| * [MLHR-1000] - Invalid IP address selected by default during upgrade |
| * [MLHR-1005] - CPU column on logical view |
| * [MLHR-1008] - twitter application failed with exception |
| * [MLHR-1015] - State is missing from physical operator overview widget |
| * [MLHR-1019] - Make gateway ip and port inputs on same line in installer |
| * [MLHR-1020] - License screen does not reflect new license upload |
| * [MLHR-1021] - Unifiers should not have link to nonexistent logical operator |
| * [MLHR-1022] - Update installer license text to reflect correct trial period |
| * [MLHR-1028] - Cannot kill application in state ACCEPTED |
| * [MLHR-1030] - Sorting on logical operators list fails until you sort on name |
| * [MLHR-1031] - UI Showing wrong total Memory usage |
| * [MLHR-1049] - Streams not showing up in dag overview |
| * [MLHR-1050] - Remove the "Development" mode |
| * [MLHR-1055] - Unnecessary gateway restart requested during installation navigation |
| * [MLHR-1057] - Overview in dag widget not accurate to visible area |
| * [MLHR-1058] - Change install wizard from widget in a page to just a page |
| * [MLHR-1066] - Install Wizard tab order and auto-focus |
| * [MLHR-1067] - Install Wizard invalid DFS directory check |
| * [MLHR-1072] - Socket input operator test fail |
| * [MLHR-1074] - Remove stram events page/widget from 0.9.5 |
| |
| |
| ### Improvement |
| * [MLHR-721] - Migrate database cache lookup to the new db framework |
| * [MLHR-985] - AdsDimension: Plug-in Dimension Unifier |
| * [MLHR-1010] - Remove info icon from installer text |
| * [MLHR-1011] - Change "property" to "item" in install step tables |
| * [MLHR-1012] - Review and edit installer text |
| * [MLHR-1013] - Installer: Rework Hadoop and System screens |
| * [MLHR-1035] - Remove darker grey background from console |
| * [MLHR-1046] - Operators/Containers not being subscribed to on App Page |
| * [MLHR-1062] - Install Wizard examples of Hadoop and DFS path |
| * [MLHR-1069] - License summary section in Install Wizard |
| |
| ### New Feature |
| * [MLHR-684] - Invalidate dashboard state from previous versions |
| * [MLHR-881] - ETL Web App - Packaging |
| * [MLHR-882] - ETL Web App - Dashboard |
| * [MLHR-885] - ETL Web App - Sample Data Generation |
| * [MLHR-997] - ETL Web App - Historical Data from MongoDB |
| * [MLHR-1014] - Remove config properties edit page, replace with config home |
| * [MLHR-1052] - Bar Chart Widget |
| * [MLHR-1053] - Widget Library - Demo Application |
| |
| ### Story |
| * [MLHR-703] - Logstream UI |
| * [MLHR-709] - Widgets Library as Independent Project |
| |
| ### Task |
| * [MLHR-661] - Migrate JDBC adapters to use the new database adapter interface |
| * [MLHR-866] - Annotate all stateless operators in Malhar as such |
| * [MLHR-868] - Demo Guide for Map-Reduce Operator - LogsCountApplication |
| * [MLHR-869] - Demo guide for Map-Reduce Operator - NewWordCountApplication. |
| * [MLHR-870] - Demo guide for Map-Reduce Operator - InvertedIndexApplication. |
| * [MLHR-922] - Machine data demo is using deprecated attributes. Gives warnings on launch |
| * [MLHR-951] - Remove deprecated warnings from demos |
| * [MLHR-954] - Need to debug older demos |
| * [MLHR-961] - Develop a Goldengate input adapter |
| * [MLHR-970] - Re-create the application that we build for iAd Poc in Malhar |
| * [MLHR-1006] - In Logical View do not include the unifier stats |
| * [MLHR-1007] - Simple user authentication/authorization support in UI |
| * [MLHR-1018] - Need a Kafka-HBase app |
| * [MLHR-1023] - Update installer text on welcome screen |
| * [MLHR-1024] - Create issues summary page under configuration |
| * [MLHR-1025] - Add progress status indicator for installer wizard |
| * [MLHR-1026] - Reorder install wizard screens |
| * [MLHR-1027] - Remove external upgrade link from licensing screen |
| * [MLHR-1041] - ETL : Consolidate input operator properties in json format |
| * [MLHR-1044] - ETL: Add input operator to ETL Application in ETL branch |
| * [MLHR-1071] - Create stateless deduper - aka deduper which forgets its state upon failure or repartitioning |
| |
| |
| |
| |
| |
| Version 0.9.4 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-455] - Cleanup maven repository workaround in install.sh |
| * [SPOI-1636] - Update all node1 demo to 0.9.3 release |
| * [SPOI-1774] - Thread local performance drop from 65M to 40 M tuplesProcessed/sec |
| * [SPOI-1775] - MROperator demo applications fail when launched from Sandbox |
| * [SPOI-1847] - Intermittent - WebSocket Publishing ignores endWindow |
| * [SPOI-1864] - POC partitioned operators not getting correct initial state |
| * [SPOI-1866] - Datatorrent applications not starting on CDH5 Vagrant cluster |
| * [SPOI-1867] - Investigate InstallAnywhere use for DataTorrent installation |
| * [SPOI-1882] - When not able to contact license agent, the application should not die |
| * [SPOI-1883] - Stram crashes with Unknown-Container issue |
| * [SPOI-1885] - Determine minimum amount of memory needed to run twitter app |
| * [SPOI-1886] - Determine minimum amount of memory needed to run mobile app |
| * [SPOI-1888] - Install license in the new console configuration |
| * [SPOI-1889] - Licensing needs to support eval and free |
| * [SPOI-1902] - Dynamic MxN partitioning does not handle scale down to single M instance |
| * [SPOI-1903] - MiniClusterTests fails because ~/.dt/dt-site.xml dependency |
| * [SPOI-1906] - history does not get flushed to the history file until the next command prompt |
| * [SPOI-1917] - Licensing error |
| * [SPOI-1922] - Loading license does not work |
| * [SPOI-1923] - Close button does not work in several pop-up windows |
| * [SPOI-1936] - No uninstall script available with the installer. |
| * [SPOI-1953] - Installation video on our website should be refreshed to reflect the latest version(s) |
| * [SPOI-1960] - application incorrectly marked as succeeded |
| * [SPOI-1968] - 404 for logical plan url |
| * [SPOI-1970] - Negative requested container count logged by Stram |
| * [SPOI-1976] - Fix AdsDimensions in certification |
| * [SPOI-1992] - the latency in the application overview freezes after a container gets killed. |
| * [SPOI-1999] - Evaluate Yarn cluster issue |
| * [SPOI-2002] - Gateway fails to load properties |
| * [SPOI-2037] - Redirect to welcome page on first run / install |
| * [SPOI-2039] - DTGateway logs to .dt/logs in service mode |
| * [SPOI-2040] - Apache logs application under contrib folder fails to run |
| * [SPOI-2047] - dtgateway service not starting when machine boots |
| * [SPOI-2049] - Better error message on invalid hadoop directory |
| * [SPOI-2051] - RPM install reports install script failure |
| * [SPOI-2052] - DTGateway restart fails when running with -service |
| * [SPOI-2055] - dtgateway service reports OK on startup failure |
| * [SPOI-2056] - LicensingAgentProtocolHelper.getLicensingAgentProtocol gets stuck when YARN is not running |
| * [SPOI-2057] - DTCLI is not working after intallation |
| * [SPOI-2058] - launch-demos macro not available after installing the platform from self extracting intaller |
| * [SPOI-2059] - Show understandable error message if the root hdfs directory could't be created |
| * [SPOI-2060] - UI Shows Nan B in Allocated Memory |
| * [SPOI-2062] - Gateway needs to check hadoop version |
| * [SPOI-2064] - dtgateway-ctl stop doesn't work |
| * [SPOI-2065] - Readme File is not udpated |
| * [SPOI-2066] - installer not recognizing -l option |
| * [SPOI-2070] - Installer: echo_success command doesn't work with Ubuntu |
| * [SPOI-2073] - Invalid Ip Address in Installer UI |
| * [SPOI-2074] - install script needs to check hadoop version |
| * [SPOI-2075] - lauch-local-demos |
| * [SPOI-2076] - Licensing agent RPC gives NPE |
| * [SPOI-2077] - Installer: we need a separate page for hadoop installation directory |
| * [SPOI-2078] - Change the certification and benchmarking script and code to use the new location of benchmarking apps |
| * [SPOI-2083] - install script, when run by rpm, complains about invalid group dtadmin |
| * [SPOI-2088] - Map Reduce demo applications still show the classnames when listed using launch-demos |
| * [SPOI-2089] - demo applications displayed after running the launch-demos command should be in alphabetical order |
| * [SPOI-2090] - Error while requesting evaluation license from Datatorrent.com |
| * [SPOI-2094] - Installer throws failed message while stopping gateway |
| * [SPOI-2097] - launch-demos macro not available after installing the platform from self extracting intaller |
| * [SPOI-2098] - App names still have full classpath |
| * [SPOI-2120] - Installer - Restart Modal is not closed after Restart Failed (Happened Once) |
| * [SPOI-2132] - Ensure HDFS does not blow up with millions of files per sec |
| * [SPOI-2133] - Delete old files to ensure NN does not crash |
| * [SPOI-2134] - Send POC1 to customer |
| * [SPOI-2143] - Spelling error and reference to $HOME/.datatorrent |
| * [SPOI-2146] - Move Kafka benchmark apps to contrib folder |
| * [SPOI-2148] - Installer - Disable Closing Modals on Click |
| * [SPOI-2149] - Address the confusion around gateway address. |
| * [SPOI-2151] - User is not able to change the defaultFS during installation |
| * [SPOI-2153] - Cryptic error message when launching app on node0 |
| * [SPOI-2157] - Getting logical plan returns error when one of the getters is bad |
| * [SPOI-2159] - gateway is polling resourcemanager for appinfo w/o subscriber |
| * [SPOI-2163] - Change directory before DTGateway launch |
| * [SPOI-2165] - Installer - add reload ability to System screen |
| * [SPOI-2170] - DTGateway classpath is duplicated after restart |
| * [SPOI-2171] - Remove reload button from System configuration screen |
| * [SPOI-2172] - Installer may display invalid port after starting DTGateway |
| * [SPOI-2173] - Installer base location change not working |
| * [SPOI-2192] - CLI command for getting list of operator classes from a jar |
| * [SPOI-2193] - CLI command for getting properties of a operator class from a jar |
| * [SPOI-2206] - dag view does not get rendered property in Firefox |
| * [SPOI-2227] - (Re)start license agent when license file is uploaded |
| * [SPOI-2229] - container local operators not redeployed |
| * [SPOI-2238] - Installer complains about sudo running as root |
| * [SPOI-2241] - DAG Firefox 28 Support |
| |
| |
| ### Improvement |
| * [SPOI-1311] - Review platform documentation |
| * [SPOI-1567] - Certify against commercial Hadoop distributions |
| * [SPOI-1769] - Trying to kill a non dt app can return a better message |
| * [SPOI-1844] - Ship project-template with a log4j.properties file with debugging level set to INFO |
| * [SPOI-1852] - WebSocket client recovery logging |
| * [SPOI-1853] - Create WebSocket clients in containers on demand |
| * [SPOI-1854] - Option to retrieve only running and recently-ended apps |
| * [SPOI-1856] - Need something that is like LicensingAgentClient but not specific for stram. |
| * [SPOI-1857] - Gateway to warn about available licensed memory being low |
| * [SPOI-1858] - CLI to directly connect to license agent to get live license info |
| * [SPOI-1859] - Gateway to directly connect to license agent to get live license info |
| * [SPOI-1861] - Gateway command to restart itself |
| * [SPOI-1869] - Add UI build script to dist build file |
| * [SPOI-1874] - the first operator that stalls for more than specific period, take it out so as to unclog the processing |
| * [SPOI-1899] - Add appmaster container to container list |
| * [SPOI-1904] - Updates needed to the README file |
| * [SPOI-1911] - Run certification as part of nighty build |
| * [SPOI-1915] - Using $DT_HOME in README |
| * [SPOI-1961] - Show the activation date of the license with list-license-agent command |
| * [SPOI-1965] - The file demos.jar should be installed by default by the Installer |
| * [SPOI-1978] - Manual eval request (by e-mail) - template |
| * [SPOI-1980] - DT Version in license request and generated license |
| * [SPOI-1981] - Approve / update license verification e-mail |
| * [SPOI-1997] - Certify against commercial Hadoop distributions |
| * [SPOI-2026] - Add support to LogicalPlan for testing dag serialization |
| * [SPOI-2150] - Update Readme file for the local install |
| * [SPOI-2155] - Installer - Validate Fields on blur event |
| * [SPOI-2156] - Installer - Navigation Code Cleanup |
| * [SPOI-2158] - Installer - CSS Classes |
| * [SPOI-2160] - Installer font size |
| * [SPOI-2166] - Configuration screen navigation panel |
| * [SPOI-2174] - Notify user with installation location and version |
| * [SPOI-2175] - Notify user about local DTGateway management during installation |
| * [SPOI-2188] - Installer - Register Instructions |
| |
| ### New Feature |
| * [SPOI-328] - Add annotation to declare the operator stateless |
| * [SPOI-393] - High Availability for STRAM |
| * [SPOI-868] - Setting operator properties as different types |
| * [SPOI-1182] - Add Key-based filter functionality to malhar library for Min, Max, SimpleMovingAverage, StandardDeviation like operators |
| * [SPOI-1654] - Logstream - aggregate top hits and bytes for URL, geo DMA, IP, URL/status code, url |
| * [SPOI-1747] - create a filter operator to output multiple records based on filter |
| * [SPOI-1756] - configuration for input adaptor |
| * [SPOI-1757] - configuration for filter operator |
| * [SPOI-1758] - configuration for dimension operator |
| * [SPOI-1759] - configuration for aggreagation operator |
| * [SPOI-1760] - configuration for web socket output |
| * [SPOI-1849] - add dt HDFS directory in configuration |
| * [SPOI-1907] - Installer: HDFS directory creation attempt via Gateway (as part of config updates) |
| * [SPOI-1909] - Port re-selection by Gateway if 9090, 9091, 9092, etc are taken |
| * [SPOI-1913] - Automate verifing the app memory for the demos |
| * [SPOI-1928] - gateway needs to be able to start with standalone hadoop jar (without hadoop installation) |
| * [SPOI-1929] - /ws/v1/about to include java version and hadoop location |
| * [SPOI-1930] - New installation script |
| * [SPOI-1966] - dtcli should be enabled to list app names (if available) as opposed to app class path |
| * [SPOI-1974] - Add throughput, totalTuplesProcessed and elapsed time to performance benchmarking |
| * [SPOI-1975] - Display throughput, tuplesProcessed per sec and latency in a tabular format. |
| * [SPOI-1987] - Copy License to Front-End Distribution |
| * [SPOI-2003] - Verify and add all the demos except Machine data to app memory usage automation |
| * [SPOI-2011] - Make a separate jar file for performance benchmarking demos |
| * [SPOI-2013] - Support for doc link as an attribute for the application |
| * [SPOI-2018] - Have a launch-performance command in dtcli |
| * [SPOI-2021] - Rename all the apps under contrib to have meaningful names |
| * [SPOI-2023] - Make a launch-contrib command available in stram cli |
| * [SPOI-2027] - Packaging benchmarking demos |
| * [SPOI-2042] - redirect user to welcome screen if dt.configStatus is not "complete" |
| * [SPOI-2044] - set property dt.configStatus to "complete" when the user has completed the config wizard |
| * [SPOI-2122] - Installer - Offline Email Template |
| * [SPOI-2147] - Provide separate dt benchmarking package scripts to throughput and hdfs operators benchmarking |
| * [SPOI-2195] - Gateway REST API to return operator classes in a jar given superclass (or not) |
| * [SPOI-2196] - Gateway REST API to return properties of an operator in a jar |
| * [SPOI-2200] - Installer - License Flow |
| |
| ### Story |
| * [SPOI-1608] - Platform Benchmarking (Platform1 and Platform2) |
| |
| ### Task |
| * [SPOI-722] - Document ads demo (add comments to codebase) |
| * [SPOI-1403] - HDFS Operator Benchmark |
| * [SPOI-1411] - Deprecate old Malhar webdemos once logstream is available |
| * [SPOI-1513] - datatorrent.com webapp development - app testing |
| * [SPOI-1610] - Develop benchmarking app for AdsDimension App (exactly once semantics) - Platform1 |
| * [SPOI-1612] - Benchmarking Ads Dimension demo app on Morado cluster (exactly once semantics) - Platform1 |
| * [SPOI-1618] - Benchmarking Machine Data app with Platform1 |
| * [SPOI-1694] - Document DT SandBox preparation |
| * [SPOI-1730] - Default License in git needs to be replaced by the license cut by the real key |
| * [SPOI-1732] - Create the real public/private key for licenses and store the private key in a safe place. |
| * [SPOI-1745] - Document Demo Application Setup |
| * [SPOI-1788] - CLI commands for licensing |
| * [SPOI-1794] - Create license info as a string |
| * [SPOI-1796] - Soft enforcement for normal paid app. 200% bump? |
| * [SPOI-1848] - Gateway to support changing and getting config parameters |
| * [SPOI-1851] - Document virtual cluster setup |
| * [SPOI-1862] - Working on creating Wire Frames for the Installation of DT Platform |
| * [SPOI-1863] - Make installer work w/o Maven |
| * [SPOI-1865] - Allow user to configure application classpath |
| * [SPOI-1868] - Support Book Keeping in the HDFSStorage |
| * [SPOI-1870] - Validate dtcli generate-license-request |
| * [SPOI-1872] - Modify generatelicense process |
| * [SPOI-1873] - License process via console |
| * [SPOI-1876] - Application Developer Guide Improvements |
| * [SPOI-1877] - Download and build JDK Standrd Doclet Source as part of DT |
| * [SPOI-1878] - DT Console Web UI Testing on Chrome |
| * [SPOI-1879] - DT Console Web UI Testing for Demo Apps on Chrome |
| * [SPOI-1884] - Operater developer guide review |
| * [SPOI-1891] - Add allatori documentation |
| * [SPOI-1892] - Add automatic build of front components to distribution |
| * [SPOI-1893] - Quick Start Guide |
| * [SPOI-1914] - cli get-app-info to include info from hadoop |
| * [SPOI-1926] - Call license web service and return license file |
| * [SPOI-1927] - Returns license request blob for UI to assemble mailto link |
| * [SPOI-1939] - Twitter Top Counter Demo Applications Guide |
| * [SPOI-1940] - RPM packaging for installer |
| * [SPOI-1941] - Include demo UI into installer |
| * [SPOI-1942] - Remove Allatori code expiration for GA |
| * [SPOI-1943] - include more info in the license request |
| * [SPOI-1945] - Add REST call to gateway to post license file |
| * [SPOI-1946] - virtual cluster configuration changes |
| * [SPOI-1947] - Create license@datatorrent.com |
| * [SPOI-1949] - Java application (with main method) that returns information given a license request blob |
| * [SPOI-1956] - License generation key pair expiration / private key protection |
| * [SPOI-1963] - Evaluate Doclava Doclet from Google |
| * [SPOI-1982] - E-mail verification success web page |
| * [SPOI-1988] - Review Quick Start Guide |
| * [SPOI-1989] - AdsDimension - Demo Applications Guide |
| * [SPOI-1990] - Twitter Rolling Top Words Counter - Demo Applications Guide |
| * [SPOI-2004] - Installer testing for GA |
| * [SPOI-2006] - Grant Google Analytics Access To Following People |
| * [SPOI-2010] - Configure the apps to use minimum memory as verified by app-memory-usage-automation. |
| * [SPOI-2015] - Get Machine data into contrib.jar |
| * [SPOI-2016] - Fraud Detection in contrib.jar |
| * [SPOI-2017] - Quick Start Guide version 2 |
| * [SPOI-2019] - List NxN performance apps (different event size vs different stream locality |
| * [SPOI-2025] - Getting Start Guide - Launch this copy. |
| * [SPOI-2032] - Certify Cloudera CDH 5.0 |
| * [SPOI-2048] - Uninstall script |
| * [SPOI-2050] - Start gateway as service flag |
| * [SPOI-2069] - Test Installer |
| * [SPOI-2091] - Update installation license agreement |
| * [SPOI-2093] - Verify demo UI is bundled with installer |
| * [SPOI-2100] - For terms of service box, change "continue" to "accept and continue" |
| * [SPOI-2106] - Change the message on 1 yr registration |
| * [SPOI-2107] - Change the message on 1 yr registration |
| * [SPOI-2108] - Put timeglass and "loading" or spinning.... while Hadoop system properties are being loaded |
| * [SPOI-2109] - Gateway down creates bad error message |
| * [SPOI-2110] - Remove errors popping on the right hand of console |
| * [SPOI-2111] - If gateway outage is discovered add a message to get them back |
| * [SPOI-2112] - Change the message on Hadoop screen |
| * [SPOI-2114] - Error if HDFS does not exist |
| * [SPOI-2115] - Create a list of issues summary screen |
| * [SPOI-2116] - Show more instructions on the completed screen |
| * [SPOI-2131] - Ingestion POC |
| * [SPOI-2136] - Do cartesian products for key, val pair |
| * [SPOI-2142] - Allow customization of cartesian product |
| * [SPOI-2161] - prereq message on welcome screen |
| * [SPOI-2162] - DFS error message |
| * [SPOI-2164] - DFS location validation |
| * [SPOI-2167] - Evaluate errors in Cloudera certification |
| * [SPOI-2223] - Test if all the applications with reduced container memory sizes run in sandbox 0.9.4. |
| * [SPOI-2230] - Uninstaller for RPM |
| * [SPOI-2231] - Provide Environment with Running Demos |
| * [SPOI-2240] - Set Up DataTorrent Demos on Dev Environment |
| |
| |
| ### Sub-task |
| * [SPOI-1682] - Too many mbassy threads!!! |
| * [SPOI-1718] - Update HA Documentation |
| * [SPOI-1729] - Restore operator recovery checkpoints in AM recovery |
| * [SPOI-1786] - Users should be able to generate license at datatorrent.com |
| * [SPOI-1931] - Installer - determine OS type and version |
| * [SPOI-1932] - installer with sudo/root user creation, service installs |
| * [SPOI-1933] - service wrappers for DTGateway |
| * [SPOI-1934] - HDFS directory creation during install |
| * [SPOI-1935] - Search for hadoop binaries standard paths |
| * [SPOI-1958] - Create HDFS Word input operator |
| * [SPOI-1962] - Add test to jekins nightly build |
| * [SPOI-1971] - Verify launch script for other apps from demo |
| * [SPOI-1972] - When some app fail, the main monitor should still keep looking at the other apps |
| * [SPOI-2028] - Provide shell script 'benchmark-throughput' to produce a single summary table |
| * [SPOI-2030] - Provide a list of individual demos used for benchmarking through dtcli benchmarking, so that the user can launch the demos individually. |
| * [SPOI-2031] - Package benchmarking suite into the installer and sandbox |
| * [SPOI-2046] - Update licenses location in installation script |
| * [SPOI-2053] - Certify CDH5.0 as part of Cloudera certification for inclusion in their process |
| * [SPOI-2054] - Certify installer on HW |
| * [SPOI-2079] - Run certification on bin install |
| * [SPOI-2085] - Change the certification resource xml files to contain the certification type. |
| |
| |
| |
| |
| ### Bug |
| * [MLHR-729] - Columns in table wrap in Firefox |
| * [MLHR-730] - Dev server does not escape double-quotes in error message |
| * [MLHR-732] - Links Outline in Firefox |
| * [MLHR-754] - JarList and DepJarList headers point to nonexistent text items |
| * [MLHR-767] - "FINISHING" has no icon in status |
| * [MLHR-785] - NaN in file size column of jar list |
| * [MLHR-809] - Inconsistent landing page across browsers |
| * [MLHR-814] - Selecting operators in the logical operators widget does not activate any actions |
| * [MLHR-816] - Names of partitions in the partitions widget should not be shown as links |
| * [MLHR-817] - Clickin on 'outputPort' leads to PageNotFound error |
| * [MLHR-824] - Don't show license agent detail when license agent is not running |
| * [MLHR-826] - Dep Jars fail to load in specify dep jars modal |
| * [MLHR-830] - Optimize RandomWordInput operator used in perfromance benhcmarking to use local final variables to improve performance. |
| * [MLHR-835] - Fix hard-coded file path in com.datatorrent.demos.wordcount.WordCountInputOperator |
| * [MLHR-836] - Need AbstractHDFSOutputAdapter |
| * [MLHR-863] - Add license headers to AbstractHdfsOutputOperator |
| * [MLHR-879] - Installer - Issues Management |
| * [MLHR-887] - Input operator that tails a growing log file in a directory |
| * [MLHR-899] - Give default name for all demo applications |
| * [MLHR-908] - Mark installation complete with dt.configStatus property |
| * [MLHR-911] - UI Shows Nan B in Allocated Memory |
| * [MLHR-912] - Installer - System Section - Show Field Specific Errors |
| * [MLHR-913] - Installer - System Section - Server-Side Error Messages |
| * [MLHR-918] - Yahoo finance with Alerts : Modify to accept multiple ticker symbols and remove hard-coded values. |
| * [MLHR-920] - Give desciptive names to benchmarking apps |
| * [MLHR-924] - Tail Operator should take care of the truncation of file |
| * [MLHR-930] - UI container list should not show time for last heartbeat for if the value is "-1" |
| * [MLHR-938] - gateway address property change |
| * [MLHR-939] - Put GatewayRestart into "actions" hash in settings.js |
| * [MLHR-940] - Delete the space in the name |
| * [MLHR-943] - Change the method name from isConnected to Connected in the db api |
| * [MLHR-991] - DAG Stream Locality |
| |
| |
| ### Improvement |
| * [MLHR-731] - Compile LESS on the fly in dev environment |
| * [MLHR-735] - Remove unused bundling make commands, update README |
| * [MLHR-745] - Add icon to left widget manager drawer |
| * [MLHR-746] - Use "cores" not % cpu for cluster metric total cpu usage |
| * [MLHR-747] - Remove "#" for "number of container", et al labels |
| * [MLHR-748] - Only load all applications from RM on demand |
| * [MLHR-749] - Improve notification history pane |
| * [MLHR-750] - Normalize labels for everything in console |
| * [MLHR-751] - Add lock where close dashboard icon would be for default dashboard |
| * [MLHR-753] - Remove avg app age in cluster metrics |
| * [MLHR-755] - Change "max alloc mem" to "peak alloc mem" in cluster metrics |
| * [MLHR-756] - Add memory levels to tooltip of license mem gauge in top right |
| * [MLHR-768] - Remove beefy from npm shrinkwrap |
| * [MLHR-770] - Clean up BaseUtil, BaseModel, BaseCollection, add tests |
| * [MLHR-781] - shorten link to log file in container page |
| * [MLHR-798] - Add "config.adsdimensions.redis.dbindex" configuration in webdemo |
| * [MLHR-873] - Add equals and hashcode to JdbcOperatorBase |
| * [MLHR-931] - ETL: Create a converter api and provide an implementation for Json Object to flat map conversion |
| * [MLHR-935] - Have "silentErrors" option for models and collections |
| * [MLHR-937] - ETL: Create a unifier for DimensionComputation operator |
| * [MLHR-947] - Improve overall look and feel of install wizard |
| * [MLHR-948] - Remove mock issues from SummaryView in installer wizard |
| * [MLHR-993] - Demo UI - Default Applications Names for Application Discovery |
| |
| ### New Feature |
| * [MLHR-688] - Discard Real-Time Updates When Page is not Active |
| * [MLHR-757] - Dashboard - Save Widget Width to Local Storage |
| * [MLHR-759] - Dashboard - Widget Definitions Collection |
| * [MLHR-763] - Line Chart Widget - hAxis options |
| * [MLHR-764] - Dashboard App - Meteor Data Source |
| * [MLHR-765] - Shutdown Container Interface |
| * [MLHR-773] - Configuration wizard page |
| * [MLHR-788] - Web Demos - Redis and MongoDB Config |
| * [MLHR-789] - Web Demos - Fraud Demo MongoDB Database Name |
| * [MLHR-790] - Web Demos - Single Configuration File |
| * [MLHR-791] - Web Demos - Start Script |
| * [MLHR-792] - Web Demos - Single Page App |
| * [MLHR-793] - Web Demos - JS/CSS Bundle |
| * [MLHR-794] - Web Demos - WebSocket Pub/Sub |
| * [MLHR-795] - Web Demos - Resources Clean Up on Scope Destroy |
| * [MLHR-796] - Web Demos - Distribution Files |
| * [MLHR-797] - Web Demos - Running Instructions |
| * [MLHR-804] - Config Page (Manage Properties) |
| * [MLHR-805] - Web Demos - Distribution Package Instructions |
| * [MLHR-806] - Installer - License Requests (REST API Calls) |
| * [MLHR-807] - Installer - License Text |
| * [MLHR-818] - Installer - License Section |
| * [MLHR-819] - Installer - System Properties Section |
| * [MLHR-820] - Console - Node.js Dev Mode |
| * [MLHR-821] - Web Demos - Distribution Package Launch Script |
| * [MLHR-831] - Installer - License Flow |
| * [MLHR-834] - Installer - License - Registration |
| * [MLHR-858] - Installer - Gateway Restart |
| * [MLHR-861] - Installer - System Properties - IP List |
| * [MLHR-864] - Installer - Handling Hadoop Not Found |
| * [MLHR-903] - Installer - WebSocket DataSource Disconnect |
| * [MLHR-909] - Installer - Restart Confirmation |
| * [MLHR-914] - Installer - Error Messages |
| * [MLHR-919] - License Bar |
| * [MLHR-923] - Installer - Properties Update "Loading" Indicator |
| * [MLHR-925] - Installer Update |
| * [MLHR-934] - Allow overrides to $.().modal(options) for Modals |
| |
| ### Story |
| * [MLHR-705] - Node.js Pub/Sub Service |
| * [MLHR-708] - Evaluate Node.js Pub/Sub Services |
| * [MLHR-710] - Pie Chart Data Model |
| * [MLHR-712] - Dashboard App - Historical Data Support |
| * [MLHR-714] - Dashboard App - MongoDB Integration |
| * [MLHR-715] - Create Dashboard from Running App - Widgets Auto Discovery |
| * [MLHR-736] - Console Firefox Support |
| * [MLHR-737] - Console Safari Support |
| * [MLHR-801] - Installer/Config UI |
| * [MLHR-802] - Web Demos - Distribution Package |
| * [MLHR-803] - Installer (Wizard) Page |
| |
| ### Task |
| * [MLHR-349] - Add build script to Malhar/front |
| * [MLHR-402] - Logstream - aggregate top hits and bytes for URL, geo DMA, IP, URL/status code, url |
| * [MLHR-651] - Use compatible version of jersey/jackson/jetty in Malhar |
| * [MLHR-659] - Migrate MongoDB adapters to use the new database adapter interface |
| * [MLHR-741] - Web Apps (Demos) Firefox Support |
| * [MLHR-744] - Web Apps (Demos) Safari Support |
| * [MLHR-760] - Dashboard App - Meteor Integration |
| * [MLHR-761] - Dashboard App - Derby.js Integration |
| * [MLHR-762] - create install script for ui |
| * [MLHR-839] - Review guide on MachineData app |
| * [MLHR-842] - Demo guide for Pi Application |
| * [MLHR-843] - Demo guide for Twitter Top URL Counter demo application |
| * [MLHR-851] - Demo guide for Fraud detection demo application |
| * [MLHR-853] - Demo guide for Mobile demo application |
| * [MLHR-854] - Demo Guide for Word-Count Application |
| * [MLHR-855] - Demo guide for Pi Calculator application |
| * [MLHR-856] - Demo guide for Twitter Rolling Top Words Application |
| * [MLHR-859] - upgrade kryo to 2.23 |
| * [MLHR-871] - Demo guide for Twitter Top URL Counter - Launch This Copy. |
| * [MLHR-872] - Demo Guide for Word-Count Application - Launch this copy. |
| * [MLHR-875] - Demo guide for Yahoo finance application |
| * [MLHR-876] - Demo guide for Yahoo finance alerting application |
| * [MLHR-877] - Demo guide for Yahoo finance application with Derby SQL |
| * [MLHR-892] - ETL logstream application - study the log stream application |
| * [MLHR-893] - ETL- Use the generic dimension operator that was created for a POC in Log stream |
| * [MLHR-900] - ETL- Operators used by logstream application need to be generic and moved to library |
| * [MLHR-904] - Fix the nightly and trigger builds broken due to removal of api.codec and api.util |
| * [MLHR-905] - Dedup: Make deduper and bucket manager part of malhar library |
| * [MLHR-910] - Demo guide for Twitter Rolling Top Words Application - Launch This Copy |
| * [MLHR-915] - CLONE - Demo guide for Pi Application - Launch this copy |
| * [MLHR-916] - CLONE - Demo guide for Pi Calculator application - Launch this copy |
| * [MLHR-917] - CLONE - Demo guide for Mobile demo application - Launch this copy |
| * [MLHR-936] - Create new Redis Store using Lettuce redis client |
| * [MLHR-949] - Add confirmation to DTGateway restart button in System Properties |
| * [MLHR-962] - ETL : Create a sifter operator |
| * [MLHR-980] - CLONE - Demo guide for Yahoo finance alerting application - launch this copy |
| * [MLHR-981] - CLONE - Demo guide for Yahoo finance application - launch this copy |
| * [MLHR-982] - CLONE - Demo guide for Yahoo finance application with Derby SQL - launch this copy |
| |
| ### Sub-task |
| * [MLHR-678] - Time Series Forecasting with Simple Linear Regression |
| * [MLHR-718] - Time Series Forecasting using Simple/Single Exponential Smoothing |
| * [MLHR-726] - Time Series Forecasting Operator using Holt's Linear Trend Model |
| * [MLHR-727] - Develop application for a telecom related use case for time series forecasting with Simple Linear Regression and CMA smoothing |
| * [MLHR-932] - Create Centered Moving Average Smoothing Operator |
| |
| |
| |
| |
| Version 0.9.3 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### New Feature |
| * [SPOI-261] - Design a general purpose read from stream and write to cassandra |
| * [SPOI-400] - Each streaming application should license check |
| * [SPOI-1622] - Input operator - XML parser |
| * [SPOI-1647] - LogStream UI |
| * [SPOI-1770] - Gateway should expose list of available topics |
| * [SPOI-1778] - Open readme on sandbox startup |
| * [SPOI-1804] - Start license app on launch app if not running |
| * [SPOI-1805] - command to show license file info in cli |
| * [SPOI-1812] - Create REST call for specific license agent, given a license id |
| * [SPOI-1823] - Gateway REST API - Get Running Applications List |
| * [SPOI-1829] - Semantic URLs for Web Apps |
| |
| ### Improvement |
| * [SPOI-1202] - Provide a way to check whether an operator is partitioned |
| * [SPOI-1783] - Add allocatedMB to main application list |
| * [SPOI-1789] - Change frequency of heartbeats to license app |
| * [SPOI-1795] - License file to clearly state hard enforcement or soft enforcement |
| * [SPOI-1816] - support simple variable substitution in the cli |
| * [SPOI-1835] - support gateway status command |
| |
| ### Story |
| * [SPOI-1320] - Support MQTT protocol |
| * [SPOI-1542] - Input operator - Directory Scan |
| |
| ### Bug |
| * [SPOI-1696] - Make de-duper dynamically partitionable |
| * [SPOI-1704] - Stram enforcement to lock physical plan changes when license memory limit is reached |
| * [SPOI-1706] - Design a enforcement format for the license policy |
| * [SPOI-1711] - Encryption/obfuscation of communication between stram and license agent |
| * [SPOI-1779] - Update sandbox documentation terminology |
| * [SPOI-1802] - Provide total license memory via stram web services |
| * [SPOI-1807] - Unknown outage on Machine data demo |
| * [SPOI-1809] - Suppress expected error message in dtgateway-ctl |
| * [SPOI-1810] - sample-stram-site.xml generates warnings |
| * [SPOI-1813] - Add SNAPSHOT repository to install pom |
| * [SPOI-1815] - Make stram memory reporting to license manager asynchronous |
| * [SPOI-1818] - Chance "className" in license file to id |
| * [SPOI-1820] - dtcli script doesn't exit when maven command fails |
| * [SPOI-1826] - Update documentation title |
| * [SPOI-1827] - Install script errors |
| * [SPOI-1831] - CLI warning about trouble with license manager |
| * [SPOI-1833] - Use encrypted byte arrays for RPC wire protocol for licensing |
| * [SPOI-1840] - Change default license memory limit to 25GB |
| * [SPOI-1841] - Make stram memory enforcement tolerances property settings for the enforcer |
| * [SPOI-1842] - Investigate the possibility of engine obfuscation jar not containing any references to license package path |
| * [SPOI-1880] - GET nonexistent container returns 500 error |
| |
| ### Task |
| * [SPOI-1375] - All sandbox apps must work in 8G VM. Need to test each |
| * [SPOI-1467] - DB lookup for Cassandra |
| * [SPOI-1507] - datatorrent.com webapp development - pilot test of Angular and WP integration |
| * [SPOI-1509] - datatorrent.com webapp development - db design |
| * [SPOI-1511] - datatorrent.com webapp development - app design |
| * [SPOI-1512] - datatorrent.com webapp development - app dev |
| * [SPOI-1515] - datatorrent.com webapp deveopment - integrate standalone app with cms |
| * [SPOI-1516] - datatorrent.com webapp development - add GA info during registration |
| * [SPOI-1517] - datatorrent.com webapp development - background jobs |
| * [SPOI-1617] - Benchmarking Performance app with Platform2 |
| * [SPOI-1641] - Benchmarking Ads Dimension App - Platform2 |
| * [SPOI-1715] - Show remainingLicensedMB and allocatedMB in UI for each application |
| * [SPOI-1763] - Provide support for Accumulo NoSQL db |
| * [SPOI-1780] - Sandbox - activate license automatically |
| * [SPOI-1781] - Sandbox - increase memory to 8GB |
| * [SPOI-1782] - License App should use much less memory (256MB or less?) |
| * [SPOI-1787] - Add license instructions to README |
| * [SPOI-1790] - Ensure update to license app on any resource change by StrAM |
| * [SPOI-1791] - Hard enforcement for free license (6GB), and eval license |
| * [SPOI-1793] - Hide sub-license and make license object behave as “what is license data right now?” |
| * [SPOI-1798] - Date format change in license file |
| * [SPOI-1799] - Change name "Sublicense" to "Section" or "LicenseSection" |
| * [SPOI-1800] - Webservice specs for Gateway for license info |
| * [SPOI-1825] - Update end user documentation |
| * [SPOI-1832] - Support CDH default log4j setup |
| |
| ### Sub-task |
| * [SPOI-1451] - Show critical path |
| * [SPOI-1721] - Augment the Partitionable interface to inform of all the partitions which actually were deployed |
| * [SPOI-1733] - Container heartbeat RPC failover |
| |
| |
| ### New Feature |
| * [MLHR-5] - UI component for license information |
| * [MLHR-6] - Google Line Chart Widget |
| * [MLHR-7] - Gauge Widget |
| * [MLHR-8] - Top N Widget |
| * [MLHR-9] - Compile Widget from HTML Template |
| * [MLHR-10] - Make widgets resizable and renameable in ui-builder |
| * [MLHR-11] - Dashboard Component Grunt Tasks |
| * [MLHR-653] - Dynamically Connect Widgets to WebSocket Topics |
| * [MLHR-655] - Create serializing mechanism for instantiated widgets and dashboard(s) |
| * [MLHR-656] - Add/Compile Widgets from templateUrl |
| * [MLHR-664] - Support MQTT protocol |
| * [MLHR-668] - Set up widget configure dialog |
| * [MLHR-669] - Visual Data Demo App |
| * [MLHR-671] - Add allocatedMB column in main application list |
| * [MLHR-673] - Dashboard App - Notification Service |
| * [MLHR-674] - Explicit Saving/Loading of Dash configurations in ui builder |
| * [MLHR-687] - Dashboard App - Filter WebSocket Topics |
| * [MLHR-689] - Dashboard App - Widget Options Modal |
| * [MLHR-690] - Dashboard App - Widgets Schema |
| * [MLHR-691] - WebSocket Topics Debugger Widget |
| * [MLHR-692] - JSON Widget |
| * [MLHR-693] - Progressbar Widget |
| * [MLHR-695] - Pie Chart Widget |
| * [MLHR-696] - Dashboard App - Development/Production Scripts |
| * [MLHR-697] - Dashboard App - Node.js Configuration |
| * [MLHR-698] - Dashboard App - WebSocket/REST API Configuration |
| |
| ### Improvement |
| * [MLHR-686] - Deglobalize the visibly component |
| |
| ### Bug |
| * [MLHR-4] - Use new livechart module for OpChart Widget |
| * [MLHR-13] - Console status column display |
| * [MLHR-648] - Update Issues section in README files |
| * [MLHR-650] - Changing metrics on logical dag fails |
| * [MLHR-654] - Some widgets' height changes when changing width |
| * [MLHR-667] - Add UI version to console |
| * [MLHR-670] - Memory leak in console |
| * [MLHR-677] - Widgets Data Models |
| * [MLHR-680] - Update license information dialog for new REST call info |
| * [MLHR-725] - WindowId Formatter |
| * [MLHR-739] - Stream Locality Toggle fails for DAG view |
| |
| ### Story |
| * [MLHR-1] - Reusable Dashboard Component with AngularJS |
| * [MLHR-2] - Dashboard Widgets |
| * [MLHR-3] - Dashboard App |
| |
| ### Task |
| * [MLHR-321] - Directory Scan operator |
| * [MLHR-452] - Create a De-duplication operator |
| * [MLHR-603] - Supports upload of dependency jars |
| * [MLHR-638] - Test streaming application for dynamic partition |
| * [MLHR-645] - More fields in drop down in logicalDAG widget |
| * [MLHR-646] - Document issue tracking location in README |
| * [MLHR-652] - Parallel Simple Linear Regression |
| * [MLHR-657] - Migrate memcache adapters to use the new database adapter interface |
| * [MLHR-662] - Migrate Redis adapters to use the new database adapter interface |
| * [MLHR-663] - Design new DB adapters interface |
| * [MLHR-666] - DB lookup for Cassandra |
| |
| |
| |
| Version 0.9.2 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-1327] - AtLeastOnceTest.testInlineOperatorsRecovery intermittent failure |
| * [SPOI-1342] - DTCli should check license and relay the information with each application launched |
| * [SPOI-1383] - Last window id and recovery window id do not update on 0.9 |
| * [SPOI-1439] - Gateway should be secured |
| * [SPOI-1445] - Add version detection for Gateway |
| * [SPOI-1456] - Free Memory in container widget changes too rapidly |
| * [SPOI-1540] - Specification of license handlers and enforcers in the license file. |
| * [SPOI-1632] - jar upload fails |
| * [SPOI-1634] - Uptime at 1Billion events/s (Machine data) |
| * [SPOI-1635] - Update node1 with latest machine data demo |
| * [SPOI-1676] - Incremental Obfuscation of dt-flume directory |
| * [SPOI-1677] - Supports uploading of dependency jars |
| * [SPOI-1678] - When loading jars, make sure they are in their separate space so they don't conflict with gateway, cli and other jars |
| * [SPOI-1679] - When uploading jar and when dependencies are not met, allow the upload with a message about dependencies |
| * [SPOI-1680] - gateway throws errors when retrieving web service info from stram |
| * [SPOI-1687] - Support launching jar and showing logical plan from HDFS |
| * [SPOI-1688] - Map Reduce Monitor Does Not Publish WebSocket Data |
| * [SPOI-1697] - Update demo configuration on node2 |
| * [SPOI-1703] - Update auto provisioning with DataTorrent 0.9.1 and GCE GA |
| * [SPOI-1707] - License agent should handle license expiry |
| * [SPOI-1708] - Stram should store license expiry |
| * [SPOI-1709] - Show license object information in gateway |
| * [SPOI-1710] - License cutting utility |
| * [SPOI-1712] - Gateway to gracefully handle stram being a newer version than itself |
| * [SPOI-1714] - Dynamic partition stop working if you start from only 1 partition |
| * [SPOI-1727] - ApplicationInfoAutoPublisher unit test error |
| * [SPOI-1728] - StramEvent exception prevents package name obfuscation |
| * [SPOI-1739] - recordingStartTime of operator stats is showing -1 from time to time |
| * [SPOI-1743] - Tuple recording on port is not showing up in web services |
| * [SPOI-1744] - Recording says ended even if the recording is still going on |
| |
| |
| ### Improvement |
| * [SPOI-1098] - event recorder logging improvements |
| * [SPOI-1370] - Make the partition logic available to the end-users |
| * [SPOI-1448] - DAG Visualization - Stream Types |
| * [SPOI-1603] - BufferServerStatsCollection - dont check against bufferserverpublisher and subscriber |
| * [SPOI-1613] - Update the User Interface guide to reflect latest version (0.9.1) |
| |
| ### New Feature |
| * [SPOI-165] - Parent jira for authentication |
| * [SPOI-170] - Stream should authenticate before allowing an operator to connect |
| * [SPOI-258] - Develop Flume Sink and corresponding DT input adapter |
| * [SPOI-327] - Parent jira for Security |
| * [SPOI-401] - Licensing alert mechanisms |
| * [SPOI-411] - Ability to modify (add, upgrade, downgrade) license while the app is running |
| * [SPOI-436] - Provide Web Service for obtaining license information (Usage limits, etc) |
| * [SPOI-729] - Include license data in DT phone home |
| * [SPOI-872] - Logical View of Running Application |
| * [SPOI-975] - Support DataLocal Functionality |
| * [SPOI-1406] - Add log file path and/or URL to each container info map |
| * [SPOI-1621] - Input operator - CDR parser using CSV |
| * [SPOI-1699] - Add locality (and maybe id?) to physical streams in REST calls |
| |
| ### Task |
| * [SPOI-1689] - Map Reduce Monitor Web App |
| * [SPOI-739] - Hadoop 2.2 certification |
| * [SPOI-763] - Competition study |
| * [SPOI-1140] - Annotate dag visualization with stream throughout and other data |
| * [SPOI-1246] - Support versioning for Gateway to STRAM communications |
| * [SPOI-1253] - Create DataTorrent Application which provides licensing server functionality |
| * [SPOI-1389] - ContainerList view should show the log file name (stderr, stdout) in the info widget |
| * [SPOI-1405] - Design macros for node0 and node1 |
| * [SPOI-1609] - Competitive analysis - DT (Platform1 and Platform2) |
| * [SPOI-1611] - Benchmarking Ads Dimension on Morado cluster (at-least-once semantics) - Platform1 |
| * [SPOI-1616] - Benchmarking Performance app with Platform1 |
| * [SPOI-1670] - Ensure that Dedup operator is fault tolerant |
| * [SPOI-1673] - use public/private key encryption for dt phone home |
| * [SPOI-1675] - Map Reduce Jobs |
| * [SPOI-1686] - Launch separate process when loading classes from application jars |
| * [SPOI-1722] - Create a utility to create default license |
| * [SPOI-1724] - Create a command line utility to generate customer license |
| * [SPOI-1736] - CLI warning on License Violation |
| * [SPOI-1742] - Update end-user documentation |
| |
| |
| ### Sub-task |
| * [SPOI-919] - Certify secure mode with Hadoop 2.2.0 |
| * [SPOI-966] - Create a licensing agent application |
| * [SPOI-1413] - Flume sink part |
| * [SPOI-1414] - DT Input Adapter for Flume |
| * [SPOI-1475] - Augment Kafka operator to dynamically adapt to load and broker/partition changes |
| * [SPOI-1538] - Develop Ads Dimension on Morado cluster (at-least-once semantics) Platform1 |
| * [SPOI-1713] - Secure communication between gateway and stram |
| * [SPOI-1720] - Ensure that the Partionable interface and StatsListener interface callbacks are made from the same thread |
| * [SPOI-1723] - The default license generation should be integrated with build |
| * [SPOI-1731] - Sync execution layer deployment state after recovery |
| |
| ### GitHub - DataTorrent/Malhar |
| * [616] - fix #615 Update Web Apps Instructions |
| * [615] - Update Web Apps Instructions |
| * [614] - corrected typo |
| * [613] - Fixes #599. Upload and specify dependency jars |
| * [612] - fixes #597 |
| * [611] - fixes #610 |
| * [610] - Telecom tests failing |
| * [609] - Github 597 |
| * [608] - #fix 607 Machine Data Demo Day Format |
| * [607] - Machine Data Demo Day Format |
| * [606] - fixes #457 added xml parse operator to parse and pick values from xml nodes and att... |
| * [605] - added the history for hadoop 1.x |
| * [603] - Map Reduce Monitor - Elapsed Time |
| * [602] - Map Reduce Monitor - Elapsed Time |
| * [601] - Map Reduce Monitor - Bootstrap JS, Server Errors Notification, Header Alignment |
| * [599] - Provide UI for uploading and specifying dependency jars |
| * [598] - Map Reduce Monitor - Server Errors Notification |
| * [597] - Map Reduce Monitor App - CPU/Memory History |
| * [595] - Map Reduce Monitor (History Charts, Animations, Readme, AngularJS Upgrade) |
| * [594] - Map Reduce Monitor - Production Files (dist) |
| * [593] - Map Reduce Monitor - AngularUI Bootstrap Progressbar issue with ngAnimate |
| * [592] - Map Reduce Monitor Update (Readme, App List, History Charts) |
| * [591] - Map Reduce Monitor - Map/Reduce History Charts |
| * [590] - Fixes #401, adds zoom to physical DAG |
| * [589] - Map Reduce Monitor - App List Columns |
| * [586] - Map Reduce Monitor - App Id AngularJS Filter |
| * [585] - Fixes #569, Cosmetic changes |
| * [584] - Map Reduce Monitor - Show Active Job First |
| * [581] - Map Reduce Monitor Update (Loading Indicator, Animations, Delayed Promise) |
| * [580] - Map Reduce Monitor - AngularJS Animations |
| * [579] - Map Reduce Monitor - Upgrade to AngularJS 1.2.6 |
| * [578] - Map Reduce Monitor - App List Loading Indicator |
| * [577] - fixes #553, fixes #575 |
| * [576] - Map Reduce Monitor - AngularJS Delayed $q Promise |
| * [575] - Map Reduce Monitor App - Send Job Stats immediately on Subscribe Request |
| * [573] - Map Reduce Monitor Update (App List Grid, Progress Bars, Job Removal) |
| * [572] - fixes #570 |
| * [571] - Map Reduce Monitor - Remove Job on WebSocket Message |
| * [570] - clipPath issue when multiple charts on the same page |
| * [569] - Various cosmetic updates for console |
| * [568] - fixes #542, tooltip no longer obstructed by graph lines |
| * [567] - Bird's Eye View for Physical DAG view |
| * [566] - fixes #544, windowIds can now handle initial value of -1 or 0 |
| * [565] - Map Reduce Monitor - Job Selection |
| * [564] - fixes #357, added logical operator page |
| * [563] - Map Reduce Monitor - App List ng-grid |
| * [561] - Map Reduce Monitor - App List Table Filter |
| * [560] - Fixed exception with KryoSerializableStreamCodec #559 |
| * [558] - CDR simulator #524 |
| * [557] - Github 525 |
| * [556] - set the name of the io threads created by ning asynchttpclient |
| * [554] - Squashed commit of the following: |
| * [553] - Map Reduce Monitor App - Store Map/Reduce Progress History |
| * [551] - fixes #550 |
| * [550] - Map Reduce Monitor App - App Should Broadcast Special Message on Unsubscription |
| * [549] - Map Reduce Monitor - Stop Updates after Job Unsubscribe |
| * [548] - Map Reduce Monitor - Found Job Notification |
| * [547] - Fix Github #545 |
| * [544] - Console does not handle initial windowId |
| * [542] - Tooltip for line graphs show up behind graphs after turning series on and off |
| * [541] - fixes #535 |
| * [539] - Map Reduce Monitor - Merge Progress Bars with Progress Table |
| * [538] - Map Reduce Monitor - Combine Map/Reduce Counters |
| * [536] - Map Reduce Monitor - App List Running Jobs Progress Bar |
| * [535] - Console breaks when switching to other page |
| * [534] - Fixes #510, Unsubscribe Logical Operators when not used by any widget |
| * [533] - Fixes #521, Refactored WindowId usage |
| * [532] - Map Reduce Monitor - Counters |
| * [530] - Map Reduce Monitor - App List Sort |
| * [526] - Map Reduce Monitor - Counters |
| * [525] - CDR processing DAG prototype |
| * [523] - Github 512 |
| * [521] - Normalize all WindowId objects by overriding "set" method of appropriate models |
| * [519] - Map Reduce Monitor - Header Alignment on Resize |
| * [518] - Map Reduce Web App |
| * [517] - Map Reduce Monitor - License Headers |
| * [516] - Map Reduce Monitor - Map Reduce Jobs List |
| * [515] - Map Reduce Monitor - AngularJS Modules Definition |
| * [514] - Map Reduce Monitor - Readme (Deployment and Running Instructions) |
| * [513] - Map Reduce Monitor - Job Query Loading Indicator |
| * [512] - Support Normalization Operator |
| * [511] - Map Reduce Monitor - AngularJS Settings Provider |
| * [510] - Unsubscribe logicalOperators on InstancePage when not in use by widget |
| * [509] - Fixes #505. Also removes one more instance of free memory metric for containers |
| * [507] - Map Reduce Monitor - Single Config (Server and Client) |
| * [505] - Add processed and emitted metrics to container overview widget |
| * [504] - fixes #356, container log url now available in container info widget |
| * [503] - Map Reduce Monitor - Active Job Highlight |
| * [502] - Map Reduce Monitor - AngularJS Parent Scope Event Propagation |
| * [501] - Fixes #364, removed free memory from container metrics |
| * [500] - Map Reduce Monitor - Mock Server |
| * [499] - Adding support for R. Basic operations - min, max and std deviation support added. Also adding support to run R scripts. |
| * [498] - Rsupport pull |
| * [496] - Map Reduce Monitor - Progress Line Chart |
| * [495] - Map Reduce Monitor - Running MAPREDUCE Applications Discovery |
| * [494] - fixes #420, can now set explicit height for widgets |
| * [493] - fix #488 added delay before reconnection |
| * [492] - CPU/RAM Metrics for Map Reduce Jobs |
| * [491] - fix #488 added delay before reconnection |
| * [489] - Map Reduce Monitor - Job Controller |
| * [488] - WebSocketOutputOperator should wait a specified number of seconds before reconnection |
| * [487] - Using uniform naming convention for applications. Fixed incorrect application names. Fixes #486. |
| * [486] - Application names are not uniform |
| * [485] - CPU/RAM Metrics for Map Reduce Jobs (Map Reduce Monitor App) |
| * [484] - Map Reduce Monitor - AngularJS UI-Router Nested Views |
| * [483] - Enhance the AbstractSlidingWindow #480, Add a SortedSlidingWindow operator #423 |
| * [482] - fixes #411. bundling on server.js, monkeypatching fs to avoid EMFILE |
| * [479] - fix #443 reconnection when the connection is dropped |
| * [478] - fix #443 Handles reconnection when the connection is dropped |
| * [477] - Improvements to LogicalDagWidget. Fixes #399, #473, #475, #476 |
| * [476] - Logical DAG Widget: Limit scroll scale extent |
| * [475] - Logical DAG Widget: add ability to reset initial dag view |
| * [474] - Map Reduce Monitor - AngularJS UI-Router |
| * [473] - Logical DAG Widget: only zoom when alt/option is held down |
| * [472] - Map Reduce Monitor App - App Does not Publish Completed Maps |
| * [471] - Map Redice Monitor - Reduce Progress Grid |
| * [470] - Map Redice Monitor - Map Progress Grid |
| * [469] - Map Redice Monitor - AngularJS Percentage Filter |
| * [468] - Map Reduce Monitor - Monitored Jobs Grid |
| * [467] - Add a general CSV parser operator to parse string, byte[] input to Map #451 |
| * [466] - Map Redice Monitor - AngularJS Util Service |
| * [465] - Map Reduce Monitor - Unsubscribe Action |
| * [464] - Map Reduce Monitor - Client-Side Settings |
| * [463] - Map Reduce Monitor - WebSocket Unsubscribe |
| * [461] - Map Reduce Monitor - Multiple Jobs Monitoring |
| * [460] - Map Reduce Monitor - Progress Bar Animation |
| * [459] - Map Reduce Monitor - Upgrade to AngularJS 1.2.4 |
| * [458] - added xml parser operator and its test, fixes #457 |
| * [456] - Github 444 |
| * [454] - Map Reduce Monitor - AngularUI Bootstrap |
| * [453] - Map Reduce Monitor - Production Build with Grunt |
| * [452] - Map Reduce Monitor - jshint |
| * [451] - CSV input operator (CDR processing) |
| * [450] - Map Reduce Monitor - Progress Bars |
| * [449] - Map Reduce Monitor App - WebSocket Query |
| * [448] - Map Reduce Monitor - Error Notifications with pnotify |
| * [444] - Map Redice Monitor App - Publish Map/Reduce Updates as Array |
| * [443] - Map Reduce Monitor App WebSocket Issue |
| * [442] - Map Reduce Monitor - Node.js Proxy for Hadoop ResourceManager |
| * [441] - Map Reduce Monitor - REST Service |
| * [439] - Map Reduce Monitor - Server Configuration |
| * [438] - Map Reduce Monitor - Settings |
| * [436] - Map Reduce Monitor - Job Progress Grid |
| * [435] - Map Reduce Monitor - WebSocket Service with AngularJS provider |
| * [434] - Map Reduce Monitor - Unit Tests |
| * [433] - Map Reduce Monitor - AngularJS Directives (widgets) |
| * [432] - Map Reduce Monitor - Page Layout with Bootstrap |
| * [431] - Map Reduce Monitor - Node.js Server |
| * [430] - Map Reduce Monitor - Yeoman Generated App |
| * [428] - Normalization operator (CDR processing) |
| * [427] - Filter operator (CDR Processing) |
| * [426] - Enrichment operator (CDR processing) |
| * [425] - Aggregator operator (CDR processing) |
| * [422] - Github 421 |
| * [421] - Create RedisOperator taking String,String for performance |
| * [420] - Allow widgets to have adjustable height |
| * [419] - DAG Styling, DAG Firefox Issue |
| * [418] - Logical DAG - Firefox Bottom Margin Issue |
| * [417] - Logical DAG Styling |
| * [416] - Fix jquery build error |
| * [415] - Organized scripts and server |
| * [414] - fix #408, fix #413 Logical DAG - Show Stream Locality on Demand |
| * [413] - Logical DAG - Right Aligned Legned and Show Locality Link |
| * [412] - Improve Front Dev Environment |
| * [411] - Improved dev environment for front |
| * [410] - Map Reduce Monitor Web App |
| * [409] - fix #393 Front Node.js Proxy |
| * [408] - Logical DAG - Show Stream Locality on Demand |
| * [407] - fixes #373 |
| * [401] - Physical DAG - Smart Zoom |
| * [399] - Physical DAG - Bird's-Eye View |
| * [393] - Front Node.js Proxy |
| * [375] - fixes #367, improves reload time during dev on front |
| * [374] - fixes #367, improves reload time during dev on front |
| * [372] - Logical DAG Legend Styling |
| * [371] - Logical DAG Legend |
| * [370] - Fix issue316 issue317 pull |
| * [369] - Logical DAG - Legend |
| * [368] - Squashed commit of the following: |
| * [367] - Precompile templates for better dev process |
| * [366] - Documenting demos pull |
| * [365] - Normalized all "processed" and "emitted" labels |
| * [364] - Remove free memory from container metrics in UI |
| * [362] - Dependency to dagre-d3 fork |
| * [361] - Logical DAG - Stream Locality |
| * [360] - Update physical operators collection to fetch from physical plan |
| * [359] - Add source and sinks to physical operator list |
| * [358] - Normalize processed/s emitted/s labels across data tables and dag view |
| * [357] - Create Logical Operator Page |
| * [356] - ContainerList view should show the log file name (stderr, stdout) in the info widget |
| * [355] - fixes #349, recently-launched app does not request operator list |
| * [354] - Make partitionable kafka input operator adjust partitions ifself for kafka partition change(del/add) |
| * [353] - Upgrade kafka to 0.8 release |
| * [352] - fixes #322 |
| * [351] - Non-partitioned operators |
| * [350] - Key/Value lookup Storage Manager changes |
| * [349] - Application launch error in console |
| * [348] - Fixes #339, switches cluster metrics to websocket topic |
| * [347] - fix #346 Physical DAG - Remove Container IDs |
| * [346] - Physical DAG - Remove Container IDs |
| * [345] - Added sensible default display for avg app age field in cluster metrics widget #341 |
| * [344] - Added build cmd to travis script, fixes #343 |
| * [343] - Build step for front not in travis script |
| * [342] - Fix for #328 |
| * [341] - Cluster overview display items (initial launch) |
| * [339] - Cluster stats should come from WebSocket topic |
| * [337] - Add 1 to N partition support for kafka input operator with simple kafka consumer #311 |
| * [336] - logstream integration with siteops ui |
| * [332] - Squashed commit of the following: |
| * [329] - Documenting demos |
| * [328] - Numbers in graph overlay illegible when close together |
| * [325] - [DB Lookup & Caching] Create Mongo-db based DB lookup operator |
| * [323] - fixes #322 |
| * [322] - Make connectionList property not null in Redisoutputoperator |
| * [313] - Marking operators non-partitionable #312 |
| * [312] - Set partitionable to false for all operators which are cannot be partitioned |
| * [311] - Dynamic partition kafka input operator to adapt to real time load |
| * [298] - LogStream - siteops conversion - totals per second ( pages / bytes) |
| * [297] - LogStream - siteops conversion - plot requests over time (per page / per server) |
| * [296] - LogStream - siteops conversion - top 10 servers with 404 response (req/sec) |
| * [295] - LogStream - siteops conversion - top 10 URL's with 404 response (req/sec) |
| * [294] - LogStream - siteops conversion - top 10 clients downloading (bytes/sec) |
| * [293] - LogStream - siteops conversion - top 10 client IP's (req/sec) |
| * [292] - LogStream - siteops conversion - server load (req/sec/server) |
| * [291] - LogStream - siteops conversion - top 10 URL's (req/sec) |
| * [290] - Kafka POM WARN during build |
| * [277] - Kafka input operator should destroy itself if the topic doesn't exist |
| * [270] - [DB Lookup & Caching] Improvement to the design of existing Storage Manager |
| * [269] - [logstream] integration with sitestats ui |
| * [111] - SPOI-1191 - #comment Added StandardDeviationKeyVal operator and test class. |
| * [104] - Map Reduce Monitor Application |
| |
| |
| Version 0.9.1 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### New Feature |
| * [SPOI-377] - Input adapter for CouchDB |
| * [SPOI-378] - Output Adapter for CouchDb |
| * [SPOI-410] - Document and test download to work with Cloudera CDH5 |
| * [SPOI-945] - Add ability to specify application properties, DAG, operator and port attributes in stram configuration file. |
| * [SPOI-1088] - Add mouseover for new graphing module |
| * [SPOI-1172] - DAG Visualization - Partition Shape |
| * [SPOI-1215] - Gauges in Container View |
| * [SPOI-1407] - Add Jar File View |
| * [SPOI-1415] - De-duplicaton operator |
| * [SPOI-1483] - Hadoop management script or service |
| * [SPOI-1501] - Widget - logical operators |
| * [SPOI-1535] - Memory Gauge Widget |
| * [SPOI-1536] - CPU Gauge Widget |
| * [SPOI-1550] - Logical DAG - Multiple Metrics |
| * [SPOI-1556] - Logical DAG Real-Time Metrics - Tracking Model Changes |
| * [SPOI-1561] - Logical DAG - Throughput |
| * [SPOI-1562] - Logical DAG - Prev/Next Metric Navigation |
| * [SPOI-1600] - Logical DAG - Show Two Metrics |
| * [SPOI-1235] - Add support for "DB lookup" functionality |
| * [SPOI-1236] - Provide support for Cassandra NoSQL database |
| |
| ### Bug |
| * [SPOI-600] - Twitter demo failed when I did two simulteneouce operator/container kill |
| * [SPOI-993] - Plan modification gives NPE when the containers aren't deployed yet |
| * [SPOI-1280] - Recording Chart widget shows up with "X" |
| * [SPOI-1303] - If the operator is setup sucessfully, ensure that it gets a chance to teardown. |
| * [SPOI-1331] - When an operator is undeployed, its input streams should be disconnected from any upstream operators. |
| * [SPOI-1388] - Nodes are showing IP instead of hostnames |
| * [SPOI-1412] - Buffer Server uses a different version of guava base jar than Malhar |
| * [SPOI-1427] - Accessing Demos From clean VM is not working |
| * [SPOI-1430] - When uploading jars, meta information should be extracted and return error if jar is not valid |
| * [SPOI-1433] - checkboxes in lists intermittently unresponsive |
| * [SPOI-1452] - GetApp web service returns 500 for apps killed via UI |
| * [SPOI-1457] - Kill container call returns 500 when request payload is not json |
| * [SPOI-1458] - After finish uploading jar from the development tab, the jar list does not get updated until browser's "refresh" button is pressed |
| * [SPOI-1459] - get recordings REST call returning null values |
| * [SPOI-1497] - Change the names of nodes in hadoop to be just hostname with the domain name. |
| * [SPOI-1498] - StreamList widget has "Name" for stream id |
| * [SPOI-1499] - CPU usage on operatorList widget is routinely over 100%, sometimes 900%? |
| * [SPOI-1548] - Logical DAG - Partition Count |
| * [SPOI-1555] - Logical DAG: Partition-Count Metric |
| * [SPOI-1563] - Moving average should be computed based on endWindow timestamps |
| * [SPOI-1570] - Update sandbox to support Hadoop 2.2 and DataTorrent 0.9.x |
| * [SPOI-1572] - gateway automatic update via websocket throws errors |
| * [SPOI-1590] - Datanode does not successfully connect to cluster after initial startup |
| * [SPOI-1614] - Front files not included during builds after migration to Malhar |
| |
| |
| ### Improvement |
| * [SPOI-1005] - CLI: create a default clirc that is controlled by the installer |
| * [SPOI-1099] - Display app master implementation version |
| * [SPOI-1110] - BufferServer Stats should be passed as part of the port stats? |
| * [SPOI-1305] - Support StringCodec for enums natively in Attributes |
| * [SPOI-1306] - Reduce the verbosity of the Context interface by changing the requirement for defaultValue argument for attrValue |
| * [SPOI-1421] - Improve default name for widgets |
| * [SPOI-1453] - Publish recently ended apps on WebSocket topics along with running app(s) |
| * [SPOI-1560] - "ERROR: mux is missing ..." messages emitted during "mvn test" on stram |
| * [SPOI-1574] - Document the additions to the config file specification |
| * [SPOI-1582] - Color non-empty filter fields for emphasis |
| * [SPOI-1583] - Limit default number of rows for physical operators |
| * [SPOI-1588] - Create service wrapper for datanode, namenode, nodemanager, resourcemanager, and historyserver |
| * [SPOI-1592] - Latency Measure Units |
| * [SPOI-1596] - Service wrapper for DTGateway |
| * [SPOI-1601] - Create meaningful names to the demos instead of providing entire path as the demo names |
| * [SPOI-1605] - Make default dashes into logical and physical for Instance |
| * [SPOI-1606] - Remove various metrics from logical operator list |
| |
| ### Task |
| * [SPOI-907] - Create Application DAG visualization |
| * [SPOI-1002] - List application names instead of class names where available |
| * [SPOI-1140] - Annotate dag visualization with stream throughout and other data |
| * [SPOI-1237] - Handle multi-container failure |
| * [SPOI-1281] - Remove TopN Widget from dashboard |
| * [SPOI-1282] - Update benchmarks |
| * [SPOI-1297] - Recover from "multiple container" failure |
| * [SPOI-1321] - Automate datanode provisioning |
| * [SPOI-1354] - Figure out a way to have charts be dramatically different for key combinations |
| * [SPOI-1360] - Create an automation framework for DataTorrent cloud provisioning |
| * [SPOI-1365] - Add ability to create, delete, and bootstrap multiple nodes with a single command |
| * [SPOI-1385] - Reduce memory used for Twitter demo containers |
| * [SPOI-1411] - Deprecate old Malhar webdemos once logstream is available |
| * [SPOI-1431] - GET /ws/v1/jars should return more meta information |
| * [SPOI-1432] - Implement /ws/v1/jars/{jarname}/meta |
| * [SPOI-1473] - Show main cluster stats in DT Console |
| * [SPOI-1477] - Automate namenode provisioning |
| * [SPOI-1478] - Recipe for Google Cloud provisioning |
| * [SPOI-1479] - Base configuration and provisioning |
| * [SPOI-1480] - Java install recipe |
| * [SPOI-1481] - Recipe to install maven |
| * [SPOI-1482] - Hadoop install recipe |
| * [SPOI-1484] - Open source UI to Malhar |
| * [SPOI-1518] - DT Console AngularJS Integration |
| * [SPOI-1519] - Logstream - sitestats UI integration |
| * [SPOI-1521] - Gateway to return error details upon status code 400s and 500s |
| * [SPOI-1523] - Run certification tests on 0.9.0 release build |
| * [SPOI-1524] - Automate performance benchmarking for different stream localities and event sizes |
| * [SPOI-1526] - Update GC stram-site.xml to reflect new version settings and customizations |
| * [SPOI-1527] - Automate stream locality perfromance benchmarking with the same tuple |
| * [SPOI-1541] - Create clirc compatible with demos |
| * [SPOI-1545] - Add documentation to chef repository |
| * [SPOI-1553] - DataTorrent performance testing on Google Cloud |
| * [SPOI-1554] - Create recipe for datatorrent local install |
| * [SPOI-1589] - Create cluster creation helper script for gc-nodes |
| * [SPOI-1591] - Create automatic installer for webapps |
| * [SPOI-1496] - Website framework selection |
| * [SPOI-1602] - Documentation index update to include Gateway API and Scalability Doc |
| * [SPOI-1607] - Update docs for 0.9.1 release |
| |
| ### Sub-task |
| * [SPOI-1447] - Show stream throughput |
| * [SPOI-1449] - DAG Visualization - Latency. Show latency per path/compute |
| * [SPOI-1450] - Show partitionable operators |
| * [SPOI-1464] - Allow operator to request/trigger repartitioning |
| * [SPOI-1466] - DB lookup for MongoDB |
| * [SPOI-1474] - Support operator specific stats in STRAM decisions |
| * [SPOI-1485] - Add license headers to all relevant files |
| * [SPOI-1486] - Move custom npm modules to node_modules folder, update npm registry |
| * [SPOI-1487] - Update .travis.yml in malhar to also build and test Front |
| * [SPOI-1488] - Create widget for displaying cluster info |
| * [SPOI-1490] - Expose gateway API calls for getting cluster information |
| * [SPOI-1529] - Understand exactly-once semantics. |
| * [SPOI-1530] - Set-up cluster on local VM |
| * [SPOI-1531] - Understand existing AdsDimension App |
| * [SPOI-1532] - Develop similar app with at-least once semantics as AdsDimension App |
| * [SPOI-1534] - Setup on Morado cluster for bench-marking |
| * [SPOI-1593] - Move front folder into Malhar repository |
| * [SPOI-1594] - Define code style standards, integrate a hinter |
| * [SPOI-1595] - Create tests for more js files |
| |
| ### GitHub - DataTorrent/Malhar |
| * [335] - Fix issue316 issue317 pull |
| * [331] - fix #330 Logical DAG Metrics |
| * [330] - Logical DAG Metrics |
| * [327] - Create a cluster stats widget |
| * [326] - Refactored DB Lookup operators and Cache Manager |
| * [324] - Add UI to repository |
| * [321] - Merged malhar dev 0.9.1 to logstream |
| * [320] - Squashed commit of the following: |
| * [316] - Provide a property to set tuple size in the RandomWord input operator |
| * [315] - fixes #314 |
| * [314] - RedisOutputOperator Exception |
| * [310] - Fixing default redis listen ports to 6379. Fixes #309 |
| * [309] - Fix default redis listen port in webapps config.js |
| * [308] - fixes #307 |
| * [307] - Change the format for passing multiple redis instances during partition |
| * [305] - Fix github 278 and 283 |
| * [304] - Added DataCache, JDBCBasedCache Operator and its test case #303 |
| * [303] - [DB Lookup & Caching] Create an operator which caches database results in memory for specified time |
| * [301] - Removing unused dependency and changing default daemon port to 9090. |
| * [300] - Update default settings for webapps |
| * [299] - A stream app to benchmark kafka partitionable input operator |
| * [287] - Use Yammer Metric to tick the kafka consumer throughput |
| * [284] - Squashed commit of the following: |
| * [283] - Monitor throughput for each kafka consumer |
| * [282] - Github 278 fix the Simple Kafka Consumer performance |
| * [278] - A performance bug in Simple kafka consumer |
| * [273] - Optimizing Redis Operator for inserting Maps |
| * [264] - Added DataCache, JDBCBasedCache Operator and its test case |
| * [262] - Marking operators non-partitionable |
| * [260] - Figure out a way to have charts be dramatically different for key combinations in Machine Demo |
| * [245] - Kafka Benchmark |
| * [159] - Squashed commit of the following: |
| * [147] - HighLow is not parameterized |
| * [130] - CouchDB Output Adaptar |
| * [289] - DimensionTimeBucket*Operator Usage and Issues |
| * [252] - Web Apps - Karma Unit Tests |
| * [251] - Machine Data Demo - Lookback Field Validation |
| * [250] - Mobile Locator Demo - Phone Range Support |
| * [232] - Show ingestion rate on MachineDemo UI |
| * [104] - Map Reduce Monitor Application |
| * [20] - Can not serialize Configuration Object |
| |
| |
| Version 0.9.0 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Improvement |
| * [SPOI-437] - Create more granular RESTful API calls in Stram for various entities in an instance |
| * [SPOI-446] - Compute aggregate values for application instance on server side |
| * [SPOI-465] - Increase unit test coverage to at least 50% |
| * [SPOI-683] - Document Active MQ operatos in library/io/ActiveMQ* |
| * [SPOI-1038] - Support more options in launch command |
| * [SPOI-1127] - Standardize "subscribe" commands for all models and collections |
| * [SPOI-1130] - Gateway to log requests and response (like access log in apache) |
| * [SPOI-1150] - Create new mrcolor instance on init of topN visualization |
| * [SPOI-1152] - CLI: add -v flag for debug logging |
| * [SPOI-1154] - Provide physical plan connections via the Daemon API |
| * [SPOI-1163] - Refactored pages and routing mechanism |
| * [SPOI-1187] - Allow 50% width for adjacent widgets |
| * [SPOI-1188] - Put widely used UI components in node_modules/datatorrent |
| * [SPOI-1224] - CLI Help enhancement |
| * [SPOI-1232] - Allow a way to assign a default value to an attribute while defining the attribute |
| * [SPOI-1295] - Make UI compatible with Backbone v1.1.0 |
| * [SPOI-1333] - Show stream locality in Streams widget |
| |
| ### New Feature |
| * [SPOI-66] - App wide "do not autorecord" parameter |
| * [SPOI-90] - Define and implement Apache server adapter node |
| * [SPOI-117] - Boolean operator node |
| * [SPOI-184] - Dashboard to have a page for one streaming application |
| * [SPOI-389] - Parent jira for "alerts" as a product |
| * [SPOI-444] - Provide a cascade merge unifier feature. |
| * [SPOI-711] - Ads Dimension Time Bucket Missing Key |
| * [SPOI-847] - Introduce Operator in Operator optimization (ThreadLocal) |
| * [SPOI-916] - Create upload jar file widget |
| * [SPOI-945] - Add ability to specify application properties, DAG, operator and port attributes in stram configuration file. |
| * [SPOI-1007] - Run tests with headless browser for CI |
| * [SPOI-1148] - Add CPU percentage to Operator List and Operator View |
| * [SPOI-1213] - Map Reduce Debugger - AngularJS - WebSocket Integration |
| * [SPOI-1227] - Map Reduce Debugger - JSHint |
| * [SPOI-1248] - DAG Visualization - Physical View - Color Coding Containers |
| * [SPOI-1323] - Pre-Deployment hook for Operator |
| * [SPOI-1332] - Logical Plan Update According to new Daemon API |
| * [SPOI-1345] - Add Alert Page - Application Name |
| * [SPOI-1377] - Create central package for language items |
| * [SPOI-1398] - Update DAG View Tool to Daemon API 0.9 |
| * [SPOI-1401] - Add AUTO_RECORD attribute on the operator level |
| * [SPOI-1402] - Add capability to look up past alerts by timestamp |
| * [SPOI-1235] - Add support for "DB lookup" functionality |
| |
| ### Bug |
| * [SPOI-440] - Change throughput calculation to use endWindow time |
| * [SPOI-727] - Document demo for charts |
| * [SPOI-904] - Tupleviewer: jump to valid index greater than total - visible fails |
| * [SPOI-992] - Javascript operator scriptBinding not serializable |
| * [SPOI-1037] - Daemon is creating a lot of threads that are not doing anything. Need to investigate and fix. |
| * [SPOI-1107] - Send the Recording information as part of appropriate OperatorStats or PortStats |
| * [SPOI-1164] - Mark non-partitionable operators on malhar |
| * [SPOI-1186] - CSS issue when a page is not using dash manager |
| * [SPOI-1189] - Misbehaving Mobile Locator Demo |
| * [SPOI-1195] - DAG Explorer POC |
| * [SPOI-1219] - Thread-Local streams should be marked as "inline" |
| * [SPOI-1220] - Create a certification test for adsdimension demo |
| * [SPOI-1221] - CLI should read configuration on app launch, not on start |
| * [SPOI-1249] - duplicate sinks in streams in physical plan returned by web service |
| * [SPOI-1283] - Fix mobile demo configuration in customer environment |
| * [SPOI-1290] - Support hadoop.socks.server setting for dtcli |
| * [SPOI-1298] - change product and package name and bump version |
| * [SPOI-1299] - Documment current DT Phone Home data |
| * [SPOI-1307] - Implement TupleRecorderCollection.deactivated method |
| * [SPOI-1308] - Document how to configure applications |
| * [SPOI-1318] - Logical DAG Display Issue in Firefox |
| * [SPOI-1324] - MobileDemo: NullPointerException in StramChild |
| * [SPOI-1327] - AtLeastOnceTest.testInlineOperatorsRecovery intermittent failure |
| * [SPOI-1328] - Container not released and no operators running in it |
| * [SPOI-1343] - Remove recordingNames from heartbeat |
| * [SPOI-1349] - support set-pager command in dtcli |
| * [SPOI-1351] - Need submit button and hour glass on machine data |
| * [SPOI-1368] - Recording never loads in tupleviewer |
| * [SPOI-1369] - Tests failing with new Daemon API changes |
| * [SPOI-1371] - Add Alert Page fails |
| * [SPOI-1374] - Drop twitter demo app from sandbox launch-demo jar |
| * [SPOI-1386] - If I click on a "finished" task, the left top nav bar says "widget" instead of "applications" |
| * [SPOI-1387] - If I click on a "finished" task, the left top nav bar says "widget" instead of "applications" |
| * [SPOI-1391] - CPU percentage for operators are 100x too large |
| |
| |
| ### Task |
| * [SPOI-634] - Enable unit tests in CI |
| * [SPOI-716] - Compare original javascript with new script operators (Python, Bash) |
| * [SPOI-720] - Coding conventions for Malhar GitHub |
| * [SPOI-817] - Implement Log Input Operator |
| * [SPOI-818] - Implement Parser Operator |
| * [SPOI-824] - Document Daemon API |
| * [SPOI-852] - Add life time per app on main dashboard |
| * [SPOI-853] - Create Pie Chart widget based on D3.js |
| * [SPOI-854] - Create Bar Chart widget based on D3.js |
| * [SPOI-856] - Create Histogram (Real-Time Traffic) widget based on D3.js |
| * [SPOI-857] - Site Stats Back-End Front-End Integration |
| * [SPOI-861] - Redis Client with Node.js for Site Stats |
| * [SPOI-862] - REST API with Node.js for Site Stats |
| * [SPOI-907] - Create Application DAG visualization |
| * [SPOI-954] - Publish Platform REST API |
| * [SPOI-998] - Throughput computations should be controllable via attributes per application |
| * [SPOI-999] - Throughput computations should be controllable per operator |
| * [SPOI-1002] - List application names instead of class names where available |
| * [SPOI-1033] - Implement OiO validations |
| * [SPOI-1062] - Build out Google Cloud cluster |
| * [SPOI-1066] - Design Versioning Scheme for forward and backward compatiblity |
| * [SPOI-1086] - Dynamically scale up and down input Operators |
| * [SPOI-1131] - Create Training Sessions |
| * [SPOI-1133] - Create Training Session 2 |
| * [SPOI-1134] - Create Training session 3 |
| * [SPOI-1139] - Productize DAG visualization |
| * [SPOI-1142] - Build a fraud detection poc |
| * [SPOI-1143] - Implement new Daemon API |
| * [SPOI-1159] - Conform to new Daemon API for REST and WebSocket |
| * [SPOI-1160] - Move REST logic from DataSource to classes |
| * [SPOI-1165] - Create a certification test for performance demo |
| * [SPOI-1166] - Create a certification test for twitter demo |
| * [SPOI-1168] - Create a certification test for mobile demo |
| * [SPOI-1193] - replace web socket client in tuple recorder with async web socket client |
| * [SPOI-1194] - remove "sync" from hdfs part file collection and implement web socket update of the newest part file |
| * [SPOI-1197] - DAG View - Application Logical Plan as Standalone Java Application/Maven Plugin |
| * [SPOI-1198] - DAG View - DAG Visualization from JSON file |
| * [SPOI-1199] - DAG View - DAG Visualization from properties file |
| * [SPOI-1207] - Common Apps Template |
| * [SPOI-1210] - Map Reduce Debugger - AngularJS Directives |
| * [SPOI-1216] - Benchmark performance benchmark with stream locality |
| * [SPOI-1222] - CLI enhancements |
| * [SPOI-1223] - CLI enhancements |
| * [SPOI-1226] - Create list of platform attributes - port, operator, stream, application |
| * [SPOI-1239] - Names for deamon and cli |
| * [SPOI-1247] - Update UI for change in logicalPlan format |
| * [SPOI-1256] - Make Machine Data Demo scalable |
| * [SPOI-1262] - Parent JIRA for "face-lift demo apps for Hadoop World" |
| * [SPOI-1265] - Web Demos - Common Application |
| * [SPOI-1268] - Document all the attributes in Documentation (Guides) |
| * [SPOI-1276] - Customer demo build tools |
| * [SPOI-1277] - Add performance demo to customer apps |
| * [SPOI-1279] - In Stream List view show DataLocality |
| * [SPOI-1282] - Update benchmarks |
| * [SPOI-1285] - keys in instanceinfo widget to be shortened |
| * [SPOI-1286] - Add GB used to instanceOverview widget |
| * [SPOI-1287] - Rename operator names and stream names on AdsCustomerApplication |
| * [SPOI-1294] - Bump up the disk space in sandbox |
| * [SPOI-1309] - Update Operations and Installation Guide by adding a section on parameter setting |
| * [SPOI-1334] - Create data for Scalability White paper |
| * [SPOI-1335] - Create Scalability White paper |
| * [SPOI-1336] - Create First cut of logstream white paper |
| * [SPOI-1337] - Create first cut for Gateway white paper |
| * [SPOI-1339] - Productize LogStream white paper |
| * [SPOI-1340] - Write Scalalability and Design Pattern white paper |
| * [SPOI-1347] - Redis periodic cleanup node8 |
| * [SPOI-1348] - Add elapsed time to demo throughput widget |
| * [SPOI-1352] - Need urls on Machine data demo to suppoer key combos |
| * [SPOI-1353] - Machine data demo UI should show which keys are not selected |
| * [SPOI-1356] - Make default look back on machine gen demo - 180 minutes |
| * [SPOI-1362] - Set up separate Redis instance and application for Machine demo |
| * [SPOI-1363] - gateway API recordings retrieval minor parameter name change |
| * [SPOI-1366] - Reduce scope of dependency versions in Front |
| * [SPOI-1367] - Name and ports not showing up in recording list |
| * [SPOI-1372] - On dashboard change "Operations" to "Applications" in the first tab |
| * [SPOI-1373] - Sandbox: size should be 8G by default, 4G causes most apps to not work |
| * [SPOI-1375] - All sandbox apps must work in 8G VM. Need to test each |
| * [SPOI-1381] - Add link to Malhar open source project in Sandbox |
| * [SPOI-1382] - Add MachineCustomerApplication to launch-customer jar on node1 |
| * [SPOI-1399] - Add two new docs to the 0.9 release, to the website |
| * [SPOI-1400] - Add timestamp for each tuple in tuple recordings |
| |
| ### Sub-task |
| * [SPOI-1109] - Change the frontend code to so that the recording information is received as part of the stats |
| * [SPOI-1135] - Beautify "Twitter Top URLs" demo |
| * [SPOI-1156] - mocha-phantomjs fails on test |
| * [SPOI-1228] - Benchmark Performance with stream locality as thread-local |
| * [SPOI-1229] - Benchmark Performance with stream locality as process local |
| * [SPOI-1230] - Benchmark Performance with stream locality as node-local |
| * [SPOI-1231] - Benchmark Performance with stream locality as rack-local |
| * [SPOI-1240] - Create Hadoop 2.2 cluster on GC |
| * [SPOI-1241] - Migrate morado cluster to Hadoop 2.1 |
| * [SPOI-1263] - Beautify "Mobile" demo app |
| * [SPOI-1264] - Beautify "Machine Generated" demo app |
| * [SPOI-1272] - Investigate Google Cluster Disks, Networking, Firewalls |
| * [SPOI-1273] - Install Chef Server for central provisioning |
| * [SPOI-1274] - Twitter demo - small changes in backend |
| * [SPOI-1275] - Mobile demo - small changes in backend |
| * [SPOI-1278] - Configure and install node0 |
| * [SPOI-1304] - Migrate core to Hadoop 2.2 |
| * [SPOI-1322] - Configure and install datanode |
| * [SPOI-1379] - Configuration changes to support Hadoop 2.2 |
| * [SPOI-1380] - Fix customer demos creation for Hadoop 2.2 |
| * [SPOI-1394] - Enable testing during build for Malhar |
| * [SPOI-1395] - Enable build testing for front |
| * [SPOI-1396] - Enable build testing from Core |
| |
| ### GitHub - DataTorrent/Malhar |
| * [288] - Machine Demo Circular Keys. Squashed commit of the following: |
| * [286] - Ads Dimensions Demo - REST API |
| * [285] - Machine Data Demo - No Data Behavior |
| * [281] - Machine Data Demo - Circular Keys Retrieval Unit Test |
| * [280] - Machine Data Demo - Retrieving Circular Keys from Redis |
| * [276] - Make topic and brokerList configurable properties for kafka input operator |
| * [275] - fixed github issue #273 |
| * [274] - Couch operators 0.9 |
| * [267] - [logstream] add logstream usecases |
| * [263] - Backend support to add a range of mobile numbers |
| * [261] - Github issues #260 and #240 |
| * [259] - Make benchmarks appear more friendly in application list |
| * [258] - Ads Dimensions Demo. Squashed commit of the following: |
| * [257] - Machine and Ads Demos Common Styling |
| * [256] - SPOI-1408 added AsyncHttpClient.close() calls |
| * [255] - Github 240 |
| * [254] - Ads Dimensions Demo - Redis Unit Test |
| * [253] - Port Ads Dimensions Demo to AngularJS |
| * [249] - Added support to mobile app for adding range of numbers at a time #242 |
| * [248] - Remove reload button from MachineData demo |
| * [247] - Web Apps Upgrade to Daemon API 0.9. Machine Demo optimizations. Squashed commit of the following: |
| * [246] - Documentation had a typo which was repeated. I fixed it and also reorganized imports. |
| * [243] - Web Apps - Upgrade to Daemon API 0.9 |
| * [241] - Add ads demo to common demo UI |
| * [240] - Make the keys in Machine Data circular |
| * [239] - Machine Demo - Negative Values in Random Data |
| * [238] - Machine Demo - Load Indicator on Slow Response |
| * [237] - Machine Demo - Instant Reload on Dimensions Change |
| * [236] - Machine Demo - Line Chart Options for Empty Chart |
| * [235] - Machine Demo - Server Polling Statistics |
| * [231] - fixed github issue #220 |
| * [230] - Machine Demo - Error Handling |
| * [228] - Machine Demo - Redis Query Optimization |
| * [227] - Machine Demo - Client Cache for Chart Data |
| * [226] - fixed github issue 219 |
| * [225] - 0.9 migration |
| * [224] - Show hourglass on the machine data demo in case of network (internet) being slow |
| * [223] - Setting Expiry Date for Redis Keys in RedisOutputOperator |
| * [222] - Couch Input Output Adaptors |
| * [221] - Web Apps - AngularJS Directive for Google Line Chart |
| * [220] - Making Redis Operator partionable |
| * [219] - Setting Expiry Date for Redis Keys in RedisOutputOperator |
| * [218] - made output port of alert escalation autorecord |
| * [217] - Add credit card fraud detection demo |
| * [216] - Squashed commit of the following: |
| * [215] - Machine demo should have 180 minutes as default in UI |
| * [211] - Fraud App Rename |
| * [210] - Web Apps - Fraud |
| * [209] - 0.9 pull |
| * [208] - Web Apps - Elapsed Time |
| * [207] - 0.9 migration |
| * [206] - SimpleMovingAverage is resetting the second last window state |
| * [205] - Fix PythonOperator |
| * [204] - Fix BashOperator |
| * [203] - Marking operators non-partitionable |
| * [202] - added new output port in JsonByteArrayOperator for emiting flattened map |
| * [201] - #198 Machine Data Demo - Dynamic Dimensions. Squashed commit of the following: |
| * [200] - added new output port in JsonByteArrayOperator for emiting flattened map |
| * [199] - making the attributes configurable from stram-site |
| * [198] - Machine Data Demo - Dynamic Dimensions |
| * [197] - Web Apps Machine Demo Update/Multiple FIxes |
| * [196] - Mobile Demo corrections |
| * [195] - Web Apps Machine Demo Update/Multiple Fixes |
| * [194] - Web Apps - Machine Data Demo - Last Minute Calculations |
| * [193] - reduced the I/O worker thread multiplier to 1 as default in ning AHC |
| * [192] - Web Apps - Express Version |
| * [191] - Web Apps - Browser Dependencies |
| * [190] - 0.9 migration |
| * [188] - Web Apps (Demos) - Squashed commit of the following: |
| * [186] - Web Apps - Styling |
| * [185] - Web Apps - Readme |
| * [184] - made the machine data and ads demo scalable |
| * [183] - Web Apps - JSHint |
| * [182] - Fixed seeds phone generator and other bugs with MobileDemo #172 |
| * [180] - suppress warning for stdout in console output operator |
| * [178] - Web Apps - License Headers |
| * [177] - Twitter demo application is called TwitterDevApplication |
| * [176] - Removed phone.html that is not being used. Fixes #175. |
| * [175] - Remove phone html that is not being used |
| * [174] - Resolving mobile demo bug |
| * [173] - bug in phonemovement |
| * [169] - Code format changes to KryoSerializableStreamCodec |
| * [168] - Web Apps - Demos Descriptions |
| * [167] - javascript operator now serializable |
| * [165] - Squashed commit of the following: |
| * [164] - Operator and stream names of adsdimension application can be clearer |
| * [163] - Web Apps - Mobile Locator Demo - Google Maps Marker Labels |
| * [162] - Web Apps - Machine Data Demo - CPU/RAM/HDD Gauges |
| * [161] - Web Apps - Machine Data Demo - Device ID Dimension |
| * [158] - Made Mobile enhancements #157 |
| * [157] - Mobile Demo: Minor enhancements |
| * [156] - Web Apps - Index Page |
| * [155] - Web Apps - Machine Data Demo |
| * [154] - Web Apps - Mobile Locator Demo |
| * [151] - change name and jump version |
| * [150] - Twitter multiplier variance #148 |
| * [149] - Checkpoint consumer offset #146 |
| * [148] - Twitter demo: change tweet multipler from 100 to Random(90-110) |
| * [146] - Commit offset at checkpoint for kafka input operator |
| * [145] - Web Apps POC |
| * [144] - MachineData: bug in MachineInfoAveragingOperator |
| * [143] - migrated to ning for web socket interface provider |
| * [142] - 0.4 migration |
| * [141] - upgraded framework version to 0.3.5 |
| * [140] - method rename in RedisOutputOperator #138 |
| * [139] - Add partitionable kafka input operator #113 |
| * [138] - Change selectDatabase in RedisOutputOperator to setDatabase |
| * [137] - Couch DB Output adapter added #130 |
| * [136] - Reducing tuple blast size #135 |
| * [135] - MachineData: reduce the number of tuples generated by the random generator |
| * [134] - Mapreduce Pull Request |
| * [133] - [logstream] logstream app flow |
| * [132] - Bug in UniqueKeyValCounter |
| * [131] - Deleted old unique value count operator #129 |
| * [129] - Delete the old UniqueValueKeyVal operator and rename the new UniqueValueCount |
| * [128] - Corrections to UniqueValueCount #127 |
| * [127] - UniqueValueCount improvements |
| * [126] - Changed deprecated call to setInline to setLocality. |
| * [125] - Changed setInline call to setLocality |
| * [123] - Clean up unused libraries |
| * [122] - squashed changed to UniqueCount #112 |
| * [121] - Operator that counts unique value per key #112 |
| * [120] - add all the locality modes to performance test |
| * [119] - Squashed changes and fixed version #99 |
| * [118] - MachineData integrated operators #99 |
| * [117] - Adding Redis input operators #43 |
| * [116] - Logstream merge 43 |
| * [115] - Adding Redis input operators |
| * [114] - Implement the automate partitioned kafka input operator(1:1) #113 |
| * [113] - Add partitionable kafka input operator (1:1) |
| * [103] - move the test residue to maven target directory so that mvn clean can clean all the files |
| * [102] - Sqaushed changes for KryoSerializableStreamCodec & Test #95 |
| * [101] - Pull request for #95 |
| * [100] - Pull request for issue #95 |
| * [99] - Add more statistics calculations to machine data |
| * [98] - Issue 95 |
| * [97] - Add machinedata demo |
| * [96] - Added output port to json byte array operator to emit JSONObjects too |
| * [95] - Create an Abstract SteamCodec which can be used for custom partitioning and uses kryo serialization |
| * [94] - added two output ports - outputMap, outputJSonObject |
| * [93] - Upgrade Kafka to 0.8 |
| * [88] - starmcli not taking the jar passed during launching application |
| * [73] - Upgrade our kafka operator api to be compatible with mvn released Kafka version 0.8 |
| * [70] - Clean kafka package in malhar library #53 |
| * [52] - addCombination doesn't work in DimensionTimeBucketOperator |
| * [46] - SiteOps Dashboard Look and Feel |
| * [45] - Move SiteOps Demo to Node.js |
| * [44] - Sliding window aggregation computations operators |
| * [43] - Create Redis Input Operator |
| * [42] - Please pull my latest changes |
| * [41] - Web Demos - License Headers |
| * [40] - Move Ads Dimensions Demo to Node.js |
| * [39] - AlertEscalationOperatorTest test failing |
| * [38] - Move Mobile Demo to Node.js |
| * [37] - Move Twitter Demo to Node.js |
| * [36] - Move Twitter, Mobile and Ads Demo to Node.js - fixes #34 |
| * [35] - Create AMQP Input Operator |
| * [34] - Move Twitter, Mobile and Ads Dimensions Demo to Node.js |
| * [33] - Ads Dimensions Demo - Common Template |
| * [32] - Common Assets for the Demos |
| * [31] - Migrate deprecated setInline calls. fixes #16 |
| * [30] - bump master version to 0.3.5-SNAPSHOT |
| * [29] - Javascript operator scriptBinding not serializable |
| * [28] - Node.js Ads Dimensions Demo - "Play" mode |
| * [27] - Node.js Ads Dimensions Demo - Node.js Daemon |
| * [26] - Node.js Ads Dimensions Demo - Dynamic Port |
| * [25] - Node.js Ads Dimensions Demo - Readme |
| * [24] - Node.js Ads Dimensions Demo Rename |
| * [23] - Mrapplication |
| * [21] - keyvalpair doing hash on key and value |
| * [18] - 404 error in the UI for logicalPlan |
| * [17] - Emitted tuples by InputOperator incorrectly shown as zero |
| * [16] - Adapt new stream locality API |
| * [11] - Create an alerts demo for testing alerts |
| * [4] - Make alert throttle operator window id based instead of wall clock time. |
| |
| |
| Version 0.3.5 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-349] - InlineStream undeploy error |
| * [SPOI-766] - Process for certification of 0.3.3 launch on demo server |
| * [SPOI-953] - Implement Exactly Once |
| * [SPOI-959] - Operator Properties Widget |
| * [SPOI-976] - Mobile apps keep dying |
| * [SPOI-1013] - Cluster clocks are not in sync |
| * [SPOI-1017] - Daemon stopped publishing over WebSocket |
| * [SPOI-1022] - Alert is removed from the list on server failure |
| * [SPOI-1023] - "Method Not Allowed" error when deleting an alert |
| * [SPOI-1030] - DT Phone Home throw NullPointerException |
| * [SPOI-1034] - 404 Not Found when getting alerts |
| * [SPOI-1035] - Cannot create alerts with stramcli |
| * [SPOI-1041] - Operator Properties fail to load |
| * [SPOI-1051] - URLs templating for REST API |
| * [SPOI-1075] - Container planned/alloc does not update in real time |
| * [SPOI-1076] - Buffer server read is a flat line |
| * [SPOI-1081] - Get common urls for all demos |
| * [SPOI-1087] - Fix container chart |
| * [SPOI-1089] - Widgets initializing twice |
| * [SPOI-1090] - Add d3 to package.json deps |
| * [SPOI-1091] - dashboard manager css issue with background color |
| * [SPOI-1092] - Widget list not rendering on initial page load |
| * [SPOI-1093] - Memory leak when switching between pages |
| * [SPOI-1097] - dashboards not being saved |
| * [SPOI-1101] - Explore Kibana 3 as UI option for site stats |
| * [SPOI-1103] - Kill button doesn't work on applist palette |
| * [SPOI-1104] - Mode switch separator in header still visible |
| * [SPOI-1105] - shutdown application command requires empty data object |
| * [SPOI-1114] - Test for AlertManagerTest.testAlertManager failed |
| * [SPOI-1118] - Upgrade grid to 4GB containers |
| * [SPOI-1121] - URL for deleting alert is incorrect |
| * [SPOI-1123] - Add alert action not sending all parameters |
| * [SPOI-1126] - Latency is whacked after adding alerts |
| * [SPOI-1137] - DAG Page View - JavaScript Error |
| * [SPOI-1144] - stram-site properties not applied to the operators (possibly?) |
| * [SPOI-1145] - ConcurrentModificationException when used launch local in stramcli |
| * [SPOI-1147] - Cannot kill "running" or relaunch "killed" application instance from AppLIst widget |
| * [SPOI-1157] - NoSuchElementException in stramcli |
| |
| ### Epic |
| * [SPOI-870] - Alerts parent jira for phase I |
| |
| ### Improvement |
| * [SPOI-462] - Feature to specify that an operator cannot be partitioned. |
| * [SPOI-523] - Document OS style guidelines |
| * [SPOI-750] - Performance Metrics Widget Unit Test |
| * [SPOI-937] - Incorporate jQuery UI into current dashboard |
| * [SPOI-962] - Daemon REST API Conventions |
| * [SPOI-963] - For exactly-once pick the correct checkpoint for recovery |
| * [SPOI-979] - Need old versions of javadoc available online |
| * [SPOI-983] - Front-End Release Branch |
| * [SPOI-1019] - switch to jquery ui tooltip |
| * [SPOI-1028] - Correctly order output port list for operator AlertThreeLevelTimedEscalationOperator |
| * [SPOI-1078] - Add appInfo to apps.list WebSocket topic |
| * [SPOI-1094] - Subscribe to app-specific websocket topic while on application |
| * [SPOI-1102] - Alerts should be a list of js object literals |
| * [SPOI-1106] - tables should resize as widget resizes by default |
| * [SPOI-1119] - Remove Malhar subtree dependency from Core |
| * [SPOI-1125] - Utilize new "optional" attribute with alert template parameters |
| |
| ### New Feature |
| * [SPOI-641] - Setting partition on operator not supporting partitioning must result in error. |
| * [SPOI-810] - Create a line charting module using d3.js |
| * [SPOI-831] - Create REST API call for uploading JAR files |
| * [SPOI-832] - Create REST API call to retrieve all uploaded jar files |
| * [SPOI-834] - Create "Top N" widget |
| * [SPOI-874] - Provide description information for filter/throttle/action classes to be used by UI |
| * [SPOI-940] - Alert List Widget - Actions (Add/View/Delete) |
| * [SPOI-987] - Add stramcli tab completion for alias and macro |
| * [SPOI-989] - Operator Properties Widget - Data Access Logic |
| * [SPOI-1011] - AlertModel - Delete |
| * [SPOI-1012] - Mock Node.js Server - Alert Delete |
| * [SPOI-1014] - AlertCollection - Retrieve |
| * [SPOI-1016] - AlertModel - Create |
| * [SPOI-1018] - GitHub issue exporter for changelog |
| * [SPOI-1024] - AlertModel - Unit Test |
| * [SPOI-1025] - AlertCollection - Unit Test |
| * [SPOI-1026] - OpPropertiesModel - Unit Test |
| * [SPOI-1029] - Unit Testing - FakeXMLHttpRequest |
| * [SPOI-1040] - Node.js back-end for Ads Dimensions Demo |
| * [SPOI-1046] - Templatized Alert Creation Page |
| * [SPOI-1055] - Node.js Redis Client (Ads Dimensions Demo) |
| * [SPOI-1080] - Node.js Daemon |
| * [SPOI-1085] - Unified Demos Page |
| * [SPOI-1095] - Move Twitter Demo to Node.js |
| * [SPOI-1096] - Move Mobile Demo to Node.js |
| * [SPOI-1111] - Get common urls for all demos - POC |
| * [SPOI-1112] - Shutdown command from UI |
| * [SPOI-1116] - Move Machine Gen Demo to Node.js |
| |
| |
| ### Task |
| * [SPOI-843] - Evaluate current market features |
| * [SPOI-846] - Implement exactly once |
| * [SPOI-855] - Dynamic re-partitioning happens on a snapshot, need to do a moving average by default |
| * [SPOI-895] - Implement REST API filterClasses for Alerts |
| * [SPOI-896] - Implement REST API escalationClasses for Alerts |
| * [SPOI-932] - CLI: support alert operations |
| * [SPOI-941] - Include latency in operator list in the dashboard. |
| * [SPOI-955] - Demo showing recurring payment check |
| * [SPOI-956] - Demo application for Machine (appliance) generated data monitoring resource usage (resources like CPU, RAM, etc) |
| * [SPOI-972] - Adapt core versioning scheme |
| * [SPOI-973] - Setup node.js as a supported technology |
| * [SPOI-982] - Version Eclipse settings |
| * [SPOI-986] - Version Nebeans settings |
| * [SPOI-995] - Unit test for AlertsManager |
| * [SPOI-996] - Unit test for AlertEscalation Operator |
| * [SPOI-1001] - Create a convenience script to start and stop hadoop cluster |
| * [SPOI-1008] - Cluster configuration versioning |
| * [SPOI-1009] - Need a signup link to Google group on Malhar first page |
| * [SPOI-1010] - Notificaitons on Malhar to include Google group |
| * [SPOI-1015] - Evaluate anomaly algorithms |
| * [SPOI-1021] - Set up user communication process |
| * [SPOI-1031] - Hide operations/development mode switch in UI |
| * [SPOI-1039] - Grid access |
| * [SPOI-1052] - Update node1 with latest release (0.3.4) |
| * [SPOI-1053] - Add users to grid |
| * [SPOI-1056] - Add create alert template REST call |
| * [SPOI-1057] - Change create alert REST call to use a template and parameters |
| * [SPOI-1058] - Retrieve alert REST call should include template name and parameters |
| * [SPOI-1060] - Get rid of stramRoot from the REST API (back end) |
| * [SPOI-1061] - Get rid of stramRoot from the REST API call (front end) |
| * [SPOI-1064] - Make machine generated data demo generic and launch on node1 |
| * [SPOI-1070] - Dashboard look and feel for demos |
| * [SPOI-1071] - Dashboard look and feel for twitter firehose demo |
| * [SPOI-1072] - Dashboard look and feel for mobile demo |
| * [SPOI-1073] - Dashboard look and feel for ads dimensional demo |
| * [SPOI-1074] - Dashboard look and feel for machine gen demo |
| * [SPOI-1077] - Dashboard look and feel for logs demo |
| * [SPOI-1079] - Evaluate Integration with third party tools |
| * [SPOI-1100] - Update https://github.com/DataTorrent/Malhar |
| * [SPOI-1117] - Allow exactly-once with downstream at-most-once only |
| * [SPOI-1132] - Create Training session 1 |
| * [SPOI-1151] - AngularJS Integration |
| |
| ### Sub-task |
| * [SPOI-876] - Allow for saving alert configurations as templates for future use, back-end |
| * [SPOI-877] - Allow for saving alert configurations as templates for future use, front-end |
| * [SPOI-978] - Fix dependency among open source and platform |
| * [SPOI-1042] - create alert template model |
| * [SPOI-1047] - create new page for add alert and re-route current url |
| * [SPOI-1048] - create parameter fill-in fields |
| * [SPOI-1113] - Remove child module poc build from framework build |
| * [SPOI-1176] - Remove malhar subtree from Core |
| |
| ### GitHub - DataTorrent/Malhar |
| * [90] - Fixed generics usage and renamed operator classes as per convention |
| * [89] - Added operator to convert json byte stream to java hash map |
| * [87] - Added operator to convert json byte stream to java hash map |
| * [86] - Web Demos Update |
| * [85] - Web Demos - Architecture Documentation |
| * [84] - Adding TravisCI build status to README #83 |
| * [83] - Add TravisCI Build status to README |
| * [82] - Add TravisCI Integration |
| * [81] - [Logstream] Reads apache logs from RabbitMQ and prints basic aggregations to console |
| * [80] - Webdemos - round time to minute |
| * [79] - Adding apps project and logstream application skeleton. #62 |
| * [78] - SiteOps Demo - Totals Calculation |
| * [77] - Operator changes for issue #76 |
| * [76] - Remove hardcoded values from RabbitMQ input operator |
| * [75] - [Issue 52]: addCombination doesn't work in DimensionTimeBucketOperator |
| * [74] - AMQP input operator for logs with sample aggregations #35 |
| * [71] - Move Machine Generated Data Demo to Node.js |
| * [70] - Clean kafka package in malhar library #53 |
| * [69] - Pull request for issue #53 |
| * [68] - Web Demos Update |
| * [67] - SiteOps Demo - Redis Service |
| * [66] - Web Demos - Describe Configuration in Readme |
| * [65] - Ads Dimensions Demo - Redis Configuration |
| * [64] - Web Demos - Relative URLs for JS/CSS |
| * [63] - Adding apps project and logstream application skeleton. #62 |
| * [62] - Create skeleton application for log stream processing |
| * [60] - Move SiteOps Demo to Node.js - Unit Tests |
| * [59] - Move SiteOps Demo to Node.js - License Headers |
| * [58] - Move SiteOps Demo to Node.js - Charts |
| * [56] - Clean input operators in malhar-library |
| * [55] - Github 54 |
| * [54] - PubSubWebSocket operators tests should be self contained and not need other helpers. |
| * [53] - Clean up Kafka input/output operator |
| * [51] - Ads webdemo is showing a drop in the graphs at the end |
| * [50] - Link to webdemos in main readme |
| * [49] - Clean input operators in lib |
| * [47] - Ads Dimensions Demo - JavaScript Loading Issue |
| * [22] - Node.js Ads Dimensions Demo |
| * [15] - Cleanup malhar-library input operator packages |
| |
| |
| Version 0.3.4 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-569] - Ads demo charting is not stable; Needs cleanup |
| * [SPOI-570] - Mobile demo does not run forever |
| * [SPOI-599] - Sometimes in the metrics charts widget some charts are flat |
| * [SPOI-617] - Change error code from 500 to 400 for bad request |
| * [SPOI-630] - Stop recording action does not work |
| * [SPOI-660] - Recording shows a red light, but no recording is being done |
| * [SPOI-674] - AppModel and AppInstanceView need clean-up |
| * [SPOI-704] - Red dots show up in the dashboard, as if the operator is recording |
| * [SPOI-714] - Ads UI load is alow |
| * [SPOI-715] - Ads demo chart is 25 mins behind |
| * [SPOI-724] - Fix LocalFileInputOperator logic and documentation |
| * [SPOI-730] - Developer version for Mac OS X |
| * [SPOI-731] - Cannot record anymore. Tuple recorder is broken |
| * [SPOI-732] - Cannot stop recording |
| * [SPOI-742] - Installation/Compilation fails with maven-eclipse plugin |
| * [SPOI-744] - Facilitate the "accepted" app state for the app instance view |
| * [SPOI-747] - StreamingContainerManager.getContainerAgents() returns inactive containers |
| * [SPOI-748] - Build Kestrel as 3rd party library and depend on it instead of including the code with operator library |
| * [SPOI-751] - browser CPU utilization excessive with larger number of updates |
| * [SPOI-755] - Evaluate ads dimensional data demo |
| * [SPOI-756] - Account for single-item, non-list responses from web services |
| * [SPOI-757] - docs directory re-link for new version install |
| * [SPOI-762] - Application Model not updating correctly, causing graph issues |
| * [SPOI-764] - Malhar engine build is including StramAppMaster in engine jar in some cases |
| * [SPOI-765] - Datasource constructing tuple GET URL incorrectly for tupleviewer |
| * [SPOI-768] - Log file rotation support issue |
| * [SPOI-804] - If a node is bad the app master should choose a different node for the containers. |
| * [SPOI-806] - list-apps shows duplicate entries |
| * [SPOI-807] - downgrade jvm on node3 |
| * [SPOI-809] - Add pig distinct semantic operator to library. |
| * [SPOI-811] - shutdown app does not work when not connecting to an app |
| * [SPOI-842] - Generated javadocs are missing several packages |
| * [SPOI-845] - Installer test fails when executing from different directory |
| * [SPOI-849] - Start recording from port list |
| * [SPOI-863] - Display application-wide latency in Application View |
| * [SPOI-883] - StramDelegationTokenManager should not be started if security is not enabled |
| * [SPOI-888] - Account for delay in Stram initialization in recently launched apps |
| * [SPOI-890] - Stram unit test is creating events data in stram/stram |
| * [SPOI-892] - getAppInfo throws an exception during the beginning of the application |
| * [SPOI-902] - Tupleviewer filter by port fails |
| * [SPOI-903] - Tupleviewer preview of loaded tuples not rendering |
| * [SPOI-905] - Review/fix license headers in open source |
| * [SPOI-911] - Sync recording call fails with 500 error |
| * [SPOI-915] - Log collection tool for hadoop cluster |
| * [SPOI-918] - Partitioning stopped on MobileLocatorDemo instance 48 |
| * [SPOI-933] - Specify application names in stram configuration file for the customer demos on the cluster. |
| * [SPOI-946] - Certify Sandbox for Ubuntu |
| * [SPOI-951] - Auto completed file name does not work with launch command |
| * [SPOI-952] - Remove operator returns error about input stream still connected. |
| * [SPOI-958] - LocalFsInputOperator test is failing |
| * [SPOI-965] - UI: Make sure empty lists and single-item lists in JSON returned from Daemon is handled properly |
| * [SPOI-969] - UI: Stop recording fails with error saying some recording name was not found with operator undefined. |
| * [SPOI-971] - Aggregation function for application stats fails intermittently |
| * [SPOI-974] - PageLoaderView Unit Test |
| * [SPOI-981] - Ensure "ports" are in the operators of the logicalPlan response |
| * [SPOI-991] - Daemon: windowIds must be sent as strings |
| * [SPOI-997] - Compute throughput as rolling average |
| |
| |
| ### Improvement |
| * [SPOI-434] - Ability to subscribe to buffer server stream at next begin window |
| * [SPOI-450] - CLI: Allow wildcard in launch file name |
| * [SPOI-463] - The appMetrics widget should remember the choices on the graphs (on/off) |
| * [SPOI-470] - Provide allocated/planned container count |
| * [SPOI-614] - Better Error Handling/Error Pages |
| * [SPOI-620] - Allow recording from operator/port page |
| * [SPOI-624] - Normalize appid/appId opid/operatorId and similar discrepancies |
| * [SPOI-677] - Setup both Core and Front-End to run on local environment |
| * [SPOI-685] - Explore other charting libraries to replace custom chart module |
| * [SPOI-687] - Improve CSS for tupleviewer |
| * [SPOI-696] - Only update whole application collection on applist page |
| * [SPOI-712] - Explicit Errors when daemon provides data in unexpected format |
| * [SPOI-753] - Overview metrics for Port View should have comma separations |
| * [SPOI-835] - CLI needs to be able to handle spaces and escape characters |
| * [SPOI-844] - operatorClass REST call should also accept "chart" or "filter" for charting and alert filters |
| * [SPOI-901] - Add palette for containers list |
| * [SPOI-938] - Centralize reused templates for links |
| |
| ### New Feature |
| * [SPOI-70] - Parent JIra: Webservices for Streaming Application |
| * [SPOI-73] - Job completed report |
| * [SPOI-74] - Reporting: Job completion report should have the list of persisted files per node |
| * [SPOI-75] - Parameter to specify if persisted file names should be included in the job completion report |
| * [SPOI-127] - CLI: Throughput data (streams) |
| * [SPOI-129] - CLI: Streaming Ap Master data |
| * [SPOI-134] - Webservice: Stream data/throughput per stream in the application |
| * [SPOI-138] - Webservices: Data on committed windows |
| * [SPOI-139] - Webservice: Latency across the dag/application |
| * [SPOI-140] - Event Logging |
| * [SPOI-143] - Webservice: Bottleneck analysis |
| * [SPOI-219] - Evaluate if we need a naming scheme to identify a physical node of a logical dnode |
| * [SPOI-337] - Do an in-node stream by user choice |
| * [SPOI-403] - Design license file format |
| * [SPOI-417] - Licensing Specification |
| * [SPOI-460] - Create Preconfigured Sandbox |
| * [SPOI-471] - Stream View |
| * [SPOI-551] - Create Video for Recording Tuples |
| * [SPOI-628] - Web service for event logging |
| * [SPOI-698] - Add streamquery operators that support expression |
| * [SPOI-699] - Add pass through streamquery operators |
| * [SPOI-710] - DatatTorrent Wiki |
| * [SPOI-718] - Implement at most once for Operators with two or more connected input ports. |
| * [SPOI-719] - Implement Dag Validations for At-Most-Once |
| * [SPOI-738] - DT Phone Home Phase II |
| * [SPOI-829] - Calculate overall latency for application |
| * [SPOI-830] - Deduce critical path in DAG for application |
| * [SPOI-833] - operator's latency should be shown in the UI |
| * [SPOI-891] - Create a web service API for describing operator class |
| * [SPOI-906] - Add palette for ports list |
| * [SPOI-908] - Selecting recordings from recording list fails |
| * [SPOI-909] - CSS issues with tupleviewer when port name too long |
| * [SPOI-920] - Application DAG UI Integration |
| * [SPOI-921] - Sandbox icons |
| * [SPOI-934] - Alert List Widget - Date/Time |
| * [SPOI-936] - Alert List Widget - Dynamic Width |
| * [SPOI-949] - Documentation processor from markdown to html |
| |
| |
| ### Task |
| * [SPOI-60] - Streaming app master logging |
| * [SPOI-191] - Enable compression for end of window data or for throttled N tuples (N sent together) |
| * [SPOI-198] - Evaluate if buffer server should retain data in a compressed state |
| * [SPOI-199] - Evaluate if we need buffer server to keep data in compressed state |
| * [SPOI-425] - Checkpointing for exactly once operator recovery |
| * [SPOI-514] - Open Source Transition for Library/Contrib/Demos |
| * [SPOI-603] - Evaluation version for Mac OS X |
| * [SPOI-631] - Make a site operations video |
| * [SPOI-633] - Need changelog for new version |
| * [SPOI-656] - Performance metrics charts does not remember preferences |
| * [SPOI-681] - Added library description to index.html |
| * [SPOI-682] - Display Application DAG in UI (evaluation) |
| * [SPOI-706] - Allow app name to be specified in launch time config file |
| * [SPOI-708] - Set up demo procedures |
| * [SPOI-713] - Technical evaluation of UIs |
| * [SPOI-717] - Create launch macros on demo server |
| * [SPOI-734] - Push 0.3.3 to demo server |
| * [SPOI-735] - Launch the latest software on customer app server |
| * [SPOI-736] - Certify customer application server |
| * [SPOI-737] - Clean up redis history on customer apps |
| * [SPOI-740] - Updated documentation generation process |
| * [SPOI-745] - DT Phone Home Server side work |
| * [SPOI-754] - Create other user for node0 launches |
| * [SPOI-759] - Technical Evaluation |
| * [SPOI-760] - Format comments for ASF project for Malhar GitHub |
| * [SPOI-761] - Format comments in API |
| * [SPOI-769] - Fix sql DeleteOperator to make it pass thru. |
| * [SPOI-770] - Fix SelectOperator to make it pass thru. |
| * [SPOI-771] - Fix sql UpdateOperator code to make it pass thru. |
| * [SPOI-773] - Add seolect expression index. |
| * [SPOI-774] - Fix sql InnnerJoin operator join condition. |
| * [SPOI-775] - Fix sql Outer Join operator to merge left/right/full join sql semantic. |
| * [SPOI-776] - Add sql Having semantic operator to library. |
| * [SPOI-777] - Add sql select top operator semantic to library. |
| * [SPOI-778] - Add sql select between condition sematic to library. |
| * [SPOI-779] - Add sql select compound condition AND/OR semanticf to library. |
| * [SPOI-780] - Add sql in condition semantic to library |
| * [SPOI-781] - Add sql like condition semantic to library. |
| * [SPOI-782] - Add having coprae value semantic to library. |
| * [SPOI-783] - Add having condition interface to support sql having operator semantic. |
| * [SPOI-784] - Add sql select unary expression semantic to library. |
| * [SPOI-785] - Create sql binary expreesion index semantic in library. |
| * [SPOI-786] - Add sql select negate index semantic to library. |
| * [SPOI-787] - Add sql slect sum index semantic to library. |
| * [SPOI-788] - Add sql select string mid index semantic to library. |
| * [SPOI-789] - Add sql slect string len semantic to library. |
| * [SPOI-790] - Add sql select string upper/lower semantic to library. |
| * [SPOI-791] - Add sql round double semantic to library. |
| * [SPOI-792] - Add sql select rtound double semantic to library. |
| * [SPOI-794] - Add sql sleect count aggregate semantic to libarry. |
| * [SPOI-795] - Add sql select min/max fucntion semantic to library. |
| * [SPOI-796] - Add sql select first/last aggregate semantic to library. |
| * [SPOI-797] - Add sql sleect sum aggregate semantic to library. |
| * [SPOI-798] - Add pig group operator semanitc to library. |
| * [SPOI-799] - Add PIG filter operator semantic to library. |
| * [SPOI-800] - Add PIG cross operator semantic to library. |
| * [SPOI-801] - Add PIG split operator semantic to library. |
| * [SPOI-812] - Design wireframes for widget for SiteStats |
| * [SPOI-814] - Prototype UI dashboard change to facilitate "app store" paradigm |
| * [SPOI-815] - Add pig join(inner) semantic operator to library. |
| * [SPOI-816] - Add pig join(outer => left/right/full) semantic operator to library. |
| * [SPOI-820] - Implement Dimension Operator that allows custom explosion |
| * [SPOI-825] - Evaluate non-Hadoop Streaming Platforms IDE |
| * [SPOI-826] - Add pig order by operator semantic to library. |
| * [SPOI-827] - Add pig limit operator semantic to library. |
| * [SPOI-828] - Add pig stream operator semantic to library. |
| * [SPOI-836] - Design Alert-related API |
| * [SPOI-840] - Add spark add flat map function semantic to operator library. |
| * [SPOI-848] - Introduce ContainerLocal as replacement for Inline |
| * [SPOI-850] - Evaluate Hadoop IDE |
| * [SPOI-851] - Evaluate JavaScript data visualization libraries |
| * [SPOI-858] - Class loader support issue: org.fusesource.hawtbuf.UTF8Buffer.class |
| * [SPOI-859] - Evaluate Streaming Platform Back-End Development Workflow |
| * [SPOI-860] - Evaluate Node.js as Back-End for Site Stats Demo |
| * [SPOI-864] - Redesign REST for altered escalation approach |
| * [SPOI-865] - Create "List of Alerts" widget |
| * [SPOI-866] - Add DataSource methods for alert REST API |
| * [SPOI-879] - Create a data list of grid nodes |
| * [SPOI-881] - Create Compute-Local api |
| * [SPOI-882] - Add compute local api to documents |
| * [SPOI-884] - Update dhcp configuration on cluster nodes. |
| * [SPOI-885] - Create prototype with Node.js + Redis + REST for Site Stats |
| * [SPOI-886] - Implement remove logical operator |
| * [SPOI-889] - Alerts persistence with Backbone.js models |
| * [SPOI-893] - Externalize UI settings |
| * [SPOI-894] - Implement JavaScript Filter operator for Alerts |
| * [SPOI-910] - Node.js Mock Server and JSONP Cross-Domain Requests |
| * [SPOI-913] - Flesh out "kill app" command from the instance view |
| * [SPOI-914] - Alerts REST API Error Handling |
| * [SPOI-935] - Make patch for flawed start/stop recording mechanism |
| * [SPOI-939] - Remove references to unfinished features for 0.3.4 release |
| * [SPOI-943] - Certify Sandbox on various OS |
| * [SPOI-944] - Certify Sandbox on Mac |
| * [SPOI-947] - Certify Sandbox for Windows |
| * [SPOI-948] - Improvements in sandbox based on OS certification feedback |
| * [SPOI-957] - Update license header |
| * [SPOI-970] - Download latest G! documents |
| * [SPOI-985] - Macro argument expansion in stramcli |
| * [SPOI-1006] - Update @since tags for 0.3.4 |
| |
| ### Sub-task |
| * [SPOI-880] - Investigate options for sandbox environment |
| * [SPOI-922] - Sandbox default hadoop configurations |
| * [SPOI-923] - Sandbox demo script and application launcher |
| * [SPOI-924] - Sandbox size and performance optimizations |
| * [SPOI-926] - Sandbox documentation |
| * [SPOI-942] - Demo launch page |
| * [SPOI-990] - Add launch-demo macro to clirc during install |
| * [SPOI-1004] - End user license agreement updates |
| |
| ### GitHub - DataTorrent/Malhar |
| * [12] - Site Stats Operator and TopNOperator issues |
| * [10] - getTopN() function returns ArrayList in com.datatorrent.lib.util.TopNSort class |
| * [9] - Another Bug in offer(E e) function of com.datatorrent.lib.util.TopNSort.java |
| * [8] - Bug in offer(E e) function of com.datatorrent.lib.util.TopNSort.java |
| * [7] - Add ability to configure timeout of RedisOutputOperator |
| * [6] - Add a continue on error functionality to AbstractKeyValueStoreOutputOperator |
| * [5] - Add rollback to AbstractKeyValueStoreOutputOperator. |
| * [3] - Map Reduce job tracker |
| * [2] - Fix application name of mobile demo |
| * [1] - Console Operators |
| |
| |
| Version 0.3.3 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-418] - Duplicate - Demos need better documentation |
| * [SPOI-438] - Remove outdated zookeeper dependency from contrib test scope |
| * [SPOI-456] - CLI: kill-container should accept container number |
| * [SPOI-464] - Update README to clarify development environment sydtem requirements |
| * [SPOI-472] - dashboard fails with empty app list |
| * [SPOI-478] - Recreate a page view on URL change |
| * [SPOI-509] - Fix UI build on Linux |
| * [SPOI-517] - Ensure inputPorts and outputPorts are not undefined before merging them |
| * [SPOI-519] - Investigate Intermittent cease of publishing from Daemon |
| * [SPOI-527] - divison by zero exception in StreamingContainerManager for calculating throughput |
| * [SPOI-554] - Fix MergeSort operator and test |
| * [SPOI-571] - Siteops demo hangs |
| * [SPOI-604] - Empty site subdirectory in dist installer |
| * [SPOI-607] - Auto publish websocket data get stuck |
| * [SPOI-618] - Start and stop recordings broke from DataSource change |
| * [SPOI-622] - Get Port REST request fails for running applications after 399 |
| * [SPOI-629] - Fix DataSource.stopOpRecording usage (options object). |
| * [SPOI-653] - Fix name in package.json |
| * [SPOI-655] - Fix "Cannot read property 'ports' of undefined" bug with RecordingModel |
| * [SPOI-659] - PortPageView should override the cleanUp method to unsubscribe to port topic |
| * [SPOI-669] - Daemon starts secondary process if already running |
| * [SPOI-673] - Operator List Widget only needs appId, not whole app model instance |
| * [SPOI-733] - Incorrect documentation in README.txt developer version |
| |
| ### Improvement |
| * [SPOI-454] - CLI: Feedback when not connected to app |
| * [SPOI-467] - Backbone.js MVC: View -> Model -> Data Source |
| * [SPOI-469] - Normalize naming convention for all modules |
| * [SPOI-500] - Move require calls to widget classes to the top of each page file |
| * [SPOI-510] - Change API for Datasource module |
| * [SPOI-512] - Remove dataSource from model attributes |
| * [SPOI-528] - PortInfoWidget Unit Test |
| * [SPOI-547] - Convert Notifier module to an Object |
| * [SPOI-563] - Move WebSocket creation out of DataSource constructor |
| * [SPOI-582] - Document Front-End Architecture (UML Diagrams) |
| * [SPOI-596] - Move require calls to the top of pages.js |
| * [SPOI-602] - Get siteops demo to work without needed high bandwidth for UI |
| * [SPOI-608] - Rename Page Views according to naming conventions |
| * [SPOI-615] - Add 'port' to breadcrumb label in port view |
| * [SPOI-616] - The 'type' attribute in portmodel is not being extrapolated in subscribeToUpdates |
| * [SPOI-621] - The DataSource should URL-Encode port name in the getPort method |
| * [SPOI-676] - Return better response for apps.list when no apps running |
| |
| ### New Feature |
| * [SPOI-135] - Webservice: Provide statistics per streaming operator |
| * [SPOI-397] - Download Parent Jira |
| * [SPOI-402] - Document demo examples |
| * [SPOI-460] - Create Preconfigured Sandbox |
| * [SPOI-466] - Design and Implement Port View |
| * [SPOI-468] - Design and Implement Container View |
| * [SPOI-476] - DT phone home first cut |
| * [SPOI-503] - Create a pipeline for handling UI feedback |
| * [SPOI-518] - Implement at most once for Operators with one or zero connected input ports. |
| * [SPOI-598] - Daemon to serve historical stats data |
| * [SPOI-700] - Remove demo/groupby |
| |
| ### Sub-task |
| * [SPOI-453] - Update the mobile demo to version with map |
| * [SPOI-513] - Move JavaSerializationStreamCodec operator to library/util package |
| * [SPOI-515] - GitHub Release: Review/format source and docs for library/io/AxctiveMQ Input Operator. |
| * [SPOI-516] - GitHub repository structure and build system changes |
| * [SPOI-520] - Test strategy for operators that currently use LocalMode |
| * [SPOI-521] - Review/fix code formatting / style issues |
| * [SPOI-524] - Refactor script operators |
| * [SPOI-526] - Modify library to use CollectorTestSink |
| * [SPOI-539] - Move PerformanceTestCategory annotation to library |
| * [SPOI-541] - Port View Unit Tests |
| * [SPOI-542] - Publish library tests jar |
| * [SPOI-543] - Remove dependency to Tuple for library tests |
| * [SPOI-545] - DataSource Unit Tests |
| * [SPOI-546] - Determine best approach to mocking require()d modules |
| * [SPOI-552] - RedisOutputOperator stops running abruptly |
| * [SPOI-553] - Test code for InnerJoin/InnerJoin2 operators |
| * [SPOI-555] - Fix Unifier in match/change operators |
| * [SPOI-556] - Graphs show up after a delay with the new ads demo web changes. |
| * [SPOI-558] - Benchmark tests depend on STRAM |
| * [SPOI-559] - Move benchmark tests to a separate module |
| * [SPOI-560] - GitHub release: SumTest math operator still has reference to STRAM |
| * [SPOI-561] - Duplicate - Separate benchmark tests into a separate module |
| * [SPOI-562] - Fix HdfsOutputTest to not depend on STRAM |
| * [SPOI-565] - GitHub release: Change Http operator tests to use mortbay jetty |
| * [SPOI-566] - Remove reference to StramTestSupport from KafkaInputOperatorTest |
| * [SPOI-567] - KafkaInputOperatorTest is using DAG |
| * [SPOI-568] - Modify contrib build to not depend on STRAM |
| * [SPOI-583] - Implement a sample PubSubWebSocket servlet for testing |
| * [SPOI-584] - Contrib test classes have dependency to bufferserver |
| * [SPOI-585] - Implement a helper OperatorContext |
| * [SPOI-586] - PageLoaderView Unit Tests |
| * [SPOI-587] - Create SQL operator base interface/class |
| * [SPOI-588] - Add SQL Select Oprator to library |
| * [SPOI-589] - Create Sql Update operator in library |
| * [SPOI-590] - Create Sql Delete operator in library |
| * [SPOI-591] - Create SQL GourpBy/OrderBy Operator in library |
| * [SPOI-593] - Create SQL Outer join operator in library |
| * [SPOI-605] - Port View - Info and Overview Widgets |
| * [SPOI-636] - Apache Open Source Release : Review cods and source code for lirary/algo operator. |
| * [SPOI-637] - Create output Unifier on library/algo/BottomNOperator. |
| * [SPOI-639] - Create output port unifier for library/algo/Distinct Operator. |
| * [SPOI-640] - Create output port unifier for library/algo/FirstMatchMap Operator. |
| * [SPOI-642] - Create output port unifier for library/algo/FirstMatchStringMap Operator. |
| * [SPOI-644] - Create output port unifier for library/algo/FirstN Operator. |
| * [SPOI-645] - HttpOutputOperatorTest is failing |
| * [SPOI-646] - Create output port unifier for library/algo/InsertSort Operator. |
| * [SPOI-647] - NavModel/Router Unit Tests |
| * [SPOI-648] - Remove hash map output port from Insert sort operator. |
| * [SPOI-649] - Create output port unifier for library/algo/InvertIndex<K, V> Operator. |
| * [SPOI-650] - Mocha global leaks issue |
| * [SPOI-651] - Create output port unifier for library/algo/InvertIndexArray<K, V> Operator. |
| * [SPOI-652] - SlidingWindowTest is failing |
| * [SPOI-654] - Fix output port unifer for LeastFrequentKey operator. |
| * [SPOI-658] - Add Mocha Console Reporter |
| * [SPOI-661] - PageLoaderView Router Navigation Unit Tests |
| * [SPOI-662] - Refactor Bash Script oPerator. |
| * [SPOI-663] - Refactor Python script operator. |
| * [SPOI-664] - Fix output port unifier for MostFrequentKey Operator |
| * [SPOI-665] - Fix output port unifier for MostFrequentKeyValMap Operator |
| * [SPOI-666] - Fix output port unifier for libarry/algo/TopN Operator |
| * [SPOI-667] - Change output port for library/algo/TopNUnique Operator, add output port unifier. |
| * [SPOI-668] - Remove demo operator TupleOperator from library. |
| * [SPOI-671] - Create unifier on output port for library/algo/UniqueKeyValOperator |
| * [SPOI-672] - Remove demo operator : libarry/algo/WindowHolder |
| * [SPOI-675] - Review source/docs for library/io operators |
| * [SPOI-684] - Remove stram dependency from hdfs input operator test. |
| * [SPOI-686] - Fix HTTP output operator and test, failing right now. |
| * [SPOI-688] - Remove TestTupleCollector from io tests, it is not a test for any thing. |
| * [SPOI-689] - Remove com.library.io.anootation empty test package. |
| * [SPOI-690] - Move com.library.io.helper test to sample library. |
| * [SPOI-691] - Review source code/docs for library/logs operator. |
| |
| ### Technical task |
| * [SPOI-557] - Review/change code for MergeSort Operator |
| |
| ### Task |
| * [SPOI-205] - Protocol for stram to change network/message bus parameters of the outstream of output adapter dnode |
| * [SPOI-324] - Performance data to be written to HDFS |
| * [SPOI-459] - Test installation on OS X |
| * [SPOI-473] - GitHub Release : Code review for library/math operators |
| * [SPOI-474] - Document mobile demo |
| * [SPOI-475] - Document twitter demo |
| * [SPOI-477] - Review/format doc for apche open source release of library operators. |
| * [SPOI-499] - Review/format doc for apche open source release of library stream operators. |
| * [SPOI-504] - GitHub Release : Review cods and source code for lirary/multi window operator. |
| * [SPOI-505] - GitHub Release: Review/Format source/docs/test for AbstractSlidignWindowKeyVal Operator |
| * [SPOI-506] - GitHub Release : Review/Format source/docs/test for library/MultiSlidingWindowKeyVal Operator |
| * [SPOI-507] - GitHub Migration: Review/Format source/docs/test for library/SimpleMovingAverage Operator |
| * [SPOI-508] - GitHub Release : Review/Format source/docs/test for library/MultiSlidingWindowRangeKeyVal Operator |
| * [SPOI-511] - GitHub Release : Review/Format source/docs/test for library/logs operator |
| * [SPOI-514] - Open Source Transition for Library/Contrib/Demos |
| * [SPOI-530] - Operator tests dependent on stram. |
| * [SPOI-532] - Remove stram dependency from com.datatorrent.lib.testbench.EventGeneratorTest |
| * [SPOI-533] - Remove stram dependency from com.datatorrent.lib.testbench.RandomEventGeneratorTest |
| * [SPOI-534] - Remove stram depnedncy from com.datatorrent.lib.testbench.SeedEventGeneratorTest |
| * [SPOI-535] - GitHub release - Remove stram dependency from com.datatorrent.lib.math.MaxKeyValTest |
| * [SPOI-536] - Remove stram dependency from com.datatorrent.lib.math.MinKeyValTest |
| * [SPOI-537] - Remove stram dependency from com.datatorrent.lib.math.MaxMapTest |
| * [SPOI-540] - Remove stram dependency from com.datatorrent.lib.io.ActiveMQInputOperatorTest |
| * [SPOI-549] - Create a process for change.log for .3.3 release |
| * [SPOI-550] - Need license file changes to add DT Phone Home |
| * [SPOI-564] - Duplicate - Need to update license text to reflect "DT Phone Home" |
| * [SPOI-572] - Create a library for sql operators |
| * [SPOI-573] - Move GroupBy to Sql library |
| * [SPOI-574] - Move innerjoin operator to Sql lib |
| * [SPOI-575] - Move util/DerbySqlStreamOperator to sql libraray |
| * [SPOI-576] - Move util/AbstractSqlStreamOperator to sql library |
| * [SPOI-577] - Move OrderByKey operator to sql library |
| * [SPOI-578] - move algo/OrderbyKeyDesc operator to sql lib |
| * [SPOI-579] - move algo/orderbyvalue to sql library |
| * [SPOI-580] - move algo/orderbyvaluedesc to sql lib |
| * [SPOI-581] - move algo/innerjoincondition to sql lib |
| * [SPOI-592] - Create SQL Inner Join Operator in library |
| * [SPOI-594] - Archive release builds |
| * [SPOI-601] - Include twitter demo on demo server |
| * [SPOI-623] - Formalize a release procedure for future release |
| * [SPOI-625] - Add footer to guides |
| * [SPOI-626] - Get site ops demo UI to scale |
| * [SPOI-627] - Site ops demo UI changes |
| * [SPOI-635] - Ensure notice in html javadocs |
| * [SPOI-670] - Create output port unifier in libarry/algo/UniqueCounter Operator |
| * [SPOI-692] - change UI version to match streaming platform version |
| * [SPOI-697] - Demo of JIRA commits feature |
| * [SPOI-702] - Create a google groups for GitHub project Malhar |
| * [SPOI-705] - Setup a server for customer apps |
| |
| |
| |
| Version 0.3.2 |
| ------------------------------------------------------------------------------------------------------------------------ |
| |
| ### Bug |
| * [SPOI-29] - Pig: A inner join node |
| * [SPOI-55] - A library module/adapter or node for creating keys from general text |
| * [SPOI-325] - need better error reporting, not sure what host it is trying to connect to |
| * [SPOI-412] - UI update rate should be 1 sec (default) and allow customization |
| * [SPOI-441] - Investigate javadoc errors in build |
| * [SPOI-452] - recompile embedded zmq with 1.6 |
| * [SPOI-482] - Fix the NPM issue when installing UI in the build process |
| * [SPOI-490] - Clear chart when switching between operators |
| * [SPOI-497] - Application dashboard starts flickering if I change the column widths too often |
| * [SPOI-498] - Update rate on UI - Default and Customize |
| |
| |
| ### Improvement |
| * [SPOI-326] - Byte code obfuscation for future releases: Allow Malhar platform with premium features to be shared with early customers |
| * [SPOI-447] - Ability to filter app list in the CLI |
| * [SPOI-451] - Convert daemon stop/start into single service script |
| * [SPOI-484] - Add server timestamp to containers info |
| |
| ### New Feature |
| * [SPOI-69] - Persistence Node: A node to persist/spool every window into storage |
| * [SPOI-89] - Define and implement a RSS input adapter node |
| * [SPOI-112] - Versioning: Protocol between StramChild to app master |
| * [SPOI-121] - Parent jira for supporting Pig programming language |
| * [SPOI-153] - Heartbeat message |
| * [SPOI-201] - Parent jira for Checkpointing support in streaming platform |
| * [SPOI-254] - Design a general purpose read from stream and write to hbase node |
| * [SPOI-259] - Create db adapters - Need one jira for each db |
| * [SPOI-330] - Add a sql operator |
| * [SPOI-332] - Do y! finance demo: calculate last price, volume, time, charts, and moving averages |
| * [SPOI-333] - Do a pi demo |
| * [SPOI-334] - Write an RSS read operator |
| * [SPOI-335] - Add persistance/recording for operator, tuples should be stored in-order |
| * [SPOI-336] - Add persistance/recording for an port of an operator, tuples should be stored in-order |
| * [SPOI-343] - Design Malhar Deamon for the UI |
| * [SPOI-344] - Design for Live as well as Historical data access via daemon/stram |
| * [SPOI-345] - New optimal buffer server to get around memory issues with netty |
| * [SPOI-347] - Develop cli commands as a foundational part of an operating system |
| * [SPOI-348] - Allow commands to run as a script through CLI |
| * [SPOI-356] - Input adapter for DRUID |
| * [SPOI-357] - Output Adapter for DRUID |
| * [SPOI-362] - Create Alert operator that does moving average and alerts if that drops by more than X% |
| * [SPOI-365] - Output Adapter for Redis |
| * [SPOI-374] - Size limits on log files (STRAM mainly) |
| * [SPOI-380] - Add security to STRAM |
| * [SPOI-383] - Implement do-As |
| * [SPOI-386] - Design latency computations |
| * [SPOI-387] - Design CPU, Memory, and Network IO usage |
| * [SPOI-388] - Added resource usage to stats and access via webservice |
| * [SPOI-391] - Design and implement at least once |
| * [SPOI-394] - Add input adapter for Redis |
| * [SPOI-395] - Create an output adapter for Redis |
| * [SPOI-398] - Test download on Amazon |
| * [SPOI-405] - Design, document, and test download to work with Apache Hadoop |
| * [SPOI-416] - Local only Download licensing |
| * [SPOI-426] - Create an attribute that forces the checkpoint to align with application window boundary |
| * [SPOI-479] - Design and implement ad data charting |
| * [SPOI-481] - UI design for application platform charting (live and historical dimensional data) |
| * [SPOI-485] - Placeholder for list of possible UI work for Summit |
| * [SPOI-486] - Enable security in the UI (need to check that the user has permission) |
| * [SPOI-487] - Design and implement new SVG charts |
| * [SPOI-489] - Alerts |
| * [SPOI-491] - Create login error pages for UI on secure cluster |
| * [SPOI-492] - Design UI for resources and latency data |
| * [SPOI-493] - Design and implement per operator view |
| * [SPOI-494] - Design and develop the "Operations" dashboard |
| * [SPOI-495] - The recording tab should show list of recordings by operator or by operator:port |
| * [SPOI-496] - Securitiy UI (sign-in, get token from deamon, etc.) |
| |
| ### Story |
| * [SPOI-371] - Review Storm features vs our open source plan |
| * [SPOI-372] - Ensure that premium feature implementation is hard for outsiders to do on the open source tree |
| |
| ### Task |
| * [SPOI-14] - Dynamic run time optimization framework: Load balancing and load shedding |
| * [SPOI-39] - Should be able to track window id through the DAG |
| * [SPOI-40] - Output messages of each node should be persisted/buffered |
| * [SPOI-63] - Output Adapter logging |
| * [SPOI-64] - Levels in logging |
| * [SPOI-231] - Demo tweeter feed analysis on Hadoop 2.0 with basic streaming setup |
| * [SPOI-273] - Setup CI and run existing demos to ensure that code does not break any |
| * [SPOI-419] - Certify our build on Amazon |
| * [SPOI-420] - Convert mobile location demo to a google maps demo |
| * [SPOI-421] - Ensure that checkpointing happens at Application window boundary |
| * [SPOI-422] - Allow an operator developer to enable checkpointing within an application window |
| * [SPOI-423] - Have a "ALLOW_CHECKPOINT_WITHIN_WINDOW" attribute (default is FALSE) |
| * [SPOI-424] - Document new checkpointing logic in app dev guide and op dev guide, add attribute to operations guide |
| * [SPOI-442] - ASF License header for downloadable sources |
| * [SPOI-443] - Packaging for dev and cluster versions |
| * [SPOI-480] - Add aggregate i/o buffer server bytes for application level |
| * [SPOI-483] - Tie up loose ends for summit version |
| * [SPOI-488] - Remove failure count column from "Live" table |
| |
