)]}' { "log": [ { "commit": "c10ae0b3978d2dcf950aa7860f2b8e8c1230b4aa", "tree": "7eedcd29086edf6156ea0e47551de0f2339872a9", "parents": [ "87834d53d92906cb1b34430113e54872d660d9b6" ], "author": { "name": "pratapaditya04", "email": "thesingh.adi.work@gmail.com", "time": "Mon Jun 08 15:22:53 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Jun 08 15:22:53 2026 +0530" }, "message": "GOBBLIN-2265: Reliable terminal job-completion status for temporal-on-YARN (AM final status + launcher hooks + exit codes + prompt un-register) (#4197)\n\n* GOBBLIN-XXXX: Reliable terminal job-completion status for temporal-on-YARN (AM final status + launcher hooks + exit codes)\n\nMake a temporal-on-YARN job\u0027s true outcome reliably observable end-to-end without depending on a\nfragile JVM shutdown hook, and expose clean extension points for a single terminal-GTE source.\n\n- AM un-registers with a FinalApplicationStatus derived from the Temporal workflow outcome\n (COMPLETED-\u003eSUCCEEDED, CANCELED-\u003eKILLED, else FAILED) instead of an unconditional SUCCEEDED, so a\n launcher polling the ApplicationReport sees the real result. Kept in lockstep with the AM JVM exit\n code via mapWorkflowStatusToFinalAppStatus/computeExitCode.\n- Remove the unreliable JVM-shutdown-hook GTE emitter (RootMetricContext closes the Kafka reporters\n in its own shutdown hook, so the GTE could be silently dropped). Status capture and exit-code\n propagation are retained.\n- Restore the in-workflow JOB_SUCCEEDED/JOB_FAILED GTEs and gate them behind a new\n gobblin.temporal.job.completion.gte.emission.enabled flag (default true), so a standalone OSS\n deployment stays self-complete; deployments that designate a launcher subclass as the single GTE\n source set it false.\n- GobblinYarnAppLauncher: surface FAILED/KILLED/UNDEFINED, lost-AM-visibility, and never-launched\n applications as a non-zero launcher exit code, and expose protected no-op hooks\n (onTerminalApplicationStatus / onLostAmVisibility / onApplicationLaunchFailure) plus\n getApplicationId/getApplicationName/getConfig accessors so a subclass can emit the single terminal\n GTE. The OSS launcher itself emits no GTE.\n- On graceful shutdown, force-kill the YARN application after the wait so the RM does not re-attempt\n the AM on cancel.\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* GOBBLIN-2265: Wake AM un-register promptly on container shutdown; drop debug logging\n\nFix a notify race in the temporal AM shutdown path: shutDown() waits on\nallContainersStopped, but the two callbacks that observe a container going\naway (AMRM onContainersCompleted -\u003e handleContainerCompletion, and NM\nonContainerStopped) did not reliably notify the waiter. DynamicScalingYarnService\nadditionally early-returned on the shutdownInProgress branch without notifying.\nAs a result the waiter slept the full wait timeout, delaying\nunregisterApplicationMaster by minutes and causing YARN to fail the attempt\neven on a successful workflow.\n\n- Add YarnService.notifyIfAllContainersStopped(); call it from\n handleContainerCompletion and onContainerStopped (base) and from both\n early-return paths of DynamicScalingYarnService.handleContainerCompletion.\n- Reduce the container-stop fallback wait from 5m to 2m (the notify makes the\n normal path immediate; this is only a missed-notify backstop).\n- Replace the temporary debug markers with normal INFO-level logging.\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* GOBBLIN-2265: Fix FindBugs ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD; clarify terminal GTE is emitted by the Temporal worker, not the AM\n\n- Route writes to the static lastTerminalStatus cache through a private static setter\n (GobblinTemporalJobLauncher) so FindBugs ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD no longer fires.\n- Rename isAmTerminalGteEmissionEnabled -\u003e isWorkflowTerminalGteEmissionEnabled and correct the\n javadoc/comments: ExecuteGobblinWorkflowImpl runs on a Temporal worker, not the YARN AM.\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* GOBBLIN-2265: Address PR review feedback on terminal job-completion status\n\n- GobblinTemporalJobLauncher.fetchWorkflowStatus: retry transient gRPC failures\n (UNAVAILABLE / DEADLINE_EXCEEDED / RESOURCE_EXHAUSTED) with bounded linear\n backoff before falling back to UNSPECIFIED, so a momentary Temporal outage is\n not mis-reported as a terminal JOB_FAILED GTE.\n- temporal YarnService.shutDown: re-check containerMap emptiness inside the\n allContainersStopped monitor and loop on the condition, fixing a lost-wakeup\n where a notify firing between the outer check and wait() stalled the AM\n un-register for the full 2-minute timeout. Also correct the\n getFinalApplicationStatusForUnregister Javadoc (a null captured status maps\n to FAILED, not SUCCEEDED).\n- GobblinYarnAppLauncher.signalGracefulShutdownAndWaitForTerminal: re-fetch the\n ApplicationReport after polling and dispatch the real terminal status if the\n app already finished, instead of an unconditional synthetic KILLED. Document\n that onTerminalApplicationStatus may receive a null ApplicationReport.\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* GOBBLIN-2265: Follow-up review feedback (drop manual describe retry; configurable container-stop wait; skip kill when app already finished)\n\n- fetchWorkflowStatus: remove the manual retry loop + Thread.sleep. Transient gRPC\n failures (UNAVAILABLE / DEADLINE_EXCEEDED / RESOURCE_EXHAUSTED) are already retried\n with backoff by the Temporal service stubs\u0027 configured RpcRetryOptions\n (gobblin.temporal.rpc.retry.options.*); the catch now only handles the\n post-retry-exhausted fallback to UNSPECIFIED.\n- YarnService.shutDown: make the bounded container-stop wait configurable via\n gobblin.temporal.containers.stop.wait.timeout.minutes (default 2).\n- GobblinYarnAppLauncher.signalGracefulShutdownAndWaitForTerminal: only force-kill\n when the app is not already finished; if it already reached a terminal state,\n dispatch the real status and skip the kill.\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* GOBBLIN-2265: Fix GobblinYarnAppLauncherTest for post-poll re-fetch\n\ntestGracefulShutdownSendsSignalAndPolls expected getApplicationReport to be\ninvoked twice, but signalGracefulShutdownAndWaitForTerminal now re-fetches the\nreport once after the poll to decide between surfacing the real terminal status\nand force-killing. Update the expected count to 3 and assert that the kill is\nskipped when the application is already terminal.\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e" }, { "commit": "87834d53d92906cb1b34430113e54872d660d9b6", "tree": "8d6d1309a87d5bc825e579decbc5efdecac37e72", "parents": [ "d62931e1e299f9c55db4da0b1fa234f8cb7be84a" ], "author": { "name": "Agam Pal Singh", "email": "32903685+agam-99@users.noreply.github.com", "time": "Mon Jun 01 14:05:02 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Jun 01 14:05:02 2026 +0530" }, "message": "GOBBLIN-2263: Refresh DagAction latency anchor (#4198)\n\nStamp each submitted job with the current DagAction store insert time\nso the launch latency metric measures per-action queueing rather than\nelapsed DAG runtime.\n\nCo-authored-by: Cursor \u003ccursoragent@cursor.com\u003e" }, { "commit": "d62931e1e299f9c55db4da0b1fa234f8cb7be84a", "tree": "3822292bda50c39452d10371478eb5c0490224ea", "parents": [ "53112596dcecd7c9cb1b827791fb3b740d5ec227" ], "author": { "name": "pratapaditya04", "email": "thesingh.adi.work@gmail.com", "time": "Tue May 19 19:54:08 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue May 19 19:54:08 2026 +0530" }, "message": "[GOBBLIN-2264] Key deadline reminders by dagAction only so DELETE events can unschedule them (#4194)\n\nDeadline reminder JobKeys previously embedded the lease event time, but the\nDagActionStore DELETE event payload omits event time, so the change monitor\ncould not cancel a still-pending deadline reminder for a dagAction whose row\nhad already been resolved.\n\nRe-key deadline reminders by (flowGroup, flowName, flowExecutionId, jobName,\ndagActionType) only and add an unscheduleReminderJob(DagAction) overload that\nthe DELETE handler can invoke. Replace any pre-existing deadline reminder on\nschedule to avoid ObjectAlreadyExistsException when the delete-then-reinsert\nduplicate-insert path in DagProcUtils.sendEnforce*DeadlineDagAction races with\nan out-of-order change event.\n\nRetry reminders still embed event time and are intentionally not unscheduled\non DELETE; their JobKey is not derivable from the DELETE payload, and orphaned\nretries fire harmlessly because the lease arbiter sees the row is gone.\n\nRe-enable the previously TODO-disabled DELETE assertion in\nDagManagementDagActionStoreChangeMonitorTest to cover the activated code path." }, { "commit": "53112596dcecd7c9cb1b827791fb3b740d5ec227", "tree": "52168d7b91b7590cee0fe3637cfa4dcbcc83fe41", "parents": [ "0ec58824bd4bb3b924152bafb4fda067a8eba77b" ], "author": { "name": "Agam Pal Singh", "email": "32903685+agam-99@users.noreply.github.com", "time": "Fri May 15 19:40:42 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri May 15 19:40:42 2026 +0530" }, "message": "GOBBLIN-2263: Plumb DagAction store-insert time onto JobSpec for downstream latency metrics (#4191)\n\n* Plumb DagAction store-insert time onto JobSpec for downstream latency metrics\n\nAdds dbUpdateTimeMillis as an optional field on DagActionStoreChangeEvent\nso MySQL CDC producers can carry the binlog EventTimestamp (true row\nmodification time) through the kafka hop into change monitors.\n\nChange monitors set the value as storeInsertTimeMillis on LeaseParams\nwhen constructing them. LaunchDagProc stamps the value onto the FlowSpec\nconfig under the new ConfigurationKeys.DAG_ACTION_LAUNCH_STORE_INSERT_TIME_MILLIS_KEY\nso downstream executors can measure true LAUNCH-to-submission latency,\nincluding CDC propagation that flow.executionId-based anchors miss\n(REST-handler time for ad-hoc, consensus DB time for scheduled).\n\nDagTask exposes the LeaseParams via a new getter so DagProc subclasses\ncan read per-event metadata. Defaults to -1 (UNKNOWN_STORE_INSERT_TIME_MILLIS)\nwhen the source timestamp isn\u0027t available, in which case LaunchDagProc\nskips the stamp and downstream code falls back to its existing anchor.\n\nCo-Authored-By: Claude Opus 4.7 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* Address review feedback and plumb storeInsertTimeMillis end-to-end\n\nReview fixes (DaisyModi):\n- Rename Avro field dbUpdateTimeMillis -\u003e storeInsertTimeMillis on\n DagActionStoreChangeEvent for naming consistency with LeaseParams.\n- Move UNKNOWN_STORE_INSERT_TIME_MILLIS from DagActionStore interface\n scope into LeaseParams, where it logically belongs. Update the two\n external references (DagActionReminderScheduler, LaunchDagProc) and\n the surrounding javadoc link.\n\nCarry the timestamp through consensus + reminders so the JobSpec stamp\nfires on the primary path:\n- MysqlMultiActiveLeaseArbiter: at the three call sites that build the\n consensus LeaseParams (CASE 2 within-epsilon, CASE 3 distinct-event,\n fresh-lease acquisition), switch from the 2-arg constructor to the\n 4-arg one and pass through leaseParams.getStoreInsertTimeMillis().\n Without this, consensus silently defaults the field to UNKNOWN and\n LaunchDagProc skips the stamp.\n- DagActionReminderScheduler: stash storeInsertTimeMillis on the Quartz\n JobDataMap when scheduling reminders and restore it when the reminder\n fires, so a host-failure-driven reattempt preserves the timestamp.\n Defaults to UNKNOWN for reminders scheduled by older code paths.\n\n* Add unit tests for storeInsertTimeMillis pass-through\n\nCovers the three sites this PR plumbs the source-DB row-insert timestamp through:\n\n- LaunchDagProcTest:\n * launchDagStampsStoreInsertTimeMillisWhenProvided asserts initialize()\n adds the DAG_ACTION_LAUNCH_STORE_INSERT_TIME_MILLIS_KEY to the FlowSpec\n config when LeaseParams carries a non-UNKNOWN value, and that the value\n is what was supplied.\n * launchDagSkipsStoreInsertTimeMillisStampWhenUnknown asserts the key is\n NOT added when the LeaseParams carries the UNKNOWN sentinel.\n * Existing three tests updated to construct a real LeaseObtainedStatus\n via the new buildLaunchDagTask helper (previously passed null, which\n NPE\u0027d after the PR added getLeaseParams() into initialize()).\n\n- MysqlMultiActiveLeaseArbiterTest:\n * testStoreInsertTimeMillisPreservedThroughConsensus exercises all three\n arbiter consensus call sites — fresh-lease (selectInfoResult), CASE 2\n (within-epsilon, same event valid), and CASE 3 (distinct event, lease\n valid) — and asserts the consensus LeaseParams preserves the inbound\n storeInsertTimeMillis instead of silently defaulting to UNKNOWN.\n\n- DagActionReminderSchedulerTest:\n * testCreateReminderJobDetailStashesStoreInsertTimeMillis verifies the\n JobDataMap on the Quartz JobDetail captures storeInsertTimeMillis.\n * testReminderJobExecuteCarriesStoreInsertTimeMillis exercises the full\n round-trip — ReminderJob.execute() reconstructs LeaseParams from the\n JobDataMap and the value survives.\n * testReminderJobExecuteFallsBackToUnknownWhenKeyAbsent asserts the\n backwards-compat path (reminder scheduled by older code without the\n new key) falls back to UNKNOWN, not throws.\n\n* Avoid Cloudera for grpc artifact resolution\n\nExclude io.grpc from the Cloudera repository so Gradle resolves grpc\nartifacts and version metadata from Maven Central instead of a flaky\nlegacy mirror.\n\nCo-authored-by: Cursor \u003ccursoragent@cursor.com\u003e\n\n* Fix tests for DagAction timestamp plumbing\n\nUpdate stale test helpers to match the new DagActionStoreChangeEvent\nschema and provide lease metadata where LaunchDagProc now reads it.\n\nCo-authored-by: Cursor \u003ccursoragent@cursor.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 4.7 (1M context) \u003cnoreply@anthropic.com\u003e\nCo-authored-by: Cursor \u003ccursoragent@cursor.com\u003e" }, { "commit": "0ec58824bd4bb3b924152bafb4fda067a8eba77b", "tree": "b299a70b2f9867ef1e5c766163f99586a4939e46", "parents": [ "4d2b91eb3552590dfdd79b032c07e563b1a204f0" ], "author": { "name": "Agam Pal Singh", "email": "32903685+agam-99@users.noreply.github.com", "time": "Wed Apr 22 10:25:37 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Apr 22 10:25:37 2026 +0530" }, "message": "[GOBBLIN-XXXX] Remove WARN log aggregation from AutomaticTroubleshooter (#4186)\n\nRaise the log appender threshold from WARN to ERROR so only ERROR and\nFATAL logs are aggregated by the troubleshooter. This reduces noise\nin the issue repository and keeps it focused on actionable errors.\n\nCo-authored-by: Claude Opus 4.6 (1M context) \u003cnoreply@anthropic.com\u003e" }, { "commit": "4d2b91eb3552590dfdd79b032c07e563b1a204f0", "tree": "e9befd91661221432b9e14f050d2069dea54c00f", "parents": [ "5af94dcd0844d98714f6827d85661014d9eda081" ], "author": { "name": "Agam Pal Singh", "email": "32903685+agam-99@users.noreply.github.com", "time": "Wed Apr 15 19:49:52 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Apr 15 19:49:52 2026 +0530" }, "message": "Skip directory entries in RecursiveCopyableDataset to fix IOException on empty source dirs (#4181)\n\n* [GOBBLIN-XXXX] Skip directory entries in RecursiveCopyableDataset to avoid IOException on empty source dirs\n\nWhen source.path is an empty directory, FileListUtils.listFilesToCopyAtPath\n(includeEmptyDirectories\u003dtrue) returns the directory itself as the sole\nFileStatus entry. The subsequent call to\nresolveReplicatedOwnerAndPermissionsRecursively passes file.getPath().getParent()\nas fromPath — which is *above* replacedPrefix — inverting the ancestry check\nand throwing an IOException.\n\nSkip FileStatus entries where isDirectory\u003dtrue; empty source directories\nproduce no copy work units by design. Log a warning so operators can\ndiagnose misconfigured source paths.\n\n* [ETL-19035] Fix ancestor path resolution for empty source directories in RecursiveCopyableDataset\n\nWhen includeEmptyDirectories\u003dtrue and the source root is empty, FileListUtils\nreturns the root directory itself as a FileStatus entry. Calling .getParent()\non it produces a path above replacedPrefix, breaking the ancestry check in\nresolveReplicatedOwnerAndPermissionsRecursively.\n\nFix: guard with isAncestor(replacedPrefix, parentPath) — fall back to the\nfile\u0027s own path only when parentPath is above the dataset root. This preserves\ncorrect ancestor permission replication for nested empty subdirs (where\n.getParent() is still within replacedPrefix) and ensures the empty root\ndirectory is replicated at the destination rather than skipped.\n\nCo-Authored-By: Claude Sonnet 4.6 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Sonnet 4.6 \u003cnoreply@anthropic.com\u003e" }, { "commit": "5af94dcd0844d98714f6827d85661014d9eda081", "tree": "3673dbb94273d5f11512fc521cdc10540d8c1565", "parents": [ "6904dd0bb22dfb4cada6a8710d376909962cd296" ], "author": { "name": "debabhishek53", "email": "debabhishek53@gmail.com", "time": "Thu Apr 09 14:52:23 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Apr 09 14:52:23 2026 +0530" }, "message": "[GOBBLIN-ICEBERG] Restore backward compat: CURRENT_DATE produces -00 … (#4185)\n\n* [GOBBLIN-ICEBERG] Restore backward compat: CURRENT_DATE produces -00 in legacy hourly mode\n\n* Address PR comments: add hourly flag check and clarify property names in Javadoc\n\n* Clarify iceberg.hourly.partition.enabled is legacy flag superseded by iceberg.partition.value.datetime.format" }, { "commit": "6904dd0bb22dfb4cada6a8710d376909962cd296", "tree": "be9eb2075fa8f8c6295dcccaa986cf882a7c4dc3", "parents": [ "c25746b9f997a195734eddb9c4f46c98342bb935" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Tue Apr 07 10:41:13 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Apr 07 10:41:13 2026 +0530" }, "message": "[GOBBLIN-2169] Fix NPE when table has no snapshot while accessing current snapshot (#4073)\n\n* throw exception incase no snapshot found for copying\n\n* added nosnapshotfound exception\n\n* add check to log exception in case of empty snapshot in source" }, { "commit": "c25746b9f997a195734eddb9c4f46c98342bb935", "tree": "3f88e797e3b51a71f669f16f9bc547daeed93203", "parents": [ "de6371d7b0e697ee97c218f6db9927fb7fc6ea2d" ], "author": { "name": "Daisy Modi", "email": "daisymodi@gmail.com", "time": "Tue Mar 31 14:39:23 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Mar 31 14:39:23 2026 +0530" }, "message": "[GOBBLIN-2247] Increase RPC retry budget to 10 minutes for Temporal gRPC throttling (#4180)\n\nThe previous defaults (~2.5 min) were insufficient to survive sustained throttle\nbursts. Tuned initialInterval to 1s, maximumInterval to 60s, and maximumAttempts\nto 15, yielding ~603s (~10 min) of cumulative retry coverage.\n\nCo-authored-by: Daisy Modi \u003cdmodi@linkedin.com\u003e\nCo-authored-by: Claude Opus 4.6 (1M context) \u003cnoreply@anthropic.com\u003e" }, { "commit": "de6371d7b0e697ee97c218f6db9927fb7fc6ea2d", "tree": "58d78f6ef5203ec666e50395949d02e1360d85cf", "parents": [ "e8b18a627cd8503338066077f8de3849ed88e91f" ], "author": { "name": "debabhishek53", "email": "debabhishek53@gmail.com", "time": "Mon Mar 30 16:15:13 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Mar 30 16:15:13 2026 +0530" }, "message": " [GOBBLIN-ICEBERG] Add configurable partition filter with hourly look… (#4171)\n\n* [GOBBLIN-ICEBERG] Add configurable partition filter with hourly lookback support\n\n - Introduce IcebergPartitionFilterGenerator utility for reusable Iceberg OR\n expression building (forDays, forHours, buildOrExpression)\n - Add iceberg.partition.value.format for arbitrary DateTimeFormatter patterns\n - Add iceberg.partition.hour (0-23) for explicit hour control in daily partitions\n - Add iceberg.lookback.hours for hour-level granularity, takes precedence over\n iceberg.lookback.days when set\n\n* Update gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/iceberg/IcebergSource.java\n\nCo-authored-by: Copilot \u003c175728472+Copilot@users.noreply.github.com\u003e\n\n* Update gobblin-data-management/src/test/java/org/apache/gobblin/data/management/copy/iceberg/IcebergPartitionFilterGeneratorTest.java\n\nCo-authored-by: Copilot \u003c175728472+Copilot@users.noreply.github.com\u003e\n\n* Updated the PR for copilot comments \u0026 build issues\n\n* Updated the PR for copilot comments \u0026 build issues\n\n* forDays now uses Expressions.startsWith so a daily value\n\n* Fixed all the comments and made some minor changes\n\n* Added the upper bound on lookback hour with few comments fix\n\n* Fix: Addressed Claude comments\n\n* parse date-only custom formats (e.g. yyyyMMdd) as LocalDate before converting to LocalDateTime\n\n---------\n\nCo-authored-by: Abhishek Deb \u003cabdeb@abdeb-mn5095.linkedin.biz\u003e\nCo-authored-by: Copilot \u003c175728472+Copilot@users.noreply.github.com\u003e" }, { "commit": "e8b18a627cd8503338066077f8de3849ed88e91f", "tree": "bd2edf702efbdc1d491449e08ed77afbcc702681", "parents": [ "dd659cd866dc27a26ee4f2ad73584ce33adabdc7" ], "author": { "name": "abhishekmjain", "email": "abhishekmjain97@gmail.com", "time": "Thu Mar 26 13:02:54 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Mar 26 13:02:54 2026 +0530" }, "message": "Add flow group prefix-based concurrency override (#4179)\n\nWhen service-level flowConcurrencyAllowed is false and a flow does not\nhave flow.allowConcurrentExecution explicitly set, allow concurrent\nexecution based on a configurable list of flow group prefixes. This\navoids bulk-updating 80k flow configs in the DB which would overwhelm\nthe Brooklin CDC stream.\n\nNew config: gobblin.service.concurrencyAllowedFlowGroupPrefixes\n\nResolution order:\n1. Per-flow flow.allowConcurrentExecution if explicitly set\n2. Service-level flowConcurrencyAllowed if true\n3. Flow group prefix match (new fallback)" }, { "commit": "dd659cd866dc27a26ee4f2ad73584ce33adabdc7", "tree": "0437aa2982b9d6666b15594c0810831b6c25ad93", "parents": [ "cb6f82be4eb00cba1284245352148978a1516c33" ], "author": { "name": "pratapaditya04", "email": "thesingh.adi.work@gmail.com", "time": "Tue Mar 24 15:53:20 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Mar 24 15:53:20 2026 +0530" }, "message": "Add batched parallel processing to ManifestBasedDataset (#4177)\n\nInstead of submitting all files to parallelStream at once, files are\nnow processed in configurable batches (default 100). Each batch is\nfully processed before the next begins, preventing unbounded concurrency\non large manifests.\n\nNew config key: gobblin.copy.manifestBased.parallelBatchSize (default 100)" }, { "commit": "cb6f82be4eb00cba1284245352148978a1516c33", "tree": "774b75292a797f541259196688dc7c30877c52d2", "parents": [ "2613bb8a19819c46eb17c756114c21f486a25e34" ], "author": { "name": "Daisy Modi", "email": "daisymodi@gmail.com", "time": "Tue Mar 24 11:16:50 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Mar 24 11:16:50 2026 +0530" }, "message": "Tune RPC retry policy to tolerate 1-2 min throttling on Temporal gRPC calls (#4176)\n\nAdd configurable RpcRetryOptions to WorkflowServiceStubsOptions in\nTemporalWorkflowClientFactory. The defaults (initialInterval\u003d500ms,\nbackoffCoefficient\u003d2.0, maximumInterval\u003d30s, maximumAttempts\u003d10)\nprovide ~151s of cumulative retry budget, enough to ride out a 2-minute\nthrottle burst without failing worker status reporting.\n\nNew config keys under gobblin.temporal.rpc.retry.options.* allow\nper-environment tuning without code changes.\n\nCo-authored-by: Daisy Modi \u003cdmodi@linkedin.com\u003e\nCo-authored-by: Claude Sonnet 4.6 (1M context) \u003cnoreply@anthropic.com\u003e" }, { "commit": "2613bb8a19819c46eb17c756114c21f486a25e34", "tree": "a170e619e2eb84ae54ca15fb601fd4eec658bd3a", "parents": [ "26406a863688e289063e92bd421bf866f84e5586" ], "author": { "name": "Daisy Modi", "email": "daisymodi@gmail.com", "time": "Mon Mar 23 15:15:04 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Mar 23 15:15:04 2026 +0530" }, "message": "[GOBBLIN-2257] Parallelize flow compilation on submission path (#4175)\n\n* Parallelize flow compilation on submission path\n\nFlow compilation happens twice: on submission and on execution. The execution\npath already runs on a thread pool of size 3 (DagProcessingEngine), but the\nsubmission path was serialized by a synchronized block in\nSpecCatalogListenersList.onAddSpec() and a single-thread executor in\nCallbacksDispatcher.\n\nThis change removes the synchronized keyword from onAddSpec() and replaces\nthe single-thread executor with a configurable thread pool (default 3 threads),\nallowing multiple flows to compile in parallel during submission. Compilation\nis still serialized per flow but multiple flows now compile concurrently.\n\nThe thread pool size is configurable via:\n gobblin.service.specCatalogListener.numThreads (default: 3)\n\nCo-Authored-By: Claude Opus 4.6 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* Use bounded ThreadPoolExecutor with CallerRunsPolicy\n\nReplace Executors.newFixedThreadPool (unbounded queue) with a bounded\nThreadPoolExecutor to prevent memory growth under heavy submission load.\nQueue capacity is set to numThreads * 10 with CallerRunsPolicy so that\nwhen the queue is full, the submitting thread runs the compilation\nitself, providing natural backpressure.\n\nCo-Authored-By: Claude Opus 4.6 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* Add comments explaining parallelization rationale\n\nAdd inline comment in FlowCatalog explaining why the submission path\nis now parallelized and Javadoc on the new SpecCatalogListenersList\nconstructor documenting the design decision and thread-safety reasoning.\n\nCo-Authored-By: Claude Opus 4.6 (1M context) \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Daisy Modi \u003cdmodi@linkedin.com\u003e\nCo-authored-by: Claude Opus 4.6 (1M context) \u003cnoreply@anthropic.com\u003e" }, { "commit": "26406a863688e289063e92bd421bf866f84e5586", "tree": "297d86fd5af39a8f380e907eae9f731ba0553e51", "parents": [ "90399ee11ae77f1a8a9fd96bf30bfe62e4ff37a1" ], "author": { "name": "pratapaditya04", "email": "thesingh.adi.work@gmail.com", "time": "Tue Mar 17 14:09:41 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Mar 17 14:09:41 2026 +0530" }, "message": "[GOBBLIN-2256]Optimize manifest discovery with paralleism (#4163)\n\n* optimizing manifest discovery" }, { "commit": "90399ee11ae77f1a8a9fd96bf30bfe62e4ff37a1", "tree": "a2b9eca7e5c1cf9998f42c43e02d366df6f2f823", "parents": [ "452283e0ed7d2e1f166303c58a1b9feb0a676225" ], "author": { "name": "Gulbarga Adithya Rao", "email": "96827791+adithya2754@users.noreply.github.com", "time": "Wed Mar 11 16:29:36 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Mar 11 16:29:36 2026 +0530" }, "message": "[GOBBLIN-2255] Temporal workflow fast failure (#4174)\n\n* fast failure for quota exceeded failures\n\n* added set of non retryable exceptions\n\n* nested getCause for non retryable failure" }, { "commit": "452283e0ed7d2e1f166303c58a1b9feb0a676225", "tree": "730f007dd4e14851220d015b4a2348f78abd6f71", "parents": [ "b594188cdfa18ff7b993221f0fc1c0b0d7cc810d" ], "author": { "name": "Agam Pal Singh", "email": "32903685+agam-99@users.noreply.github.com", "time": "Wed Mar 11 12:56:52 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Mar 11 12:56:52 2026 +0530" }, "message": "Orchestration Layer Issue Capturing (#4167)\n\nCapture service layer errors and emit them as issues\n\nCo-Authored-By: Claude Opus 4.6 (1M context) \u003cnoreply@anthropic.com\u003e" }, { "commit": "b594188cdfa18ff7b993221f0fc1c0b0d7cc810d", "tree": "505bfe91b1f1898efa99f0b2589be3e2394f2e5f", "parents": [ "f34dc82fd660928f083278cde61b548d4266fd2b" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Wed Mar 11 10:40:04 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Mar 11 10:40:04 2026 +0530" }, "message": "[GOBBLIN-2254] Add timeout in gobblintaskrunner shutdown and move metrics reporting … (#4173)\n\n* add timeout in gobblintaskrunner shutdown and move metrics reporting shutdown after stopping helix task closing\n\n* address review comments" }, { "commit": "f34dc82fd660928f083278cde61b548d4266fd2b", "tree": "175c6a2660e495b18613346d78ee09ca0c880926", "parents": [ "4a041b8b96d891da7f779e4e201089278eddbe2a" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Tue Mar 10 10:24:58 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Mar 10 10:24:58 2026 +0530" }, "message": "add fsUri in dataset descriptor for iceberg source for lineage (#4172)" }, { "commit": "4a041b8b96d891da7f779e4e201089278eddbe2a", "tree": "2b1443ef3a19844a294b511f16fdac72c9752f63", "parents": [ "1c875ce7982851779abb711136a421df1f1bec6c" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Fri Mar 06 19:44:47 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Mar 06 19:44:47 2026 +0530" }, "message": "added missing source and destinatin data required for lineage (#4170)" }, { "commit": "1c875ce7982851779abb711136a421df1f1bec6c", "tree": "295c1208e45aff5bc50f8e693fd6f40b752c6f46", "parents": [ "dcbe779c69c30c1d9941cc717fac5f63e5b9f774" ], "author": { "name": "Vyom Aggarwal", "email": "aggarwalvyom@gmail.com", "time": "Thu Mar 05 23:13:38 2026 -0800" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Mar 06 12:43:38 2026 +0530" }, "message": "Skip Permission Check for files existing in target, (#4162)\n\n- Added skipPermissionCheck flag to skip I/O operations for files that already exist in the target location\n - When enabled, skips expensive file status checks and permission resolutions for existing files\n - Improves performance for discovery by avoiding unnecessary I/O operations\n - Disabled by default to maintain backward compatibility" }, { "commit": "dcbe779c69c30c1d9941cc717fac5f63e5b9f774", "tree": "19f796078749b252dcba7cfcb334ec4379125a78", "parents": [ "5e128c909bfc095f07b58077d12f9e4f83b229cf" ], "author": { "name": "pratapaditya04", "email": "thesingh.adi.work@gmail.com", "time": "Fri Feb 27 12:25:55 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Feb 27 12:25:55 2026 +0530" }, "message": "[GOBBLIN-2211] Use correct jobExecId when fetching jobStatus (#4169)\n\nJobStatusRetriever.getJobExecutionId was reading from JOB_EXECUTION_ID_FIELD instead of GAAS_JOB_EXEC_ID_HASH, causing incorrect job execution IDs to be resolved in some flows. This fix reads the correct\nproperty with safe fallback handling for missing or non-numeric values, and adds unit tests covering the various input cases." }, { "commit": "5e128c909bfc095f07b58077d12f9e4f83b229cf", "tree": "e97354b7ac51eee1c6e7c1a8448f3789bbf58dbb", "parents": [ "9e557b872f626d3d054fb9d29ba499bd4ca0488e" ], "author": { "name": "pratapaditya04", "email": "thesingh.adi.work@gmail.com", "time": "Fri Feb 27 11:00:26 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Feb 27 11:00:26 2026 +0530" }, "message": "[GOBBLIN-2247] Add cleanup logic for Staging directories in temporal flow (#4166)\n\nImplements graceful shutdown support across Gobblin Yarn and Temporal by:\n\nYarn Launcher: Sends GRACEFUL_SHUTDOWN signal to AM container and waits for terminal state\nTemporal Job Launcher: Executes shutdown hook to clean up staging and working directories\nThis ensures proper cleanup during job termination while allowing the Application Master time to finish cleanup tasks." }, { "commit": "9e557b872f626d3d054fb9d29ba499bd4ca0488e", "tree": "89b897d6adf3a9706305416282b34a6001254a6f", "parents": [ "d445b1e2fecf014662c335f90c3a937f967016d1" ], "author": { "name": "Gulbarga Adithya Rao", "email": "96827791+adithya2754@users.noreply.github.com", "time": "Thu Feb 19 16:35:21 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Feb 19 16:35:21 2026 +0530" }, "message": "[GOBBLIN-2246] Make Orchestrator MDM metric dimesnion configurable (#4164)\n\nAdded configurable mdm dimensions to jobSucceeded" }, { "commit": "d445b1e2fecf014662c335f90c3a937f967016d1", "tree": "f3796151953f4bb49a3a2b1652c087e8c8b990cb", "parents": [ "fc24ec493fababdc8167d685734507d6fee18bd4" ], "author": { "name": "Agam Pal Singh", "email": "32903685+agam-99@users.noreply.github.com", "time": "Wed Feb 18 20:18:27 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Feb 18 20:18:27 2026 +0530" }, "message": "Update Jar cache directory partitioning bi-monthly from monthly (#4165)\n\n* updated cache directory partitioning to bi-monthly" }, { "commit": "fc24ec493fababdc8167d685734507d6fee18bd4", "tree": "735e67b7e3b6981f00858cc5e76a3dbc85aa4063", "parents": [ "8e9146ee4e19daee303606d33007a81d4f961e73" ], "author": { "name": "Gulbarga Adithya Rao", "email": "96827791+adithya2754@users.noreply.github.com", "time": "Tue Feb 17 22:41:49 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Feb 17 22:41:49 2026 +0530" }, "message": "bumped docker testcontainers version (#4168)\n\n" }, { "commit": "8e9146ee4e19daee303606d33007a81d4f961e73", "tree": "d065685a028783f4fff04ede12cf0f86b979497a", "parents": [ "4ee5b3efcfe8e2604683fa8523a0ebd250eb1471" ], "author": { "name": "Agam Pal Singh", "email": "32903685+agam-99@users.noreply.github.com", "time": "Thu Jan 15 11:41:40 2026 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Jan 15 11:41:40 2026 +0530" }, "message": "[GOBBLIN-2245] Independent Dynamic Scaling for different Activities in Temporal WorkFlow (#4159)\n\n* independent memory and container configuration for temporal stages\n\n* route non execute activities to default queue\n\n* code refactoring\n\n* code cleanup\n\n* fix: spin up 1 container initially by default\n\n* proper queue routing for commit phase\n\n* code cleanup\n\n* - spin up initial container with execution worker\n- unit tests\n\n* - only forward NestingExecWorkflow to execution queue\n\n* removed redundant unit tests\n\n* removed redundant unit tests\n\n* copilot review fixes\n\n* PR review fixes\n\n* undo whitespace change\n\n* PR review comments addressing\n\n* PR review comments addressing\n\n* updated property name\n\n* removed unused import\n\n---------\n\nCo-authored-by: Agam Pal Singh \u003cagsingh@linkedin.com\u003e" }, { "commit": "4ee5b3efcfe8e2604683fa8523a0ebd250eb1471", "tree": "e7511ef1d938d8bf4e62ed8f344d19e667206911", "parents": [ "6619cb1253572523a9953d73859552a4086556e4" ], "author": { "name": "pratapaditya04", "email": "thesingh.adi.work@gmail.com", "time": "Thu Dec 18 11:48:41 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Dec 18 11:48:41 2025 +0530" }, "message": "[GOBBLIN-2243]Add Fallback dir for Jar caching (#4160)\n\n* added fallback caching dir\n\n* addressed comments\n\n* refactor\n\n* resolved comments\n\n* refactor" }, { "commit": "6619cb1253572523a9953d73859552a4086556e4", "tree": "4829ffae3d68aebe56c3406c3542cf8e9104a84b", "parents": [ "663929cfed42466b491bd4b5b9739d9d8d8cf2ef" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Wed Dec 17 12:00:49 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Dec 17 12:00:49 2025 +0530" }, "message": "Populate file set property in CopyableFile for IcebergFileStreamExtractor (#4161)\n\n" }, { "commit": "663929cfed42466b491bd4b5b9739d9d8d8cf2ef", "tree": "8df5b3726935001f5e033136dfc58ad8cb40f89d", "parents": [ "85dbbc675caf1e51c6b302c7f4aa29589130d0c9" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Wed Dec 17 08:21:03 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Dec 17 08:21:03 2025 +0530" }, "message": "[GOBBLIN-2209] Emit GaaS Executor Otel Metrics (#4118)\n\n" }, { "commit": "85dbbc675caf1e51c6b302c7f4aa29589130d0c9", "tree": "b06193b06a78e3a7fd6d86c2080112477d5ef29c", "parents": [ "a5a893585a88dbe3a12f654f1dd111ddb09d82d0" ], "author": { "name": "pratapaditya04", "email": "thesingh.adi.work@gmail.com", "time": "Wed Dec 03 15:50:04 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Dec 03 15:50:04 2025 +0530" }, "message": "added cert based authentication in mysql gaas (#4158)\n\n" }, { "commit": "a5a893585a88dbe3a12f654f1dd111ddb09d82d0", "tree": "ba9b21d4b940e21e28f6201695095054a57ce4df", "parents": [ "7923a96979296bcf2478b258445c9cb14a4bcece" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Tue Dec 02 08:59:12 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Dec 02 08:59:12 2025 +0530" }, "message": "attempt partial commit only in case commit policy is partial commit (#4157)\n\n" }, { "commit": "7923a96979296bcf2478b258445c9cb14a4bcece", "tree": "b562e2f22e8222a28acc2c591cd4c4a9c5ceb76d", "parents": [ "00d0ab99dd7e3f3620c1c7b11efaf2d8a6b29652" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Mon Nov 24 16:51:58 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Nov 24 16:51:58 2025 +0530" }, "message": "Add pre-publish step for IcebergSource and fix other issues in Iceberg file based copy (#4155)\n\n* Workaround for Iceberg file based copy issues\n\n* Add pre-publish step for Iceberg file based copy for file deletion" }, { "commit": "00d0ab99dd7e3f3620c1c7b11efaf2d8a6b29652", "tree": "e4a01225f3f8e6d065fdea2b0ff1543fb2d76089", "parents": [ "9ad7ad22b2f59613354db934818f9475b7d47486" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Thu Nov 20 11:59:40 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Nov 20 11:59:40 2025 +0530" }, "message": "[GOBBLIN-2239] Introduce pushMessagesSync for sync GTE emission (#4156)\n\n* introduce pushMessagesSync for GTE emission\n\n* add missing java doc\n\n* some refactoring\n\n* updated java doc" }, { "commit": "9ad7ad22b2f59613354db934818f9475b7d47486", "tree": "7a4e44b6e5ce4e27676270c91eec06333f8efe19", "parents": [ "e98452a0495e38e029b056d8953ba3ab417ae7af" ], "author": { "name": "Prateek Khandelwal", "email": "prkhandelwal@linkedin.com", "time": "Tue Nov 11 19:50:47 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Nov 11 19:50:47 2025 +0530" }, "message": "[GOBBLIN-2231] Added extractor for partition-aware file copy from Iceberg to any dest (#4154)\n\n* Added IcebergFileStreamExtractor for partition-aware file copy from Iceberg to any dest\n\n* Address review comments and fix indentation\n\n* Add IcebergFileStreamHelper change\n\n* Fix test failure" }, { "commit": "e98452a0495e38e029b056d8953ba3ab417ae7af", "tree": "5662a681f22689e9cb8fe4f6d3b811fedd95a0e5", "parents": [ "93be76b3574c9d7401a95aae43e0ff8ab52ebde6" ], "author": { "name": "Prateek Khandelwal", "email": "prkhandelwal@linkedin.com", "time": "Mon Nov 10 22:59:47 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Nov 10 22:59:47 2025 +0530" }, "message": "[GOBBLIN-2231] Implement IcebergSource to enable copying Iceberg data files to any dest (#4146)\n\n* Implement IcebergSource to enable copying Iceberg data files to any dest\n\n* Introduce partition-aware discovery \u0026 lookback period based copy for IcebergSource\n\n* Added unit tests for IcebergSource\n\n* Added unit tests for IcebergFileStreamHelper\n\n* Add FilePathWithPartition class in IcebergTable\n\n* Address review comments\n\n* Fix indentation\n\n* Handling for yyyy-MM-dd-00 partition format\n\n* Remove unused import\n\n* Address review comment\n\n* Fix core tests error\n\n* Fix failing test\n\n* Fix failing test" }, { "commit": "93be76b3574c9d7401a95aae43e0ff8ab52ebde6", "tree": "8940095fe78da5c18834ecd79d3f64d060161db1", "parents": [ "858f706fa01e5446fc5e1c261183d8827dce1063" ], "author": { "name": "pratapaditya04", "email": "thesingh.adi.work@gmail.com", "time": "Thu Nov 06 16:41:05 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Nov 06 16:41:05 2025 +0530" }, "message": "[GOBBLIN-2238]Remove unsupported ChaCha20 cipher suites from default SSL configuration (#4153)\n\n* removed chacha cipher suites\n\n* added java 8 support matrix" }, { "commit": "858f706fa01e5446fc5e1c261183d8827dce1063", "tree": "d63d4df5b0f21f32ce21b130a3e3ff5fc22a078a", "parents": [ "c12e3211ae9eab7229a9d61224c57cf11d6c7fdb" ], "author": { "name": "abhishekmjain", "email": "abhishekmjain97@gmail.com", "time": "Mon Nov 03 14:38:12 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Nov 03 14:38:12 2025 +0530" }, "message": "[GOBBLIN-2234] Fix removal of existing ACLs issue for path existing in destination (#4149)\n\n* Fix removal of existing ACL issue for path existing in destination" }, { "commit": "c12e3211ae9eab7229a9d61224c57cf11d6c7fdb", "tree": "29fe1cdf6116d59a9ecae455a154e35f573df075", "parents": [ "8ba6c1fc8721fd245101829b232530bc2ccecc7f" ], "author": { "name": "pratapaditya04", "email": "thesingh.adi.work@gmail.com", "time": "Mon Nov 03 12:37:14 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Nov 03 12:37:14 2025 +0530" }, "message": "[GOBBLIN-2237]Make AvroUtils compatible with avro 1.10 (#4152)\n\n* made avro get field compatible with avro 1.10\n\n* addressed comments\n\n* addressed comments\n\n* made debug logs" }, { "commit": "8ba6c1fc8721fd245101829b232530bc2ccecc7f", "tree": "70baed9e1a6c235eda170747fe87a9ef06de8e01", "parents": [ "7c54b95fd34d57249a1fb351b504713ed43ccce1" ], "author": { "name": "abhishekmjain", "email": "abhishekmjain97@gmail.com", "time": "Mon Nov 03 11:10:20 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Nov 03 11:10:20 2025 +0530" }, "message": "[GOBBLIN-2236] Remove dataset urn specific properties from CommitStats (#4151)\n\n* Remove dataset urn specific properties from CommitStats and ExecGobblinStats" }, { "commit": "7c54b95fd34d57249a1fb351b504713ed43ccce1", "tree": "f638c5855a58632a4ebf49e73cc9ffdc791a31f7", "parents": [ "66e95ce792f8dc43a70715e1135e2ece52dc44df" ], "author": { "name": "abhishekmjain", "email": "abhishekmjain97@gmail.com", "time": "Tue Oct 28 16:20:21 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 28 16:20:21 2025 +0530" }, "message": "Fix owner execute permission bit issue for path existing in destination (#4147)\n\n" }, { "commit": "66e95ce792f8dc43a70715e1135e2ece52dc44df", "tree": "b2f1eacd9230bcfa4592addc39a83b646128aba1", "parents": [ "179baac40d09d1542129f6158f46c016e3247fc5" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Fri Oct 24 18:08:35 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 24 18:08:35 2025 +0530" }, "message": "Evolve iceberg table schema for partition copy (#4142)\n\n" }, { "commit": "179baac40d09d1542129f6158f46c016e3247fc5", "tree": "5ca9d6e95ebf72bf5996775a01a4d6125fb0b89b", "parents": [ "20731d542a467750dcbb5b1e1c45d63ff410c6b5" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Tue Oct 07 21:36:37 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 07 21:36:37 2025 +0530" }, "message": "Emit job metrics at RM level (#4143)\n\n" }, { "commit": "20731d542a467750dcbb5b1e1c45d63ff410c6b5", "tree": "fd35558d09a3b3691654d689d82bcd2cdea89498", "parents": [ "756c5801a902f96547d40ce4166b5eda2576a815" ], "author": { "name": "pratapaditya04", "email": "thesingh.adi.work@gmail.com", "time": "Fri Sep 26 12:45:43 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Sep 26 12:45:43 2025 +0530" }, "message": "[GOBBLIN-2229] Enable access to Dataset state store during work discovery (#4144)\n\nEnable access to dataset state store during work discovery" }, { "commit": "756c5801a902f96547d40ce4166b5eda2576a815", "tree": "4c2613c2a4c9e3699dfce534fcc9a27a67d955ba", "parents": [ "5ad93db8dcefb859934ef3a31a88738a14a95b1b" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Fri Sep 19 11:55:18 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Sep 19 11:55:18 2025 +0530" }, "message": "[GOBBLIN-2227] Make dag action and spec store monitor initialization config driven (#4141)\n\n* make consumer initialization config driven\n\n* add header lines\n\n* added java docs" }, { "commit": "5ad93db8dcefb859934ef3a31a88738a14a95b1b", "tree": "6bfc38c446ef8d0127c87bddd0de4d86d0c544c4", "parents": [ "3617c8f7d75ed1f8ad55842b983f77f2113ae393" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Mon Sep 08 15:57:07 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 08 15:57:07 2025 +0530" }, "message": "Construct iceberg data files during commit step (#4140)\n\n" }, { "commit": "3617c8f7d75ed1f8ad55842b983f77f2113ae393", "tree": "7b8296f3e9832e08fc7d032d7652b163f36524c4", "parents": [ "6849776cf605fc30051830daaa333623972bf5e9" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Thu Sep 04 10:06:01 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Sep 04 10:06:01 2025 +0530" }, "message": "Revert \"[GOBBLIN-2223] Optimise writing of serialised Work Unit to File syste…\" (#4138)\n\nThis reverts commit 88555c24cc99a3de07e8fabae7475b6878560e34." }, { "commit": "6849776cf605fc30051830daaa333623972bf5e9", "tree": "0158bb411c69347f45b0d7e7874689a12dae9b6c", "parents": [ "88555c24cc99a3de07e8fabae7475b6878560e34" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Wed Sep 03 12:46:41 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 03 12:46:41 2025 +0530" }, "message": "[GOBBLIN-2225] Use random id for Workunit if recovery helper is not initialised (#4134)\n\n* Add unique id to CopyEntity instead of computing it from serialised json string\n\n* Add unique id to CopyEntity instead of computing it from serialised json string" }, { "commit": "88555c24cc99a3de07e8fabae7475b6878560e34", "tree": "5f3d0ab19774e619217d696c5420dc42fc8390d8", "parents": [ "1ab8993124cf46e37e430af384fff652c350b0e6" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Tue Sep 02 19:58:38 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Sep 02 19:58:38 2025 +0530" }, "message": "[GOBBLIN-2223] Optimise writing of serialised Work Unit to File system (#4133)\n\n* Optimise writing of serialised Work Unit to File system\n\n* Optimise writing of serialised Work Unit to File system" }, { "commit": "1ab8993124cf46e37e430af384fff652c350b0e6", "tree": "1159b49808ccfa2d2d88db16def3df73439a9133", "parents": [ "877a0f8f1a728ec70eb02f1a5e504ee62768f16c" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Wed Aug 27 08:11:08 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Aug 27 08:11:08 2025 +0530" }, "message": "Remove unnecessary CopyEntity deserialisation (#4132)\n\n" }, { "commit": "877a0f8f1a728ec70eb02f1a5e504ee62768f16c", "tree": "506fe046530d2bccc8c376a3e9ecd442760b38a1", "parents": [ "f7fe367c4089ae7b0895c442d5e6a18b872b5761" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Tue Aug 26 19:44:51 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Aug 26 19:44:51 2025 +0530" }, "message": "Change data files from byte[] to list of base64 encoded string (#4131)\n\n" }, { "commit": "f7fe367c4089ae7b0895c442d5e6a18b872b5761", "tree": "3727a0bfdac7b6a96e9da98f0843d47c1cf3853a", "parents": [ "08272be25e8efc69b6fb7c33386e0da02b5fbf17" ], "author": { "name": "vsinghal85", "email": "vsinghal85@gmail.com", "time": "Mon Aug 25 12:08:39 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Aug 25 12:08:39 2025 +0530" }, "message": "DQ handling for no bytes read scenarios and metrics for bytes read/written (#4135)\n\n* introduce not evaluated flag for null bytes read case and bytes read/written metrics\n\n---------\n\nCo-authored-by: Vaibhav Singhal \u003cvaibsing@vaibsing-mn7618.linkedin.biz\u003e" }, { "commit": "08272be25e8efc69b6fb7c33386e0da02b5fbf17", "tree": "fdcabdb24252c3bd493b29572938cae61613cf1d", "parents": [ "5c5db48a9c04a071dc95bc270cd6055381ddd8c6" ], "author": { "name": "Suryakant Agarwal", "email": "agarwalsuryakant561@gmail.com", "time": "Wed Aug 13 20:36:09 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Aug 13 20:36:09 2025 +0530" }, "message": "[GOBBLIN-2218] Fix bug for executions failing with one-time schedule (#4130)\n\n* Fix Bug For Executions Failing With Only One Schedule\n---------\n\nCo-authored-by: Suryakant Agarwal \u003csuragarwal@linkedin.com\u003e" }, { "commit": "5c5db48a9c04a071dc95bc270cd6055381ddd8c6", "tree": "bc8fd93ae552aabbe780686700a05e809e9c9da7", "parents": [ "57d728a34d96422f52c081dd65190bbccf6bb5de" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Wed Aug 13 16:07:23 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Aug 13 16:07:23 2025 +0530" }, "message": "Emit metrics if dag act throws exception (#4129)\n\n" }, { "commit": "57d728a34d96422f52c081dd65190bbccf6bb5de", "tree": "b3d52a7c3c1c5d50c1bfd7a105ff37140034fb62", "parents": [ "8c6710322f8cba2a0d7d5897867b46cd2da51cfe" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Mon Aug 11 17:06:00 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Aug 11 17:06:00 2025 +0530" }, "message": "Update job status before adding REEVALUATE dag action (#4128)\n\n" }, { "commit": "8c6710322f8cba2a0d7d5897867b46cd2da51cfe", "tree": "526dae7995697606cb4ba849797c44d6380c2531", "parents": [ "26b3b778f71b47531bfca2c88758b0a82801077e" ], "author": { "name": "vsinghal85", "email": "vsinghal85@gmail.com", "time": "Fri Aug 08 13:55:06 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Aug 08 13:55:06 2025 +0530" }, "message": "Evaluate data quality only when Policies are applied (#4127)\n\n* add shouldEvaluate data quality check at dataset level\n\n* Refactor checks for invoking evaluateAndEmitDataQuality\n\n---------\n\nCo-authored-by: Vaibhav Singhal \u003cvaibsing@vaibsing-mn7618.linkedin.biz\u003e" }, { "commit": "26b3b778f71b47531bfca2c88758b0a82801077e", "tree": "1aa3fd8700ef0bf5e52d6fb49370829dc04cb39f", "parents": [ "4c4d2ba61261c25de4930d96ba186874bf312082" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Thu Aug 07 18:06:49 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Aug 07 18:06:49 2025 +0530" }, "message": "Send flow event if flow deadline is crossed (#4126)\n\n" }, { "commit": "4c4d2ba61261c25de4930d96ba186874bf312082", "tree": "d8753ca6e60e5b6f5ef340d7dd59f73e6f1341a2", "parents": [ "9151f93668cbb403d319d4474a4d313beb81ef70" ], "author": { "name": "vsinghal85", "email": "vsinghal85@gmail.com", "time": "Tue Aug 05 20:55:11 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Aug 05 20:55:11 2025 +0530" }, "message": "[GOBBLIN-2204] Implement FileSize Data Quality for FileBasedCopy\n\n* Compute data quality and update task states\n\n* Retry for failed data quality check\n\n* Only do data quality check if data quality flag is enabled\n\n* Computing overall dataquality of data set and adding it to dataset summary\n\n* Changes for testing failed data quality changes\n\n* Updating task state and computing overall data quality\n\n* Refactor and checkstyle fix\n\n* Propogate data quality from fork to task state\n\n* Refactoring and unit tests\n\n* Add unit tests for FileAwareInputStreamDataWriterTest\n\n* refactor changes\n\n* Compute task level data quality\n\n* Some refactoring and code cleanup\n\n* Update tests\n\n* Fix checkstyle failures\n\n* Code cleanup\n\n* Adding metrics for overall data quality\n\n* Address PR comments and add metric for no of files evaluated for data quality\n\n* Refactor dq evaluation\n\n* Address PR comments\n\n* Introduce data quality flag, to control and manage job failures due to data quality\n\n* Address PR comments\n\n* Switch back to CounterBuilder approach for metrics\n\n* Evaluate data quality only for commit via CommitActivity\n\n* Remove unused enum\n\n* move getDataQuality to JobState\n\n* Address PR comments\n\n* Fix the policy evaluation logic when both optional and mandatory policies are provided\n\n* Remove codestyle errors and remove flag check at task level\n\n---------\n\nCo-authored-by: Vaibhav Singhal \u003cvaibsing@vaibsing-mn7618.linkedin.biz\u003e" }, { "commit": "9151f93668cbb403d319d4474a4d313beb81ef70", "tree": "8fe7254744bfff3553571c087ebf8deb63199b9b", "parents": [ "ddd9468b3deda7ca0935c48796e05ed68f85d85c" ], "author": { "name": "NamsB7", "email": "126075176+NamsB7@users.noreply.github.com", "time": "Mon Jul 28 18:55:10 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Jul 28 18:55:10 2025 +0530" }, "message": "Insert Final summary issue entry in issues table (#4125)\n\n" }, { "commit": "ddd9468b3deda7ca0935c48796e05ed68f85d85c", "tree": "0b63ddd14f64fe057662c013f30e03b6c54de899", "parents": [ "51773abba115e1bebec413662063286789c712db" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Mon Jul 28 10:14:27 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Jul 28 10:14:27 2025 +0530" }, "message": "Propagate file system job properties to Work units (#4124)\n\n" }, { "commit": "51773abba115e1bebec413662063286789c712db", "tree": "9fe4b372ce258be809d881c59afdc3f698f0c0e1", "parents": [ "c6e2112c06aa46061351636039204141162d1f05" ], "author": { "name": "NamsB7", "email": "126075176+NamsB7@users.noreply.github.com", "time": "Fri Jul 25 15:38:00 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Jul 25 15:38:00 2025 +0530" }, "message": "[GOBBLIN-2211] Implement Error Classification based on execution issues (#4121)\n\nImplemented Error Classification for jobs based on execution issues\n\nAdded 2 new tables for ErrorPatternStore along with its in memory implementation" }, { "commit": "c6e2112c06aa46061351636039204141162d1f05", "tree": "df6d4d5c039810b6467c157b76d066555b8a0d11", "parents": [ "5da75cebec5238656b3818a52d63ec68d630b714" ], "author": { "name": "thisisArjit", "email": "Thisisarjit@gmail.com", "time": "Wed Jul 16 16:03:20 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Jul 16 16:03:20 2025 +0530" }, "message": "Fix AvroCompactionTaskTest \u0026 OrcCompactionTaskTest (#4122)\n\n" }, { "commit": "5da75cebec5238656b3818a52d63ec68d630b714", "tree": "9e4094935020385830bae6f074c85a76c6e9f124", "parents": [ "1c6c24fe47c1e4df3bfbdd60c161751248c9702c" ], "author": { "name": "vsinghal85", "email": "vsinghal85@gmail.com", "time": "Wed May 21 15:01:32 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed May 21 15:01:32 2025 +0530" }, "message": "[GOBBLIN-2208] Acl preservation fix (#4117)\n\n* Preserving ACL for directories" }, { "commit": "1c6c24fe47c1e4df3bfbdd60c161751248c9702c", "tree": "b0b61200606889171e0ea14be3c7de7671768da6", "parents": [ "8d71c89cc16a1b8d2d4bf7f19182fca32a3608c4" ], "author": { "name": "pratapaditya04", "email": "thesingh.adi.work@gmail.com", "time": "Mon May 12 12:08:11 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon May 12 12:08:11 2025 +0530" }, "message": "[GOBBLIN-2207] Make temporal unique classifier GaaS attempt aware (#4116)\n\n* made temporal unique classifier gaas attempt aware" }, { "commit": "8d71c89cc16a1b8d2d4bf7f19182fca32a3608c4", "tree": "1e7e781e43371c4f37500877149180deb5707d62", "parents": [ "be7c1a7f51c096f7c35746a29987fd09581d6556" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Mon May 05 13:11:35 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon May 05 13:11:35 2025 +0530" }, "message": "[GOBBLIN-2206] Revert extra execute bit getting set in ManifestDistcp (#4115)\n\n* added check to add extra set permission step\n\n* add copy constructor for ownerandpermission\n\n* remove extra null check for checkstyle failure" }, { "commit": "be7c1a7f51c096f7c35746a29987fd09581d6556", "tree": "075783f74ec48011d4fd919a176ccfe2e3c96ff8", "parents": [ "b535daf55455cd0260d5af06fd21462046bb1727" ], "author": { "name": "pratapaditya04", "email": "thesingh.adi.work@gmail.com", "time": "Fri Apr 25 15:38:59 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Apr 25 15:38:59 2025 +0530" }, "message": "[GOBBLIN-2200] Onboard a new job executionId to deprecate Azkaban executionId (#4108)\n\n* created a new unique jobExecutionIdentifier to deprecate Azkaban executionId" }, { "commit": "b535daf55455cd0260d5af06fd21462046bb1727", "tree": "b3032f1da82884b6425b61148741d6f9dc951da8", "parents": [ "4cfa2f733c913a71338286e477af027dec9e7889" ], "author": { "name": "vsinghal85", "email": "vsinghal85@gmail.com", "time": "Fri Apr 25 15:00:56 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Apr 25 15:00:56 2025 +0530" }, "message": "Add data quality field in GaaSJobObservabilityEventSchema (#4114)\n\n* Add data quality field in GaaSJobObservabilityEventSchema\n\n---------\n\nCo-authored-by: Vaibhav Singhal \u003cvaibsing@vaibsing-mn7618.linkedin.biz\u003e" }, { "commit": "4cfa2f733c913a71338286e477af027dec9e7889", "tree": "795d91cc735333a11ce1089f3915f1643f488c48", "parents": [ "05487669f25d0323095cf4a555d3ef902eaf2870" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Wed Apr 23 15:54:56 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Apr 23 15:54:56 2025 +0530" }, "message": "[GOBBLIN-2203] Add Support for Delete Manifests in Iceberg Full Table Replication (#4112)\n\n* added support for delete manifests in iceberg full table replication\n\n* remove unused imports" }, { "commit": "05487669f25d0323095cf4a555d3ef902eaf2870", "tree": "1bcb9c59c3f26e0aa53824d41f98e00310bc787c", "parents": [ "4a4c1711e247de774baf34a1ae0041d1cd43a30a" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Mon Apr 21 10:00:33 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Apr 21 10:00:33 2025 +0530" }, "message": "change aggregation temporality to DELTA instead of CUMULATIVE (#4111)\n\n" }, { "commit": "4a4c1711e247de774baf34a1ae0041d1cd43a30a", "tree": "eba82b967b7a798b699f9b0122d0ea26dbab21ce", "parents": [ "60a9d31720bd9216f4a09a11880b84d3d82d9e1c" ], "author": { "name": "vsinghal85", "email": "vsinghal85@gmail.com", "time": "Sat Apr 05 01:46:18 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sat Apr 05 01:46:18 2025 +0530" }, "message": "Checkstyle fix (#4110)\n\nCo-authored-by: Vaibhav Singhal" }, { "commit": "60a9d31720bd9216f4a09a11880b84d3d82d9e1c", "tree": "cdfc667dfc83944fc969827a1990c548b9070fb8", "parents": [ "a978b299ba1f2762786f5fecee5a960269db219b" ], "author": { "name": "vsinghal85", "email": "vsinghal85@gmail.com", "time": "Fri Apr 04 19:37:01 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Apr 04 19:37:01 2025 +0530" }, "message": "DagProcessingThread bug fix (#4109)\n\n* DagProcessingThread bug fix and enhance logging\n\n---------\n\nCo-authored-by: Vaibhav Singhal" }, { "commit": "a978b299ba1f2762786f5fecee5a960269db219b", "tree": "539dcba6c8c2fc35049d7b484b4b671894489152", "parents": [ "8b250d4ec1d1df554c62da9e15b4d0e7e29cbd27" ], "author": { "name": "Prateek Khandelwal", "email": "prkhandelwal@linkedin.com", "time": "Tue Mar 18 13:02:32 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Mar 18 13:02:32 2025 +0530" }, "message": "[GOBBLIN-2199] Support dynamic container scaling based on target completion time \u0026 WU count (#4106)\n\n" }, { "commit": "8b250d4ec1d1df554c62da9e15b4d0e7e29cbd27", "tree": "4b56a6e2f93a6c2c4bb94777000d182855ea4e1e", "parents": [ "a37a2411fcabe9d83676d4fd073296fe6bbd3634" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Fri Mar 07 10:49:35 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Mar 07 10:49:35 2025 +0530" }, "message": "[GOBBLIN-2147] Added lookback time fetch in partitioned filesource (#4044)\n\nAdd lookback time fetch in partitioned filesource\nIn case no lookback time is passed, default it to min_watermark" }, { "commit": "a37a2411fcabe9d83676d4fd073296fe6bbd3634", "tree": "681dc7c6411312197cf0e93d6cc69c1ca764140c", "parents": [ "e64f7077dd9d7e33e032db9484751a5602840801" ], "author": { "name": "Prateek Khandelwal", "email": "prkhandelwal@linkedin.com", "time": "Thu Mar 06 20:01:42 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Mar 06 20:01:42 2025 +0530" }, "message": "Prevent dagProcessingEngine thread termination by handling exceptions gracefully (#4105)\n\n" }, { "commit": "e64f7077dd9d7e33e032db9484751a5602840801", "tree": "5b687aef9f29a4bb5fc1e9e0245daf4fda2ca7ee", "parents": [ "1d9398214e03ebed55b6d00a7b8eb39ec1fcce22" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Thu Feb 27 11:03:58 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Feb 27 11:03:58 2025 +0530" }, "message": "[GOBBLIN-2190] Added param to create Activityoptions without HeartbeatTimeout (#4103)\n\n" }, { "commit": "1d9398214e03ebed55b6d00a7b8eb39ec1fcce22", "tree": "2be5fba75f2e50475ec77449eae21f5a44117aec", "parents": [ "f2bcdc7329dedd5a0d19fb923f60de278ed92d6c" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Wed Feb 26 16:11:29 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Feb 26 16:11:29 2025 +0530" }, "message": "[GOBBLIN-2190] Added SUBMITGTE activity to ActivityType enum (#4102)\n\n" }, { "commit": "f2bcdc7329dedd5a0d19fb923f60de278ed92d6c", "tree": "ca973065da0610da6acab07278d8df27ca8e2dc8", "parents": [ "eb60a2c01a38f2df0faabe2ebf9325cf51d87e2d" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Tue Feb 25 14:19:41 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Feb 25 14:19:41 2025 +0530" }, "message": "[GOBBLIN-2190] Implement ActivityType \u0026 add HeartBeat for Temporal Activities (#4093)\n\nMade StartToClose timeout configurable and added heartbeat" }, { "commit": "eb60a2c01a38f2df0faabe2ebf9325cf51d87e2d", "tree": "94404e400a39acf85edf9923af309f01da6473d4", "parents": [ "8dde23a26e06c83cc027fb8ed725aaeccfd8b134", "db9cf7ed56a937b5cb4f36834cd6b6d456e5e6ea" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Thu Feb 20 15:22:22 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Feb 20 15:22:22 2025 +0530" }, "message": "[GOBBLIN-2189] Implemented ContainerCompletion callback in DynamicScalingYarnService\n\n" }, { "commit": "db9cf7ed56a937b5cb4f36834cd6b6d456e5e6ea", "tree": "a71fc18dbb877e83fdd5bcdd20dc62f0f8069487", "parents": [ "dee3fe4c4e533eb9eecb0d00918bc9375ee9cd91" ], "author": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Thu Feb 20 12:34:36 2025 +0530" }, "committer": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Thu Feb 20 12:34:36 2025 +0530" }, "message": "corrected log statement\n" }, { "commit": "dee3fe4c4e533eb9eecb0d00918bc9375ee9cd91", "tree": "f85cc1050b151d7dc39e62118f9cc82d6c84970d", "parents": [ "f341c1e59e1d11731ad2f1b15c3e64b3cfd282a6" ], "author": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Thu Feb 20 12:30:55 2025 +0530" }, "committer": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Thu Feb 20 12:30:55 2025 +0530" }, "message": "added a comment\n" }, { "commit": "8dde23a26e06c83cc027fb8ed725aaeccfd8b134", "tree": "87dc9e34dc09f40d4fbd074eb5bdbe7e01290348", "parents": [ "23c4481c7d00bdc3fcef3e1aaad8679acca951cf" ], "author": { "name": "abhishekmjain", "email": "abhishekmjain97@gmail.com", "time": "Thu Feb 20 10:20:24 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Feb 20 10:20:24 2025 +0530" }, "message": "Upgrade hadoop-azure-datalake dependency (#4099)\n\n" }, { "commit": "23c4481c7d00bdc3fcef3e1aaad8679acca951cf", "tree": "dbe3c3622b62ed9c2b373206c84658718aa908f9", "parents": [ "adef734100aa3eb076a0aeea8d202c4131543d02" ], "author": { "name": "abhishekmjain", "email": "abhishekmjain97@gmail.com", "time": "Mon Feb 17 23:07:12 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Feb 17 23:07:12 2025 +0530" }, "message": "[GOBBLIN-2194] Close temporal metrics scope on job completion (#4097)\n\nClose temporal metrics scope on job completion\r\nOverride close in GobblinTemporalJobLauncher" }, { "commit": "f341c1e59e1d11731ad2f1b15c3e64b3cfd282a6", "tree": "80f1bd38c102788ad11758ea209951590303c854", "parents": [ "861916aa5485bc9d3f02943047bd807088445ec2" ], "author": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Mon Feb 17 10:34:55 2025 +0530" }, "committer": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Mon Feb 17 10:34:55 2025 +0530" }, "message": "addressed comments\n" }, { "commit": "861916aa5485bc9d3f02943047bd807088445ec2", "tree": "7c8072f1d523c520e4c94e189b645d99384e27eb", "parents": [ "729bebbfe452389335d46932fb5ae1849955654a" ], "author": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Tue Feb 11 08:24:06 2025 +0530" }, "committer": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Tue Feb 11 08:24:06 2025 +0530" }, "message": "removed extra spaces\n" }, { "commit": "729bebbfe452389335d46932fb5ae1849955654a", "tree": "b364a76d707b28fb8f5de01b69505d51b72ea753", "parents": [ "29891fd62fc15123fdebf0c51e0a5110b6726401" ], "author": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Mon Feb 10 21:51:39 2025 +0530" }, "committer": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Mon Feb 10 21:51:39 2025 +0530" }, "message": "added test for non-replacement case\n" }, { "commit": "29891fd62fc15123fdebf0c51e0a5110b6726401", "tree": "87e01f0ff84079af67b516bb6342c03898923d30", "parents": [ "4c7b019d72bab5c6a8f6c6296b80e150fac50fa6" ], "author": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Mon Feb 10 21:30:16 2025 +0530" }, "committer": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Mon Feb 10 21:30:16 2025 +0530" }, "message": "added test for handle container completion\n" }, { "commit": "4c7b019d72bab5c6a8f6c6296b80e150fac50fa6", "tree": "e2c7182bb50eb2c093d417738339222fa8b3e853", "parents": [ "90b7e6c66f6dc7da2287131319bb917e5a469d71" ], "author": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Fri Feb 07 10:55:04 2025 +0530" }, "committer": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Fri Feb 07 10:55:04 2025 +0530" }, "message": "fixed typo\n" }, { "commit": "90b7e6c66f6dc7da2287131319bb917e5a469d71", "tree": "c89e4b2b45d69b23d99cb08119847bbc75bd8cc6", "parents": [ "7a80eb7910a5a48908988ea799fbc5306cd393bb" ], "author": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Thu Feb 06 10:14:23 2025 +0530" }, "committer": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Thu Feb 06 10:14:23 2025 +0530" }, "message": "addressed review comments\n" }, { "commit": "adef734100aa3eb076a0aeea8d202c4131543d02", "tree": "00eaae803986cebdfa76a4b91ddfd231b88eb1b4", "parents": [ "09a3e21add10f3a42a3c89c531db9e7bc84d5f2e" ], "author": { "name": "abhishekmjain", "email": "abhishekmjain97@gmail.com", "time": "Wed Feb 05 20:27:16 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Feb 05 20:27:16 2025 +0530" }, "message": "Add metrics support for Temporal Workflow (#4095)\n\n" }, { "commit": "7a80eb7910a5a48908988ea799fbc5306cd393bb", "tree": "192ad6b2b48736a6cc3bd39443a23ab5f5aa1023", "parents": [ "e69a77ea19353c0dbf4c08cd381bf281eef51022" ], "author": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Fri Jan 31 11:51:14 2025 +0530" }, "committer": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Fri Jan 31 11:51:14 2025 +0530" }, "message": "addressed review comments\n" }, { "commit": "09a3e21add10f3a42a3c89c531db9e7bc84d5f2e", "tree": "d5028bbd58eda8c1528db2fb134503576519daf0", "parents": [ "e584e5b49d24aaed398d082cf1e07bac8c13f453" ], "author": { "name": "abhishekmjain", "email": "abhishekmjain97@gmail.com", "time": "Wed Jan 29 22:59:40 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Jan 29 22:59:40 2025 +0530" }, "message": "[GOBBLIN-2191] Shutdown GobblinJobLauncher executor (#4094)\n\nShutdown GobblinJobLauncher executor to fix AM graceful shutdown" }, { "commit": "e69a77ea19353c0dbf4c08cd381bf281eef51022", "tree": "be793532f189f601fb85c076e3033f8a7180238b", "parents": [ "089e2e0b18ce902258811234f5e1d0700a911692" ], "author": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Tue Jan 21 22:48:25 2025 +0530" }, "committer": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Tue Jan 21 22:48:25 2025 +0530" }, "message": "improved dummy tests\n" }, { "commit": "e584e5b49d24aaed398d082cf1e07bac8c13f453", "tree": "36c57aa743cc98e9469f9c6935840c290f1ea8f1", "parents": [ "a0cef2822d9558c7817bc9ff78f786b8bd58a8db" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Mon Jan 20 09:52:45 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Jan 20 09:52:45 2025 +0530" }, "message": "[GOBBLIN-2182] Cleanup workdir in Gobblin-on-Temporal execution (#4085)\n\n* added cleanup for got controlled with a config" }, { "commit": "089e2e0b18ce902258811234f5e1d0700a911692", "tree": "a1ff4ae93f42bd7207057003b3fee4b628d7e16a", "parents": [ "e8f757412f64fca8d0aa2866fd8c6269aa997739" ], "author": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Tue Jan 14 19:30:30 2025 +0530" }, "committer": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Tue Jan 14 19:30:30 2025 +0530" }, "message": "fix updating workforce plan while containerCompletion\n" }, { "commit": "e8f757412f64fca8d0aa2866fd8c6269aa997739", "tree": "1079459db517a56f8c4f028596e1a5d968ff92c8", "parents": [ "a0cef2822d9558c7817bc9ff78f786b8bd58a8db" ], "author": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Mon Jan 13 15:43:52 2025 +0530" }, "committer": { "name": "Vivek Rai", "email": "vivekrai5839@gmail.com", "time": "Mon Jan 13 15:43:52 2025 +0530" }, "message": "initial dynamicscalingyarnservice changes for handleContainerCompletion\n" }, { "commit": "a0cef2822d9558c7817bc9ff78f786b8bd58a8db", "tree": "ef9db271259bc26e9462ae73a6784ac09143e1c3", "parents": [ "87c8ab4a4297c213058fabfb1752f33a2aa47c39" ], "author": { "name": "Kip Kohn", "email": "ckohn@linkedin.com", "time": "Tue Jan 07 02:46:55 2025 -0800" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Jan 07 02:46:55 2025 -0800" }, "message": "[GOBBLIN-2188] Define `Initializer.AfterInitializeMemento` for GoT to tunnel state from `GenerateWorkUnits` to `CommitActivity` (#4091)\n\n" }, { "commit": "87c8ab4a4297c213058fabfb1752f33a2aa47c39", "tree": "bd259ab02d47e432b836c9685b6d63aae439d760", "parents": [ "3269764a1141acbeeb9f5f8f867ccfeb803b22d5" ], "author": { "name": "Vivek Rai", "email": "43493515+Blazer-007@users.noreply.github.com", "time": "Mon Jan 06 19:08:05 2025 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Jan 06 05:38:05 2025 -0800" }, "message": "[GOBBLIN-2187] Prevent NaN while generating `WorkUnitsSizeSummary.Distillation` (#4090)\n\n" }, { "commit": "3269764a1141acbeeb9f5f8f867ccfeb803b22d5", "tree": "de0cb0f3f12d70d065311b1d453cf9f6ee72a097", "parents": [ "7dbeebf7fecc748ea3ef90cc318214cf26ba5afa" ], "author": { "name": "Kip Kohn", "email": "ckohn@linkedin.com", "time": "Wed Jan 01 21:39:23 2025 -0800" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Jan 01 21:39:23 2025 -0800" }, "message": "[GOBBLIN-2186] Emit GoT GTEs to time `WorkUnit` prep and to record volume of Work Discovery (#4089)\n\n" }, { "commit": "7dbeebf7fecc748ea3ef90cc318214cf26ba5afa", "tree": "53a84e1c64afdbfdcd439f92c19e2b62908fe578", "parents": [ "faecb356003bfb1909e026428372feee47f36d48" ], "author": { "name": "Kip Kohn", "email": "ckohn@linkedin.com", "time": "Mon Dec 23 23:14:22 2024 -0800" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Dec 23 23:14:22 2024 -0800" }, "message": "[GOBBLIN-2185] Recommend GoT Dynamic Auto-Scaling using heuristics based on `WorkUnitsSizeSummary` (#4087)\n\n* Implement GoT Dynamic auto-scaling PoC of `WorkUnitsSizeSummary`-driven linear heuristic\r\n\r\n* Do not generate `@Setter`s for `@Data` POJOs, for which deserialization support prevents having `final` members\r\n\r\n* Align choice of directory between `FsScalingDirectivesRecipient` and `FsScalingDirectivesSource`, and ensure various handles get closed\r\n" }, { "commit": "faecb356003bfb1909e026428372feee47f36d48", "tree": "5e3055b85c058ac35797f87a12dab17fa28cb5fe", "parents": [ "2595f6c328df95599634e0f95d9bf6ddf8150fb2" ], "author": { "name": "vsinghal85", "email": "vsinghal85@gmail.com", "time": "Sat Dec 14 01:59:04 2024 +0530" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Dec 13 12:29:04 2024 -0800" }, "message": "[GOBBLIN-2181] Ensure `DagTask::conclude` after a non-transient exception (#4084)\n\n" } ], "next": "2595f6c328df95599634e0f95d9bf6ddf8150fb2" }