site/src/site/releasenotes/groovy-4.0.adoc

:source-highlighter: pygments
:pygments-style: emacs
:icons: font

Groovy 4 builds upon existing features and streamlines various legacy options.

[width="80%",align="center"]
|===
a| NOTE: _WARNING:_
Material on this page is still under development! +
We are currently releasing alpha versions of Groovy 4.0 with a goal
of gathering feedback on the language changes from our community. In addition, early versions assist other projects
and tool vendors within the Groovy ecosystem to begin assessing the impact of moving to/supporting Groovy 4.0.
Caution should be exercised if using new features as the details may change before final release. +
Some features described here as "incubating" may become stable before 4.0.0 final is released,
others are expected to remain incubating for version 4.
We don't recommend using alpha versions or incubating features for production systems.
|===


[[Groovy4.0-naming-changes]]
== Important naming/structuring changes

[[Groovy4.0-maven-coordinates]]
=== Maven coordinate change

In Groovy 4.0, the _groupId_ of the maven coordinates for Groovy have changed from `org.codehaus.groovy`
to `org.apache.groovy`. Please update your Gradle/Maven/other build settings appropriately.

[[Groovy4.0-split-package-renaming]]
=== Legacy package removal

The Java Platform Module System (JPMS) requires that classes in distinct modules
have distinct package names. Groovy has its own "modules" but these haven't
historically been structured according to the above requirement.

Groovy 3 provided duplicate versions of numerous classes (in old and new packages)
to allow Groovy users to migrate towards the new JPMS compliant package names.
See the link:http://groovy-lang.org/releasenotes/groovy-3.0.html#Groovy3.0releasenotes-Splitpackages[Groovy 3 release notes]
for more details.
In short, time to stop using `groovy.util.XmlSlurper` and start using `groovy.xml.XmlSlurper`.
Similarly, you should now be using `groovy.xml.XmlParser`, `groovy.ant.AntBuilder`, `groovy.test.GroovyTestCase`
and the other classes mentioned in the prior mentioned Groovy 3 release notes.

[[Groovy4.0-module-changes]]
=== Module changes for groovy-all

Based on user feedback and download statistics, we rejigged which modules are included in the `groovy-all` pom
(link:https://issues.apache.org/jira/browse/GROOVY-9647[GROOVY-9647]).
The `groovy-yaml` module is fairly widely used and is now included in `groovy-all`.
The `groovy-testng` module is less widely used and is no longer included in `groovy-all`.
Please adjust your build script dependencies if needed.
If you are using the Groovy distribution, no changes are required since it
includes the optional modules.

[[Groovy4.0-consolidation]]
== Legacy consolidation

[[Groovy4.0-parrot-only]]
=== Old parser removal

Groovy 3 introduced the new "Parrot" parser which supports lambdas, method
references, and numerous other tweaks. In Groovy 3, You could still revert back to the old parser
if you wanted. In Groovy 4, the old Antlr2 based parser is removed.
Please use older versions of Groovy if you require the old parser.

[[Groovy4.0-indy-only]]
=== Classic bytecode generation removal

For many versions, Groovy could generate classic _call-site based_ bytecode
or bytecode targeting the JDK7+ invoke dynamic ("indy") bytecode instructions.
You could switch between them with a compiler switch and we had two sets of
jars ("normal" and "-indy") built with and without the switch enabled.
In Groovy 4.0, only bytecode using the latter approach can be generated.
There is now one set of jars and they happen to be indy flavored.

Currently, the Groovy runtime still contains any necessary support for
classes compiled using older versions of Groovy.
Please use Groovy versions up to 3.x if you need to generate the older
style bytecode.

This work was originally planned for Groovy 3.0, but there were numerous places
where "indy" code was noticeably slower than "classic" bytecode.
We have made numerous speed improvements (starting with https://issues.apache.org/jira/browse/GROOVY-8298[GROOVY-8298])
and have some ability to tune internal thresholds (search the code base for
`groovy.indy.optimize.threshold` and `groovy.indy.fallback.threshold`).
That work gave us useful speed improvements, but we welcome further feedback
to help improve overall performance of the indy bytecode.

[[Groovy4.0-new]]
== New features

[[Groovy4.0-new-checkers]]
=== Built-in type checkers

Groovy's static nature includes an extensible type-checking mechanism.
This mechanism allows users to:

* selectively weaken type checking to allow more dynamic style code to parse static checking, or
* strengthen type checking, allowing Groovy to be much stricter than Java in scenarios where that is desirable

So far, we know this feature has been used internally by companies (e.g. type-checked DSLs),
but we haven't seen widespread sharing of type checker extensions.
From Groovy 4, we plan to bundle some select type checkers within the optional
`groovy-typecheckers` module,
to encourage further use of this feature.

The first inclusion is a checker for regular expressions. Consider the following code:

[source,groovy]
--------------------------------------
def newYearsEve = '2020-12-31'
def matcher = newYearsEve =~ /(\d{4})-(\d{1,2})-(\d{1,2}/
--------------------------------------

This passes compilation but fails at runtime with a `PatternSyntaxException`
since we "accidentally" left off the final closing bracket.
We can get this feedback at compilation time using the new checker as follows:

[source,groovy]
--------------------------------------
import groovy.transform.TypeChecked

@TypeChecked(extensions = 'groovy.typecheckers.RegexChecker')
def whenIs2020Over() {
    def newYearsEve = '2020-12-31'
    def matcher = newYearsEve =~ /(\d{4})-(\d{1,2})-(\d{1,2}/
}
--------------------------------------

Which gives this expected compilation error:

--------------------------------------
1 compilation error:
[Static type checking] - Bad regex: Unclosed group near index 26
(\d{4})-(\d{1,2})-(\d{1,2}
 at line: 6, column: 19
--------------------------------------

As usual, Groovy's compiler customization mechanisms would allow you to
simplify application of such checkers, e.g. make it apply globally
using a compiler configuration script, as just one example.

We welcome further feedback on additional type checker extensions to include within Groovy.

[[Groovy4.0-new-macro-builtins]]
=== Built-in macro methods

Groovy macros were introduced in Groovy 2.5 to make it easier to create AST transforms
and other code which manipulates the compiler AST data structures.
One part of macros, known as macro methods, allows what looks like a global method call
to be replaced with transformed code during compilation.

A bit like type checker extensions, we know this feature has been used in numerous places,
but so far, we haven't seen widespread sharing of macro methods.
From Groovy 4, we plan to bundle some select macro methods within the optional
`groovy-macro-library` module,
to encourage further use of this feature.

The first inclusions assist with old-school debugging (poor man's serialization?).
Suppose during coding you have defined numerous variables:

[source,groovy]
--------------------------------------
def num = 42
def list = [1 ,2, 3]
def range = 0..5
def string = 'foo'
--------------------------------------

Suppose now you want to print those out for debugging purposes.
You could write some appropriate `println` statements and maybe sprinkle in some
calls to `format()`. You might even have an IDE help you do that.
Alternatively, the `NV` macro method comes to the rescue:

[source,groovy]
--------------------------------------
println NV(num, list, range, string)
--------------------------------------

which outputs:

--------------------------------------
num=42, list=[1, 2, 3], range=[0, 1, 2, 3, 4, 5], string=foo
--------------------------------------

Here, the `NV` macro method springs into action during the compilation process.
The compiler replaces the apparent global `NV` method with a `println` statement
which outputs the variables name and `toString()` value.

Two other variations exist. `NVI` calls Groovy's `inspect()` method rather than
`toString()` and `NVD` calls Groovy's `dump()` method. So this code:

[source,groovy]
--------------------------------------
println NVI(range)
--------------------------------------

produces the following output:

--------------------------------------
range=0..5
--------------------------------------

And this code:

[source,groovy]
--------------------------------------
println NVD(range)
--------------------------------------

yields:

--------------------------------------
range=<groovy.lang.IntRange@14 from=0 to=5 reverse=false inclusive=true modCount=0>
--------------------------------------

We welcome further feedback on additional macro methods to include within Groovy.
If you do enable this optional module but want to limit which macro methods are enabled,
there is now a mechanism to disable individual macro methods (and extension methods)
link:https://issues.apache.org/jira/browse/GROOVY-9675[GROOVY-9675].

[[Groovy4.0-new-javashell]]
=== JavaShell (incubating)

A Java equivalent of GroovyShell, allowing to more easily work with snippets of Java code.
As an example, the following snippet shows compiling a _record_ (JDK14) and checking its `toString` with Groovy:

[source,groovy]
--------------------------------------
import org.apache.groovy.util.JavaShell
def opts = ['--enable-preview', '--release', '14']
def src = 'record Coord(int x, int y) {}'
Class coordClass = new JavaShell().compile('Coord', opts, src)
assert coordClass.newInstance(5, 10).toString() == 'Coord[x=5, y=10]'
--------------------------------------

This feature is used in numerous places within the Groovy codebase for testing purposes.
Various code snippets are compiled using both Java and Groovy to ensure the compiler is behaving as intended.
We also use this feature to provide a productivity enhancement for polyglot developers allowing
Java code to be run (as Java) within the Groovy Console:

image:img/groovyconsole_run_as_java.png[image] +

[[Groovy4.0-new-pojo]]
=== POJO Annotation (incubating)

Groovy supports both dynamic and static natures.
Dynamic Groovy's power and flexibility comes from making (potentially extensive) use of the runtime.
Static Groovy relies on the runtime library much less. Many method calls will have bytecode
corresponding to direct JVM method calls (similar to Java bytecode)
while the Groovy runtime is often bypassed altogether.
But even for static Groovy, hard-links to the Groovy jars remain.
All Groovy classes still implement the `GroovyObject` interface (and so have methods like `getMetaClass` and `invokeMethod`)
and there are some other places which call into the Groovy runtime.

The `@POJO` marker interface is used to indicate that the generated class is more like a plain old Java object
than an enhanced Groovy object. The annotation is currently ignored unless combined with `@CompileStatic`.
For such a class, the compiler won't generate methods typically needed by Groovy, e.g. `getMetaClass()`.
This feature is typically used for generating classes which need to be used with Java or Java frameworks
in situations where Java might become confused by Groovy's "plumbing" methods.

The feature is incubating. Currently, the presence of the annotation should be
treated like a hint to the compiler to produce bytecode not relying on the Groovy runtime
if it can but not a guarantee. Users of `@CompileStatic` will know that certain dynamic
features aren't possible when they switch to static Groovy.
Even so, while the desire is to produce Java-like code,
there are numerous Groovy feature which still work for some scenarios.

Consider the following example. First a Groovy `Point` class:

[source,groovy]
--------------------------------------
@CompileStatic
@POJO
@Canonical(includeNames = true)
class Point {
    Integer x, y
}
--------------------------------------

And now a Groovy `PointList` class:

[source,groovy]
--------------------------------------
@CompileStatic
@POJO
class PointList {
    @Delegate
    List<Point> points
}
--------------------------------------

We can then compile the following Java code.
We do not need the Groovy jars available for `javac` or `java`,
we only need the class files produced from the previous step.

[source,java]
--------------------------------------
Predicate<Point> xNeqY = p -> p.getX() != p.getY();  // <1>

Point p13 = new Point(1, 3);
List<Point> pts = List.of(p13, new Point(2, 2), new Point(3, 1));
PointList list = new PointList();
list.setPoints(pts);

System.out.println(list.size());
System.out.println(list.contains(p13));

list.forEach(System.out::println);

long count = list.stream().filter(xNeqY).collect(counting());  // <2>
System.out.println(count);
--------------------------------------
<1> Check whether x not equal to y
<2> Count points where x neq y

Note that while our `PointList` class has numerous list methods available
(`size`, `contains`, `forEach`, `stream`, etc.) courtesy of Groovy's `@Delegate` transform,
these are baked into the class file, not calling into any Groovy library or runtime code.

When run, the following output is produced:

--------------------------------------
3
true
Point(x:1, y:3)
Point(x:2, y:2)
Point(x:3, y:1)
2
--------------------------------------

In essence, this opens up the possibility to use Groovy
as a kind of pre-processor similar to https://projectlombok.org/[Lombok] but backed by the Groovy language.

[width="80%",align="center"]
|===
a| NOTE: _WARNING:_
Not all parts of the compiler and not all AST transforms yet know about `POJO`.
Your mileage may vary as to whether using this approach will or won't require
the Groovy jars to be on the classpath. While we anticipate some improvements over time
allowing more Groovy constructs to work with `@POJO`, we currently make no guarantees that
all constructs will eventually be supported. Hence the incubating status.
|===

[[Groovy4.0-new-records]]
=== Record-like classes (incubating)

Java 14 introduced _records_ as a _preview_ feature.
As per this https://www.infoq.com/articles/java-14-feature-spotlight/[records spotlight article],
records "model _plain data_ aggregates with less ceremony".
Groovy has features like the `@Immutable` transformation which already support
modeling data aggregates with less ceremony, and while these features overlap to some degree
with the design of records, they are not a direct equivalent.
Records are a slight re-mixing of the features of `@Immutable` with a fex variations added to the mix.

Luckily for us, Groovy's `@Immutable` is itself a meta-annotation (also known as annotation collector)
which combines more fine-grained features. It is relatively simple to provide a record-like re-mix
of those features and that is what Groovy 4 provides with its `@RecordType` transform.
It combines the following transforms/marker annotations:

[source,groovy]
--------------------------------------
@RecordBase
@ToString(cache = true, includeNames = true)
@EqualsAndHashCode(cache = true, useCanEqual = false)
@ImmutableOptions
@PropertyOptions(propertyHandler = ImmutablePropertyHandler)
@TupleConstructor(defaults = false)
@MapConstructor
@KnownImmutable
@POJO
--------------------------------------

You define and use a record-like class as follows:

[source,groovy]
--------------------------------------
@groovy.transform.RecordType
class Cyclist {
    String firstName
    String lastName
}

def richie = new Cyclist('Richie', 'Porte')
--------------------------------------

This produces a class with the following characteristics:

* it is implicitly final
* it has a private final field `firstName` with an accessor method `firstName()`; ditto for `lastName`
* it has a default `Cyclist(String, String)` constructor
* it has a default `serialVersionUID` of 0L
* it has implicit `toString()`, `equals()` and `hashCode()` methods

In future releases, we may provide some syntactic sugar to allow the above definition to be written like this:

[source,groovy]
--------------------------------------
record Cyclist(String firstName, String lastName) { }    // possible future syntax
--------------------------------------

This syntax is *not* currently supported.
We are seeking feedback on the implementation details in the meantime
and are keen to understand where our users might use records or record-like structures.

[width="80%",align="center"]
|===
a| NOTE: _WARNING:_
The implementation of records is not final, hence the incubating status.
We are still weighing up numerous options.
We can make our record-like classes much more flexible than the Java variant,
so long as we follow the rules such classes are expected to follow.
However, at some point, if we provide too many differences, it may cause
future issues as Java records evolve.
|===

[[Groovy4.0-new-contracts]]
=== Groovy Contracts (incubating)

This optional module supports design-by-contract style of programming.
More specifically, it provides contract annotations that support the
specification of class-invariants, pre-conditions, and post-conditions
on Groovy classes and interfaces.
Here is an example:

[source,groovy]
--------------------------------------
import groovy.contracts.*

@Invariant({ speed() >= 0 })
class Rocket {
    int speed = 0
    boolean started = true

    @Requires({ isStarted() })
    @Ensures({ old.speed < speed })
    def accelerate(inc) { speed += inc }

    def isStarted() { started }

    def speed() { speed }
}

def r = new Rocket()
r.accelerate(5)
--------------------------------------

This causes checking logic, corresponding to the contract declarations, to be injected
as required in the classes methods and constructors.
The checking logic will ensure that any pre-condition is satisfied
before a method executes, that any post-condition holds after any method executes
and that any class invariant is true before and after a method is called.

This module replaces the previously external `gcontracts` project which is now archived.

[[Groovy4.0-other]]
== Other improvements

=== GString performance improvements

GString internals were revamped to improve performance.
When safe to do so, GString toString values are now automatically cached.
While infrequently used, GStrings do permit their internal data structures to
be viewed (and even changed!). In such circumstances, caching is disabled.
If you wish to view and not change the internal data structures, you can
call a `freeze()` method in `GStringImpl` to disallow changing of the internal
data structures which allows caching to remain active.
link:https://issues.apache.org/jira/browse/GROOVY-9637[GROOVY-9637]

As an example, the following script takes about 10s to run with Groovy 3 and about 0.1s with Groovy 4:

[source,groovy]
--------------------------------------
def now = java.time.LocalDateTime.now()
def gs = "integer: ${1}, double: ${1.2d}, string: ${'x'}, class: ${Map.class}, boolean: ${true}, date: ${now}"
long b = System.currentTimeMillis()
for (int i = 0; i < 10000000; i++) {
    gs.toString()
}
long e = System.currentTimeMillis()
println "${e - b}ms"
--------------------------------------

[[Groovy4.0-ongoing]]
== On-going work still in progress or undergoing investigation

=== JSR308 improvements (work in progress)

Groovy has been improving JSR-308 support over recent versions.
In Groovy 4.0, additional support has been added.

=== Java-inspired changes (under investigation)

The following changes are being explored as candidate for Groovy 4:

* Switch expressions
* Module definitions written in Groovy

[[Groovy4.0-breaking]]
== Other breaking changes

* Numerous classes previously "leaked" ASM constants which are essentially an internal implementation detail by virtue of
implementing an `Opcodes` interface. This will not normally affect the majority of
Groovy scripts but might impact code which manipulates AST nodes such as AST transforms.
Before compiling with Groovy 4, some of these may need one or more appropriate static import statements added.
AST transforms which extend `AbstractASTTransformation` are one example of potentially affected classes.
(link:https://issues.apache.org/jira/browse/GROOVY-9736[GROOVY-9736]).
* `ASTTest` previously had `RUNTIME` retention but now has `SOURCE` retention.
We know of no users making use of the old retention but are aware of various
issues keeping the old value.
link:https://issues.apache.org/jira/browse/GROOVY-9702[GROOVY-9702]
* There were some inconsistencies with JavaBean property naming conventions
for various edge cases, e.g. for a field with a name being a single uppercase `X` and having a `getX` accessor,
then the field was given priority over the accessor.
link:https://issues.apache.org/jira/browse/GROOVY-9618[GROOVY-9618]
* Numerous mostly internal data structure classes, e.g. AbstractConcurrentMapBase, AbstractConcurrentMap, ManagedConcurrentMap
were deprecated and their usage replaced with better alternatives.
This should be mostly invisible but some changes might impact users using
internal Groovy classes directly.
link:https://issues.apache.org/jira/browse/GROOVY-9631[GROOVY-9631]
* We bumped our Picocli version. This resulted in minor formatting changes
of some CLI help messages. We recommend not relying on the exact format of such messages.
link:https://issues.apache.org/jira/browse/GROOVY-9627[GROOVY-9627]
* We are currently attempting to improve how Groovy code accesses private fields
in certain scenarios where such access is expected but problematic, e.g. within closure definitions where
subclasses or inner classes are involved
(link:https://issues.apache.org/jira/browse/GROOVY-5438[GROOVY-5438]).
You may notice breakages in Groovy 4 code in such scenarios until this issue is progressed.
As a workaround in the meantime, you may be able to use local variable outside a closure
to reference the relevant fields and then reference those local variables in the closure.

[[Groovy4.0-requirements]]
== JDK requirements

Groovy 4.0 requires JDK9+ to build and JDK8 is the minimum version of the JRE that we support.
Groovy has been tested on JDK versions 8 through 16.

[[Groovy4.0-more-info]]
== More information

You can browse all the link:../changelogs/changelog-4.0.0-unreleased.html[tickets closed for Groovy 4.0 in JIRA].