Google Git
Sign in
apache / groovy-dev-site / 05adfa18bee595b4cce2dce8113ff1e42e8ffe78 / . / blog / detecting-objects-with-groovy-the.html
blob: 9bbc378958fee6cef43c600e3c16352edcc412e1 [file] [log] [blame]
<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]-->
<!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]-->
<!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]--><head>
<meta charset='utf-8'/><meta http-equiv='X-UA-Compatible' content='IE=edge'/><meta name='viewport' content='width=device-width, initial-scale=1'/><meta name='keywords' content='data science, deep learning, groovy, neural networks, apache mxnet, DJL, artifical intelligence, machine learning, gradle'/><meta name='description' content='This post looks at using Apache Groovy, DLJ and Apache MXNet to use neural networks to detect objects within an image.'/><title>The Apache Groovy programming language - Blogs - Detecting objects with Groovy, the Deep Java Library (DJL), and Apache MXNet</title><link href='../img/favicon.ico' type='image/x-ico' rel='icon'/><link rel='stylesheet' type='text/css' href='../css/bootstrap.css'/><link rel='stylesheet' type='text/css' href='../css/font-awesome.min.css'/><link rel='stylesheet' type='text/css' href='../css/style.css'/><link rel='stylesheet' type='text/css' href='https://cdnjs.cloudflare.com/ajax/libs/prettify/r298/prettify.min.css'/>
</head><body>
<div id='fork-me'>
<a href='https://github.com/apache/groovy'>
<img style='position: fixed; top: 20px; right: -58px; border: 0; z-index: 100; transform: rotate(45deg);' src='/img/horizontal-github-ribbon.png'/>
</a>
</div><div id='st-container' class='st-container st-effect-9'>
<nav class='st-menu st-effect-9' id='menu-12'>
<h2 class='icon icon-lab'>Socialize</h2><ul>
<li>
<a href='https://groovy-lang.org/mailing-lists.html' class='icon'><span class='fa fa-envelope'></span> Discuss on the mailing-list</a>
</li><li>
<a href='https://twitter.com/ApacheGroovy' class='icon'><span class='fa fa-twitter'></span> Groovy on Twitter</a>
</li><li>
<a href='https://groovy-lang.org/events.html' class='icon'><span class='fa fa-calendar'></span> Events and conferences</a>
</li><li>
<a href='https://github.com/apache/groovy' class='icon'><span class='fa fa-github'></span> Source code on GitHub</a>
</li><li>
<a href='https://groovy-lang.org/reporting-issues.html' class='icon'><span class='fa fa-bug'></span> Report issues in Jira</a>
</li><li>
<a href='http://stackoverflow.com/questions/tagged/groovy' class='icon'><span class='fa fa-stack-overflow'></span> Stack Overflow questions</a>
</li><li>
<a href='http://groovycommunity.com/' class='icon'><span class='fa fa-slack'></span> Slack Community</a>
</li>
</ul>
</nav><div class='st-pusher'>
<div class='st-content'>
<div class='st-content-inner'>
<!--[if lt IE 7]>
<p class="browsehappy">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/">upgrade your browser</a> to improve your experience.</p>
<![endif]--><div><div class='navbar navbar-default navbar-static-top' role='navigation'>
<div class='container'>
<div class='navbar-header'>
<button type='button' class='navbar-toggle' data-toggle='collapse' data-target='.navbar-collapse'>
<span class='sr-only'></span><span class='icon-bar'></span><span class='icon-bar'></span><span class='icon-bar'></span>
</button><a class='navbar-brand' href='../index.html'>
<i class='fa fa-star'></i> Apache Groovy
</a>
</div><div class='navbar-collapse collapse'>
<ul class='nav navbar-nav navbar-right'>
<li class=''><a href='https://groovy-lang.org/learn.html'>Learn</a></li><li class=''><a href='https://groovy-lang.org/documentation.html'>Documentation</a></li><li class=''><a href='/download.html'>Download</a></li><li class=''><a href='https://groovy-lang.org/support.html'>Support</a></li><li class=''><a href='/'>Contribute</a></li><li class=''><a href='https://groovy-lang.org/ecosystem.html'>Ecosystem</a></li><li class=''><a href='/blog'>Blog posts</a></li><li class=''><a href='https://groovy.apache.org/events.html'></a></li><li>
<a data-effect='st-effect-9' class='st-trigger' href='#'>Socialize</a>
</li><li class=''>
<a href='../search.html'>
<i class='fa fa-search'></i>
</a>
</li>
</ul>
</div>
</div>
</div><div id='content' class='page-1'><div class='row'><div class='row-fluid'><div class='col-lg-3'><ul class='nav-sidebar'><li><a href='./'>Blog index</a></li><li class='active'><a href='#doc'>Detecting objects with Groovy, the Deep Java Library (DJL), and Apache MXNet</a></li><li><a href='#_deep_learning' class='anchor-link'>Deep Learning</a></li><li><a href='#_deep_java_library_djl_apache_mxnet' class='anchor-link'>Deep Java Library (DJL) &amp; Apache MXNet</a></li><li><a href='#_using_djl_with_groovy' class='anchor-link'>Using DJL with Groovy</a></li><li><a href='#_building_and_running_our_application' class='anchor-link'>Building and running our application</a></li><li><a href='#_further_information' class='anchor-link'>Further Information</a></li><li><a href='#_conclusion' class='anchor-link'>Conclusion</a></li></ul><br/><ul class='nav-sidebar'><li style='padding: 0.35em 0.625em; background-color: #eee'><span>Related posts</span></li><li><a href='./classifying-iris-flowers-with-deep'>Classifying Iris Flowers with Deep Learning, Groovy and GraalVM</a></li><li><a href='./parsing-json-with-groovy'>Parsing JSON with Groovy</a></li><li><a href='./deep-learning-and-eclipse-collections'>Deep Learning and Eclipse Collections</a></li><li><a href='./matrix-calculations-with-groovy-apache'>Matrix calculations with Groovy, Apache Commons Math, ojAlgo, Nd4j and EJML</a></li><li><a href='./whiskey-clustering-with-groovy-and'>Whiskey Clustering with Groovy and Apache Ignite</a></li><li><a href='./reading-and-writing-csv-files'>Reading and Writing CSV files with Groovy</a></li><li><a href='./using-groovy-with-apache-wayang'>Using Groovy with Apache Wayang and Apache Spark</a></li><li><a href='./fruity-eclipse-collections'>Fruity Eclipse Collections</a></li></ul></div><div class='col-lg-8 col-lg-pull-0'><a name='doc'></a><h1>Detecting objects with Groovy, the Deep Java Library (DJL), and Apache MXNet</h1><p><span>Author: <i>Paul King</i></span><br/><span>Published: 2022-08-01 11:52AM</span></p><hr/><div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>This blog posts looks at using <a href="https://groovy-lang.org/">Apache Groovy</a> with the
<a href="https://djl.ai/">Deep Java Library (DJL)</a> and backed by the
<a href="https://mxnet.incubator.apache.org/">Apache MXNet</a> engine to detect objects within an image. (Apache MXNet is an <a href="https://incubator.apache.org/">incubating project</a> at <a href="https://www.apache.org/">the ASF</a>.)</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_deep_learning">Deep Learning</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Deep learning falls under the branches of <a href="https://en.wikipedia.org/wiki/Machine_learning">machine learning</a> and <a href="https://en.wikipedia.org/wiki/Artificial_intelligence">artificial intelligence</a>. It involves multiple layers (hence the "deep") of an <a href="https://en.wikipedia.org/wiki/Artificial_neural_network">artificial neural network</a>. There are lots of ways to configure such networks and the details are beyond the scope of this blog post, but we can give some basic details. We will have four input nodes corresponding to the measurements of our four characteristics. We will have three output nodes corresponding to each possible class (species). We will also have one or more additional layers in between.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="img/deep_network.png" alt="Deep learning network" width="630"></span></p>
</div>
<div class="paragraph">
<p>Each node in this network mimics to some degree a neuron in the human brain. Again, we&#8217;ll simplify the details. Each node has multiple inputs, which are given a particular weight, as well as an activation function which will determine whether our node "fires". Training the model is a process which works out what the best weights should be.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="img/deep_node.png" alt="Node in a deep learning network" width="580"></span></p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_deep_java_library_djl_apache_mxnet">Deep Java Library (DJL) &amp; Apache MXNet</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Rather than writing your own neural networks, libraries such as <a href="https://djl.ai/">DJL</a> provide high-level abstractions which automate to some degree the creation of the necessary neural network layers. DJL is engine agnostic, so it&#8217;s capable of supporting different backends including Apache MXNet, PyTorch, TensorFlow and ONNX Runtime. We&#8217;ll use the default engine which for our application (at the time of writing) is Apache MXNet.</p>
</div>
<div class="paragraph">
<p><a href="https://mxnet.apache.org/">Apache MXNet</a> provides the underlying engine. It has support for imperative and symbolic execution, distributed training of your models using multi-gpu or multi-host hardware, and multiple language bindings. Groovy is fully compatible with the Java binding.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_using_djl_with_groovy">Using DJL with Groovy</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Groovy uses the Java binding. Consider looking at the DJL beginner tutorials for Java - they will work almost unchanged for Groovy.</p>
</div>
<div class="paragraph">
<p>For our example, the first thing we need to do is download the image we want to run the object detection model on:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="groovy">Path tempDir = Files.createTempDirectory("resnetssd")
def imageName = 'dog-ssd.jpg'
Path localImage = tempDir.resolve(imageName)
def url = new URL("https://s3.amazonaws.com/model-server/inputs/$imageName")
DownloadUtils.download(url, localImage, new ProgressBar())
Image img = ImageFactory.instance.fromFile(localImage)</code></pre>
</div>
</div>
<div class="paragraph">
<p>It happens to be a well-known already available image. We&#8217;ll store a local copy of the image in a temporary directory, and we&#8217;ll use a utility class that comes with DJL to provide a nice progress bar while the image is downloading. DJL provides its own image classes, so we&#8217;ll create an instance using the appropriate class from the downloaded image.</p>
</div>
<div class="paragraph">
<p>Next we want to configure our neural network layers:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="groovy">def criteria = Criteria.builder()
.optApplication(Application.CV.OBJECT_DETECTION)
.setTypes(Image, DetectedObjects)
.optFilter("backbone", "resnet50")
.optEngine(Engine.defaultEngineName)
.optProgress(new ProgressBar())
.build()</code></pre>
</div>
</div>
<div class="paragraph">
<p>DLJ supports numerous model <em>applications</em> including image classification, word recognition, sentiment analysis, linear regression, and others. We&#8217;ll select <em>object detection</em>. This kind of application looks for the bounding box of known objects within an image. The <em>types</em> configuration option identifies that our input will be an image and the output will be detected objects. The <em>filter</em> option indicates that we will be using ResNet-50 (a 50-layers deep convolutional neural network often used as a backbone for many computer vision tasks). We set the <em>engine</em> to be the default engine which happens to be Apache MXNet. We also configure an optional progress bar to provide feedback of progress while our model is running.</p>
</div>
<div class="paragraph">
<p>Now that we have our configuration sorted, we&#8217;ll use it to load a model and then use the model to make object predictions:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="groovy">def detection = criteria.loadModel().withCloseable { model -&gt;
model.newPredictor().predict(img)
}
detection.items().each { println it }
img.drawBoundingBoxes(detection)</code></pre>
</div>
</div>
<div class="paragraph">
<p>For good measure, we&#8217;ll draw the bounding boxes into our image.</p>
</div>
<div class="paragraph">
<p>Next, we save our image into a file and display it using Groovy&#8217;s SwingBuilder.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="groovy">Path imageSaved = tempDir.resolve('detected.png')
imageSaved.withOutputStream { os -&gt; img.save(os, 'png') }
def saved = ImageIO.read(imageSaved.toFile())
new SwingBuilder().edt {
frame(title: "$detection.numberOfObjects detected objects",
size: [saved.width, saved.height],
defaultCloseOperation: DISPOSE_ON_CLOSE,
show: true) { label(icon: imageIcon(image: saved)) }
}</code></pre>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_building_and_running_our_application">Building and running our application</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Our code is stored on a source file called <code>ObjectDetect.groovy</code>.</p>
</div>
<div class="paragraph">
<p>We used <a href="https://gradle.org/">Gradle</a> for our build file:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="groovy">apply plugin: 'groovy'
apply plugin: 'application'
repositories {
mavenCentral()
}
application {
mainClass = 'ObjectDetect'
}
dependencies {
implementation "ai.djl:api:0.18.0"
implementation "org.apache.groovy:groovy:4.0.4"
implementation "org.apache.groovy:groovy-swing:4.0.4"
runtimeOnly "ai.djl:model-zoo:0.18.0"
runtimeOnly "ai.djl.mxnet:mxnet-engine:0.18.0"
runtimeOnly "ai.djl.mxnet:mxnet-model-zoo:0.18.0"
runtimeOnly "ai.djl.mxnet:mxnet-native-auto:1.8.0"
runtimeOnly "org.apache.groovy:groovy-nio:4.0.4"
runtimeOnly "org.slf4j:slf4j-jdk14:1.7.36"
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>We run the application with the gradle <code>run</code> task:</p>
</div>
<div class="listingblock">
<div class="content">
<pre><span class="lime">paulk@pop-os</span>:<span class="blue">/extra/projects/groovy-data-science</span>$ ./gradlew DLMXNet:run
> Task :DeepLearningMxnet:run
Downloading: 100% |<span class="olive">████████████████████████████████████████</span>| dog-ssd.jpg
Loading: 100% |<span class="olive">████████████████████████████████████████</span>|
...
class: <strong class="green">"car"</strong>, probability: <strong>0.99991</strong>, bounds: [x=0.611, y=0.137, width=0.293, height=0.160]
class: <strong class="green">"bicycle"</strong>, probability: <strong>0.95385</strong>, bounds: [x=0.162, y=0.207, width=0.594, height=0.588]
class: <strong class="green">"dog"</strong>, probability: <strong>0.93752</strong>, bounds: [x=0.168, y=0.350, width=0.274, height=0.593]</pre>
</div>
</div>
<div class="paragraph">
<p>The displayed image looks like this:</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="img/detected_objects.png" alt="Detected objects" width="468"></span></p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_further_information">Further Information</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The full source code can be found in the following repo:<br>
<a href="https://github.com/paulk-asert/groovy-data-science/subprojects/DeepLearningMxnet" class="bare">https://github.com/paulk-asert/groovy-data-science/subprojects/DeepLearningMxnet</a></p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_conclusion">Conclusion</h2>
<div class="sectionbody">
<div class="paragraph">
<p>We have examined using Apache Groovy, DLJ and Apache MXNet to detect objects
within an image. We&#8217;ve used a model based on a rich deep learning model, but we
didn&#8217;t need to get into the details of the model or its neural network layers.
DLJ and Apache MXNet did the hard lifting for us. Groovy provided a simple coding
experience for building our application.</p>
</div>
</div>
</div></div></div></div></div><footer id='footer'>
<div class='row'>
<div class='colset-3-footer'>
<div class='col-1'>
<h1>Groovy</h1><ul>
<li><a href='https://groovy-lang.org/learn.html'>Learn</a></li><li><a href='https://groovy-lang.org/documentation.html'>Documentation</a></li><li><a href='/download.html'>Download</a></li><li><a href='https://groovy-lang.org/support.html'>Support</a></li><li><a href='/'>Contribute</a></li><li><a href='https://groovy-lang.org/ecosystem.html'>Ecosystem</a></li><li><a href='/blog'>Blog posts</a></li><li><a href='https://groovy.apache.org/events.html'></a></li>
</ul>
</div><div class='col-2'>
<h1>About</h1><ul>
<li><a href='https://github.com/apache/groovy'>Source code</a></li><li><a href='https://groovy-lang.org/security.html'>Security</a></li><li><a href='https://groovy-lang.org/learn.html#books'>Books</a></li><li><a href='https://groovy-lang.org/thanks.html'>Thanks</a></li><li><a href='http://www.apache.org/foundation/sponsorship.html'>Sponsorship</a></li><li><a href='https://groovy-lang.org/faq.html'>FAQ</a></li><li><a href='https://groovy-lang.org/search.html'>Search</a></li>
</ul>
</div><div class='col-3'>
<h1>Socialize</h1><ul>
<li><a href='https://groovy-lang.org/mailing-lists.html'>Discuss on the mailing-list</a></li><li><a href='https://twitter.com/ApacheGroovy'>Groovy on Twitter</a></li><li><a href='https://groovy-lang.org/events.html'>Events and conferences</a></li><li><a href='https://github.com/apache/groovy'>Source code on GitHub</a></li><li><a href='https://groovy-lang.org/reporting-issues.html'>Report issues in Jira</a></li><li><a href='http://stackoverflow.com/questions/tagged/groovy'>Stack Overflow questions</a></li><li><a href='http://groovycommunity.com/'>Slack Community</a></li>
</ul>
</div><div class='col-right'>
<p>
The Groovy programming language is supported by the <a href='http://www.apache.org'>Apache Software Foundation</a> and the Groovy community.
</p><div text-align='right'>
<img src='../img/asf_logo.png' title='The Apache Software Foundation' alt='The Apache Software Foundation' style='width:60%'/>
</div><p>Apache&reg; and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p>
</div>
</div><div class='clearfix'>&copy; 2003-2023 the Apache Groovy project &mdash; Groovy is Open Source: <a href='http://www.apache.org/licenses/LICENSE-2.0.html' alt='Apache 2 License'>license</a>, <a href='https://privacy.apache.org/policies/privacy-policy-public.html'>privacy policy</a>.</div>
</div>
</footer></div>
</div>
</div>
</div>
</div><script src='../js/vendor/jquery-1.10.2.min.js' defer></script><script src='../js/vendor/classie.js' defer></script><script src='../js/vendor/bootstrap.js' defer></script><script src='../js/vendor/sidebarEffects.js' defer></script><script src='../js/vendor/modernizr-2.6.2.min.js' defer></script><script src='../js/plugins.js' defer></script><script src='https://cdnjs.cloudflare.com/ajax/libs/prettify/r298/prettify.min.js'></script><script>document.addEventListener('DOMContentLoaded',prettyPrint)</script><script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-257558-10', 'auto');
ga('send', 'pageview');
</script>
</body></html>