| <!doctype html> |
| <html lang="en" dir="ltr" class="blog-wrapper blog-tags-post-list-page plugin-blog plugin-id-default" data-has-hydrated="false"> |
| <head> |
| <meta charset="UTF-8"> |
| <meta name="generator" content="Docusaurus v3.1.1"> |
| <title data-rh="true">One post tagged with "federated" | Apache Wayang (incubating)</title><meta data-rh="true" name="viewport" content="width=device-width,initial-scale=1"><meta data-rh="true" name="twitter:card" content="summary_large_image"><meta data-rh="true" property="og:url" content="https://wayang.apache.org/blog/tags/federated"><meta data-rh="true" property="og:locale" content="en"><meta data-rh="true" name="docusaurus_locale" content="en"><meta data-rh="true" name="docsearch:language" content="en"><meta data-rh="true" property="og:title" content="One post tagged with "federated" | Apache Wayang (incubating)"><meta data-rh="true" name="docusaurus_tag" content="blog_tags_posts"><meta data-rh="true" name="docsearch:docusaurus_tag" content="blog_tags_posts"><link data-rh="true" rel="icon" href="/img/wayang-logo.jpg"><link data-rh="true" rel="canonical" href="https://wayang.apache.org/blog/tags/federated"><link data-rh="true" rel="alternate" href="https://wayang.apache.org/blog/tags/federated" hreflang="en"><link data-rh="true" rel="alternate" href="https://wayang.apache.org/blog/tags/federated" hreflang="x-default"><link rel="alternate" type="application/rss+xml" href="/blog/rss.xml" title="Apache Wayang (incubating) RSS Feed"> |
| <link rel="alternate" type="application/atom+xml" href="/blog/atom.xml" title="Apache Wayang (incubating) Atom Feed"><link rel="stylesheet" href="/assets/css/styles.ecf70413.css"> |
| <script src="/assets/js/runtime~main.bdab3f22.js" defer="defer"></script> |
| <script src="/assets/js/main.454575b0.js" defer="defer"></script> |
| </head> |
| <body class="navigation-with-keyboard"> |
| <script>!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){try{return new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}}()||function(){try{return localStorage.getItem("theme")}catch(t){}}();t(null!==e?e:"light")}(),function(){try{const a=new URLSearchParams(window.location.search).entries();for(var[t,e]of a)if(t.startsWith("docusaurus-data-")){var n=t.replace("docusaurus-data-","data-");document.documentElement.setAttribute(n,e)}}catch(t){}}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())</script><div id="__docusaurus"><div role="region" aria-label="Skip to main content"><a class="skipToContent_fXgn" href="#__docusaurus_skipToContent_fallback">Skip to main content</a></div><div class="announcementBar_mb4j" style="background-color:#fafbfc;color:#091E42" role="banner"><div class="announcementBarPlaceholder_vyr4"></div><div class="content_knG7 announcementBarContent_xLdY">⭐️ If you like Apache Wayang, give it a star on <a target="_blank" href="https://github.com/apache/incubator-wayang">GitHub</a>! ⭐ </div><button type="button" aria-label="Close" class="clean-btn close closeButton_CVFx announcementBarClose_gvF7"><svg viewBox="0 0 15 15" width="14" height="14"><g stroke="currentColor" stroke-width="3.1"><path d="M.75.75l13.5 13.5M14.25.75L.75 14.25"></path></g></svg></button></div><nav aria-label="Main" class="navbar navbar--fixed-top"><div class="navbar__inner"><div class="navbar__items"><button aria-label="Toggle navigation bar" aria-expanded="false" class="navbar__toggle clean-btn" type="button"><svg width="30" height="30" viewBox="0 0 30 30" aria-hidden="true"><path stroke="currentColor" stroke-linecap="round" stroke-miterlimit="10" stroke-width="2" d="M4 7h22M4 15h22M4 23h22"></path></svg></button><a class="navbar__brand" href="/"><div class="navbar__logo"><img src="/img/wayang.png" alt="Wayang Logo" class="themedComponent_mlkZ themedComponent--light_NVdE"><img src="/img/wayang.png" alt="Wayang Logo" class="themedComponent_mlkZ themedComponent--dark_xIcU"></div><b class="navbar__title text--truncate"></b></a></div><div class="navbar__items navbar__items--right"><a class="navbar__item navbar__link" href="/docs/start/download">Download</a><a class="navbar__item navbar__link" href="/docs/introduction/about">About</a><a class="navbar__item navbar__link" href="/docs/guide/installation">Developers</a><div class="navbar__item dropdown dropdown--hoverable dropdown--right"><a href="#" aria-haspopup="true" aria-expanded="false" role="button" class="navbar__link">Community</a><ul class="dropdown__menu"><li><a aria-current="page" class="dropdown__link dropdown__link--active" href="/blog/">Blog</a></li><li><a class="dropdown__link" href="/docs/community/mailinglist">Project</a></li></ul></div><div class="navbar__item dropdown dropdown--hoverable dropdown--right"><a href="#" aria-haspopup="true" aria-expanded="false" role="button" class="navbar__link">ASF</a><ul class="dropdown__menu"><li><a href="https://www.apache.org/" target="_blank" rel="noopener noreferrer" class="dropdown__link">Foundation</a></li><li><a href="https://www.apache.org/licenses/" target="_blank" rel="noopener noreferrer" class="dropdown__link">License</a></li><li><a href="https://www.apache.org/events/current-event.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Events</a></li><li><a href="https://privacy.apache.org/policies/privacy-policy-public.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Privacy</a></li><li><a href="https://www.apache.org/security/" target="_blank" rel="noopener noreferrer" class="dropdown__link">Security</a></li><li><a href="https://www.apache.org/foundation/sponsorship.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Sponsorship</a></li><li><a href="https://www.apache.org/foundation/thanks.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Thanks</a></li><li><a href="https://www.apache.org/foundation/policies/conduct.html" target="_blank" rel="noopener noreferrer" class="dropdown__link">Code of Conduct</a></li></ul></div><a href="https://github.com/apache/incubator-wayang" target="_blank" rel="noopener noreferrer" class="navbar__item navbar__link header-github-link" aria-label="GitHub repository"></a><div class="toggle_vylO colorModeToggle_DEke"><button class="clean-btn toggleButton_gllP toggleButtonDisabled_aARS" type="button" disabled="" title="Switch between dark and light mode (currently light mode)" aria-label="Switch between dark and light mode (currently light mode)" aria-live="polite"><svg viewBox="0 0 24 24" width="24" height="24" class="lightToggleIcon_pyhR"><path fill="currentColor" d="M12,9c1.65,0,3,1.35,3,3s-1.35,3-3,3s-3-1.35-3-3S10.35,9,12,9 M12,7c-2.76,0-5,2.24-5,5s2.24,5,5,5s5-2.24,5-5 S14.76,7,12,7L12,7z M2,13l2,0c0.55,0,1-0.45,1-1s-0.45-1-1-1l-2,0c-0.55,0-1,0.45-1,1S1.45,13,2,13z M20,13l2,0c0.55,0,1-0.45,1-1 s-0.45-1-1-1l-2,0c-0.55,0-1,0.45-1,1S19.45,13,20,13z M11,2v2c0,0.55,0.45,1,1,1s1-0.45,1-1V2c0-0.55-0.45-1-1-1S11,1.45,11,2z M11,20v2c0,0.55,0.45,1,1,1s1-0.45,1-1v-2c0-0.55-0.45-1-1-1C11.45,19,11,19.45,11,20z M5.99,4.58c-0.39-0.39-1.03-0.39-1.41,0 c-0.39,0.39-0.39,1.03,0,1.41l1.06,1.06c0.39,0.39,1.03,0.39,1.41,0s0.39-1.03,0-1.41L5.99,4.58z M18.36,16.95 c-0.39-0.39-1.03-0.39-1.41,0c-0.39,0.39-0.39,1.03,0,1.41l1.06,1.06c0.39,0.39,1.03,0.39,1.41,0c0.39-0.39,0.39-1.03,0-1.41 L18.36,16.95z M19.42,5.99c0.39-0.39,0.39-1.03,0-1.41c-0.39-0.39-1.03-0.39-1.41,0l-1.06,1.06c-0.39,0.39-0.39,1.03,0,1.41 s1.03,0.39,1.41,0L19.42,5.99z M7.05,18.36c0.39-0.39,0.39-1.03,0-1.41c-0.39-0.39-1.03-0.39-1.41,0l-1.06,1.06 c-0.39,0.39-0.39,1.03,0,1.41s1.03,0.39,1.41,0L7.05,18.36z"></path></svg><svg viewBox="0 0 24 24" width="24" height="24" class="darkToggleIcon_wfgR"><path fill="currentColor" d="M9.37,5.51C9.19,6.15,9.1,6.82,9.1,7.5c0,4.08,3.32,7.4,7.4,7.4c0.68,0,1.35-0.09,1.99-0.27C17.45,17.19,14.93,19,12,19 c-3.86,0-7-3.14-7-7C5,9.07,6.81,6.55,9.37,5.51z M12,3c-4.97,0-9,4.03-9,9s4.03,9,9,9s9-4.03,9-9c0-0.46-0.04-0.92-0.1-1.36 c-0.98,1.37-2.58,2.26-4.4,2.26c-2.98,0-5.4-2.42-5.4-5.4c0-1.81,0.89-3.42,2.26-4.4C12.92,3.04,12.46,3,12,3L12,3z"></path></svg></button></div><div class="navbarSearchContainer_Bca1"><div class="navbar__search"><span aria-label="expand searchbar" role="button" class="search-icon" tabindex="0"></span><input id="search_input_react" type="search" placeholder="Loading..." aria-label="Search" class="navbar__search-input search-bar" disabled=""></div></div></div></div><div role="presentation" class="navbar-sidebar__backdrop"></div></nav><div id="__docusaurus_skipToContent_fallback" class="main-wrapper mainWrapper_z2l0"><div class="container margin-vert--lg"><div class="row"><aside class="col col--3"><nav class="sidebar_re4s thin-scrollbar" aria-label="Blog recent posts navigation"><div class="sidebarItemTitle_pO2u margin-bottom--md">All our posts</div><ul class="sidebarItemList_Yudw clean-list"><li class="sidebarItem__DBe"><a class="sidebarItemLink_mo7H" href="/blog/wayang-federated-ai">Wayang and the Federated AI</a></li><li class="sidebarItem__DBe"><a class="sidebarItemLink_mo7H" href="/blog/wayang-python-api">Pywayang - Apache Wayang's Python API</a></li><li class="sidebarItem__DBe"><a class="sidebarItemLink_mo7H" href="/blog/kafka-meets-wayang-3">Apache Kafka meets Apache Wayang - Part 3</a></li><li class="sidebarItem__DBe"><a class="sidebarItemLink_mo7H" href="/blog/wayang-vs-trino">Apache Wayang vs. Presto/Trino</a></li><li class="sidebarItem__DBe"><a class="sidebarItemLink_mo7H" href="/blog/kafka-meets-wayang-2">Apache Kafka meets Apache Wayang - Part 2</a></li><li class="sidebarItem__DBe"><a class="sidebarItemLink_mo7H" href="/blog/kafka-meets-wayang-1">Apache Kafka meets Apache Wayang - Part 1</a></li><li class="sidebarItem__DBe"><a class="sidebarItemLink_mo7H" href="/blog/website_update">Website updated</a></li></ul></nav></aside><main class="col col--7" itemscope="" itemtype="https://schema.org/Blog"><header class="margin-bottom--xl"><h1>One post tagged with "federated"</h1><a href="/blog/tags">View All Tags</a></header><article class="margin-bottom--xl" itemprop="blogPost" itemscope="" itemtype="https://schema.org/BlogPosting"><meta itemprop="description" content="AI systems and applications are widely used nowadays, from assisting grammar spellings to"><header><h2 class="title_f1Hy" itemprop="headline"><a itemprop="url" href="/blog/wayang-federated-ai">Wayang and the Federated AI</a></h2><div class="container_mt6G margin-vert--md"><time datetime="2024-04-17T00:00:00.000Z" itemprop="datePublished">April 17, 2024</time> · <!-- -->3 min read</div><div class="margin-top--md margin-bottom--sm row"><div class="col col--6 authorCol_Hf19"><div class="avatar margin-bottom--sm"><a href="https://github.com/glauesppen" target="_blank" rel="noopener noreferrer" class="avatar__photo-link"><img class="avatar__photo" src="https://avatars.githubusercontent.com/glauesppen" alt="Gláucia Esppenchutz" itemprop="image"></a><div class="avatar__intro" itemprop="author" itemscope="" itemtype="https://schema.org/Person"><div class="avatar__name"><a href="https://github.com/glauesppen" target="_blank" rel="noopener noreferrer" itemprop="url"><span itemprop="name">Gláucia Esppenchutz</span></a></div><small class="avatar__subtitle" itemprop="description">(P)PMC Apache Wayang</small></div></div></div></div></header><div class="markdown" itemprop="articleBody"><p>AI systems and applications are widely used nowadays, from assisting grammar spellings to |
| detecting early signs of cancer cells. Building an AI requires a lot of data and training to achieve |
| the desired results, and federated learning is an approach to make AI training more viable. |
| Federated learning (or collaborative learning) is a technique that trains AI models on data |
| distributed across multiple serves or devices. It does so without centralizing data on a single |
| place or storage. It also prevents the possibility of data breaches and protects sensitive |
| personal data. One of the significant challenges in working with AI is the variety of tools found |
| in the market or the open-source community. Each tool provides results in a different form; |
| integrating them can be pretty challenging. Let's talk about Apache Wayang (incubating) and |
| how it can help to solve this problem.</p> |
| <h2 class="anchor anchorWithStickyNavbar_LWe7" id="apache-wayang-in-the-federated-ai-world">Apache Wayang in the Federated AI world<a href="#apache-wayang-in-the-federated-ai-world" class="hash-link" aria-label="Direct link to Apache Wayang in the Federated AI world" title="Direct link to Apache Wayang in the Federated AI world"></a></h2> |
| <p>Apache Wayang (Wayang, for short), a project in an incubation phase at Apache Software |
| Foundation (ASF), integrates big data platforms and tools by removing the complexity of |
| worrying about low-level details. Interestingly, even if it was not designed for, Wayang could |
| also serve as a scalable platform for federated learning: the Wayang community is starting to |
| work on integrating federated learning capabilities. In a federated learning approach, Wayang |
| would allow different local models to be built and exchange its model results across other data |
| centers to combine them into a single enhanced model.</p> |
| <h2 class="anchor anchorWithStickyNavbar_LWe7" id="a-real-world-example">A real-world example<a href="#a-real-world-example" class="hash-link" aria-label="Direct link to A real-world example" title="Direct link to A real-world example"></a></h2> |
| <p>Let's consider a real-world scenario. Hospitals and health organizations have increased their |
| investments in machine/deep learning initiatives to learn more and predict diagnostics. |
| However, due to legal frameworks, sharing patients' information or diagnostics is impossible, |
| and the solution would be to apply federated learning. To solve this problem, we could use |
| Wayang to help to train the models. See the diagram 1 below:</p> |
| <br> |
| <img width="75%" alt="wayang stack" src="/img/architecture/federated-ai-architecture-1.png"> |
| <br> |
| <br> |
| <p>As a first step, the data scientists would send an ML task to Wayang, which will work as an |
| abstraction layer to connect to different data processing platforms, sparing the time to build |
| integration code for each. Then, the data platforms process and generate the results that will |
| be sent back to Wayang. Wayang aggregates the results into one "global result" and sends it |
| back to the requestor as a next step.</p> |
| <br> |
| <img width="75%" alt="wayang stack" src="/img/architecture/federated-ai-architecture-2.png"> |
| <br> |
| <br> |
| <p>The process repeats until the desired results are achieved. |
| Although it is very much like a Federated learning pipeline, Wayang removes a considerable |
| layer of complexity from the developers by integrating with diverse types of data platforms. It |
| also brings fast development and reduces the need for a deep understanding of data |
| infrastructure or integrations. Developers can focus on the logic and how to execute tasks |
| instead of details about data processors.</p> |
| <h3 class="anchor anchorWithStickyNavbar_LWe7" id="follow-wayang">Follow Wayang<a href="#follow-wayang" class="hash-link" aria-label="Direct link to Follow Wayang" title="Direct link to Follow Wayang"></a></h3> |
| <p>Apache Wayang is in an incubation phase and has a potential roadmap of implementations |
| coming soon (including the federated learning aspect as well as an SQL interface and a novel |
| data debugging functionality). If you want to hear or join the community, consult the link |
| <a href="https://wayang.apache.org/community/" target="_blank" rel="noopener noreferrer">https://wayang.apache.org/community/</a> , join the mailing lists, contribute with new ideas, |
| write documentation, or fix bugs.</p> |
| <br> |
| <h5 class="anchor anchorWithStickyNavbar_LWe7" id="thank-you">Thank you!<a href="#thank-you" class="hash-link" aria-label="Direct link to Thank you!" title="Direct link to Thank you!"></a></h5> |
| <p>I (Gláucia) want to thank professor Jorge Quiané for the guidance to write this blog post. |
| Thanks for incentivate me to join the project and for the knowledge shared. I will always remember you.</p></div><footer class="row docusaurus-mt-lg"><div class="col"><b>Tags:</b><ul class="tags_jXut padding--none margin-left--sm"><li class="tag_QGVx"><a class="tag_zVej tagRegular_sFm0" href="/blog/tags/wayang">wayang</a></li><li class="tag_QGVx"><a class="tag_zVej tagRegular_sFm0" href="/blog/tags/federated">federated</a></li><li class="tag_QGVx"><a class="tag_zVej tagRegular_sFm0" href="/blog/tags/ai">ai</a></li></ul></div></footer></article><nav class="pagination-nav" aria-label="Blog list page navigation"></nav></main></div></div></div><footer class="footer footer--dark"><div class="container container-fluid"><div class="row footer__links"><div class="col footer__col"><div class="footer__title">Community</div><ul class="footer__items clean-list"><li class="footer__item"><a href="https://lists.apache.org/list.html?dev@wayang.apache.org" target="_blank" rel="noopener noreferrer" class="footer__link-item">Mailing list<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li><li class="footer__item"><a href="https://www.youtube.com/@apachewayang" target="_blank" rel="noopener noreferrer" class="footer__link-item">YouTube<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li><li class="footer__item"><a href="https://www.linkedin.com/company/apachewayang" target="_blank" rel="noopener noreferrer" class="footer__link-item">LinkedIn<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li><li class="footer__item"><a href="https://www.reddit.com/r/ApacheWayang" target="_blank" rel="noopener noreferrer" class="footer__link-item">Reddit<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li><li class="footer__item"><a href="https://twitter.com/apachewayang" target="_blank" rel="noopener noreferrer" class="footer__link-item">Twitter<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li></ul></div><div class="col footer__col"><div class="footer__title">Docs</div><ul class="footer__items clean-list"><li class="footer__item"><a class="footer__link-item" href="/docs/start/download">Install</a></li><li class="footer__item"><a class="footer__link-item" href="/docs/introduction/features">Features</a></li><li class="footer__item"><a class="footer__link-item" href="/docs/introduction/benchmark">Benchmark</a></li></ul></div><div class="col footer__col"><div class="footer__title">Repositories</div><ul class="footer__items clean-list"><li class="footer__item"><a href="https://github.com/apache/incubator-wayang" target="_blank" rel="noopener noreferrer" class="footer__link-item">Wayang<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li><li class="footer__item"><a href="https://github.com/apache/incubator-wayang-website" target="_blank" rel="noopener noreferrer" class="footer__link-item">Website<svg width="13.5" height="13.5" aria-hidden="true" viewBox="0 0 24 24" class="iconExternalLink_nPIU"><path fill="currentColor" d="M21 13v10h-21v-19h12v2h-10v15h17v-8h2zm3-12h-10.988l4.035 4-6.977 7.07 2.828 2.828 6.977-7.07 4.125 4.172v-11z"></path></svg></a></li></ul></div></div><div class="footer__bottom text--center"><div class="margin-bottom--sm"><a href="https://incubator.apache.org/" rel="noopener noreferrer" class="footerLogoLink_BH7S"><img src="/img/apache-incubator.svg" alt="Apache Incubator logo" class="footer__logo themedComponent_mlkZ themedComponent--light_NVdE" width="200"><img src="/img/apache-incubator.svg" alt="Apache Incubator logo" class="footer__logo themedComponent_mlkZ themedComponent--dark_xIcU" width="200"></a></div><div class="footer__copyright"><div> |
| <p> Apache Wayang is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. </p> |
| <p> |
| Copyright © 2024 The Apache Software Foundation, Licensed under the Apache License, Version 2.0. <br> |
| Apache, the names of Apache projects, and the feather logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. |
| </p> |
| </div></div></div></div></footer></div> |
| </body> |
| </html> |
