| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| |
| # Daffodil-VSCode Developer's Guide |
| |
| ## Build Status |
| |
| [](https://github.com/apache/daffodil-vscode/actions/workflows/CI.yml) |
| [](https://github.com/apache/daffodil-vscode/actions/workflows/nightly.yml) |
| |
| ## Table of Contents |
| |
| - [Daffodil-VSCode Developer's Guide](#daffodil-vscode-developers-guide) |
| - [Build Status](#build-status) |
| - [Table of Contents](#table-of-contents) |
| - [Welcome](#welcome) |
| - [Prerequisites](#prerequisites) |
| - [Install Git](#install-git) |
| - [Install Visual Studio Code (VSCode)](#install-visual-studio-code-vscode) |
| - [Installing Build Requirements](#installing-build-requirements) |
| - [Summary of Build Requirements](#summary-of-build-requirements) |
| - [Step-by-step Guide for Installing Build Requirements](#step-by-step-guide-for-installing-build-requirements) |
| - [Installing Node](#installing-node) |
| - [Installing Node on Windows Note](#installing-node-on-windows-note) |
| - [Installing Java (JDK 17)](#installing-java-jdk-17) |
| - [Build Issues with Higher JDK Versions](#build-issues-with-higher-jdk-versions) |
| - [Switching Java Versions on Linux](#switching-java-versions-on-linux) |
| - [Installing SBT](#installing-sbt) |
| - [Enabling Yarn from Within Node](#enabling-yarn-from-within-node) |
| - [Do Not Use the Latest Version of Yarn](#do-not-use-the-latest-version-of-yarn) |
| - [How to Enable](#how-to-enable) |
| - [Contributing and Setup](#contributing-and-setup) |
| - [Forking the Project](#forking-the-project) |
| - [Cloning the Project to Local Environment](#cloning-the-project-to-local-environment) |
| - [Setting up SSH Keys](#setting-up-ssh-keys) |
| - [General Workflow](#general-workflow) |
| - [Pull Request Approval and Merge Process](#pull-request-approval-and-merge-process) |
| - [Approval Requirements](#approval-requirements) |
| - [Review Cycle](#review-cycle) |
| - [Merging Process](#merging-process) |
| - [Post-Merge Actions](#post-merge-actions) |
| - [Before Starting Work](#before-starting-work) |
| - [Additional Resources](#additional-resources) |
| - [Opening the Repository in VSCode](#opening-the-repository-in-vscode) |
| - [Recommended VSCode Extensions](#recommended-vscode-extensions) |
| - [Verifying Setup Can Build](#verifying-setup-can-build) |
| - [Yarn Package](#yarn-package) |
| - [Automated Testing Suite](#automated-testing-suite) |
| - [Testing Against a Specific Version of VS Code](#testing-against-a-specific-version-of-vs-code) |
| - [Debugging the Extension](#debugging-the-extension) |
| - [Changing the sampleWorkspace folder name and location](#changing-the-sampleworkspace-folder-name-and-location) |
| - [Test a Local Version of Apache Daffodil Debugger](#test-a-local-version-of-apache-daffodil-debugger) |
| - [Debugging UI Source](#debugging-ui-source) |
| - [Troubleshooting](#troubleshooting) |
| - [Yarn Package Issues](#yarn-package-issues) |
| - [If Yarn Keeps Updating to The Latest Version](#if-yarn-keeps-updating-to-the-latest-version) |
| - [Yarn Test Issues](#yarn-test-issues) |
| - [Data Editor Opens Test Case Failing](#data-editor-opens-test-case-failing) |
| - [TLS Certificate Issues](#tls-certificate-issues) |
| - [Tests Failing When Connected Through SSH](#tests-failing-when-connected-through-ssh) |
| - [Debugging Issues](#debugging-issues) |
| - [SELinux Port Functionality](#selinux-port-functionality) |
| - [Windows Yarn Test Window Not Loading Extensions](#windows-yarn-test-window-not-loading-extensions) |
| - [Variable Values Not Displaying in VSCode Debugger](#variable-values-not-displaying-in-vscode-debugger) |
| - [Using Symlink Paths](#using-symlink-paths) |
| - [Re-trying Verification After Errors](#re-trying-verification-after-errors) |
| - [Inspecting the Log Files](#inspecting-the-log-files) |
| - [Building the Documentation](#building-the-documentation) |
| - [Install Pandoc](#install-pandoc) |
| - [Command to build the Documentation](#command-to-build-the-documentation) |
| - [Adding Release Candidate Information to Apache Daffodil Site](#adding-release-candidate-information-to-apache-daffodil-site) |
| - [Reviewing and Verifying Dependency Bot Updates](#reviewing-and-verifying-dependency-bot-updates) |
| - [Testing Information](#testing-information) |
| - [Monitoring Project Status](#monitoring-project-status) |
| - [DFDL Intellisense Suggestion Cross Reference](#dfdl-intellisense-suggestion-cross-reference) |
| - [DFDL Schema Definition Error Listing](#dfdl-schema-definition-error-listing) |
| - [Thank you for your interest in contributing to this project!](#thank-you-for-your-interest-in-contributing-to-this-project) |
| |
| ## Welcome |
| |
| This guide contains developer-oriented instructions on how to setup your local development environment for contributing to this project. |
| |
| If you would like the latest stable release of the extension, please reference the [README file](README.md). |
| |
| The project is currently growing, please refer to the [wiki](https://github.com/apache/daffodil-vscode/wiki) for an overview of what the extension includes. There is also user documentation on the right pane with release specific use case guides. |
| |
| ## Prerequisites |
| |
| ### Install Git |
| |
| If your system doesn’t have Git installed, [install Git](https://git-scm.com/downloads). Select the appropriate operating system and follow the listed instructions. |
| To verify git is installed on your system, enter git -v into your system’s terminal and it should display the version of git if properly installed. |
| |
| ### Install Visual Studio Code (VSCode) |
| |
| Install VSCode per [official VSCode documentation](https://code.visualstudio.com/docs/setup/setup-overview): |
| |
| - [Windows](https://code.visualstudio.com/docs/setup/windows) |
| - [Linux](https://code.visualstudio.com/docs/setup/linux) |
| - [MacOS](https://code.visualstudio.com/docs/setup/mac) |
| |
| ## Installing Build Requirements |
| |
| This section will provide describe what requirements are needed to build with a step-by-step guide to assist developers of varying skill levels. |
| |
| ### Summary of Build Requirements |
| |
| - Java Development Kit (JDK) between 8 and 17 |
| - SBT 0.13.8 or higher |
| - Node 20 or higher |
| - [Yarn Classic](https://classic.yarnpkg.com/en/docs/install#windows-stable) |
| |
| ### Step-by-step Guide for Installing Build Requirements |
| |
| #### Installing Node |
| |
| Navigate to the [Node Download page](https://nodejs.org/en/download). Select the instructions for installing the latest LTS version of Node.js for your operating system using fnm with npm. |
| |
|  |
| |
| ##### Installing Node on Windows Note |
| |
| Make sure you have `winget` installed. If it’s not installed, you can install it by typing in `Install-Module -Name Microsoft.WinGet.Client` into PowerShell. |
| After following all the steps outlined on Node’s page, you want to [create a PowerShell profile](https://lazyadmin.nl/powershell/powershell-profile/) which is a PowerShell script that gets executed every time a new instance of PowerShell is opened. Then append `fnm env --use-on-cd --shell powershell | Out-String | Invoke-Expression` per [fnm's guidance](https://github.com/Schniz/fnm?tab=readme-ov-file#powershell). |
| |
| #### Installing Java (JDK 17) |
| |
| You can install [Oracle's JDK 17](https://www.oracle.com/java/technologies/javase/jdk17-archive-downloads.html) or alternatively install the OpenJDK equivalent. |
| |
| ##### Build Issues with Higher JDK Versions |
| |
| Note that higher versions of the JDK may cause extension building issues and is not advised. |
| |
| ##### Switching Java Versions on Linux |
| |
| You might have Java already installed. You can change the default Java provider and version by running `sudo update-alternatives --config java`. |
| |
| #### Installing SBT |
| |
| Install SBT for your appropriate operating system: |
| |
| - [Windows Instructions](https://www.scala-sbt.org/1.x/docs/Installing-sbt-on-Windows.html) |
| - Preferable method: install via .msi |
| - [Linux Instructions](https://www.scala-sbt.org/1.x/docs/Installing-sbt-on-Linux.html) |
| - [MacOS Instructions](https://www.scala-sbt.org/1.x/docs/Installing-sbt-on-Mac.html) |
| |
| #### Enabling Yarn from Within Node |
| |
| ##### Do Not Use the Latest Version of Yarn |
| |
| Do not upgrade versions of Yarn for repository. Do not follow the [instructions on Yarn’s official website](https://yarnpkg.com/getting-started/install). This will break the extension packaging process. |
| |
| ##### How to Enable |
| |
| Type into the console `corepack enable yarn`. |
| |
| ## Contributing and Setup |
| |
| ### Forking the Project |
| |
| This is an Apache Project. If you would like to contribute, you will need to fork the [daffodil-vscode main branch](https://github.com/apache/daffodil-vscode) to your own repo of choice and create pull requests to the main branch with your code changes. |
|  |
| |
| ### Cloning the Project to Local Environment |
| |
| Once forked, you can clone that forked repository to your own local environment. This can be done by using `git clone`, followed by copy and pasting the https or SSH urls shown under the “Code” dropdown. We recommend using SSH. Instructions for setting up SSH can be found in [Setting up SSH Keys](#setting-up-ssh-keys). |
| |
|  |
| |
|  |
| |
| Once cloned, you can now create branches, commits, and push changes back to your remote fork. You may make changes with any IDE, but we highly encourage using VSCode for testing for maximum compatability. |
| |
| #### Setting up SSH Keys |
| |
| Enter ssh-keygen into your terminal to generate an RSA key. Follow the prompts for naming the public and private key files (optional) and giving the keys an optional passphrase. |
| |
| The public and private keys are stored in a folder called .ssh in your users folder. The public key file is denoted with a .pub whereas the private key doesn’t have a file extension. |
| |
| Navigate to [SSH and GPG keys GitHub settings](https://github.com/settings/keys). Log into your GitHub account if needed. Click on New SSH Key. |
| |
|  |
| |
| Give your new SSH Key a name and paste the contents of the .pub file into the key textbox |
| |
|  |
| |
| Note that the above image has parts of the key blurred out for confidentiality. |
| |
| Click on Add SSH key and follow GitHub prompts for verification. |
| |
| [GitHub has their own guide on setting up SSH keys](https://docs.github.com/en/authentication/connecting-to-github-with-ssh) which can be alternatively followed. |
| |
| ### General Workflow |
| |
| Ensure that you keep your fork synced with the daffodil-vscode main by using the sync fork button, this ensures that you are developing with up to date code, so that you can be sure your changes work with present code. |
| |
|  |
| |
| If any changes were made while you were working on yours, you will need to pull these changes down to your local environment and merge them with your changes before pushing back to remote. |
| |
| Once changes are pushed, you can make pull requests with completed changes back to the main daffodil-vscode branch. You can use the contribute drop down to create a pull request back to the main branch. |
| |
|  |
| |
| ### Pull Request Approval and Merge Process |
| |
| Once you've created a pull request to the main daffodil-vscode branch, it must go through the Apache Daffodil project's approval process before it can be merged. |
| |
| #### Approval Requirements |
| |
| **All pull requests require at least two +1 votes (green approvals) from PMC members or committers before they can be merged.** This ensures that changes are properly reviewed and meet the project's quality standards. The approvals must come from committers with write access to the repository. |
| |
| #### Review Cycle |
| |
| The review process typically follows this workflow: |
| |
| 1. **Submit Pull Request**: Create your PR with your proposed changes to the main branch |
| 2. **Reviewer Feedback**: PMC members, committers, and other contributors will review your code and provide feedback |
| 3. **Address Feedback**: Add new commits to address review comments |
| - **Important**: Do not squash previously reviewed commits during the review cycle |
| - Each review iteration should involve a separate, distinct commit to preserve the review commentary |
| - The pull request will automatically update with new commits pushed to your branch |
| 4. **Approval**: Once reviewers are satisfied, they will provide +1 votes (approvals) |
| 5. **Merge**: After receiving at least two +1 votes from PMC members or committers, your PR is ready to be merged. **Squash all of your commits on your branch into one commit** before the final merge, as our PR pipeline has a check that only allows for PRs with a single commit |
| |
| #### Merging Process |
| |
| Only committers with merged GitHub and ASF (Apache Software Foundation) accounts have the authority to merge pull requests. |
| |
| **Merge Method**: When merging, committers must: |
| - Click the "Merge pull request" dropdown |
| - Select **"Rebase and merge"** |
| - Do **not** use "Squash and merge" or "Create a merge commit" |
| |
| This maintains a clean, linear commit history while preserving individual commits from the pull request. |
| |
| #### Post-Merge Actions |
| |
| After a pull request is merged, the committer who performed the merge should mark the associated GitHub issue as "Closed" and/or move it to the appropriate project board status. |
| |
| #### Before Starting Work |
| |
| To signal to the community that you are actively working on an issue and avoid duplicate efforts: |
| - **Assign the GitHub issue to yourself** before beginning development |
| - Add a comment on the issue to let others know you're working on it |
| - If the issue is part of a milestone project board, move it to "In Progress" |
| |
| #### Additional Resources |
| |
| The [Apache Daffodil Code Contributor Workflow](https://cwiki.apache.org/confluence/display/DAFFODIL/Code+Contributor+Workflow) documentation provides additional context about the broader Apache Daffodil contribution process. While the daffodil-vscode workflow is similar in many respects, there are some differences specific to this project, such as the merge method and commit requirements described above. |
| |
| ### Opening the Repository in VSCode |
| |
| There are multiple ways of opening the repository that you have cloned in VSCode: |
| |
| 1. In a terminal, type in `code <PATH TO THE CLONED REPOSITORY>` |
| 2. Clicking on open folder at the VSCode homescreen |
| |
|  |
| |
| 3. At the top options bar, File -> Open Folder |
| |
|  |
| |
| #### Recommended VSCode Extensions |
| |
| Upon opening, VSCode may prompt you to install recommended extensions. Below are the following reasons why you may need to install the recommended extensions. |
| |
| - [Prettier - Code formatter - Visual Studio Marketplace](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode) - Code formatter needed to pass CI formatting checks for PRs |
| |
| - [JAR Viewer - Visual Studio Marketplace](https://marketplace.visualstudio.com/items?itemName=wmanth.jar-viewer) - DFDL schema files can be packed in jar files when passed to Daffodil. This extension allows the debugger to show these schema files while stepping through them. |
| |
| - [Highlight Matching Tag - Visual Studio Marketplace](https://marketplace.visualstudio.com/items?itemName=vincaslt.highlight-matching-tag) - Convenience extension to better highlight opening and closing tags in DFDL schema files |
| |
| ### Verifying Setup Can Build |
| |
| Navigate inside your cloned folder in a terminal. Enter `yarn` to install required packages. If it prompts you to install yarn 1.22.XX, type `y`. |
| |
| Then type in `yarn package`. If there’s new .vsix file in the folder, then you have successfully set up your development environment correctly. For more information about `yarn package`, read [Yarn Package](#yarn-package). |
| |
|  |
| |
| Lastly, run `yarn test`. All tests should pass without any errors. More information can be found under [Automated Testing Suite (Yarn Test)](#automated-testing-suite) |
| |
|  |
| |
| Alternatively, you can run all of the commands in a single line by running `yarn && yarn package && yarn test && echo "All good!"`. |
| |
| #### Yarn Package |
| |
| If you would like to confirm that your changes compile, you can run the extension through the VSCode debugger as shown below. Under the run and debugger, you should see a launch.json already loaded for the extension, just hit the green play button. This will open a new debug window of vscode that will have the extension built and running. You can then test any changes made and ensure it is operating as intended. For more information on the debugging in VSCode, please reference [Debugging the Extension section](#debugging-the-extension). |
| |
|  |
| |
| You can also do this manually on the CLI with the following command: |
| `yarn package` |
| |
| This command will perform the following tasks: |
| |
| - Create sbt zip package and unzip it |
| - Install dependencies |
| - Compiles Extension |
| - Packages Extension |
| - Creates File |
| - apache-daffodil-vscode-*.vsix |
| |
| You can then take this .vsix file and install the extension into your vscode instance. Be sure you don’t already have the extension installed from the marketplace or the versioning could cause issues with seeing changes from your build. |
| |
|  |
| |
| #### Automated Testing Suite |
| |
| The Apache Daffodil VS Code Extension comes with an automated test suite. Run it as follows: |
| |
| ```shell |
| yarn test |
| ``` |
| |
| ##### Testing Against a Specific Version of VS Code |
| |
| By default, the test suite will use the earliest supported release of VS Code. To test against any _specific_ version of VS Code (in this example, VS Code version 1.74.3), execute the test suite as follows, setting `DAFFODIL_TEST_VSCODE_VERSION` to the desired version: |
| |
| ```shell |
| DAFFODIL_TEST_VSCODE_VERSION=1.74.3 yarn test |
| ``` |
| |
| Set `DAFFODIL_TEST_VSCODE_VERSION` to `stable` to use the latest stable release, or to `insiders` to use the latest (nightly) insiders build. |
| |
| ### Debugging the Extension |
| |
| Create a `sampleWorkspace` folder in the folder one level higher than where `daffodil-vscode` currently resides. For example, if you have your `daffodil-vscode` folder stored in a folder called repos, then make a folder in repos called `sampleWorkspace`. |
| |
|  |
| |
| It’s advised to copy sample data files and sample DFDL schemas (.dfdl.xsd) in here. You can find DFDL schemas at [DFDL Schemas for Commercial and Scientific Data Formats](https://github.com/DFDLSchemas). |
| |
| Next start debugging the extension. Make sure the “Extension” debug configuration is currently selected. You can press the green run button or alternatively press F5. |
| |
|  |
| |
| Starting the debugging session creates a terminal inside of VSCode running `yarn watch` and builds the extension. `yarn watch` automatically recompiles changes to the code in real-time. It also displays problems that occur in the code in real-time. For the changes to take effect however, you will have to stop and restart debugging. |
| |
| You may see this window pop up |
| |
|  |
| |
| Click on debug anyway once `yarn watch` says vite is fully compiled. |
| |
|  |
| |
| A new VSCode window should’ve popped up with the sampleWorkspace opened. |
| |
|  |
| |
| Open the command palette with `Ctrl + Shift + P` and type in `Daffodil Debug`. Select the `Configure launch.json` option. |
| |
|  |
| |
| Your window should look like this. |
| |
|  |
| |
| Scroll down and check these following options |
| |
|  |
| |
| Click save |
| |
|  |
| |
| Click on the Run and Debug Icon. Wizard Config should show at the top. |
| |
|  |
| |
| Then select a schema file and a data file. (Note, if you want to hard code where the files are, you’re able to change the DFDL and data file path in the Daffodil Configure launch.json window) |
| |
|  |
| |
|  |
| |
| Your window should look like the following. Note that you may have to move some tabs around. |
| |
|  |
| |
| Here’s an example view of the data editor (OmegaEdit) with the schema on the left. |
| |
|  |
| |
| Here’s the infoset diff view. The infoset is the resulting output XML/JSON file. |
| |
|  |
| |
| #### Changing the sampleWorkspace folder name and location |
| |
| If you would like to specify a different name and/or location for the sampleWorkspace, modify the following line in `.vscode/launch.json`: |
| |
|  |
| |
| #### Test a Local Version of Apache Daffodil Debugger |
| |
| The local Apache Daffodil™ Extension for Visual Studio Code downloads and caches the Apache Daffodil™ Debugger corresponding to the latest extension release. If you want to test a _local_ version of the Apache Daffodil Debugger, you need to: |
| |
| - add `"useExistingServer": true` to the configuration in your `launch.json` in the sample workspace; |
| - launch the backend debugger locally, using a launch configuration like below: |
| |
| ```json |
| { |
| "type": "scala", |
| "name": "DAPodil", |
| "request": "launch", |
| "mainClass": "org.apache.daffodil.debugger.dap.DAPodil", |
| "args": [] |
| } |
| ``` |
| |
| This will start the debug adapter and await a connection from the Apache Daffodil VS Code Extension (usually on TCP port 4711); and |
| - debug your schema file, as long as your `launch.json` has the `useExistingServer` setting above. |
| |
| #### Debugging UI Source |
| |
| ##### Inspecting the HTML |
| |
| To inspect the HTML, open up the UI window (e.g., Configure launch.json, Data Editor) that you wish to inspect and open the Webview Developer Tools with `Ctrl + Shift + P` and typing `Developer: Open Webview Developer Tools`. |
| |
| **NOTE:** This only debugs the UI from the DOM. It does not add any debugging functionality within the extension's JS/TS source files. |
| |
|  |
| |
| From the Elements tab, you can inspect the HTML by expanding and hovering elements to highlight them in the UI. |
| |
|  |
| |
| Likewise, you can use the Inspect tool to click on items in the UI to see the corresponding HTML element. |
| |
|  |
| |
|  |
| |
| ##### Debugging the JavaScript |
| |
| To debug the JavaScript, add a `debugger` statement to the source code where you want to debug. The debugger statement acts like a breakpoint and pauses execution when reached. |
| |
|  |
| |
| Start the Extension debug window and open the UI window (e.g., Configure launch.json, Data Editor) that you are debugging, and open the Webview Developer Tools with `Ctrl + Shift + P` and typing `Developer: Open Webview Developer Tools`. Then, complete the action necessary to reach the `debugger` statement if need be. After the `debugger` statement is reached, the execution will pause, and the source will be shown in the Webview Developer Tools window. |
| |
|  |
| |
| You can then add breakpoints by clicking the line number. |
| |
|  |
| |
| You can also use the bar on the right side to continue or step through the execution. |
| |
|  |
| |
| To examine the value of a variable while stepping through the execution, you can add a watch expression in the right pane. |
| |
|  |
| |
|  |
| |
| ### Troubleshooting |
| |
| You may run into issues with building, running tests, or debugging the extension. The follow sections will describe some issues you may encounter and discuss remedies. |
| |
| #### Yarn Package Issues |
| |
| ##### If Yarn Keeps Updating to The Latest Version |
| |
| As of typing this document (Feb 2025), the latest version of yarn is 4.6.0. If you type `yarn` and your console outputs the following or anything similar: |
| |
| ```PowerShell |
| ➤ YN0087: Migrated your project to the latest Yarn version �� |
| ➤ YN0000: · Yarn 4.6.0 |
| ➤ YN0000: ┌ Resolution step |
| ➤ YN0085: │ + @omega-edit/client@npm:0.9.83, @tsconfig/svelte@npm:5.0.2, @types/glob@npm:8.1.0, @types/mocha@npm:10.0.6, @types/node@npm:20.11.30, @types/vscode-webview@npm:1.57.4, @types/vscode@npm:1.95.0, @viperproject/locate-java-home@npm:1.1.15, @vscode/debugadapter-testsupport@npm:1.65.0, @vscode/debugadapter@npm:1.67.0, @vscode/test-electron@npm:2.3.8, @vscode/vsce@npm:2.22.0, @vscode/webview-ui-toolkit@npm:1.4.0, await-notify@npm:1.0.1, chai@npm:4.4.1, and 703 more. |
| ➤ YN0000: └ Completed in 9s 958ms |
| ➤ YN0000: ┌ Post-resolution validation |
| ➤ YN0002: │ apache-daffodil-vscode@workspace:. doesn't provide react (pa7c88), requested by @vscode/webview-ui-toolkit. |
| ➤ YN0086: │ Some peer dependencies are incorrectly met by your project; run yarn explain peer-requirements <hash> for details, where <hash> is the six-letter p-prefixed code. |
| ➤ YN0000: └ Completed |
| ➤ YN0000: ┌ Fetch step |
| ➤ YN0013: │ 696 packages were added to the project (+ 397.25 MiB). |
| ➤ YN0000: └ Completed in 4m 47s |
| ➤ YN0000: ┌ Link step |
| ➤ YN0007: │ esbuild@npm:0.19.9 must be built because it never has been before or the last one failed |
| ➤ YN0007: │ svelte-preprocess@npm:5.1.1 [5fe27] must be built because it never has been before or the last one failed |
| ➤ YN0007: │ svelte-preprocess@npm:5.1.1 [ab741] must be built because it never has been before or the last one failed |
| ➤ YN0007: │ keytar@npm:7.9.0 must be built because it never has been before or the last one failed |
| ➤ YN0007: │ protobufjs@npm:7.4.0 must be built because it never has been before or the last one failed |
| ➤ YN0000: └ Completed in 2m 40s |
| ➤ YN0000: · Done with warnings in 7m 37s |
| ``` |
| |
| this means that there is a yarn project that is initialized in a folder that’s higher up from the cloned repository folder. This will negatively affect the extension packaging process. A remedy for the issue is to remove all yarn files and the package.json file in the higher folder that the yarn project is initialized in. |
| |
| To remedy this, you need to change versions of Yarn. Use `yarn set version 1.22.22` to change versions of Yarn. [Documentation for set-version](https://yarnpkg.com/cli/set/version). |
| |
| #### Yarn Test Issues |
| |
| ##### Data Editor Opens Test Case Failing |
| |
| This means port 9000 is Occupied. See the current workarounds section in ["data editor opens" test fails if Port 9000 is Occupied · Issue #1175 · apache/daffodil-vscode](https://github.com/apache/daffodil-vscode/issues/1175). |
| |
| ##### TLS Certificate Issues |
| |
| HTTPS TLS certificates are verified by default. When running the test suite in certain environments (e.g., company VPN that uses endpoint protection), TLS certificate verifications may fail with a self-signed certificate error. If this is the case, either have node trust the endpoint protection certificate, or use one of these workarounds to disable the certificate verification: |
| |
| ```shell |
| NODE_TLS_REJECT_UNAUTHORIZED=0 yarn test |
| ``` |
| |
| or |
| |
| ```shell |
| node ./out/tests/runTest.js --disable_cert_verification |
| ``` |
| |
| **WARNING:** Do not `export NODE_TLS_REJECT_UNAUTHORIZED=0` into your environment as it will disable TLS certificate verification on _all_ node HTTPS connections done in that shell session. |
| |
| ##### Tests Failing When Connected Through SSH |
| |
| Unit tests may not work when connected through SSH. |
| |
| #### Debugging Issues |
| |
| ##### SELinux Port Functionality |
| |
| If you’re running into frequent issues with connectivity or VSCode freezing, it may be worth it to disable SELInux enforcing mode. |
| To check to see if SELinux OS is in enforcing mode, you can type `getenforce` in a console. If it outputs Enforcing, you’ll want to set it to Permissive by using `sudo setenforce Permissive`. |
| |
| ##### Windows Yarn Test Window Not Loading Extensions |
| |
| If you run `yarn test`, and are running into an issue where the test window is saying there’s a missing dependent extension |
| |
|  |
| |
| and yarn tests under the `getCommands` section are failing, |
| |
|  |
| |
| Inside of src\tests\runTest.ts, replace |
| |
| ```TypeScript |
| { |
| encoding: 'utf-8', |
| stdio: 'inherit', |
| } |
| ``` |
| |
| with |
| |
| ```TypeScript |
| { |
| encoding: 'utf-8', |
| stdio: 'inherit', |
| shell: os.platform().toLowerCase().startsWith('win'), |
| } |
| ``` |
| |
| and add |
| `import os from 'os'` |
| near the top of the file along with the other import statements. |
| |
| ##### Variable Values Not Displaying in VSCode Debugger |
| |
| When debugging in VSCode, if you're noticing that it's not possible to view variables' values or they're appearing as uncaught references when they aren't supposed to, as shown below, |
| |
|  |
| |
| the yarn script `"vite:dev"`, under `"scripts"` in `package.json`, should be utilized for building the |
| extension for debugging capabilities. |
| |
| ##### Using Symlink Paths |
| |
| Another important issue to note when choosing working directories is that VSCode, like many IDEs, seems to not like projects that are on paths reachable via symlinks. Using symlinks in paths is highly likely to cause problems and thus should be avoided. |
| |
| #### Re-trying Verification After Errors |
| |
| Type in `git clean -fdx`. Then run `yarn && yarn package && yarn test && echo "All good!"`. |
| If issues persist, you may want to uninstall Node and reinstall it. If that doesn’t remedy the issue, you may have to create a fresh VM. |
| |
| #### Inspecting the Log Files |
| |
| For issues not mentioned in this section, the log files for the debugger are located in your OS temp directory (e.g. `/tmp` for Ubuntu 24.04.2). The log file for the debugger should be named `daffodil-debugger-<port>.log` when debugging the extension or `yarn-test-daffodil-debugger-<port>.log` when running yarn test. |
| |
| The log files for the data editor are located in your OS app data path (e.g. `/home/<user>/.local/share/omega_edit`). The log files for the data editor should be named `serv-<port>.log` for the server logs and `dataEditor-<port>.log` for the data editor logs when using the data editor or `test-serv-<port>.log` and `test-dataEditor-<port>.log` when running yarn test. |
| |
| ## Development Specific |
| |
| ### Dependency Licensing Information |
| |
| The extension has a pipeline for ensuring that `yarn` packages and other relevant dependencies are in compliance with accepted licenses. Licenses that are not compliant or don't have their licenses noted in the repository will be flagged in CI. |
| |
| Dependencies' licensing information is compiled under `./build/package`. Contents of the dependencies' LICENSE, NONOTICE, NOTICE files are to be appended to the corresponding file in following the existing |
| licensing file's format. Refer to the prior appended entries as a guideline for the formatting. |
| |
| Developers may have to modify contents of the files under `./build/package` in events where dependencies may be affected, such as package updates, additions, etc. |
| |
| ## Building the Documentation |
| |
| ### Install Pandoc |
| |
| Before running commands to build the documentation, be sure to [install Pandoc](https://pandoc.org/installing.html). |
| |
| ### Command to build the Documentation |
| |
| To build `docx` (Word formatted) documentation, from the top of the cloned repository, run: |
| |
| ```shell |
| cd docs && make all |
| ``` |
| |
| ## Adding Release Candidate Information to Apache Daffodil Site |
| |
| On every release candidate the following process needs to be done on the [apache/daffodil-site](https://github.com/apache/daffodil-site): |
| |
| - Create a new file under `site/_vscode` with the release version as the name, e.g. `1.4.1.md` using the following example. Look at the closed tickets for the milestone version to add bullets to the `New Features & Fixes` section. |
| ```Markdown |
| --- |
| |
| title: <version> |
| release: rc1 (replace with current rc version or "final" if rc moved to final release) |
| apache: true |
| date: <date-rc-was-created-and-put-out-for-vote> |
| summary: > |
| <summary of release candidate> |
| |
| source-dist: |
| - "apache-daffodil-vscode-<version>-src.zip" |
| |
| binary-dist: |
| - "apache-daffodil-vscode-<version>.vsix" |
| --- |
| |
| This release contains new features and fixes listed below. |
| |
| # New Features & Fixes |
| |
| * <category>: (Debugger, Data Editor, etc.) |
| * <bulleted list of features/fixes> |
| * <category>: (Debugger, Data Editor, etc.) |
| * <bulleted list of features/fixes> |
| * Known Issues: |
| * <bulleted list of known issues> |
| |
| # Closed Issues |
| |
| [GitHub v<version> Closed Issues](<link to milestone's closed issues>) |
| |
| ``` |
| |
| - Create a PR to the daffodil-site repository |
| - After each RC and and when an RC is promoted to final release, the file needs to be updated to keep track of where the release process is. |
| - Once that initial PR is merged that release page will not be shown unless the value released: is set to final which is done after promoting a RC to final release. |
| |
| ## Reviewing and Verifying Dependency Bot Updates |
| |
| For GitHub CI action updates (pull requests that start with **Bump actions/...**), make sure the affected workflows still operate as expected (they are automatically CI tested). GitHub CI actions update workflow YAML files, and are part of the CI infrastructure and not a code dependency. These should be relatively quick and easy to assess compared to code dependencies. |
| |
| If the updates are not GitHub CI action updates, then additional scrutiny is required. When reviewing and verifying dependency bot updates that are part the software supply chain being distributed, please use the following checklist: |
| |
| - [ ] **Do all automated continuous integration checks pass?** |
| - [ ] **Is the update a patch, minor, or major update?** |
| - [ ] **Is the license still compatible with ASF License Policy?** |
| - [ ] **Have any changes been made to LICENSE/NOTICE files that need to be incorporated?** |
| - [ ] **Have any transitive dependencies been added or changed?** |
| |
| ## Testing Information |
| |
| Testing for this extension comprises of unit testing, CI/CD tests for pull requests (PRs), and manual tests. |
| |
| List of manual tests can be found in the [testing README](https://github.com/apache/daffodil-vscode/blob/8c70937f6badc8b0e8eec5b4d34d3657e0676a32/src/tests/README.md). |
| |
| ## Monitoring Project Status |
| |
| Milestone-level project status can be monitored using the [Projects tab](https://github.com/apache/daffodil-vscode/projects) in the [Project's GitHub repository](https://github.com/apache/daffodil-vscode). |
| |
| ## DFDL Intellisense Suggestion Cross Reference |
| |
| Documents exist that attempt to map the DFDL schema elements and attributes in the [Apache DFDL Specification](https://daffodil.apache.org/docs/dfdl/) that can be used as a guide to determine the intellisense suggestions that the extension should make based on the element or attribute marked by the cursor when intellisense is triggered. The documents include an excel spreadsheet, markdown file exports of the spreadsheets, and html exports of the spreadsheets. These files can be found in the source code at src/language/dfdl-xref and src/language/dfdl-xref/html. |
| |
| |
| ## DFDL Schema Definition Error Listing |
| |
| A document exists that shows possible schema definition errors that can occur when developing DFDL schemas and the conditions that create those errors. This file can be found in the source code at src/language/dfdl-xref/DFDL_Schema_Defintion_Errors.md. |
| |
| ## Thank you for your interest in contributing to this project |
| |
| You can ask questions on the <dev@daffodil.apache.org> or <users@daffodil.apache.org> mailing lists. You can report bugs via GitHub Issues. |
