Updated package versions to eliminate vulnerable and deprecated transitive dependencies (#976)
* NuGet.config: Added preview feeds for J2N and ICU4N
* .build/dependencies.props: Bumped J2N to 2.1.0-alpha-0103
* .build/dependencies.props: Bumped ICU4N to 60.1.0-alpha.430. Removed dependencies on ICU4N.Collation, ICU4N.CurrencyData, ICU4N.LanguageData, ICU4N.RegionData, and ICU4N.Transliterator because these have all been merged into the main assembly. Did minimal integration to fix compile errors.
* .build/dependencies.props: Bumped ICU4N to 60.1.0-alpha.431
* SWEEP: Removed dependency on Microsoft.Extensions.Options, as this was just to pin the version
* .build/dependencies.props + Lucene.Net.TestFramework.csproj: Removed dependency on System.Text.Json, since it was only used to pin the version
* .build/dependencies.props: Use 6.0.0 for all Microsoft.Extensions packages on net6.0 and 8.0.0 for Microsoft.Extensions packages on net8.0
* .build/dependencies.props: Bumped Microsoft.NETFramework.ReferenceAssemblies to 1.0.3
* .build/dependencies.props + Lucene.Net.csproj: Removed dependency on System.Runtime.CompilerServices.Unsafe as it was only used to pin the version
* .build/dependencies.props: Use System.Security.Cryptography.Xml 8.0.2 on net8.0. Only use 6.0.1 on net6.0 because lucene-cli is the only consumer. 6.0.0 has a vulnerability, so we must pin the version since we own the distribution.
* Lucene.Net.csproj: Added package dependency on System.Runtime.InteropServices.RuntimeInformation and reference to System.Net.Http in net462
* .build/dependencies.props: Bumped J2N to 2.1.0-alpha-0105
* .build/dependencies.props: Bumped ICU4N to 60.1.0-alpha.434
* Lucene.Net.sln: For Lucene.Net.csproj, added build dependencies on Lucene.Net.CodeAnalysis.CSharp and Lucene.Net.CodeAnalysis.VisualBasic to ensure it is built prior to Lucene.Net
* .build/dependencies.props + Lucene.Net.CodeAnalysis (CSharp + VisualBasic): Added package references on System.Net.Http and System.Text.RegularExpressions
* Lucene.Net.Expressions.csproj: Added package references on transitive dependencies NETStandardLibrary 1.6.1, System.Net.Http 4.3.4, and System.Text.RegularExpressions 4.3.1 because they have vulnerabilities.
* Lucene.Net.Expressions.csproj: Don't reference NETStandard.Library
* .build/dependencies.props + lucene-cli.csproj: Consistently use Microsoft.Extensions.Configuration 8.x. In Lucene.Net.TestFramework and lucene-cli, we must reference Microsoft.Extensions.Configuration.Json 8.0.1 to avoid pulling in vulnerable transitive dependencies.
* .build/dependencies.props: Bumped Microsoft.Extensions.DependencyInjection.Abstractions to 8.0.0 and Microsoft.Extensions.DependencyInjection to 8.0.1 to be consistent with Microsoft.Extensions.Configuration
* .build/dependencies.props: Bumped Microsoft.AspNetCore.Http.Abstractions to 2.1.1 because 2.0.0 has been deprecated
* .build/dependencies.props: Bumped Microsoft.AspNetCore.TestHost to 2.1.1 on .NET Framework and 8.0.0 on other target frameworks
* .build/dependenceis.props + Lucene.Net.Replicator.AspNetCore: Added dependency on System.Text.Encodings.Web to upgrade the version, since the version referenced by Microsoft.AspNetCore.Http.Abstractions is vulnerable and there is no upgrade.
* .build/dependencies.props: Use appropriate version of Microsoft.AspNetCore.TestHost for the test target framework
* .build/dependencies.props + Lucene.Net.Tests.Replicator.csproj: On net472, added references to Microsoft.AspNetCore.Http and System.IO.Pipelines because the versions that Microsoft.AspNetCore.TestHost 2.1.1 references are vulnerable
* .build/dependencies.props + Lucene.Net.Analysis.OpenNLP: Added reference to System.Text.Json because the version that IKVM references transitively is vulnerable and we are blocked from upgrading IKVM due to disk space limitations on Azure DevOps.
* .build/dependencies.props: Bumped Microsoft.Extensions.Configuration.Xml to 8.0.1 to avoid bringing in vulnerable version of System.Formats.Asn1 by default.
* .build/dependencies.props: Bumped J2N to 2.1.0 and added a version constraint so we cannot depend on 3.x or higher (since it will break binary compatibility)
* .build/dependencies.props: Bumped ICU4N to 60.1.0-alpha.435
* .build/dependencies.props: Bumped ICU4N to 60.1.0-alpha.436
diff --git a/.build/dependencies.props b/.build/dependencies.props
index 0915432..4be8d11 100644
--- a/.build/dependencies.props
+++ b/.build/dependencies.props
@@ -32,34 +32,31 @@
https://github.com/apache/lucene-solr/tree/31d7ec7bbfdcd2c4cc61d9d35e962165410b65fe/lucene/analysis/icu/src/data/utr30
Just make sure they are adjusted to the right version of ICU/Lucene.
<ICU4NPackageVersion>[60.1,60.2)</ICU4NPackageVersion> -->
- <ICU4NPackageVersion>60.1.0-alpha.356</ICU4NPackageVersion>
- <ICU4NCollationPackageVersion>$(ICU4NPackageVersion)</ICU4NCollationPackageVersion>
- <ICU4NCurrencyDataPackageVersion>$(ICU4NPackageVersion)</ICU4NCurrencyDataPackageVersion>
- <ICU4NLanguageDataPackageVersion>$(ICU4NPackageVersion)</ICU4NLanguageDataPackageVersion>
- <ICU4NRegionDataPackageVersion>$(ICU4NPackageVersion)</ICU4NRegionDataPackageVersion>
- <ICU4NTransliteratorPackageVersion>$(ICU4NPackageVersion)</ICU4NTransliteratorPackageVersion>
+ <ICU4NPackageVersion>[60.1.0-alpha.436]</ICU4NPackageVersion>
<IKVMPackageVersion>8.7.5</IKVMPackageVersion>
<IKVMMavenSdkPackageVersion>1.6.7</IKVMMavenSdkPackageVersion>
- <J2NPackageVersion>2.0.0</J2NPackageVersion>
+ <!-- J2N will break binary compatibility in 3.0.0 to fix the APIs of collection types -->
+ <J2NPackageVersion>[2.1.0, 3.0.0)</J2NPackageVersion>
<LiquidTestReportsMarkdownPackageVersion>1.0.9</LiquidTestReportsMarkdownPackageVersion>
- <MicrosoftAspNetCoreHttpAbstractionsPackageVersion>2.0.0</MicrosoftAspNetCoreHttpAbstractionsPackageVersion>
- <MicrosoftAspNetCoreTestHostPackageVersion>5.0.0</MicrosoftAspNetCoreTestHostPackageVersion>
- <MicrosoftAspNetCoreTestHostPackageVersion Condition=" $(TargetFramework.StartsWith('net4')) Or $(TargetFramework.StartsWith('netcoreapp')) ">2.0.0</MicrosoftAspNetCoreTestHostPackageVersion>
+ <MicrosoftAspNetCoreHttpAbstractionsPackageVersion>2.1.1</MicrosoftAspNetCoreHttpAbstractionsPackageVersion>
+ <MicrosoftAspNetCoreHttpPackageVersion>2.1.34</MicrosoftAspNetCoreHttpPackageVersion>
+ <MicrosoftAspNetCoreTestHostPackageVersion>6.0.0</MicrosoftAspNetCoreTestHostPackageVersion>
+ <MicrosoftAspNetCoreTestHostPackageVersion Condition=" $(TargetFramework.StartsWith('net4')) ">2.1.1</MicrosoftAspNetCoreTestHostPackageVersion>
+ <MicrosoftAspNetCoreTestHostPackageVersion Condition=" '$(TargetFramework)' == 'net5.0' ">3.1.32</MicrosoftAspNetCoreTestHostPackageVersion>
<MicrosoftCodeAnalysisAnalyzersPackageVersion>2.9.8</MicrosoftCodeAnalysisAnalyzersPackageVersion>
<MicrosoftCodeAnalysisCSharpPackageVersion>2.6.1</MicrosoftCodeAnalysisCSharpPackageVersion>
<MicrosoftCodeAnalysisCSharpWorkspacesPackageVersion>$(MicrosoftCodeAnalysisCSharpPackageVersion)</MicrosoftCodeAnalysisCSharpWorkspacesPackageVersion>
<MicrosoftCodeAnalysisVisualBasicPackageVersion>2.6.1</MicrosoftCodeAnalysisVisualBasicPackageVersion>
<MicrosoftCodeAnalysisVisualBasicWorkspacesPackageVersion>$(MicrosoftCodeAnalysisVisualBasicPackageVersion)</MicrosoftCodeAnalysisVisualBasicWorkspacesPackageVersion>
- <MicrosoftExtensionsConfigurationPackageVersion>3.1.6</MicrosoftExtensionsConfigurationPackageVersion>
+ <MicrosoftExtensionsConfigurationPackageVersion>8.0.0</MicrosoftExtensionsConfigurationPackageVersion>
<MicrosoftExtensionsConfigurationAbstractionsPackageVersion>$(MicrosoftExtensionsConfigurationPackageVersion)</MicrosoftExtensionsConfigurationAbstractionsPackageVersion>
<MicrosoftExtensionsConfigurationCommandLinePackageVersion>$(MicrosoftExtensionsConfigurationPackageVersion)</MicrosoftExtensionsConfigurationCommandLinePackageVersion>
<MicrosoftExtensionsConfigurationEnvironmentVariablesPackageVersion>$(MicrosoftExtensionsConfigurationPackageVersion)</MicrosoftExtensionsConfigurationEnvironmentVariablesPackageVersion>
- <MicrosoftExtensionsConfigurationJsonPackageVersion>$(MicrosoftExtensionsConfigurationPackageVersion)</MicrosoftExtensionsConfigurationJsonPackageVersion>
- <MicrosoftExtensionsConfigurationXmlPackageVersion>$(MicrosoftExtensionsConfigurationPackageVersion)</MicrosoftExtensionsConfigurationXmlPackageVersion>
- <MicrosoftExtensionsDependencyInjectionAbstractionsPackageVersion>2.0.0</MicrosoftExtensionsDependencyInjectionAbstractionsPackageVersion>
- <MicrosoftExtensionsDependencyInjectionPackageVersion>$(MicrosoftExtensionsDependencyInjectionAbstractionsPackageVersion)</MicrosoftExtensionsDependencyInjectionPackageVersion>
- <MicrosoftExtensionsOptionsPackageVersion>2.0.0</MicrosoftExtensionsOptionsPackageVersion>
- <MicrosoftNETFrameworkReferenceAssembliesPackageReferenceVersion>1.0.2</MicrosoftNETFrameworkReferenceAssembliesPackageReferenceVersion>
+ <MicrosoftExtensionsConfigurationJsonPackageVersion>8.0.1</MicrosoftExtensionsConfigurationJsonPackageVersion>
+ <MicrosoftExtensionsConfigurationXmlPackageVersion>8.0.1</MicrosoftExtensionsConfigurationXmlPackageVersion>
+ <MicrosoftExtensionsDependencyInjectionAbstractionsPackageVersion>8.0.0</MicrosoftExtensionsDependencyInjectionAbstractionsPackageVersion>
+ <MicrosoftExtensionsDependencyInjectionPackageVersion>8.0.1</MicrosoftExtensionsDependencyInjectionPackageVersion>
+ <MicrosoftNETFrameworkReferenceAssembliesPackageReferenceVersion>1.0.3</MicrosoftNETFrameworkReferenceAssembliesPackageReferenceVersion>
<MicrosoftNETTestSdkPackageVersion>17.9.0</MicrosoftNETTestSdkPackageVersion>
<MicrosoftSourceLinkAzureReposGitPackageReferenceVersion>1.1.1</MicrosoftSourceLinkAzureReposGitPackageReferenceVersion>
<MicrosoftSourceLinkGitHubPackageReferenceVersion>$(MicrosoftSourceLinkAzureReposGitPackageReferenceVersion)</MicrosoftSourceLinkGitHubPackageReferenceVersion>
@@ -73,16 +70,18 @@
<RandomizedTestingGeneratorsPackageVersion>2.7.8</RandomizedTestingGeneratorsPackageVersion>
<SharpZipLibPackageVersion>1.4.2</SharpZipLibPackageVersion>
<Spatial4nPackageVersion>0.4.1.1</Spatial4nPackageVersion>
+ <SystemIOPipelinesPackageVersion>8.0.0</SystemIOPipelinesPackageVersion>
<SystemMemoryPackageVersion>4.5.5</SystemMemoryPackageVersion>
+ <SystemNetHttpPackageVersion>4.3.4</SystemNetHttpPackageVersion>
<SystemReflectionEmitPackageVersion>4.3.0</SystemReflectionEmitPackageVersion>
<SystemReflectionEmitILGenerationPackageVersion>4.3.0</SystemReflectionEmitILGenerationPackageVersion>
<SystemReflectionTypeExtensionsPackageVersion>4.3.0</SystemReflectionTypeExtensionsPackageVersion>
- <SystemRuntimeCompilerServicesUnsafePackageVersion>6.0.0</SystemRuntimeCompilerServicesUnsafePackageVersion>
<SystemRuntimeInteropServicesRuntimeInformationPackageVersion>4.3.0</SystemRuntimeInteropServicesRuntimeInformationPackageVersion>
- <SystemSecurityCryptographyXmlPackageVersion>6.0.1</SystemSecurityCryptographyXmlPackageVersion>
<SystemTextEncodingCodePagesPackageVersion>4.3.0</SystemTextEncodingCodePagesPackageVersion>
<SystemTextEncodingCodePagesPackageVersion Condition=" '$(TargetFramework)' == 'net472' ">5.0.0</SystemTextEncodingCodePagesPackageVersion>
- <SystemTextJsonPackageVersion>6.0.6</SystemTextJsonPackageVersion>
+ <SystemTextEncodingsWebPackageVersion>8.0.0</SystemTextEncodingsWebPackageVersion>
+ <SystemTextJsonPackageVersion>6.0.10</SystemTextJsonPackageVersion>
+ <SystemTextRegularExpressionsPackageVersion>4.3.1</SystemTextRegularExpressionsPackageVersion>
<TimeZoneConverterPackageVersion>6.1.0</TimeZoneConverterPackageVersion>
</PropertyGroup>
<PropertyGroup Label="Maven Package Reference Versions">
diff --git a/Lucene.Net.sln b/Lucene.Net.sln
index a7a55a7..487e23f 100644
--- a/Lucene.Net.sln
+++ b/Lucene.Net.sln
@@ -119,6 +119,10 @@
EndProjectSection
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Lucene.Net", "src\Lucene.Net\Lucene.Net.csproj", "{3A0AA37E-2B7B-4416-B528-DA4E0E6A6706}"
+ ProjectSection(ProjectDependencies) = postProject
+ {441876AF-F691-408C-85EC-6A934E60F627} = {441876AF-F691-408C-85EC-6A934E60F627}
+ {5CD4D4E8-6132-4384-98FC-6AB1C97E0B80} = {5CD4D4E8-6132-4384-98FC-6AB1C97E0B80}
+ EndProjectSection
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Lucene.Net.Analysis.Common", "src\Lucene.Net.Analysis.Common\Lucene.Net.Analysis.Common.csproj", "{3D0366A8-515D-44F0-835F-4118853CFA14}"
EndProject
@@ -284,7 +288,7 @@
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "docs", "docs", "{42599646-275F-4970-BC60-A3349F6498CC}"
EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "LuceneDocsPlugins", "src\docs\LuceneDocsPlugins\LuceneDocsPlugins.csproj", "{FED4A824-1F32-4948-8D37-2B7610804DB5}"
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "LuceneDocsPlugins", "src\docs\LuceneDocsPlugins\LuceneDocsPlugins.csproj", "{FED4A824-1F32-4948-8D37-2B7610804DB5}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
diff --git a/NuGet.config b/NuGet.config
index afae527..d475c20 100644
--- a/NuGet.config
+++ b/NuGet.config
@@ -1,4 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?>
+<?xml version="1.0" encoding="utf-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
@@ -21,6 +21,8 @@
<configuration>
<packageSources>
+ <add key="ICU4N preview feed" value="https://www.myget.org/F/icu4n/api/v3/index.json" />
+ <add key="J2N preview feed" value="https://www.myget.org/F/j2n-preview/api/v3/index.json" />
<add key="NuGet official package source" value="https://api.nuget.org/v3/index.json" />
</packageSources>
</configuration>
diff --git a/src/Lucene.Net.Analysis.Common/Analysis/Util/OpenStringBuilder.cs b/src/Lucene.Net.Analysis.Common/Analysis/Util/OpenStringBuilder.cs
index 7955cbd..8de7197 100644
--- a/src/Lucene.Net.Analysis.Common/Analysis/Util/OpenStringBuilder.cs
+++ b/src/Lucene.Net.Analysis.Common/Analysis/Util/OpenStringBuilder.cs
@@ -28,7 +28,7 @@
/// <summary>
/// A StringBuilder that allows one to access the array.
/// </summary>
- public class OpenStringBuilder : IAppendable, ICharSequence
+ public class OpenStringBuilder : IAppendable, ISpanAppendable, ICharSequence // LUCENENET specific - implemented ISpanAppendable to support ReadOnlySpan<char>
{
protected char[] m_buf;
protected int m_len;
@@ -93,6 +93,13 @@
return this;
}
+ public virtual OpenStringBuilder Append(ReadOnlySpan<char> value) // LUCENENET specific - added to support ReadOnlySpan<char>
+ {
+ EnsureCapacity(value.Length);
+ UnsafeWrite(value);
+ return this;
+ }
+
// LUCENENET specific - overload for string (more common in .NET than ICharSequence)
public virtual OpenStringBuilder Append(string csq)
{
@@ -192,6 +199,13 @@
UnsafeWrite((char)b);
}
+ public virtual void UnsafeWrite(ReadOnlySpan<char> b) // LUCENENET specific - added to support ReadOnlySpan<char>
+ {
+ int len = b.Length;
+ b.CopyTo(m_buf.AsSpan(this.m_len, len));
+ this.m_len += len;
+ }
+
public virtual void UnsafeWrite(char[] b, int off, int len)
{
Arrays.Copy(b, off, m_buf, this.m_len, len);
@@ -319,6 +333,23 @@
return new string(m_buf, 0, Length);
}
+ public virtual OpenStringBuilder Remove(int startIndex, int length) // LUCENENET specific - added missing remove method
+ {
+ if (m_len == length && startIndex == 0)
+ {
+ m_len = 0;
+ return this;
+ }
+
+ if (length > 0)
+ {
+ int endIndex = startIndex + length;
+ m_buf.AsSpan(endIndex).CopyTo(m_buf.AsSpan(startIndex));
+ m_len -= length;
+ }
+ return this;
+ }
+
#region IAppendable Members
IAppendable IAppendable.Append(char value)
@@ -366,6 +397,8 @@
return Append(value, startIndex, count);
}
+ ISpanAppendable ISpanAppendable.Append(ReadOnlySpan<char> value) => Append(value);
+
#endregion IAppendable Members
}
-}
\ No newline at end of file
+}
diff --git a/src/Lucene.Net.Analysis.ICU/Analysis/Icu/ICUNormalizer2CharFilter.cs b/src/Lucene.Net.Analysis.ICU/Analysis/Icu/ICUNormalizer2CharFilter.cs
index 5dc99b0..577fa46 100644
--- a/src/Lucene.Net.Analysis.ICU/Analysis/Icu/ICUNormalizer2CharFilter.cs
+++ b/src/Lucene.Net.Analysis.ICU/Analysis/Icu/ICUNormalizer2CharFilter.cs
@@ -1,5 +1,6 @@
// Lucene version compatibility level < 7.1.0
using J2N;
+using J2N.Text;
using ICU4N.Text;
using Lucene.Net.Analysis.CharFilters;
using Lucene.Net.Analysis.Util;
@@ -37,7 +38,7 @@
public sealed class ICUNormalizer2CharFilter : BaseCharFilter
{
private readonly Normalizer2 normalizer;
- private readonly StringBuilder inputBuffer = new StringBuilder();
+ private readonly OpenStringBuilder inputBuffer = new OpenStringBuilder(); // LUCENENET specific - Using OpenStringBuilder because StringBuilder cannot be directly converted to ReadOnlySpan<char>
private readonly StringBuilder resultBuffer = new StringBuilder();
private bool inputFinished;
@@ -162,10 +163,10 @@
private int ReadFromInputWhileSpanQuickCheckYes()
{
- int end = normalizer.SpanQuickCheckYes(inputBuffer);
+ int end = normalizer.SpanQuickCheckYes(inputBuffer.Array.AsSpan(0, inputBuffer.Length));
if (end > 0)
{
- resultBuffer.Append(inputBuffer.ToString(0, end));
+ resultBuffer.Append(inputBuffer.Array.AsSpan(0, end));
inputBuffer.Remove(0, end);
checkedInputBoundary = Math.Max(checkedInputBoundary - end, 0);
charCount += end;
@@ -212,7 +213,7 @@
private int NormalizeInputUpto(int length)
{
int destOrigLen = resultBuffer.Length;
- normalizer.NormalizeSecondAndAppend(resultBuffer, inputBuffer.ToString(0, length));
+ normalizer.NormalizeSecondAndAppend(resultBuffer, inputBuffer.Array.AsSpan(0, length));
inputBuffer.Remove(0, length);
checkedInputBoundary = Math.Max(checkedInputBoundary - length, 0);
diff --git a/src/Lucene.Net.Analysis.ICU/Analysis/Icu/ICUNormalizer2Filter.cs b/src/Lucene.Net.Analysis.ICU/Analysis/Icu/ICUNormalizer2Filter.cs
index 0af6c4c..b081c51 100644
--- a/src/Lucene.Net.Analysis.ICU/Analysis/Icu/ICUNormalizer2Filter.cs
+++ b/src/Lucene.Net.Analysis.ICU/Analysis/Icu/ICUNormalizer2Filter.cs
@@ -1,7 +1,9 @@
// Lucene version compatibility level 7.1.0
using ICU4N.Text;
using Lucene.Net.Analysis.TokenAttributes;
+using Lucene.Net.Analysis.Util;
using Lucene.Net.Support;
+using System;
using System.Text;
namespace Lucene.Net.Analysis.Icu
@@ -60,7 +62,7 @@
{
private readonly ICharTermAttribute termAtt;
private readonly Normalizer2 normalizer;
- private readonly StringBuilder buffer = new StringBuilder();
+ private readonly OpenStringBuilder buffer = new OpenStringBuilder(); // LUCENENET specific - Using OpenStringBuilder because StringBuilder cannot be directly converted to ReadOnlySpan<char>
/// <summary>
/// Create a new <see cref="ICUNormalizer2Filter"/> that combines NFKC normalization, Case
@@ -88,10 +90,10 @@
{
if (m_input.IncrementToken())
{
- if (normalizer.QuickCheck(termAtt) != QuickCheckResult.Yes)
+ if (normalizer.QuickCheck(termAtt.Buffer.AsSpan(0, termAtt.Length)) != QuickCheckResult.Yes)
{
buffer.Length = 0;
- normalizer.Normalize(termAtt, buffer);
+ normalizer.Normalize(termAtt.Buffer.AsSpan(0, termAtt.Length), buffer);
termAtt.SetEmpty().Append(buffer);
}
return true;
diff --git a/src/Lucene.Net.Analysis.ICU/Analysis/Icu/ICUTransformFilter.cs b/src/Lucene.Net.Analysis.ICU/Analysis/Icu/ICUTransformFilter.cs
index 4ff71f8..bff9ada 100644
--- a/src/Lucene.Net.Analysis.ICU/Analysis/Icu/ICUTransformFilter.cs
+++ b/src/Lucene.Net.Analysis.ICU/Analysis/Icu/ICUTransformFilter.cs
@@ -2,6 +2,7 @@
using ICU4N.Text;
using Lucene.Net.Analysis.TokenAttributes;
using Lucene.Net.Support;
+using System;
namespace Lucene.Net.Analysis.Icu
{
@@ -128,7 +129,7 @@
this.length = token.Length;
}
- public int Char32At(int pos) => UTF16.CharAt(buffer, 0, length, pos);
+ public int Char32At(int pos) => UTF16.CharAt(buffer.AsSpan(0, length), pos);
public char this[int pos] => buffer[pos];
@@ -139,6 +140,11 @@
Replace(destinationIndex, destinationIndex - destinationIndex, text, 0, length); // LUCENENET: Corrected length & charsLen
}
+ public void CopyTo(int sourceIndex, Span<char> destination, int count)
+ {
+ buffer.AsSpan(sourceIndex, count).CopyTo(destination);
+ }
+
public void CopyTo(int sourceIndex, char[] destination, int destinationIndex, int count)
{
Arrays.Copy(buffer, sourceIndex, destination, destinationIndex, count);
@@ -157,6 +163,15 @@
token.Length = (this.length = newLength);
}
+ public void Replace(int start, int length, ReadOnlySpan<char> text) // LUCENENET: Changed 2nd parameter from limit to length
+ {
+ int charsLen = text.Length;
+ int newLength = ShiftForReplace(start, length + start, charsLen); // LUCENENET: Changed 2nd parameter to calculate limit
+ // insert the replacement text
+ text.CopyTo(buffer.AsSpan(start, charsLen));
+ token.Length = (this.length = newLength);
+ }
+
public void Replace(int start, int length, char[] text, int charsStart,
int charsLen)
{
diff --git a/src/Lucene.Net.Analysis.ICU/Analysis/Icu/Segmentation/BreakIteratorWrapper.cs b/src/Lucene.Net.Analysis.ICU/Analysis/Icu/Segmentation/BreakIteratorWrapper.cs
index d01aacc..4be727a 100644
--- a/src/Lucene.Net.Analysis.ICU/Analysis/Icu/Segmentation/BreakIteratorWrapper.cs
+++ b/src/Lucene.Net.Analysis.ICU/Analysis/Icu/Segmentation/BreakIteratorWrapper.cs
@@ -1,6 +1,7 @@
// Lucene version compatibility level 8.6.1
using ICU4N.Text;
using J2N;
+using System;
namespace Lucene.Net.Analysis.Icu.Segmentation
{
@@ -79,7 +80,7 @@
{
int begin = start + current;
int end = start + next;
- int codepoint = UTF16.CharAt(text, 0, end, begin);
+ int codepoint = UTF16.CharAt(text.AsSpan(0, end), begin); // LUCENENET: Checked 2nd argument
if (EMOJI.Contains(codepoint))
{
if (EMOJI_RK.Contains(codepoint))
diff --git a/src/Lucene.Net.Analysis.ICU/Analysis/Icu/Segmentation/ScriptIterator.cs b/src/Lucene.Net.Analysis.ICU/Analysis/Icu/Segmentation/ScriptIterator.cs
index 1228c5d..bf80745 100644
--- a/src/Lucene.Net.Analysis.ICU/Analysis/Icu/Segmentation/ScriptIterator.cs
+++ b/src/Lucene.Net.Analysis.ICU/Analysis/Icu/Segmentation/ScriptIterator.cs
@@ -2,6 +2,7 @@
using ICU4N;
using ICU4N.Globalization;
using ICU4N.Text;
+using System;
namespace Lucene.Net.Analysis.Icu.Segmentation
{
@@ -93,7 +94,7 @@
while (index < limit)
{
- int ch = UTF16.CharAt(text, start, limit, index - start);
+ int ch = UTF16.CharAt(text.AsSpan(start, limit), index - start);
int sc = GetScript(ch);
/*
diff --git a/src/Lucene.Net.Analysis.OpenNLP/Lucene.Net.Analysis.OpenNLP.csproj b/src/Lucene.Net.Analysis.OpenNLP/Lucene.Net.Analysis.OpenNLP.csproj
index 33920cd..c8a8759 100644
--- a/src/Lucene.Net.Analysis.OpenNLP/Lucene.Net.Analysis.OpenNLP.csproj
+++ b/src/Lucene.Net.Analysis.OpenNLP/Lucene.Net.Analysis.OpenNLP.csproj
@@ -52,6 +52,10 @@
<PackageReference Include="ICU4N" Version="$(ICU4NPackageVersion)" />
<PackageReference Include="IKVM" Version="$(IKVMPackageVersion)" />
<PackageReference Include="IKVM.Maven.Sdk" Version="$(IKVMMavenSdkPackageVersion)" />
+
+ <!-- This is a transitive dependency of IKVM, but the version it references is vulnerable. We are currently blocked from upgrading
+ IKVM due to the Azure DevOps limitation of 10GB for a single build agent, so we have to add this reference instead. -->
+ <PackageReference Include="System.Text.Json" Version="$(SystemTextJsonPackageVersion)" />
</ItemGroup>
<ItemGroup>
diff --git a/src/Lucene.Net.Benchmark/Lucene.Net.Benchmark.csproj b/src/Lucene.Net.Benchmark/Lucene.Net.Benchmark.csproj
index 762ae85..250ca43 100644
--- a/src/Lucene.Net.Benchmark/Lucene.Net.Benchmark.csproj
+++ b/src/Lucene.Net.Benchmark/Lucene.Net.Benchmark.csproj
@@ -53,7 +53,7 @@
<ItemGroup>
<PackageReference Include="J2N" Version="$(J2NPackageVersion)" />
- <PackageReference Include="ICU4N.Collation" Version="$(ICU4NCollationPackageVersion)" />
+ <PackageReference Include="ICU4N" Version="$(ICU4NPackageVersion)" />
<PackageReference Include="SharpZipLib" Version="$(SharpZipLibPackageVersion)" />
<PackageReference Include="Spatial4n" Version="$(Spatial4nPackageVersion)" />
</ItemGroup>
diff --git a/src/Lucene.Net.Expressions/Lucene.Net.Expressions.csproj b/src/Lucene.Net.Expressions/Lucene.Net.Expressions.csproj
index 60a437c..d32f09f 100644
--- a/src/Lucene.Net.Expressions/Lucene.Net.Expressions.csproj
+++ b/src/Lucene.Net.Expressions/Lucene.Net.Expressions.csproj
@@ -46,6 +46,10 @@
<ItemGroup>
<PackageReference Include="Antlr3.Runtime" Version="$(Antlr3RuntimePackageVersion)" />
+
+ <!-- These are transitive dependencies, but Antlr3.Runtime doesn't provide a secure alternative on netstandard2.0, so we need to include them. -->
+ <PackageReference Include="System.Net.Http" Version="$(SystemNetHttpPackageVersion)" />
+ <PackageReference Include="System.Text.RegularExpressions" Version="$(SystemTextRegularExpressionsPackageVersion)" />
</ItemGroup>
<ItemGroup Condition=" '$(TargetFramework)' == 'netstandard2.1' ">
diff --git a/src/Lucene.Net.TestFramework/Lucene.Net.TestFramework.csproj b/src/Lucene.Net.TestFramework/Lucene.Net.TestFramework.csproj
index 4eb51d7..712ec3b 100644
--- a/src/Lucene.Net.TestFramework/Lucene.Net.TestFramework.csproj
+++ b/src/Lucene.Net.TestFramework/Lucene.Net.TestFramework.csproj
@@ -67,11 +67,6 @@
</ItemGroup>
<ItemGroup Condition=" '$(TargetFramework)' == 'net462' ">
- <!-- This is a transitive dependency of Microsoft.Extensions.Configuration.Json and IKVM. It is just here to pin the version to avoid conflicts. -->
- <PackageReference Include="System.Text.Json" Version="$(SystemTextJsonPackageVersion)" />
- </ItemGroup>
-
- <ItemGroup Condition=" '$(TargetFramework)' == 'net462' ">
<Reference Include="System.IO.Compression" />
<Reference Include="System.Numerics" />
<Reference Include="System.ServiceModel" />
diff --git a/src/Lucene.Net.Tests.Analysis.SmartCn/Lucene.Net.Tests.Analysis.SmartCn.csproj b/src/Lucene.Net.Tests.Analysis.SmartCn/Lucene.Net.Tests.Analysis.SmartCn.csproj
index 79d6f9a..abe9d1c 100644
--- a/src/Lucene.Net.Tests.Analysis.SmartCn/Lucene.Net.Tests.Analysis.SmartCn.csproj
+++ b/src/Lucene.Net.Tests.Analysis.SmartCn/Lucene.Net.Tests.Analysis.SmartCn.csproj
@@ -59,12 +59,7 @@
<Import Project="$(SolutionDir).build/TestReferences.Common.targets" />
- <ItemGroup Condition=" '$(TargetFramework)' == 'net48' ">
- <PackageReference Include="Microsoft.Extensions.Options" Version="$(MicrosoftExtensionsOptionsPackageVersion)"/>
- </ItemGroup>
-
<ItemGroup Condition=" '$(TargetFramework)' == 'net472' ">
- <PackageReference Include="Microsoft.Extensions.Options" Version="$(MicrosoftExtensionsOptionsPackageVersion)"/>
<PackageReference Include="System.Text.Encoding.CodePages" Version="$(SystemTextEncodingCodePagesPackageVersion)" />
</ItemGroup>
diff --git a/src/Lucene.Net.Tests.Benchmark/Lucene.Net.Tests.Benchmark.csproj b/src/Lucene.Net.Tests.Benchmark/Lucene.Net.Tests.Benchmark.csproj
index afca29a..6973045 100644
--- a/src/Lucene.Net.Tests.Benchmark/Lucene.Net.Tests.Benchmark.csproj
+++ b/src/Lucene.Net.Tests.Benchmark/Lucene.Net.Tests.Benchmark.csproj
@@ -87,12 +87,4 @@
<PackageReference Include="ICU4N" Version="$(ICU4NPackageVersion)" />
</ItemGroup>
- <ItemGroup Condition=" '$(TargetFramework)' == 'net48' ">
- <PackageReference Include="Microsoft.Extensions.Options" Version="$(MicrosoftExtensionsOptionsPackageVersion)"/>
- </ItemGroup>
-
- <ItemGroup Condition=" '$(TargetFramework)' == 'net472' ">
- <PackageReference Include="Microsoft.Extensions.Options" Version="$(MicrosoftExtensionsOptionsPackageVersion)"/>
- </ItemGroup>
-
</Project>
diff --git a/src/Lucene.Net.Tests.Replicator/Lucene.Net.Tests.Replicator.csproj b/src/Lucene.Net.Tests.Replicator/Lucene.Net.Tests.Replicator.csproj
index a518765..a794ca3 100644
--- a/src/Lucene.Net.Tests.Replicator/Lucene.Net.Tests.Replicator.csproj
+++ b/src/Lucene.Net.Tests.Replicator/Lucene.Net.Tests.Replicator.csproj
@@ -67,6 +67,12 @@
<PackageReference Include="Newtonsoft.Json" Version="$(NewtonsoftJsonPackageVersion)" />
</ItemGroup>
+ <ItemGroup Condition="'$(TargetFramework)' == 'net472'">
+ <!-- These are transitive dependencies of Microsoft.AspNetCore.TestHost that we are explicitly upgrading to packages without known vulnerabilities -->
+ <PackageReference Include="Microsoft.AspNetCore.Http" Version="$(MicrosoftAspNetCoreHttpPackageVersion)" />
+ <PackageReference Include="System.IO.Pipelines" Version="$(SystemIOPipelinesPackageVersion)" />
+ </ItemGroup>
+
<ItemGroup Condition="$(TargetFramework.StartsWith('net4'))">
<Reference Include="System.Net.Http" />
</ItemGroup>
diff --git a/src/Lucene.Net/Lucene.Net.csproj b/src/Lucene.Net/Lucene.Net.csproj
index 2491ecc..b04cb5b 100644
--- a/src/Lucene.Net/Lucene.Net.csproj
+++ b/src/Lucene.Net/Lucene.Net.csproj
@@ -60,14 +60,15 @@
<ItemGroup Condition=" '$(TargetFramework)' == 'netstandard2.0' ">
<PackageReference Include="System.Memory" Version="$(SystemMemoryPackageVersion)" />
- <!-- NOTE: This is a transitive dependency only, but we are forcing an upgrade to ensure there are no conflicts with dependencies. -->
- <PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="$(SystemRuntimeCompilerServicesUnsafePackageVersion)" />
</ItemGroup>
<ItemGroup Condition=" '$(TargetFramework)' == 'net462' ">
<PackageReference Include="System.Memory" Version="$(SystemMemoryPackageVersion)" />
- <!-- NOTE: This is a transitive dependency only, but we are forcing an upgrade to ensure there are no conflicts with dependencies. -->
- <PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="$(SystemRuntimeCompilerServicesUnsafePackageVersion)" />
+ <PackageReference Include="System.Runtime.InteropServices.RuntimeInformation" Version="$(SystemRuntimeInteropServicesRuntimeInformationPackageVersion)" />
+ </ItemGroup>
+
+ <ItemGroup Condition=" '$(TargetFramework)' == 'net462' ">
+ <Reference Include="System.Net.Http" />
</ItemGroup>
<ItemGroup>
diff --git a/src/dotnet/Lucene.Net.CodeAnalysis.CSharp/Lucene.Net.CodeAnalysis.CSharp.csproj b/src/dotnet/Lucene.Net.CodeAnalysis.CSharp/Lucene.Net.CodeAnalysis.CSharp.csproj
index 2905dd3..a631150 100644
--- a/src/dotnet/Lucene.Net.CodeAnalysis.CSharp/Lucene.Net.CodeAnalysis.CSharp.csproj
+++ b/src/dotnet/Lucene.Net.CodeAnalysis.CSharp/Lucene.Net.CodeAnalysis.CSharp.csproj
@@ -34,6 +34,10 @@
<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="$(MicrosoftCodeAnalysisCSharpPackageVersion)" />
<PackageReference Include="Microsoft.CodeAnalysis.CSharp.Workspaces" Version="$(MicrosoftCodeAnalysisCSharpWorkspacesPackageVersion)" PrivateAssets="all" />
<PackageReference Update="NETStandard.Library" PrivateAssets="all" />
+
+ <!-- These are transitive dependencies, but the version being resolved is vulnerable, so we are upgrading here. -->
+ <PackageReference Include="System.Net.Http" Version="$(SystemNetHttpPackageVersion)" />
+ <PackageReference Include="System.Text.RegularExpressions" Version="$(SystemTextRegularExpressionsPackageVersion)" />
</ItemGroup>
</Project>
diff --git a/src/dotnet/Lucene.Net.CodeAnalysis.VisualBasic/Lucene.Net.CodeAnalysis.VisualBasic.csproj b/src/dotnet/Lucene.Net.CodeAnalysis.VisualBasic/Lucene.Net.CodeAnalysis.VisualBasic.csproj
index 6c2d113..f76bace 100644
--- a/src/dotnet/Lucene.Net.CodeAnalysis.VisualBasic/Lucene.Net.CodeAnalysis.VisualBasic.csproj
+++ b/src/dotnet/Lucene.Net.CodeAnalysis.VisualBasic/Lucene.Net.CodeAnalysis.VisualBasic.csproj
@@ -34,6 +34,10 @@
<PackageReference Include="Microsoft.CodeAnalysis.VisualBasic" Version="$(MicrosoftCodeAnalysisVisualBasicPackageVersion)" />
<PackageReference Include="Microsoft.CodeAnalysis.VisualBasic.Workspaces" Version="$(MicrosoftCodeAnalysisVisualBasicWorkspacesPackageVersion)" PrivateAssets="all" />
<PackageReference Update="NETStandard.Library" PrivateAssets="all" />
+
+ <!-- These are transitive dependencies, but the version being resolved is vulnerable, so we are upgrading here. -->
+ <PackageReference Include="System.Net.Http" Version="$(SystemNetHttpPackageVersion)" PrivateAssets="all" />
+ <PackageReference Include="System.Text.RegularExpressions" Version="$(SystemTextRegularExpressionsPackageVersion)" PrivateAssets="all" />
</ItemGroup>
</Project>
diff --git a/src/dotnet/Lucene.Net.ICU/Lucene.Net.ICU.csproj b/src/dotnet/Lucene.Net.ICU/Lucene.Net.ICU.csproj
index 3da8fcb..827d421 100644
--- a/src/dotnet/Lucene.Net.ICU/Lucene.Net.ICU.csproj
+++ b/src/dotnet/Lucene.Net.ICU/Lucene.Net.ICU.csproj
@@ -72,11 +72,6 @@
<ItemGroup>
<PackageReference Include="ICU4N" Version="$(ICU4NPackageVersion)" />
- <PackageReference Include="ICU4N.Collation" Version="$(ICU4NCollationPackageVersion)" />
- <PackageReference Include="ICU4N.CurrencyData" Version="$(ICU4NCurrencyDataPackageVersion)" />
- <PackageReference Include="ICU4N.LanguageData" Version="$(ICU4NLanguageDataPackageVersion)" />
- <PackageReference Include="ICU4N.RegionData" Version="$(ICU4NRegionDataPackageVersion)" />
- <PackageReference Include="ICU4N.Transliterator" Version="$(ICU4NTransliteratorPackageVersion)" />
</ItemGroup>
<ItemGroup>
diff --git a/src/dotnet/Lucene.Net.Replicator.AspNetCore/Lucene.Net.Replicator.AspNetCore.csproj b/src/dotnet/Lucene.Net.Replicator.AspNetCore/Lucene.Net.Replicator.AspNetCore.csproj
index 946defd..e415034 100644
--- a/src/dotnet/Lucene.Net.Replicator.AspNetCore/Lucene.Net.Replicator.AspNetCore.csproj
+++ b/src/dotnet/Lucene.Net.Replicator.AspNetCore/Lucene.Net.Replicator.AspNetCore.csproj
@@ -42,6 +42,9 @@
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.Http.Abstractions" Version="$(MicrosoftAspNetCoreHttpAbstractionsPackageVersion)" />
+
+ <!-- This is a transitive dependency of Microsoft.AspNetCore.Http.Abstractions, but the referenced version of System.Text.Encodings.Web has high severity vulnerabilities, so we are upgrading it. -->
+ <PackageReference Include="System.Text.Encodings.Web" Version="$(SystemTextEncodingsWebPackageVersion)" />
</ItemGroup>
</Project>
diff --git a/src/dotnet/Lucene.Net.Tests.ICU/Lucene.Net.Tests.ICU.csproj b/src/dotnet/Lucene.Net.Tests.ICU/Lucene.Net.Tests.ICU.csproj
index 94873e7..6666ad2 100644
--- a/src/dotnet/Lucene.Net.Tests.ICU/Lucene.Net.Tests.ICU.csproj
+++ b/src/dotnet/Lucene.Net.Tests.ICU/Lucene.Net.Tests.ICU.csproj
@@ -77,19 +77,6 @@
<ItemGroup>
<PackageReference Include="ICU4N" Version="$(ICU4NPackageVersion)" />
- <PackageReference Include="ICU4N.Collation" Version="$(ICU4NCollationPackageVersion)" />
- <PackageReference Include="ICU4N.CurrencyData" Version="$(ICU4NCurrencyDataPackageVersion)" />
- <PackageReference Include="ICU4N.LanguageData" Version="$(ICU4NLanguageDataPackageVersion)" />
- <PackageReference Include="ICU4N.RegionData" Version="$(ICU4NRegionDataPackageVersion)" />
- <PackageReference Include="ICU4N.Transliterator" Version="$(ICU4NTransliteratorPackageVersion)" />
- </ItemGroup>
-
- <ItemGroup Condition=" '$(TargetFramework)' == 'net48' ">
- <PackageReference Include="Microsoft.Extensions.Options" Version="$(MicrosoftExtensionsOptionsPackageVersion)" />
- </ItemGroup>
-
- <ItemGroup Condition=" '$(TargetFramework)' == 'net472' ">
- <PackageReference Include="Microsoft.Extensions.Options" Version="$(MicrosoftExtensionsOptionsPackageVersion)" />
</ItemGroup>
</Project>
diff --git a/src/dotnet/tools/lucene-cli/lucene-cli.csproj b/src/dotnet/tools/lucene-cli/lucene-cli.csproj
index 73a1446..53bf6cf 100644
--- a/src/dotnet/tools/lucene-cli/lucene-cli.csproj
+++ b/src/dotnet/tools/lucene-cli/lucene-cli.csproj
@@ -77,7 +77,6 @@
<PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" Version="$(MicrosoftExtensionsConfigurationEnvironmentVariablesPackageVersion)" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="$(MicrosoftExtensionsConfigurationJsonPackageVersion)" />
<PackageReference Include="Microsoft.Extensions.Configuration.Xml" Version="$(MicrosoftExtensionsConfigurationXmlPackageVersion)" />
- <PackageReference Include="System.Security.Cryptography.Xml" Version="$(SystemSecurityCryptographyXmlPackageVersion)" />
</ItemGroup>
<ItemGroup>
