Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port Time Series #977

Merged
merged 48 commits into from
Oct 2, 2018
Merged

Port Time Series #977

merged 48 commits into from
Oct 2, 2018

Conversation

codemzs
Copy link
Member

@codemzs codemzs commented Sep 21, 2018

Port of time series prediction.

  • ExponentialAverage
  • IidChangePointDetector
  • IidSpikeDetector
  • PercentileThresholdTransform
  • PValueTransform
  • SlidingWindowTransform
  • SsaChangePointDetector
  • SsaSpikeDetector

fixes #978, #1092 and #1103

}

private const string DllName = "MklImports.dll";
private const string DllProxyName = "MklImports.dll";
Copy link
Contributor

@Ivanidzo4ka Ivanidzo4ka Sep 21, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need one of them. #Closed

FreeDescriptor(ref descriptor);
}

// REVIEW saamizad: for some reason the native backward scaling for DFTI in MKL does not work.
Copy link
Contributor

@Ivanidzo4ka Ivanidzo4ka Sep 21, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REVIEW saamizad [](start = 15, length = 15)

Cn you clean code from aliases? #Closed

@markusweimer
Copy link
Member

markusweimer commented Sep 21, 2018

This PR doesn't have a useful title, nor does it have a description. Please address this, and the missing Issue # before merging. #Resolved

@codemzs
Copy link
Member Author

codemzs commented Sep 21, 2018

Yes, sir. I was planning on doing that! :)


In reply to: 423611446 [](ancestors = 423611446)

@codemzs codemzs requested a review from Zruty0 September 21, 2018 18:38
@codemzs codemzs changed the title Timeseries Time Series Sep 21, 2018
@shauheen shauheen self-requested a review September 21, 2018 18:40
Copy link
Contributor

@shauheen shauheen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please complete the necessary design documentation on this work first.

@@ -0,0 +1,148 @@
//------------------------------------------------------------------------------
// <copyright company="Microsoft Corporation">
Copy link
Contributor

@Zruty0 Zruty0 Sep 24, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Microsoft [](start = 23, length = 9)

fix headers #Closed


namespace Microsoft.ML.Runtime.TimeSeriesProcessing
{
// REVIEW saamizad: This base class and its children classes generate one output column of type VBuffer<Double> to output 3 different anomaly scores as well as
Copy link
Contributor

@Zruty0 Zruty0 Sep 24, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

saamizad [](start = 14, length = 8)

remove aliases #Resolved

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep the 'REVIEW' though


In reply to: 219996812 [](ancestors = 219996812)

Copy link
Contributor

@Zruty0 Zruty0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🕐

@codemzs codemzs changed the title Time Series Port Time Series Sep 25, 2018
@codemzs codemzs changed the title Port Time Series WIP: Port Time Series Sep 25, 2018
</PropertyGroup>

<ItemGroup>
<Content Include="$(PackagesDir)\mlnetmkldeps\$(MlNetMklDepsPackageVersion)\LICENSE.txt" Pack="true" PackagePath=".\NOTICE.txt" />
Copy link
Member

@eerhardt eerhardt Oct 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need the license file here since we are not packaging MKL in this nuget. #Resolved

@@ -754,6 +754,11 @@ private void CheckLabel(RoleMappedData examples, out int weightSetCount)

private static unsafe class Native
{
static Native()
{
var t = ErrorMessage(0);
Copy link
Member

@eerhardt eerhardt Oct 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use:

_ = ErrorMessage(0);

instead of var t. _ signifies "I want to capture the variable, but I don't actually use the variable". #Resolved

@@ -754,6 +754,11 @@ private void CheckLabel(RoleMappedData examples, out int weightSetCount)

private static unsafe class Native
{
static Native()
{
var t = ErrorMessage(0);
Copy link
Member

@eerhardt eerhardt Oct 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add the reasoning/comment here on why we are doing this. #Resolved

@@ -18,4 +18,8 @@ endif()
add_library(SymSgdNative SHARED ${SOURCES} ${RESOURCES})
target_link_libraries(SymSgdNative PUBLIC ${MKL_LIBRARY})

if(APPLE)
set_target_properties(SymSgdNative PROPERTIES INSTALL_RPATH "@loader_path;@loader_path/../../../../../${MKL_LIB_RPATH};${MKL_LIB_PATH}")
Copy link
Member

@eerhardt eerhardt Oct 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

${MKL_LIB_PATH} shouldn't be necessary, right? When we are running in our repo, these assemblies should live together, so @loader_path should just take care of it. #Closed

target_link_libraries(MklProxyNative PUBLIC ${MKL_LIBRARY})

if(APPLE)
set_target_properties(MklProxyNative PROPERTIES INSTALL_RPATH "@loader_path;${MKL_LIB_PATH}")
Copy link
Member

@eerhardt eerhardt Oct 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't need MKL_LIB_PATH here, right? MklProxyNative will always be in the same directory as MklImports. #Closed

<ItemGroup>
<NativePackageAsset Include="$(NativeAssetsBuiltPath)\$(NativeLibPrefix)MklImports$(NativeLibExtension)"
RelativePath="Microsoft.ML.HalLearners\runtimes\$(PackageRid)\native" />
<NativePackageAsset Include="$(NativeAssetsBuiltPath)\$(NativeLibPrefix)MklProxyNative$(NativeLibExtension)"
Copy link
Member

@eerhardt eerhardt Oct 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line needs to go above the previous ItemGroup. The reason this ItemGroup exists down here is because the one above adds the symbols for the assemblies we produce in this repo. Since MklImports is not produced by this repo, we have no symbols for it. But MklProxyNative IS produced by this repo, so we need to package the $(NativeLibSymbolExtension) files as well. #Closed

@@ -53,7 +53,7 @@
DependsOnTargets="GenerateVersionHeader">

<PropertyGroup>
<BuildArgs>$(Configuration) $(TargetArchitecture) --mkllibpath $(PackagesDir)mlnetmkldeps\$(MlNetMklDepsPackageVersion)\runtimes\$(PackageRid)\native</BuildArgs>
<BuildArgs>$(Configuration) $(TargetArchitecture) --mkllibpath $(PackagesDir)mlnetmkldeps\$(MlNetMklDepsPackageVersion)\runtimes\$(PackageRid)\native --mkllibrpath mlnetmkldeps/$(MlNetMklDepsPackageVersion)/runtimes/$(PackageRid)/native</BuildArgs>
Copy link
Member

@eerhardt eerhardt Oct 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no such thing as 'rpath' on Windows, right? We shouldn't need to use/pass it on Windows. #Closed

--mkllibrpath)
shift
__mkllibrpath=$1
;;
Copy link
Member

@eerhardt eerhardt Oct 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit) tabs at the end of this line #Resolved

CceFormat = 57 /* Complex conjugate-even */
};

EXPORT_API(int) DftiSetValue(void *handle, ConfigParam config_param, ...);
Copy link
Member

@eerhardt eerhardt Oct 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice having a comment here explaining why we need to make these "proxy" methods. #Closed

@@ -18,4 +18,8 @@ endif()
add_library(SymSgdNative SHARED ${SOURCES} ${RESOURCES})
target_link_libraries(SymSgdNative PUBLIC ${MKL_LIBRARY})

if(APPLE)
set_target_properties(SymSgdNative PROPERTIES INSTALL_RPATH "@loader_path;@loader_path/../../../../../${MKL_LIB_RPATH};${MKL_LIB_PATH}")
Copy link
Member

@eerhardt eerhardt Oct 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@loader_path/../../../../../${MKL_LIB_RPATH} [](start = 78, length = 44)

How about let's put the ../../.. outside of the CMake files as well? The whole relative path should be passed into ${MKL_LIB_RPATH}, so here we can just say:

@loader_path/${MKL_LIB_RPATH} #Closed

</PropertyGroup>

<ItemGroup>
<Content Include="..\common\CommonPackage.props" Pack="true" PackagePath="build\netstandard2.0\$(MSBuildProjectName).props" />
Copy link
Member

@eerhardt eerhardt Oct 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TimeSeries doesn't ship any native assemblies, so it doesn't need this .props file. #Resolved

@codemzs
Copy link
Member Author

codemzs commented Oct 2, 2018

    [Fact(Skip = "TEMP: Missing MKL Binary.")]

We need a real dataset for this. Opened issue #1120


In reply to: 426390719 [](ancestors = 426390719)


Refers to: test/Microsoft.ML.Core.Tests/UnitTests/TestEntryPoints.cs:3753 in be7ccff. [](commit_id = be7ccff, deletion_comment = False)

@eerhardt
Copy link
Member

eerhardt commented Oct 2, 2018

    [Fact(Skip = "TEMP: Missing MKL Binary.")]

(minor) Can we add that issue link into the code?


In reply to: 426421800 [](ancestors = 426421800,426390719)


Refers to: test/Microsoft.ML.Core.Tests/UnitTests/TestEntryPoints.cs:3753 in be7ccff. [](commit_id = be7ccff, deletion_comment = False)

Copy link
Member

@eerhardt eerhardt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

Copy link
Contributor

@shauheen shauheen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

Copy link
Contributor

@shauheen shauheen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@codemzs codemzs merged commit cde7038 into dotnet:master Oct 2, 2018
@codemzs codemzs deleted the timeseries branch October 2, 2018 23:57
@ghost ghost locked as resolved and limited conversation to collaborators Mar 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Time Series
7 participants