Does Harmony Core support aggregation from OData v4?

[NathanLyonAsi]

I'm working on developing a service API that uses OData to query our data. While attempting to use some of the advanced aggregate features of OData V4 I ran into some errors that originated with the Harmony.Core DbContext objects. Am I doing something wrong or is V4 not yet supported? If it isn't currently supported, is this a planned enhancement and, if so, when can we expect it?

[hippiehunter]

There is some minimal support for Aggregation in the current version, can you share the query and what you're trying to accomplish?

[NathanLyonAsi]

There is some minimal support for Aggregation in the current version, can you share the query and what you're trying to accomplish?

I don't think $apply seems to be working at all. Even this simple OData query is throwing an exception: $apply=aggregate(Balance with sum as TotalBalance,$count as AccountCount)

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> System.InvalidCastException: Unable to cast object of type 'Harmony.Core.EF.Query.Internal.HarmonyTableExpression' to type 'System.Linq.Expressions.MethodCallExpression'.
at Harmony.Core.EF.Query.Internal.HarmonyQueryExpression.ApplyGrouping(Expression groupingKey, ShapedQueryExpression shaperExpression).

I have others if you need more examples, but they're all using $apply

[hippiehunter]

Unfortunately it looks like this feature isnt available. There was some work done specifically for $apply=aggregate(Field with sum as Name) but it didnt work well and no one was asking for it so it was dropped. Is this just exploration on your part or is this critical for your usecase?

[matt-canaday]

We are working with a 3rd party to create a UI/UX that will be making ODATA v4 calls to our Harmony Core service. So, our hope was that it could be upgraded to support ODATA v4 calls that includes the aggregators. While I can't say it's critical, it is very much desired so we can continue to work with our UI/UX developers without downgrading to ODATA v3 if possible.

[hippiehunter]

Hi Matt,
To clarify, the odata v4 spec is pretty vast and our support for it is not 100% but $apply isnt actually part of the core odata v4 spec its an extension beyond odata v4. here's the spec document for its extension https://docs.oasis-open.org/odata/odata-data-aggregation-ext/v4.0/cs02/odata-data-aggregation-ext-v4.0-cs02.html#_Toc435016565

There is a route through to the aggregation extension support and we're headed there anyway for unrelated reasons discussed at the summit this year. Unfortunately I dont have anything like a firm timeline for you right now.

[matt-canaday]

Thanks! We'll see about working around the calls with our UI/UX partner and know that not all functions are available right now, so we can get the necessary data and do the parts not supported via code.

[matt-canaday]

I'm bringing this up in case there are any suggestions you can provide on how we might be able to use some basic aggregator functions with Harmony Core. Specifically: Group by, sum, and average. Count is available in the basic ODATA and we can use that, but we have need for some grouping and summing at least. Any help is appreciated.

[hippiehunter]

its going to take a few days to weave its way through into a release but here's the PR so you can track its progress #389 I wouldn't recommend you use this for data files accessed remotely via xfServer but for data that is local to the harmony core server and result sets that are smaller than total available system memory this implementation should work well. A more optimized/larger than memory implementation will need to wait for the larger changes.

[matt-canaday]

Fantastic news! Thank you.

[hippiehunter]

ok, this version is live on nuget.org. You should be able to upgrade to it using the latest harmony core cli tool. pay careful attention to any error messages during the upgrade, this is using a new msbuild version for the tooling and its been a little bit iffy.

@hippiehunter I'm getting the following when trying to use the latest harmony core cli tool to upgrade:

D:\Source\wip\InsiteHarmony>harmonycore upgrade-latest
This utility will make significant changes to projects and other source files in your Harmony Core development environm.

Type YES to proceed:
yes
Checking for current version info
SolutionDir '' is not set. Using the current working directory.
Scanning 'D:\Source\wip\InsiteHarmony' for HarmonyCore project files
Failed to locate MSBuild path
Unhandled exception. System.AggregateException: One or more errors occurred. (Object reference not set to an instance o)
---> System.NullReferenceException: Object reference not set to an instance of an object.
at HarmonyCore.CliTool.Program.UpgradeLatest(SolutionInfo solution, VersionTargetingInfo versionInfo, String overrid5
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.Wait()
at HarmonyCore.CliTool.Program.<>c__DisplayClass11_0.b__1(UpgradeLatestOptions opts) in C:\Users4
at CommandLine.ParserResultExtensions.MapResult[T1,T2,T3,T4,T5,T6,T7,T8,T9,T10,TResult](ParserResult1 result, Func)
at HarmonyCore.CliTool.Program.ProcessCommandArgs(String[] args) in C:\Users\Jeff.Greene\agent_work\1\s\HarmonyCore2
at HarmonyCore.CliTool.Program.Main(String[] args) in C:\Users\Jeff.Greene\agent_work\1\s\HarmonyCore.CliTool\Progr2

D:\Source\wip\InsiteHarmony>

D:\Source\wip\InsiteHarmony>msbuild /version
MSBuild version 17.14.23+b0019275e for .NET Framework
17.14.23.42201

[hippiehunter]

can you try directly installing the .NET 8 SDK https://dotnet.microsoft.com/en-us/download/dotnet/8.0 the version installed by Visual Studio is weirdly incomplete and upgrades to VisualStudio sometimes even break the manually installed SDK

Thank you! Directly installing the .NET 8 SDK took care of the problem for me.