add minimum data types and rank range for operations#910
Conversation
BruceDai
changed the title
BruceDai
changed the title
@BruceDai : Remerge with main. My PR adds an extra blank line that pacifies the build error (and yes, that line was already correct, and nothing changed there for months, but bikeshed changed). |
reillyeon
left a comment
reillyeon
left a comment
There was a problem hiding this comment.
General wording looks good. I have not validated the specific operator limits specified.
fdwr
left a comment
fdwr
left a comment
There was a problem hiding this comment.
👀 Thanks for the updates Bruce - I'm sure they were tedious :b.
| <tr> | ||
| <td>{{input}}</td> | ||
| <td>[=/any data type|any=]</td> | ||
| <td>{{MLOperandDataType/"float32"}}, {{MLOperandDataType/"float16"}}, {{MLOperandDataType/"int32"}}</td> |
There was a problem hiding this comment.
ORT doesn't support int32
There was a problem hiding this comment.
🤔 An ONNX BitCast operator could help with cases of pure data movement (since no actual math happens) like this one when the operator supports a different data type of the same bit size (sizeof(int32) == sizeof(uint32) == sizeof(float32)). For example, we could call BitCast(int32input, type=float32) -> Triangular -> BitCast(input, type=int32). I'll comment on that ONNX PR.
(update) BitCast operator is pending: onnx/onnx#7527
There was a problem hiding this comment.
(update) BitCast is in ONNX https://onnx.ai/onnx/operators/onnx__BitCast.html, now enabling callers of the ORT backend to support all data types of the same bit size for data movement operators, so long as the EP supports BitCast and at least one data type of a given bit size (e.g. supporting int16 with Gather also enables uint16, float16, bfloat16...).
|
Thanks @huningxin and @fdwr for review and identifications. 👍 |
BruceDai
force-pushed
the
minimum_datatype_rankrange
branch
2 times, most recently
from
949a42f to
b6b300f
Compare
BruceDai
force-pushed
the
minimum_datatype_rankrange
branch
from
b6b300f to
3f29bff
Compare
mwyrzykowski
left a comment
mwyrzykowski
left a comment
There was a problem hiding this comment.
Conflict with the WG meeting, but ✅
| If an operation supports only a subset of {{MLOperandDataType}}s, the <dfn>allowed data types</dfn> for each of the operation's input operands, including both positional arguments and options, are given as either an explicit list of {{MLOperandDataType}}s, or a constraint that the operand's [=MLOperand/dataType=] must be the <dfn lt="same type as">same as</dfn> the [=MLOperand/dataType=] of another input operand, or <dfn lt="any data type">any</dfn> to allow any {{MLOperandDataType}}. | ||
|
|
||
| Implementations may support fewer data types for operands than specified. This can be queried for each operation using the {{MLContext/opSupportLimits()}} method on {{MLContext}} and inspecting the {{MLTensorLimits/dataTypes}} value of the corresponding member for the operation. | ||
| Implementations MAY support fewer data types for operands than specified, but MUST support at least the specified <dfn>required data types</dfn>. Support can be queried for each operation using the {{MLContext/opSupportLimits()}} method on {{MLContext}} and inspecting the {{MLTensorLimits/dataTypes}} value of the corresponding member for the operation across DirectML, ONNX Runtime, LiteRT, and CoreML backends in Chromium prototype now. |
There was a problem hiding this comment.
Chromium impl has removed DirectML backends.
| If an operation requires input operands with a particular [=MLOperand/rank=], the <dfn>allowed ranks</dfn> for each of the operation's input operands, including both positional arguments and options, are given as an explicit rank (e.g. 1), or <dfn lt="any rank">N</dfn> to allow any dimensionality, or the <dfn lt="same rank as">same as</dfn> another operand. More specific constraints are common, such as when an input operand's shape must be [=/unidirectionally broadcastable=] to or [=/bidirectionally broadcastable=] with another input operand; in these cases, the [=/allowed ranks=] are listed as a range, with specific validation given as steps in the operation. | ||
|
|
||
| Implementations may impose a more restricted lower bound and/or upper bound on the [=MLOperand/rank=] of operands than specified. This can be queried for each operation using the {{MLContext/opSupportLimits()}} method on {{MLContext}} and inspecting the {{MLTensorLimits/rankRange}}.{{MLRankRange/min}} and {{MLTensorLimits/rankRange}}.{{MLRankRange/max}} values of the corresponding member for the operation. | ||
| Implementations MAY impose a more restricted lower bound and/or upper bound on the [=MLOperand/rank=] of operands than specified, but MUST support at least the specified <dfn>required ranks</dfn>. Support can be queried for each operation using the {{MLContext/opSupportLimits()}} method on {{MLContext}} and inspecting the {{MLTensorLimits/rankRange}}.{{MLRankRange/min}} and {{MLTensorLimits/rankRange}}.{{MLRankRange/max}} values of the corresponding member for the operation across DirectML, ONNX Runtime, LiteRT, and CoreML backends in Chromium prototype now. |
| <td>{{MLOperandDataType/"uint8"}}</td> | ||
| <td>{{MLOperandDataType/"uint8"}}</td> | ||
| <td>[=/any rank|N=]</td> | ||
| <td>0 to 5</td> |
There was a problem hiding this comment.
I filed a chromium issue [TFLite] Support the maximum ranks of input for logicalAnd, logicalOr and logicalXor be greater than 4 for it.
@BruceDai , what's your plan for fixing this issue? Generalizing them to 0 to 5D with backend reshapes sounds good to me.
5 participants
This PR is to fix #896.
@huningxin @fdwr @reillyeon PTAL, thanks!
Preview | Diff