Commit 6111b7b
fix: Prevent silent conversion of float array to int (#96)
#### Relevant issue or PR
N/A
#### Description of changes
Tesseract `Arrays` are intended to prevent conversion from float to int
due to the following line in
`runtime.array_encoding.coerce_shape_dtype`:
```python
if not np.can_cast(arr.dtype, expected_dtype, casting=**"same_kind"**):
```
However, this does not work as intended due to casting happening at an
earlier stage in `runtime.array_encoding.python_to_array`
```python
arr = np.asarray(val, dtype=**expected_dtype**, order="C")
```
This PR removes the initial casting by changing the `dtype` kwarg to
`None` (note that this is its default so we could remove entirely if
preferred). Unfortunately, `asarray` does not offer a `casting` kwarg so
we can't pass `"same_kind"` any earlier.
#### Testing done
- [x] Manually tested validation errors raised when trying to read json
with float arrays into an int array:
```
╭─ Error ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Invalid value for ApplyInputSchema: 4 validation errors for ApplyInputSchema │
│ inputs.a.`chain[EncodedArrayModel__any__int32__DIFFERENTIABLE,function-plain[functools.partial(<function decode_array at │
│ 0x10542a980>, expected_shape=(None,), expected_dtype='int32')()]]` │
│ Input should be a valid dictionary or instance of EncodedArrayModel__any__int32__DIFFERENTIABLE [type=model_type, input_value=[1.1, │
│ 2.5, 3.9], input_type=list] │
│ For further information visit https://errors.pydantic.dev/2.10/v/model_type │
│ inputs.a.`function-plain[functools.partial(<function python_to_array at 0x10542aca0>, expected_shape=(None,), │
│ expected_dtype='int32')()]` │
│ Value error, Dtype mismatch: float64 cannot be cast to int32 [type=value_error, input_value=[1.1, 2.5, 3.9], input_type=list] │
│ For further information visit https://errors.pydantic.dev/2.10/v/value_error │
│ inputs.b.`chain[EncodedArrayModel__any__int32__DIFFERENTIABLE,function-plain[functools.partial(<function decode_array at │
│ 0x10542a980>, expected_shape=(None,), expected_dtype='int32')()]]` │
│ Input should be a valid dictionary or instance of EncodedArrayModel__any__int32__DIFFERENTIABLE [type=model_type, input_value=[4.1, │
│ 5.5, 6.9], input_type=list] │
│ For further information visit https://errors.pydantic.dev/2.10/v/model_type │
│ inputs.b.`function-plain[functools.partial(<function python_to_array at 0x10542aca0>, expected_shape=(None,), │
│ expected_dtype='int32')()]` │
│ Value error, Dtype mismatch: float64 cannot be cast to int32 [type=value_error, input_value=[4.1, 5.5, 6.9], input_type=list] │
│ For further information visit https://errors.pydantic.dev/2.10/v/value_error │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```
- [x] Manually tested that validation errors are now raised when
converting from float to int
```python
>>> import numpy as np
>>> from tesseract_core.runtime import Array, Int32
>>> from pydantic import TypeAdapter
>>> TypeAdapter(Array[..., 'int32']).validate_python(2.5*np.ones((3,3)))
Traceback (most recent call last):
File "<python-input-3>", line 1, in <module>
TypeAdapter(Array[..., 'int32']).validate_python(2.5*np.ones((3,3)))
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/Users/jonathanbrodrick/.virtualenvs/ergodic/lib/python3.13/site-packages/pydantic/type_adapter.py", line 412, in validate_python
return self.validator.validate_python(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
object,
^^^^^^^
...<3 lines>...
allow_partial=experimental_allow_partial,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
pydantic_core._pydantic_core.ValidationError: 2 validation errors for json-or-python[json=chain[EncodedArrayModel__anyrank__int32__noflags,function-plain[functools.partial(<function decode_array at 0x105d6a340>, expected_shape=Ellipsis, expected_dtype='int32')()]],python=union[chain[EncodedArrayModel__anyrank__int32__noflags,function-plain[functools.partial(<function decode_array at 0x105d6a340>, expected_shape=Ellipsis, expected_dtype='int32')()]],function-plain[functools.partial(<function python_to_array at 0x105d6a660>, expected_shape=Ellipsis, expected_dtype='int32')()]]]
`chain[EncodedArrayModel__anyrank__int32__noflags,function-plain[functools.partial(<function decode_array at 0x105d6a340>, expected_shape=Ellipsis, expected_dtype='int32')()]]`
Input should be a valid dictionary or instance of EncodedArrayModel__anyrank__int32__noflags [type=model_type, input_value=array([[2.5, 2.5, 2.5],
... [2.5, 2.5, 2.5]]), input_type=ndarray]
For further information visit https://errors.pydantic.dev/2.10/v/model_type
`function-plain[functools.partial(<function python_to_array at 0x105d6a660>, expected_shape=Ellipsis, expected_dtype='int32')()]`
Value error, Dtype mismatch: float64 cannot be cast to int32 [type=value_error, input_value=array([[2.5, 2.5, 2.5],
... [2.5, 2.5, 2.5]]), input_type=ndarray]
For further information visit https://errors.pydantic.dev/2.10/v/value_error
```
- [x] CI passes
#### License
- [x] By submitting this pull request, I confirm that my contribution is
made under the terms of the [Apache 2.0
license](https://pasteurlabs.github.io/tesseract/LICENSE).
- [x] I sign the Developer Certificate of Origin below by adding my name
and email address to the `Signed-off-by` line.
<details>
<summary><b>Developer Certificate of Origin</b></summary>
```text
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
```
</details>
Signed-off-by: Jonathan Brodrick <jonathan.brodrick@simulation.science>
---------
Co-authored-by: Dion Häfner <dion.haefner@simulation.science>1 parent 1578ddb commit 6111b7b
File tree
3 files changed
+92
-11
lines changed- tesseract_core/runtime
- tests/runtime_tests
3 files changed
+92
-11
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
356 | 356 | | |
357 | 357 | | |
358 | 358 | | |
359 | | - | |
360 | | - | |
361 | | - | |
362 | | - | |
363 | | - | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
364 | 367 | | |
365 | 368 | | |
366 | 369 | | |
| |||
381 | 384 | | |
382 | 385 | | |
383 | 386 | | |
384 | | - | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
385 | 396 | | |
386 | 397 | | |
387 | 398 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
408 | 408 | | |
409 | 409 | | |
410 | 410 | | |
411 | | - | |
| 411 | + | |
412 | 412 | | |
413 | 413 | | |
414 | 414 | | |
| |||
427 | 427 | | |
428 | 428 | | |
429 | 429 | | |
430 | | - | |
| 430 | + | |
431 | 431 | | |
432 | 432 | | |
433 | 433 | | |
| |||
436 | 436 | | |
437 | 437 | | |
438 | 438 | | |
439 | | - | |
| 439 | + | |
440 | 440 | | |
441 | 441 | | |
442 | | - | |
443 | | - | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
447 | 447 | | |
448 | 448 | | |
449 | 449 | | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
0 commit comments