tinshift: support TIN GeoPackage files#4603
Conversation
rouault
force-pushed
the
tinshift_gpkg
branch
from
a200e1f to
48f13ec
Compare
rouault
force-pushed
the
tinshift_gpkg
branch
2 times, most recently
from
6ef4d73 to
29eacff
Compare
jjimenezshaw
left a comment
jjimenezshaw
left a comment
There was a problem hiding this comment.
I didn't review the complete algorithm yet. But one usual optimization is to start searching on the last triangle successfully returned. If searched points an near, they will probably be in the same triangle. Is it something to include in the API somehow? Or just an implementation detail? (I want to think it is an implementation detail, but I am not sure. In that case we should store the last used triangle threadsafe somewhere)
rouault
force-pushed
the
tinshift_gpkg
branch
from
29eacff to
d769dc3
Compare
good idea. Implemented now |
rouault
force-pushed
the
tinshift_gpkg
branch
3 times, most recently
from
52e3960 to
73575dc
Compare
|
|
||
| ------------------------------------------------------------------------------- | ||
| =============================================================================== | ||
| Test +proj=tinshift with GeoPackage files on network |
There was a problem hiding this comment.
Is this using curl or sqlite3 directly? Is it getting the whole file, or is it "cloud optimized"?
There was a problem hiding this comment.
We use our NetworkFile implementation that uses curl underneath and range request, and plug it into sqlite3 I/O layer. That way we can have piece wise access to the parts of the file we need, and benefit from the local caching in our ~/.local/share/proj/cache.db of the chunks we have already read.
From my test on a synthetic grid of 500x500 nodes, with each square divided into 2 triangles, that generates a GeoPackage file of ~ 64 MB. The first access to extract a triangle involves ~ 20 network requests of 16 KB each. Accesses to other parts (not close to each other) of the file afterwards take between 6 and 14 requests.
It seems that FlatGeoBuf spatial index (packed Hilbert RTree) would involve less requests, but FlatGeoBuf supports only one layer per file, and we need 2 layers. The FlatGeoBuf files would also be larger than the GeoPackage one (presumably because both the triangle geometry and the RTree need to be stored, whereas with GeoPackage we only store the RTree of the bbox of the triangles, but not the geometry of the triangles themselves, just the indices to the vertices), although file size is probably less of a concern. We could possibly have a .tar of 2 flatgeobuf files? I'm not sure if that's worth the complication.
rouault
force-pushed
the
tinshift_gpkg
branch
3 times, most recently
from
4c89256 to
863f356
Compare
Such files are equivalent to already supported TIN JSON files, but scale better for arbitrarily large triangulations, in particular for network-based access. The format is defined in source/specifications/tin_gpkg.rst and the https://github.com/OSGeo/PROJ-data/blob/master/grid_tools/tin_json_to_gpkg_tin.py Python script may be used o convert an existing TIN JSON into a TIN GeoPackage.
rouault
force-pushed
the
tinshift_gpkg
branch
from
863f356 to
50856a7
Compare
|
@GlenRice-NOAA Did you get a chance to test this feature? |
|
No, I have not. I will try to get to it in the next couple of weeks. |
|
@GlenRice-NOAA any progress? Release 9.8.0 is coming soon |
|
I'm planning on having it tested by the end of the week. Thanks for checking in. |
|
Can we bump this to the next release? |
you mean 9.9 (likely to be released towards end of this year) ? Sure, no emergency from my side |
|
Thank you. |
|
The PROJ project highly values your contribution and would love to see this work merged! Unfortunately this PR has not had any activity in the last two months and is being automatically marked as "stale". If you think this pull request should be merged, please check
|
|
Keep open |
|
@GlenRice-NOAA Any ETA on when you will be able to give feedback on this? |
|
We have someone new working on it (again). We think we can provide feedback in the next month. |
5 participants
Such files are equivalent to already supported TIN JSON files, but scale better for arbitrarily large triangulations, in particular for network-based access.
The format is defined in source/specifications/tin_gpkg.rst and the https://github.com/rouault/PROJ-data/blob/tin_json_to_tin_gpkg/grid_tools/tin_json_to_tin_gpkg.py Python script may be used o convert an existing TIN JSON into a TIN GeoPackage.
$ echo 3432087 6995748 0 | PROJ_NETWORK=ON PROJ_DATA=data bin/cct +proj=tinshift +file=fi_nls_ykj_etrs35fin.gpkg
431943.0905 6992816.7826 0.0000 inf
$ echo 3432087 6995748 0 | PROJ_NETWORK=ON PROJ_DATA=data bin/cct +proj=tinshift +file=fi_nls_ykj_etrs35fin.json
431943.0905 6992816.7826 0.0000 inf
$ echo 3432087 6995748 0 | PROJ_NETWORK=ON PROJ_DATA=data bin/cct +proj=tinshift +file=fi_nls_n60_n2000.gpkg
3432087.0000 6995748.0000 0.3161 inf
$ echo 3432087 6995748 0 | PROJ_NETWORK=ON PROJ_DATA=data bin/cct +proj=tinshift +file=fi_nls_n60_n2000.json
3432087.0000 6995748.0000 0.3161 inf
Fixes #3732