Btree index build fail with "No partition metadata files found in index directory"

[jackye1995]

Copy Discord issue reported:

Ran

updated_dataset = lr.create_scalar_index(
            dataset=dataset_uri,
            column=column,
            index_type=index_type,
            replace=True,
            num_workers=effective_workers,
            ray_remote_args=ray_remote_args,
    )

18:06:11 | INFO     | Ray initialized: {'node:__internal_head__': 1.0, 'memory': 697043922944.0, 'GPU': 4.0, 'accelerator_type:GB200': 1.0, 'CPU': 140.0, 'object_store_memory': 200000000000.0, 'node:xxx': 1.0}
18:06:11 | INFO     | Total fragments: 5000
18:06:11 | INFO     | Ray workers: 16
18:06:11 | INFO     | Creating BTREE index on 'video_uuid' using 16 Ray workers...

18:10:11 | ERROR    | Operation failed: Encountered internal error. Please file a bug report at https://github.com/lance-format/lance/issues. No partition metadata files found in index directory: debug/lance_db/yyy/xxx/_indices/8e4fea4f-92b3-4cc4-affc-911313f95a22 (page_files: 0, lookup_files: 0), /runner/_work/lance/lance/rust/lance-index/src/scalar/btree.rs:1494:23

Some conversations:

I can see that Ray created 16 actors and running but it just fails after some time with no concrete logs/info on what went wrong.
This is a remote Lance table in GCS. I am able to query data from the table (to validate access/operations) at path gs://{bucket_name}/debug/lance_db/yyy/xxx

[chenghao-guo]

It could be potentially caused by pylance>=1.0.0 while create_scalar_index should use index_uuid

while lance-ray 0.0.8 + pylance 0.39 was using fragment_uuid.

pylance modified this parameter in pylance 0.40 version and caused this breaking change:
See this PR fix in lance-ray: #62

Solution:
Try upgrading lance-ray to 0.1.0 if the user was using the pylance 1.0.0 version