Skip to content

Custom lmdb path #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
lfoppiano wants to merge 9 commits into
base: master
Choose a base branch
from
Draft

Custom lmdb path #10

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
581c18d
add support for processing datatet TEI files
lfoppiano Oct 13, 2024
5d79f8a
apply the correct URL when using datastet
lfoppiano Oct 13, 2024
719928b
add also time/document in the runtime
lfoppiano Nov 20, 2024
f9f5dc4
update client
lfoppiano Jan 15, 2025
78b4a93
use custom LMDB path if provided
lfoppiano Jun 8, 2025
570a2e1
use custom LMDB path also when resetting
lfoppiano Jun 8, 2025
a1b0fb1
fix
lfoppiano Jun 12, 2025
55e3aad
fix again the db paths
lfoppiano Jun 16, 2025
948e957
update documentation
lfoppiano Oct 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions Readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Finally install the project in editable state
```
usage: client.py [-h] [--repo-in REPO_IN] [--file-in FILE_IN] [--file-out FILE_OUT] [--config CONFIG]
[--reprocess] [--reset] [--load] [--diagnostic-mongo] [--diagnostic-files]
[--scorched-earth] [--datastet]
[--scorched-earth] [--datastet] [--data-path DATA_PATH]

Softcite software mention recognizer client

Expand All @@ -68,6 +68,7 @@ optional arguments:
--file-out FILE_OUT path to a single output the software mentions in JSON format, extracted from
the PDF file-in
--config CONFIG path to the config file, default is ./config.json
--data-path DATA_PATH path to the directory containing the LMDB database.
--reprocess reprocessed failed PDF or XML fulltexts
--reset ignore previous processing states and re-init the annotation process from the
beginning
Expand Down Expand Up @@ -137,7 +138,7 @@ Anntations will be added along the PDF and XML files, with extension `*.software
-rw-rw-r-- 1 lopez lopez 485 Aug 8 03:41 0100a44b-6f3f-4cf7-86f9-8ef5e8401567.software.json
```

### Reprocess failed PDF or XML fulltexts
### Reprocess failed PDF or XML fulltext

Just add `--reprocess` to the command line, the processing will be limited to the PDF and XML files that failed when processing them:

Expand Down Expand Up @@ -175,7 +176,7 @@ python3 -m software_mentions_client.client --diagnostic-mongo

## Configuration

By default, the concurreny of the parallelized calls to a service is `8`. This parameter can be changed in the configuration file `config.json`.
By default, the concurrency of the parallelized calls to a service is `8`. This parameter can be changed in the configuration file `config.json`.

Other important configuration parameter are the URL of the Software mention recognition web service `software_mention_url`, the optional URL of a DataStet server if used `dataset_mention_url`, the MongoDb instance information if you wish to load the produced annotations in MongoDB.

Expand Down
Loading