Skip to content

feat: implement AI integration services with NER-based data anonymization#13

Open
Juste-Leo2 wants to merge 4 commits into
mainfrom
dev3
Open

feat: implement AI integration services with NER-based data anonymization#13
Juste-Leo2 wants to merge 4 commits into
mainfrom
dev3

Conversation

@Juste-Leo2

Copy link
Copy Markdown
Owner

No description provided.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces local Named Entity Recognition (NER) based anonymization to mask sensitive data (such as names, locations, emails, and phone numbers) before sending requests to external AI APIs. Key changes include adding @huggingface/transformers and onnxruntime-react-native dependencies, configuring Metro to resolve model assets, creating a model download script, and integrating anonymization toggles in the UI and AI service. The review highlights several critical issues: a file descriptor leak in the download script during redirects, React Native runtime path resolution failures for local assets, permanent caching of rejected promises in the NER service singleton, a regex matching failure due to unhandled SentencePiece space characters, and a privacy risk where NER failures silently leak un-anonymized data to external APIs.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread scripts/download_model.js
Comment on lines +22 to +44
const downloadFile = (fileUrl, destPath) => {
return new Promise((resolve, reject) => {
console.log(`Téléchargement de ${fileUrl} ...`);
const file = fs.createWriteStream(destPath);
https.get(fileUrl, (response) => {
if (response.statusCode === 302 || response.statusCode === 301) {
// Redirection HuggingFace
downloadFile(response.headers.location, destPath).then(resolve).catch(reject);
return;
}
if (response.statusCode !== 200) {
reject(new Error(`Erreur HTTP ${response.statusCode} pour ${fileUrl}`));
return;
}
response.pipe(file);
file.on('finish', () => {
file.close(() => resolve());
});
}).on('error', (err) => {
fs.unlink(destPath, () => reject(err));
});
});
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

In the downloadFile function, the write stream file is created on line 25 before checking the HTTP status code. If a redirect (301/302) or an error (non-200) occurs, this stream is left open and never closed or destroyed. This leaks file descriptors and can cause concurrent write conflicts on destPath during the recursive downloadFile call. You should only create the write stream after confirming a successful 200 OK response.

const downloadFile = (fileUrl, destPath) => {
  return new Promise((resolve, reject) => {
    console.log(`Téléchargement de ${fileUrl} ...`);
    https.get(fileUrl, (response) => {
      if (response.statusCode === 302 || response.statusCode === 301) {
        // Redirection HuggingFace
        downloadFile(response.headers.location, destPath).then(resolve).catch(reject);
        return;
      }
      if (response.statusCode !== 200) {
        reject(new Error(`Erreur HTTP ${response.statusCode} pour ${fileUrl}`));
        return;
      }
      const file = fs.createWriteStream(destPath);
      response.pipe(file);
      file.on('finish', () => {
        file.close(() => resolve());
      });
      file.on('error', (err) => {
        fs.unlink(destPath, () => reject(err));
      });
    }).on('error', (err) => {
      reject(err);
    });
  });
};

Comment thread src/services/NerService.ts Outdated
// Mettre le chemin vers le dossier contenant le modèle (relatif pour le bundler RN)
// Avec transformers.js et React Native, on peut utiliser des URL ou des chemins d'assets locaux
// selon la configuration du plugin babel / metro.
env.localModelPath = 'assets/models/';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

In React Native, local assets are bundled into the native binary and are not accessible via standard relative file paths like 'assets/models/' at runtime. Using @huggingface/transformers directly with local paths will fail in production. Consider using a React Native-compatible wrapper like @automatalabs/react-native-transformers or a custom native asset copier to copy the model files to the document/cache directory before loading them.

Comment thread src/services/anonymizer.ts Outdated

for (const entity of sortedEntities) {
if (entity.entity_group === 'PER' || entity.entity_group === 'LOC') {
let originalWord = entity.word.trim();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

ALBERT uses the SentencePiece tokenizer, which represents spaces as the special character (U+2581). The word returned by the pipeline will contain this character (e.g., Montpellier or Jean-Claude). Since trim() only removes standard whitespace characters, it will NOT remove (U+2581). As a result, the subsequent regex replacement will fail to match the original text because the original text contains standard spaces, not (U+2581). To fix this, we must replace \u2581 with a standard space (or remove it) before trimming and matching.

Suggested change
let originalWord = entity.word.trim();
let originalWord = entity.word.replace(/\u2581/g, ' ').trim();

Comment on lines +46 to +47
try {
const entities = await extractEntities(anonymizedText);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-high high

If the local NER model fails to load or run, extractEntities silently catches the error and returns an empty array. This causes anonymize to silently succeed without masking any names or locations, sending the raw, un-anonymized sensitive data (PII) to the external LLM API. Since the user explicitly enabled anonymization for privacy, this silent fallback is a major privacy risk. Consider propagating the error or returning a status indicating failure, so that the calling service can abort the request or warn the user before sending un-anonymized data.

Comment on lines +17 to +35
class NERInference {
static instance: any = null;

static async getInstance() {
if (this.instance === null) {
console.log("Chargement du modèle ONNX...");

// Initialisation du pipeline
this.instance = pipeline(
'token-classification',
'albert-wikiner-fr-onnx',
{
quantized: false // Utiliser false car le fichier s'appelle model.onnx (pas model_quantized.onnx)
}
);
}
return this.instance;
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The pipeline function is asynchronous and returns a Promise. Since this.instance is assigned the Promise directly without await, any concurrent calls to getInstance() will return the same Promise, which is good. However, if the Promise rejects (e.g., due to a loading error), this.instance will permanently hold a rejected Promise, preventing any future retries. We can make the singleton pattern more robust by handling concurrent initialization and allowing retries on failure.

class NERInference {
  static instance: any = null;
  static initPromise: Promise<any> | null = null;

  static async getInstance() {
    if (this.instance) return this.instance;

    if (!this.initPromise) {
      console.log("Chargement du modèle ONNX...");
      this.initPromise = pipeline(
        'token-classification',
        'albert-wikiner-fr-onnx',
        {
          quantized: false
        }
      ).then(classifier => {
        this.instance = classifier;
        return classifier;
      }).catch(err => {
        this.initPromise = null; // Allow retry on failure
        throw err;
      });
    }
    return this.initPromise;
  }
}

@Juste-Leo2

Copy link
Copy Markdown
Owner Author

Okay, the system is finally working and anonymizes the entries. Just wanted to mention a minor issue with “\n” that I found, but it's easy to fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant