diff --git a/docs/en/06-masking.md b/docs/en/06-masking.md new file mode 100644 index 0000000..f9fd833 --- /dev/null +++ b/docs/en/06-masking.md @@ -0,0 +1,268 @@ +# Masking + +Masking provides the ability to hide some of the information in the logs without modifying the logs themselves in the `seq-db`. + +Masking applies to search, export, and aggregation operations. + +See `handlers.seq_api.masking` section in [config](./02-configuration.md). + +## Examples + +### Simple + +All log fields will be masked. The masks will be applied in sequence. + +```yaml +masking: + masks: + - re: '(\d{3})-(\d{3})-(\d{4})' + mode: 'mask' + - re: '@[a-z]+' + mode: 'mask' +``` + +Before: +```json +{ + "message": "request from @host123", + "user": "@ivan", + "phone": "123-456-7890" +} +``` + +After: +```json +{ + "message": "request from ********", + "user": "*****", + "phone": "************" +} +``` + +### Process/Ignore fields + +You can specify a list of fields that will be processed/ignored during masking. +The list can be either global for all masks, or local for each mask (local has the higher priority). + +```yaml +masking: + masks: + - re: '(\d{3})-(\d{3})-(\d{4})' + mode: 'mask' + process_fields: + - 'private_phone' +``` + +Before: +```json +{ + "public_phone": "098-765-4321", + "fake_phone": "123-456-7890", + "private_phone": "123-456-7890" +} +``` + +After: +```json +{ + "public_phone": "098-765-4321", + "fake_phone": "123-456-7890", + "private_phone": "************" +} +``` + +--- + +```yaml +masking: + masks: + - re: '(\d{3})-(\d{3})-(\d{4})' + mode: 'mask' + ignore_fields: + - 'fake_phone' + process_fields: + - 'fake_phone' +``` + +Before: +```json +{ + "public_phone": "098-765-4321", + "fake_phone": "123-456-7890", + "private_phone": "123-456-7890" +} +``` + +After: +```json +{ + "public_phone": "************", + "fake_phone": "123-456-7890", + "private_phone": "************" +} +``` + +### Groups + +For partial masking, you must use the `groups` field. + +```yaml +masking: + masks: + - re: '(\d{3})-(\d{3})-(\d{4})' + groups: [1, 3] + mode: 'mask' +``` + +Before: +```json +{ + "phone": "123-456-7890" +} +``` + +After: +```json +{ + "phone": "***-456-****" +} +``` + +### Mask modes + +There are 3 masking modes: `mask`, `replace` and `cut`. The `mask` mode was used in the examples above. + +```yaml +masking: + masks: + - re: '(\d{3})-(\d{3})-(\d{4})' + mode: 'replace' + replace_word: +``` + +Before: +```json +{ + "phone": "123-456-7890" +} +``` + +After: +```json +{ + "phone": "" +} +``` + +--- + +```yaml +masking: + masks: + - re: '(\d{3})-(\d{3})-(\d{4})' + mode: 'cut' +``` + +Before: +```json +{ + "message": "phone: 123-456-7890;" +} +``` + +After: +```json +{ + "message": "phone: ;" +} +``` + +## Field filters + +Field filters provide the ability to apply masks only for those events whose fields fall under the filtering conditions. + +### Field filter set + +Field filter set is a set of filters that are interconnected by a logical condition (`or`, `and`, `not`). +Even if you need to apply only one filter, you must specify the `condition` field, but in this case it is ignored (except `not`). + +```yaml +masking: + masks: + - ... + field_filters: + - condition: 'or' + filters: [, , ...] +``` + +### Examples + +```yaml +masking: + masks: + - ... + field_filters: + condition: 'or' + filters: + - filed: 'level' + mode: 'equal' + values: ['0', '1', '2', '3'] + - field: 'message' + mode: 'contains' + vaules: ['error', 'panic'] +``` + +Masked: +```json +{ + "level": "3", + "message": "request failed" +} +``` +```json +{ + "level": "6", + "message": "parsing error occured" +} +``` + +Not masked: +```json +{ + "level": "4", + "message": "request failed" +} +``` + +--- + +```yaml +masking: + masks: + - ... + field_filters: + condition: 'not' + filters: + - filed: 'version' + mode: 'suffix' + values: ['test', 'rc'] +``` + +Masked: +```json +{ + "version": "1.23.4" +} +``` + +Not masked: +```json +{ + "version": "1.23.4-test" +} +``` +```json +{ + "version": "1.23.4-rc" +} +``` \ No newline at end of file diff --git a/docs/ru/05-dashboards-api.md b/docs/ru/05-dashboards-api.md index 1212082..92d39a6 100644 --- a/docs/ru/05-dashboards-api.md +++ b/docs/ru/05-dashboards-api.md @@ -1,6 +1,6 @@ # Dashboards API -Dashboards API pпредоставляет возможность управлять дашбордами. +Dashboards API предоставляет возможность управлять дашбордами. Дашборд состоит из: - поискового запроса diff --git a/docs/ru/06-masking.md b/docs/ru/06-masking.md new file mode 100644 index 0000000..2f860ea --- /dev/null +++ b/docs/ru/06-masking.md @@ -0,0 +1,268 @@ +# Маскирование + +Маскирование предоставляет возможность скрывать часть информации в логах без изменения самих логов в `seq-db`. + +Маскирование применяется к операциям поиска, экспорта и агрегирования. + +Смотрите раздел `handlers.seq_api.masking` в [конфигурации](./02-configuration.md). + +## Примеры + +### Простой + +Все поля лога будут замаскированы. Маски будут применяться последовательно. + +```yaml +masking: + masks: + - re: '(\d{3})-(\d{3})-(\d{4})' + mode: 'mask' + - re: '@[a-z]+' + mode: 'mask' +``` + +До: +```json +{ + "message": "request from @host123", + "user": "@ivan", + "phone": "123-456-7890" +} +``` + +После: +```json +{ + "message": "request from ********", + "user": "*****", + "phone": "************" +} +``` + +### Обрабатываемые/Игнорируемые поля + +Вы можете указать список полей, которые будут обрабатываться/игнорироваться во время маскирования. +Список может быть как глобальным для всех масок, так и локальным для каждой отдельной маски (локальный имеет более высокий приоритет). + +```yaml +masking: + masks: + - re: '(\d{3})-(\d{3})-(\d{4})' + mode: 'mask' + process_fields: + - 'private_phone' +``` + +До: +```json +{ + "public_phone": "098-765-4321", + "fake_phone": "123-456-7890", + "private_phone": "123-456-7890" +} +``` + +После: +```json +{ + "public_phone": "098-765-4321", + "fake_phone": "123-456-7890", + "private_phone": "************" +} +``` + +--- + +```yaml +masking: + masks: + - re: '(\d{3})-(\d{3})-(\d{4})' + mode: 'mask' + ignore_fields: + - 'fake_phone' + process_fields: + - 'fake_phone' +``` + +До: +```json +{ + "public_phone": "098-765-4321", + "fake_phone": "123-456-7890", + "private_phone": "123-456-7890" +} +``` + +После: +```json +{ + "public_phone": "************", + "fake_phone": "123-456-7890", + "private_phone": "************" +} +``` + +### Группы + +Для частичного маскирования используется поле `groups`. + +```yaml +masking: + masks: + - re: '(\d{3})-(\d{3})-(\d{4})' + groups: [1, 3] + mode: 'mask' +``` + +До: +```json +{ + "phone": "123-456-7890" +} +``` + +После: +```json +{ + "phone": "***-456-****" +} +``` + +### Режимы маскирования + +Существует 3 режима маскирования: `mask`, `replace` и `cut`. В приведенных выше примерах использовался режим `mask`. + +```yaml +masking: + masks: + - re: '(\d{3})-(\d{3})-(\d{4})' + mode: 'replace' + replace_word: +``` + +До: +```json +{ + "phone": "123-456-7890" +} +``` + +После: +```json +{ + "phone": "" +} +``` + +--- + +```yaml +masking: + masks: + - re: '(\d{3})-(\d{3})-(\d{4})' + mode: 'cut' +``` + +До: +```json +{ + "message": "phone: 123-456-7890;" +} +``` + +После: +```json +{ + "message": "phone: ;" +} +``` + +## Фильтрация по полям + +Фильтрация по полям предоставляет возможность применять маски только для тех событий, поля которых попадают под условия фильтрации. + +### Набор фильтров по полям + +Набор фильтров по полям - это набор фильтров, которые связаны между собой логическим условием (`or`, `and`, `not`). +Даже если вам нужно применить только один фильтр, вы должны задать `condition`, но в этом случае оно игнорируется (за исключением `not`). + +```yaml +masking: + masks: + - ... + field_filters: + - condition: 'or' + filters: [, , ...] +``` + +### Примеры + +```yaml +masking: + masks: + - ... + field_filters: + condition: 'or' + filters: + - filed: 'level' + mode: 'equal' + values: ['0', '1', '2', '3'] + - field: 'message' + mode: 'contains' + vaules: ['error', 'panic'] +``` + +Будет замаскировано: +```json +{ + "level": "3", + "message": "request failed" +} +``` +```json +{ + "level": "6", + "message": "parsing error occured" +} +``` + +Не будет замаскировано: +```json +{ + "level": "4", + "message": "request failed" +} +``` + +--- + +```yaml +masking: + masks: + - ... + field_filters: + condition: 'not' + filters: + - filed: 'version' + mode: 'suffix' + values: ['test', 'rc'] +``` + +Будет замаскировано: +```json +{ + "version": "1.23.4" +} +``` + +Не будет замаскировано: +```json +{ + "version": "1.23.4-test" +} +``` +```json +{ + "version": "1.23.4-rc" +} +``` \ No newline at end of file