Need help with configuration Veeam 13 & OAuth2

[sztomin]

Hello,

I need help with authentication configuration.
My company uses Veeam 13, which uses OAuth2. If I understand correctly, httpapi_exporter does not natively support OAuth2.
https://helpcenter.veeam.com/references/vbr/13/rest/1.3-rev1/tag/SectionOverview#section/Authorization-and-Security

Unfortunately, the ready-made configuration available in the repository is not adapted to the newer Veeam. I tried to modify it, but despite my efforts, I find it difficult to understand.

login:
- name: init login loop
  vars:
	login_retry: 0
	status_code: 0
  until: 'js: login_retry < queryRetry'

  actions:
	- name: login phase
	  query:
		url: /oauth2/token
		data: 'js: "user="+user+"&password="+password+"&grant_type=password"'
		method: post
		auth_config:
		  mode: basic
		ok_status: 201
		var_name: _
	- name: analyze login response
	  play_script: auth_check

In the login script, I am able to get access_token and refresh_token by sending a payload. How could I save these tokens so that I can use them later to read metrics?
How to implement OAuth2 authorizations in the configuration?

[peekjef72]

Hi Szymon,

Sorry to hear that you’re having problems. It would probably be a good idea to improve the documentation regarding debugging and troubleshooting login issues. Maybe we could write a tutorial based on your use case!

I’m not very familiar with OAuth2’s internal mechanisms, but usually, after providing a login/password, it sends a token that must be set in the request headers as Authorization: Bearer wACCESS_TOKEN>.

So, the first approach is to do it yourself in the "auth_check" script, modifying the current one as follows:

current script (see etc/veeam/config.yml) :

      auth_check:
        - name: analyze login response ok
          set_fact:
            headers: 
              - name: "X-RestSvcSessionId"
                value: 'js: exporter.getHeader( response_headers, "X-RestSvcSessionId" )'
            logged: true
            login_retry: $queryRetry
            # remove .login entry from symbols table
            # login: ~
          when:
            - 'js: status_code == 201'
        - name: analyze login response
          set_fact:
            # auth_token: ""
            logged: false
            login_retry: $queryRetry
            headers: 
              - name: "Auto"
                action: delete
          when:
            - 'js: (status_code == 401) || (status_code == 403)'

        - name: analyze login response not ok with retry
          set_fact:
            # auth_token: ""
            logged: false
            login_retry: 'js: ++login_retry'
          when:
            - 'js: (status_code != 201) && (status_code != 401) && (status_code != 403)'

My suggestion is to alter the script to update the condtions and actions:

• OK condition : js: status_code == 201 (seem correct according to your ok_status)

  • collect the token ( header usually)
  • set the header directcly similar to the current script:

         set_fact:
           headers: 
             - name: "Authorization"
               value: >
                  js: 
                      "Bearer " + exporter.getHeader( response_headers, "<ACCESS_TOKEN_HEADER>" )
           logged: true
           login_retry: $queryRetry

• wrong auth condition : js: (status_code == 401) || (status_code == 403)' ) :
  try an invalid auth to see if the resulting code is really 401 or 403

• wrong others conds (to retry) : js: (status_code != 201) && (status_code != 401) && (status_code != 403) :
  change only if previous code (401 403) are invalid.

A second approch is to set the "reserved/internal" variable "auth_token" so that it is used for all queries; The script in the OK condition would look something like this :

     set_fact:
       auth_token: 'js: exporter.getHeader( response_headers, "<ACCESS_TOKEN_HEADER>"  )'
       logged: true
       login_retry: $queryRetry

I currently don't have an access to an OAuth2 use case/appi, so Ican't test these suggestions.

I encourage you to add the `debug: true flag on the query that allow to receive the full server response.

  actions:
    - name: login phase
      query:
    	url: /oauth2/token
    	data: 'js: "user="+user+"&password="+password+"&grant_type=password"'
    	method: post
    	auth_config:
    	  mode: basic
    	ok_status: 201
        debug: true
    	var_name: _

And use the command line option --dry-run in conjonction with --target=... to have direct logs in the console.

Hope this helps you gain access. If not, please provide some anonymized logs.

JFPIK-

[sztomin]

Hi,

Thanks for your reply.

I managed to figure it out and understand it a little better. I won't lie, I used Gemini, so if there's something strange in the config, that might be why. But I looked through it and it looks okay.

I managed to make a simple configuration. As you wrote, it only uses the access_token passed to the header, so it will retrieve it again every hour via /api/oauth2/token.
I also made a mistake earlier, because when retrieving the token, you need to specify the username parameter, not user.
I don't know if the endpoint for ping should be /api/v1/serverTime, but I don't know what else I could replace it with.

Is auth_check have to be?

global:
  scrape_timeout: 30s
  scrape_timeout_offset: 500ms
  min_interval: 0s
  query_retry: 3
  metric_prefix: veeam_em
  invalid_auth_code:
    - 401
    - 403
  exporter_name: veeam_exporter
profiles:
  veeam:
    scripts:
      init:
        - name: default headers
          set_fact:
            headers:
              - name: Content-Type
                value: application/x-www-form-urlencoded
              - name: Accept
                value: application/json
            scheme: https
            port: 9419
            verifySSL: false
            base_url: /api
            jobHistory: '-12h'
            taskHistory: '-12h'
      login:
        - name: perform login and store token
          actions:
            - name: enforce x-www-form-urlencoded
              set_fact:
                headers:
                  - name: Content-Type
                    value: application/x-www-form-urlencoded
            - name: execute login query
              query:
                url: /oauth2/token
                data: 'js: "username=" + user + "&password=" + password + "&grant_type=password"'
                method: post
                ok_status: 200
                var_name: login_res
            - name: apply token and json headers
              set_fact:
                logged: true
                access_token: 'js: login_res.access_token'
                headers:
                  - name: Content-Type
                    value: application/json
                  - name: Accept
                    value: application/json
                  - name: x-api-version
                    value: 1.3-rev1
                  - name: Authorization
                    value: 'js: "Bearer " + login_res.access_token'
      ping:
        - name: check ping
          query:
            url: /v1/serverTime
            method: get
            ok_status: 200
targets:
  - name: default
    scheme: https
    host: <HOST>
    port: 9419
    auth_config:
      mode: script
      user: <USERNAME>
      password: <PASSWORD>
    profile: veeam
    collectors:
      - ~.*_metrics
  - targets_files:
      - ./targets/*.yaml
collector_files:
  - metrics/*.collector.yml

I also thought earlier that only the authentication had changed, but in fact the entire Veeam API has changed, and as a result, the entire ready-made configs (including collectors) is not compatible with this API.
The configs in the repository is for the Veeam Backup Enterprise Manager REST API. And I'm dealing with the Veeam Backup & Replication REST API.

So, I made a simple collector that gives metrics with job names.

collector_name: job_last_run_status
metric_prefix: veeam_vbr_jobs

scripts:
  get_jobs_data:
    - name: collect vbr jobs
      query:
        url: /v1/jobs/states
        method: get
        ok_status: 200
        var_name: jobs_states

    - name: export jobs info
      with_items: 'js: jobs_states.data'
      metrics:
        - metric_name: info
          help: "Jobs status"
          type: gauge
          key_labels:
            job_id: '{{ .id }}'
            job_name: '{{ .name }}'
            job_type: '{{ .type }}'
          values:
            _: 1

One last question, less related to the matter at hand. Is there any way to convert the date (ISO 8601) to a UNIX timestamp?
For example: "lastRun": "2025-07-28T10:44:39.694588+02:00"

https://en.wikipedia.org/wiki/ISO_8601
https://helpcenter.veeam.com/references/vbr/13/rest/1.3-rev1/tag/Jobs#operation/GetAllJobsStates

Do you have any suggestions on what I should change?

[sztomin]

And yes, if I find some time, I can help with the documentation or a tutorial.😃

[mateuszdrab]

Hi Szymon, sorry to hijack this thread but I wonder if you got the exporter working at all since upgrading to Veeam v13 Enterprise Manager?

For me, the exporter worked fine (I was using a local account on the Veeam Enterprise Manager server); however, since the upgrade to v13 I keep getting 403s, this would make sense if the API changed.

However, I'm also seeing that you're attempting to use the exporter against VBR API (Not VEM).

I wanted to figure out if we're facing the same or different issues so that I can make a new issue if needed.

Thanks!

[peekjef72]

Hi,

Sorry, for the delay.
I did not read the message with enought attention, so I forget to answer your question:

I managed to make a simple configuration. As you wrote, it only uses the access_token passed to the header, so it will retrieve it again every hour via /api/oauth2/token. I also made a mistake earlier, because when retrieving the token, you need to specify the username parameter, not user. I don't know if the endpoint for ping should be /api/v1/serverTime, but I don't know what else I could replace it with.

The ping script has two goals (at least):

• to produce an "up" metric representing the availability of the target.
• to check if the authentication is valid and not expired, and if expired to try to reconnect.

So any url on target sytem is valid!
Sometimes you use that url to collect metrics from the response; if not you should specify "parser: none" in the query parameters to avoid parsing the response. If I remerber the veeam documentation there was an endpoint with version... it should be an another candidate.

I also thought earlier that only the authentication had changed, but in fact the entire Veeam API has changed, and as a result, the entire ready-made configs (including collectors) is not compatible with this API. The configs in the repository is for the Veeam Backup Enterprise Manager REST API. And I'm dealing with the Veeam Backup & Replication REST API.

Maybe can you share this and the further works on VBR here with a PR in contribs parts ?

One last question, less related to the matter at hand. Is there any way to convert the date (ISO 8601) to a UNIX timestamp? For example: "lastRun": "2025-07-28T10:44:39.694588+02:00"

https://en.wikipedia.org/wiki/ISO_8601 https://helpcenter.veeam.com/references/vbr/13/rest/1.3-rev1/tag/Jobs#operation/GetAllJobsStates

Do you have any suggestions on what I should change?

My referene is the browser dev console for javascript (and AI): so

mytime: >
  js:
    my_date = new Date(date_str)
    my_date.getTime() / 1000

below a text copy of my tests in a js console:

cur_date = new Date( "2025-07-28T10:44:39.694588+02:00" )
Mon Jul 28 2025 10:44:39 GMT+0200 (heure d’été d’Europe centrale)
cur_date.getTime()
1753692279694

and a reverse test under UNIX with date command:

$ date -d@1753692279
Mon Jul 28 10:44:39 CEST 2025

[sztomin]

Hi,

Sorry for the delay, I've been a bit busy.

Mateusz, I only had a brief chance to work with the previous API (Enterprise Manager). Also I don't have any experience working with Veeam itself. From my perspective, API had changed.

https://forums.veeam.com/restful-api-f30/some-api-questions-t78595.html

From the link above, it appears that EM is independent of VBR. Maybe there’s a way to enable or configure EM, but I’m not sure.

The ping script has two goals (at least):

• to produce an "up" metric representing the availability of the target.
• to check if the authentication is valid and not expired, and if expired to try to reconnect.

So any url on target sytem is valid!
Sometimes you use that url to collect metrics from the response; if not you should specify "parser: none" in the query parameters to avoid parsing the response. If I remerber the veeam documentation there was an endpoint with version... it should be an another candidate.

Thanks for the advice, I added parser: none. Unfortunately, I couldn’t find an endpoint with a version in the Veeam documentation.

Maybe you could share this and your further work on VBR here with a PR in the contribs section?

Sure, but I’ll have to fix the collectors because the descriptions aren’t in English and generalize it - there are a few things that are set up specifically for us.
I’ll try to prepare something based on the existing Veeam configuration and submit a PR by the end of next week. Unfortunately, I don’t have access to all endpoints, so I couldn't test everything.

My reference is the browser dev console for JavaScript (and AI): so

mytime: >
js:
my_date = new Date(date_str)
my_date.getTime() / 1000
Below is a text copy of my tests in a JavaScript console:

cur_date = new Date( “2025-07-28T10:44:39.694588+02:00” )
Mon Jul 28 2025 10:44:39 GMT+0200 (Central European Summer Time)
cur_date.getTime()
1753692279694
and a reverse test under UNIX using the date command:

$ date -d@1753692279
Mon Jul 28 10:44:39 CEST 2025

Thanks, it works great.

[mateuszdrab]

Hi @peekjef72

I'm surprised to see that no reports have been raised so far with regards to V13 EM not working with the exporter. For me, ever since the upgrade the exporter never worked.

Now after the 0.4.2 update, the exporter completely fails to start with:

ts=2026-05-23T15:13:24.817Z level=error caller=httpapi_exporter.go:155 msg="Error creating exporter: registry for script is nil"  

Would you be able to help me fix up the issues so that it works with Veeam again?

[peekjef72]

Hi Mateusz,
You're right this is a regression introduced in version 0.4.2. Sorry about that, and thank you for the quick feedback!
I've fixed the behavior for the specific case used in the default Veeam profile. (see CHANGELOG)

Concerning EM V13, I will check with the backup admins at my company, but last time I've asked they still used an older version.

To be continued.

[peekjef72]

@mateuszdrab

Could you be more precise when you say it doesn't work ?
Maybe can you send anonymized logs from the exporter in debug mode ?

when I've built the first configuration I've recorded the manual curl command that I've used as a starting point:

curl -vk -u "domain\user" \
    -XPOST "https://host.domain:port/api/sessionMngr/?v=latest" \
    -H "Content-Type: application/json" 
    --data ""

curl -vk "https://host.domain:port/api/reports/summary/overview" \
    -H "Accept: application/json" \
    -H "X-RestSvcSessionId: MGRmZTQ0MjctNjkwOC00MDIzLTkyYjctMmJkMTQ0OTQ5YTE5"

The first command is for authentication, to obtain a valid token received in header line X-RestSvcSession.
The second to obtain the "first" data from veeam (veeam_overview_collector.yml)

I don't see changes metionned in the API doc for that GET /reports/summary/overview

[mateuszdrab]

Hi Mateusz, You're right this is a regression introduced in version 0.4.2. Sorry about that, and thank you for the quick feedback! I've fixed the behavior for the specific case used in the default Veeam profile. (see CHANGELOG)

Concerning EM V13, I will check with the backup admins at my company, but last time I've asked they still used an older version.

To be continued.

Thank you, this resolved the start-up crash and allowed the exporter to start again.

I noticed my configuration was quite out of date with the Veeam contrib code, I have updated my code (most particularly, the files in the metrics directory) to match exactly your current code. I also updated the config file as I didn't have the latest syntax in there so no profiles were defined.

With the configuration updated, I verified also that the credentials the exporter uses still work in Veeam EM.

Currently, seeing the following errors when running the exporter:

ts=2026-05-24T08:48:54.221Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: BackupServers not found" coll=veeam_overview_metrics script="get veeam_overview" name="proceed elements"
ts=2026-05-24T08:48:54.221Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ProxyServers not found" coll=veeam_overview_metrics script="get veeam_overview" name="proceed elements"
ts=2026-05-24T08:48:54.221Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: RepositoryServers not found" coll=veeam_overview_metrics script="get veeam_overview" name="proceed elements"
ts=2026-05-24T08:48:54.221Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ScheduledJobs not found" coll=veeam_overview_metrics script="get veeam_overview" name="proceed elements"
ts=2026-05-24T08:48:54.221Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ProtectedVms not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.221Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: SuccessfulVmLastestStates not found" coll=veeam_overview_metrics script="get veeam_overview" name="proceed elements"
ts=2026-05-24T08:48:54.221Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: BackedUpVms not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ReplicatedVms not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: RestorePoints not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: FullBackupPointsSize not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements total_bytes"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: RunningJobs not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: IncrementalBackupPointsSize not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements total_bytes"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ScheduledJobs not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ReplicaRestorePointsSize not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements total_bytes"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ScheduledBackupJobs not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: SourceVmsSize not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements total_bytes"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ScheduledReplicaJobs not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.221Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: WarningVmLastestStates not found" coll=veeam_overview_metrics script="get veeam_overview" name="proceed elements"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: FailedVmLastestStates not found" coll=veeam_overview_metrics script="get veeam_overview" name="proceed elements"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: TotalJobRuns not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: SuccessfulJobRuns not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: WarningsJobRuns not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.222Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: FailedJobRuns not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.236Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'taskname'" errmsg="getVarError 1: taskname not found" coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.236Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'vmname'" errmsg="getVarError 1: vmname not found" coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.236Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'backupserver'" errmsg="getVarError 1: backupserver not found" coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.236Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'jobname'" errmsg="getVarError 1: jobname not found" coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.237Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.237Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.237Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.237Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.237Z level=warn caller=collector.go:220 msg="invalid value for 'when' {{or (EQ (toString .task.state) \"2\") (EQ (toString .task.state) \"3\")}}: getVarError 2: invalid template: template: field:1:26: executing \"field\" at <.task.state>: can't evaluate field state in type interface {}" coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.251Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'backupserver'" errmsg="getVarError 1: backupserver not found" coll=backup_jobs_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.251Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'name'" errmsg="getVarError 1: name not found" coll=backup_jobs_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.251Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'jobname'" errmsg="getVarError 1: jobname not found" coll=backup_jobs_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.251Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'jobtype'" errmsg="getVarError 1: jobtype not found" coll=backup_jobs_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.251Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.251Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.251Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.251Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.735Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ProtectedVms not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.735Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: BackedUpVms not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.735Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ReplicatedVms not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.735Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: RestorePoints not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.735Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: FullBackupPointsSize not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements total_bytes"
ts=2026-05-24T08:48:54.735Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: IncrementalBackupPointsSize not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements total_bytes"
ts=2026-05-24T08:48:54.735Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ReplicaRestorePointsSize not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements total_bytes"
ts=2026-05-24T08:48:54.736Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: SourceVmsSize not found" coll=veeam_overview_vms_metrics script="get veeam_overview" name="proceed elements total_bytes"
ts=2026-05-24T08:48:54.736Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: BackupServers not found" coll=veeam_overview_metrics script="get veeam_overview" name="proceed elements"
ts=2026-05-24T08:48:54.736Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ProxyServers not found" coll=veeam_overview_metrics script="get veeam_overview" name="proceed elements"
ts=2026-05-24T08:48:54.736Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: RepositoryServers not found" coll=veeam_overview_metrics script="get veeam_overview" name="proceed elements"
ts=2026-05-24T08:48:54.736Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ScheduledJobs not found" coll=veeam_overview_metrics script="get veeam_overview" name="proceed elements"
ts=2026-05-24T08:48:54.736Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: SuccessfulVmLastestStates not found" coll=veeam_overview_metrics script="get veeam_overview" name="proceed elements"
ts=2026-05-24T08:48:54.736Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: WarningVmLastestStates not found" coll=veeam_overview_metrics script="get veeam_overview" name="proceed elements"
ts=2026-05-24T08:48:54.736Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: FailedVmLastestStates not found" coll=veeam_overview_metrics script="get veeam_overview" name="proceed elements"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: RunningJobs not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ScheduledJobs not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ScheduledBackupJobs not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'jobname'" errmsg="getVarError 1: jobname not found" coll=backup_jobs_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'jobtype'" errmsg="getVarError 1: jobtype not found" coll=backup_jobs_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'backupserver'" errmsg="getVarError 1: backupserver not found" coll=backup_jobs_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'name'" errmsg="getVarError 1: name not found" coll=backup_jobs_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.821Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.821Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.821Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'backupserver'" errmsg="getVarError 1: backupserver not found" coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'jobname'" errmsg="getVarError 1: jobname not found" coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'taskname'" errmsg="getVarError 1: taskname not found" coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'vmname'" errmsg="getVarError 1: vmname not found" coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session" name="build labels"
ts=2026-05-24T08:48:54.821Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: ScheduledReplicaJobs not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: TotalJobRuns not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.821Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: SuccessfulJobRuns not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.821Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: WarningsJobRuns not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.822Z level=warn caller=yaml_script.go:341 msg="error building map value for key 'value'" errmsg="getVarError 1: FailedJobRuns not found" coll=job_overview_metrics script="get jobs_overview" name="proceed elements count"
ts=2026-05-24T08:48:54.822Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.822Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.822Z level=warn caller=metric.go:286 msg="invalid type need map[string[][string]" type_error=string coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.822Z level=warn caller=collector.go:220 msg="invalid value for 'when' {{or (EQ (toString .task.state) \"2\") (EQ (toString .task.state) \"3\")}}: getVarError 2: invalid template: template: field:1:26: executing \"field\" at <.task.state>: can't evaluate field state in type interface {}" coll=backup_jobs_tasks_sessions_metrics script="get backup_job_session"
ts=2026-05-24T08:48:54.968Z level=error caller=exporter.go:137 msg="target has panic-ed: send on closed channel" coll=veeam-backup
ts=2026-05-24T08:48:54.968Z level=error caller=promhttp.go:136 msg="Error gathering metrics for 'veeam-backup': 5 error(s) occurred:\n* collected metric \"veeam_em_jobs_tasks_sessions_duration\" { counter:{value:0}} was collected before with the same name and label values\n* collected metric \"veeam_em_overview_jobs_max_duration\" { label:{name:\"jobname\" value:\"\"} label:{name:\"type\" value:\"job\"} counter:{value:0}} was collected before with the same name and label values\n* collected metric \"veeam_em_overview_jobs_max_duration\" { label:{name:\"jobname\" value:\"\"} label:{name:\"type\" value:\"backupjob\"} counter:{value:0}} was collected before with the same name and label values\n* collected metric \"veeam_em_overview_jobs_max_duration\" { label:{name:\"jobname\" value:\"\"} label:{name:\"type\" value:\"replicajob\"} counter:{value:0}} was collected before with the same name and label values\n* collected metric \"veeam_em_jobs_sessions_duration\" { counter:{value:0}} was collected before with the same name and label values"

[mateuszdrab]

I might have found potentially the culprit

ts=2026-05-24T08:55:06.282Z level=warn caller=query_action.go:461 msg="internal method returns error: 'Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority'" coll=veeam-backup script=ping name="check ping cnx"
ts=2026-05-24T08:55:06.282Z level=error caller=target.go:1005 msg="Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority" coll=ping/veeam-backup
ts=2026-05-24T08:56:06.189Z level=warn caller=query_action.go:461 msg="internal method returns error: 'Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority'" coll=veeam-backup script=ping name="check ping cnx"
ts=2026-05-24T08:56:06.189Z level=error caller=target.go:1005 msg="Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority" coll=ping/veeam-backup
ts=2026-05-24T08:57:06.182Z level=warn caller=query_action.go:461 msg="internal method returns error: 'Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority'" coll=veeam-backup script=ping name="check ping cnx"
ts=2026-05-24T08:57:06.182Z level=error caller=target.go:1005 msg="Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority" coll=ping/veeam-backup
ts=2026-05-24T08:58:06.220Z level=warn caller=query_action.go:461 msg="internal method returns error: 'Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority'" coll=veeam-backup script=ping name="check ping cnx"
ts=2026-05-24T08:58:06.220Z level=error caller=target.go:1005 msg="Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority" coll=ping/veeam-backup
ts=2026-05-24T08:59:06.178Z level=warn caller=query_action.go:461 msg="internal method returns error: 'Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority'" coll=veeam-backup script=ping name="check ping cnx"
ts=2026-05-24T08:59:06.178Z level=error caller=target.go:1005 msg="Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority" coll=ping/veeam-backup
ts=2026-05-24T09:00:06.174Z level=warn caller=query_action.go:461 msg="internal method returns error: 'Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority'" coll=veeam-backup script=ping name="check ping cnx"
ts=2026-05-24T09:00:06.174Z level=error caller=target.go:1005 msg="Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority" coll=ping/veeam-backup
ts=2026-05-24T09:01:06.239Z level=warn caller=query_action.go:461 msg="internal method returns error: 'Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority'" coll=veeam-backup script=ping name="check ping cnx"
ts=2026-05-24T09:01:06.239Z level=error caller=target.go:1005 msg="Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority" coll=ping/veeam-backup
ts=2026-05-24T09:02:06.197Z level=warn caller=query_action.go:461 msg="internal method returns error: 'Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority'" coll=veeam-backup script=ping name="check ping cnx"
ts=2026-05-24T09:02:06.197Z level=error caller=target.go:1005 msg="Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority" coll=ping/veeam-backup
ts=2026-05-24T09:03:06.181Z level=warn caller=query_action.go:461 msg="internal method returns error: 'Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority'" coll=veeam-backup script=ping name="check ping cnx"
ts=2026-05-24T09:03:06.182Z level=error caller=target.go:1005 msg="Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority" coll=ping/veeam-backup
ts=2026-05-24T09:04:06.248Z level=warn caller=query_action.go:461 msg="internal method returns error: 'Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority'" coll=veeam-backup script=ping name="check ping cnx"
ts=2026-05-24T09:04:06.248Z level=error caller=target.go:1005 msg="Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority" coll=ping/veeam-backup
ts=2026-05-24T09:05:06.180Z level=warn caller=query_action.go:461 msg="internal method returns error: 'Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority'" coll=veeam-backup script=ping name="check ping cnx"
ts=2026-05-24T09:05:06.181Z level=error caller=target.go:1005 msg="Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority" coll=ping/veeam-backup
ts=2026-05-24T09:06:06.199Z level=warn caller=query_action.go:461 msg="internal method returns error: 'Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority'" coll=veeam-backup script=ping name="check ping cnx"
ts=2026-05-24T09:06:06.199Z level=error caller=target.go:1005 msg="Get \"https://veeam-backup:443/api/\": tls: failed to verify certificate: x509: certificate signed by unknown authority" coll=ping/veeam-backup

Edit, added verifySSL: false (supported according to copilot) to profile but the same issue continues.

[peekjef72]

for tls, I remember a similar problem with netscaler exporter:

v 0.4.1

...

2025-03-26 - not release

• added: global config parameter tls_version that allows to add old tls ciphers, because of golang change since 1.22: see config; update the netscaler default config file to use tls_version: all

So maybe can you add in the config.yml in global section

global:
  # max timeout for the exporter: if prometheus sends a value greater than scrape_timeout, scrape_timeout will be used
  scrape_timeout: 30s
  # Subtracted from Prometheus' scrape_timeout to give us some headroom and prevent Prometheus from timing out first.
  scrape_timeout_offset: 500ms
  # Minimum interval between collector runs: by default (0s) collectors are executed on every scrape.
  min_interval: 0s
  # all unsuccessful queries will be retried this number of times
  query_retry: 3
  # all metrics will be named "[metric_prefix]_[metric_name]"
  metric_prefix: "veeam_em"
  # all http codes that will consider the connection has an invalid auth and must do a Login()
  invalid_auth_code: [401,403]
  exporter_name: veeam_exporter  # list of allowed tls version, meaning authorized ciphers for https connections
  #   all, tls_upto_1.2, tls_1.2 ,tls_1.3
  tls_version: all

...

and give feedbacks.

[peekjef72]

any way, I consider there is a bug in the query action because there is not result so return code is not valid ( must be 200). I've to investigate this point and probably to create a new issue!

[mateuszdrab]

Unfortunately, no improvement with tls_version: all - I'd not expect this to make a difference much since Veeam is up to date so would be using latest ciphers/tls version.

Let me raise a new issue.