Skip to content

Quota error messages will cause NUnit tests to fail when CaptureInEditor = true #2538

@TheCrossBoy

Description

@TheCrossBoy

Environment

Found on sentry v4.1.0, but likely applies to any version

Steps to Reproduce

I think any quota error message will trigger this, but here's the steps that should trigger it easily with structured logs

  1. Install Sentry to a project and configure it to capture in editor and send structured logs for all types
  2. Setup NUnity tests in unity that run when a build happens
  3. Go over the Logs quota for the organization
  4. Trigger a build in unity

Expected Result

Hitting a quota/rate limit error does not cause NUnit tests to fail.

Actual Result

If a quota limit log message is triggered during the test, it will cause it to fail. Example:

11:24:33 Failed Test: TestName with message: Unhandled log message: '[Error] Sentry: (Error) HttpTransport: Sentry rejected the envelope ''. Status code: TooManyRequests. Error detail: Sentry dropped data due to a quota or internal rate limit being reached. This will not affect your application. See https://docs.sentry.io/product/accounts/quotas/ for more information.. Error causes: . '. Use UnityEngine.TestTools.LogAssert.Expect

Additional Info

I believe this is happening because LogAssert.ignoreFailingMessages isn't set to true for our tests, and the quota message is logged with LogError. I wouldn't consider "just enable this option for your tests" to be a good solution, as I do still want the tests to fail with other unexpected log messages, and I don't think there's any kind of filter for it.

This may not be able to be fixed directly, other than making the message not at the error level (which may not be desired).

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugSomething isn't workingUnity

    Projects

    Status

    No status

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions