Explore AI powered lints

[benalleng]

Rather than have manual review catch this every time, this is the sort of probabilistic lint we may be able to lean on LLMs for. I'm proposing an issue. I think we can add this one and then use the robot to check as we see repeated slips. I think it could hold us to this type of checklist with much less effort than a human reviewer following this checklist rust-bitcoin/rust-bitcoin#843.

To pull on this thread further I think we could also try to observe some common catches either from our personal mistakes or something that the LLM finds frequently to hard code allowing the LLM to focus on smaller surface areas.

[DanGould]

Take a casual look at my thinking on this for a first blush https://claude.ai/share/c7733f88-cbf9-4e13-bdc3-ab2d39a15fa9

And see how I used coderabit as a poor man's (it's free for OSS) custom api endpoint that removes all the actual coderabbit stuff but just prompts it to review this one lint DanGould#31 (review). I think a real, thorough setup would need multiple sub agents to at least chunk the massive API checklist for guidelines rust gives us, but I'm increasingly convinced this isn't only possible but also valuable

[DanGould]

Every AI lint needs to have extremely strict acceptance criteria. So in the case of monitoring the public API, we need to make sure we tell the AIs what the public API actually is. If we can do that generating rustdoc, we can definitely define it accurately for AIs #1373 (comment)