Bluesky has a proposal to declare user intention for things like archiving and AI training. Looks pretty good. Maybe we should mirror this in robots.txt? It stretches the original purpose of the file but it should be somewhere outside of a specific protocol. There was also CC-NT for one narrow use.

@manton They should attend the IETF meetings for AIPREFS and such instead of inventing new proposals and chucking them up on GitHub, potentially splintering common practice.

@alexr Thanks for the pointer, I wasn’t aware of the AI pref working group either.