“OpenAI’s hunger for data is coming back to bite it”
These methods, and the sheer size of the data set, mean tech companies tend to have a very limited understanding of what has gone into training their models.
@baldur OpenAI could have used their funding to develop methods for examining what sources the language model uses in its answers. Though difficult, this would have been real innovation and a contribution to the field. Instead, they are monetizing data they harvested illegally.