Thanks for replying. I appreciate it. I just wondered if there is anything that can be done to reduce the impact such as rate limiting or sleeping using celery or something similar.
Example here:
https://stackoverflow.com/a/50294531
I apologise in advance if this is an example of a newb teaching you to suck eggs. Just trying to see if the metadata retrieval process can be improved.