Surrendering to FOMO on important AI “breakthroughs”, I subscribed to Arxiv mailing list. Like all of the subscribers, I can’t read all the submissions in the daily email. To be completely honest, I simply archive the email from my inbox everyday without even opening the email at all. There is a slight chance that I try to browse through titles and abstracts one by one. But I wouldn’t be able to read more than 10 abstracts. There are simply too many to read through.
I ran into this GPU memory leak issue when building a PyTorch training pipeline. After spending quite some time, I finally figured out this minimal reproducible example. Kicking off the training, it shows constantly increasing allocated GPU memory. This “AverageMeter” has been used in many popular repositories (e.g., https://github.com/facebookresearch/moco). It’s by-design tracking the average of […]