Here is some unofficial info that I compiled for a customer some time ago. Maybe you can use it as a starting point.
ftp.drupal.org , Port 443 (for update module)
git.drupal.org , Port 22, 80, 443 (for patches)
packages.drupal.org , Port 22, 80, 443 (for Drupal Composer packages)
updates.drupal.org , Port 80, 443 (for update module)
www.drupal.org , Port 443 (for patches)
cgit.drupalcode.org , Port 443 (for patches)
git.drupalcode.org , Port 22, 80, 443 (for patches)
gitlab.drupalcode.org , Port 22, 80, 443 (for patches or installing dev branches using Composer)
github.com 19 , Port 22, 80, 443 (for libraries and installing dev branches of those libraries using Composer)
repo.packagist.org , Port 22, 80, 443 (for Drupal Composer packages and dependencies of those packages)
Some it you'll need only if you apply patches from there. Some of it might no longer be needed, depending on your setup. Some of it you might need only, if you have the update module enabled.
At the time, hosts provided by Drupal Assoc were behind a CDN and it was Fastly. IP ranges of Fastly: https://docs.fastly.com/guides/securing-communications/accessing-fastlys-ip-ranges
Here are GitHub's ip addresses, if you use packages/libraries from there: https://help.github.com/articles/about-github-s-ip-addresses/