Using GitHubPrivateRepositoryReleaseDownloadStrategy removed in brew v2

hoge.rb

require "formula"
require_relative "lib/private_strategy"

class Hoge < Formula
  homepage "https://github.com/yourcompany/hoge"
  url "https://github.com/yourcompany/hoge/releases/download/v0.1.0/hoge_v0.1.0_darwin_amd64.tar.gz", :using => GitHubPrivateRepositoryReleaseDownloadStrategy
  sha256 "6de411ff3e4b1658a413dd6181fcXXXXXXXXXXXXXXXXXXXX"
  head "https://github.com/yourcompany/hoge.git"
  version "0.1.0"

  def install
    bin.install "hoge"
  end
end

install.sh

brew tap yourcompany/tap [email protected]:yourcompany/homebrew-tap.git
export HOMEBREW_GITHUB_API_TOKEN=xxx
brew install hoge

private_strategy.rb

# Save this file as `lib/private_strategy.rb`
# Add `require_relative "lib/private_strategy"` to your formula.
# 
# This is based on the following, with minor fixes.
# https://github.com/Homebrew/brew/blob/193af1442f6b9a19fa71325160d0ee2889a1b6c9/Library/Homebrew/compat/download_strategy.rb#L48-L157

# BSD 2-Clause License
#
# Copyright (c) 2009-present, Homebrew contributors
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright notice, this
#   list of conditions and the following disclaimer.
#
# * Redistributions in binary form must reproduce the above copyright notice,
#   this list of conditions and the following disclaimer in the documentation
#   and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

# GitHubPrivateRepositoryDownloadStrategy downloads contents from GitHub
# Private Repository. To use it, add
# `:using => GitHubPrivateRepositoryDownloadStrategy` to the URL section of
# your formula. This download strategy uses GitHub access tokens (in the
# environment variables `HOMEBREW_GITHUB_API_TOKEN`) to sign the request.  This
# strategy is suitable for corporate use just like S3DownloadStrategy, because
# it lets you use a private GitHub repository for internal distribution.  It
# works with public one, but in that case simply use CurlDownloadStrategy.
class GitHubPrivateRepositoryDownloadStrategy < CurlDownloadStrategy
  require "utils/formatter"
  require "utils/github"

  def initialize(url, name, version, **meta)
    super
    parse_url_pattern
    set_github_token
  end

  def parse_url_pattern
    unless match = url.match(%r{https://github.com/([^/]+)/([^/]+)/(\S+)})
      raise CurlDownloadStrategyError, "Invalid url pattern for GitHub Repository."
    end

    _, @owner, @repo, @filepath = *match
  end

  def download_url
    "https://#{@github_token}@github.com/#{@owner}/#{@repo}/#{@filepath}"
  end

  private

  def _fetch(url:, resolved_url:)
    curl_download download_url, to: temporary_path
  end

  def set_github_token
    @github_token = ENV["HOMEBREW_GITHUB_API_TOKEN"]
    unless @github_token
      raise CurlDownloadStrategyError, "Environmental variable HOMEBREW_GITHUB_API_TOKEN is required."
    end

    validate_github_repository_access!
  end

  def validate_github_repository_access!
    # Test access to the repository
    GitHub.repository(@owner, @repo)
  rescue GitHub::HTTPNotFoundError
    # We only handle HTTPNotFoundError here,
    # becase AuthenticationFailedError is handled within util/github.
    message = <<~EOS
      HOMEBREW_GITHUB_API_TOKEN can not access the repository: #{@owner}/#{@repo}
      This token may not have permission to access the repository or the url of formula may be incorrect.
    EOS
    raise CurlDownloadStrategyError, message
  end
end

# GitHubPrivateRepositoryReleaseDownloadStrategy downloads tarballs from GitHub
# Release assets. To use it, add
# `:using => GitHubPrivateRepositoryReleaseDownloadStrategy` to the URL section of
# your formula. This download strategy uses GitHub access tokens (in the
# environment variables HOMEBREW_GITHUB_API_TOKEN) to sign the request.
class GitHubPrivateRepositoryReleaseDownloadStrategy < GitHubPrivateRepositoryDownloadStrategy
  def initialize(url, name, version, **meta)
    super
  end

  def parse_url_pattern
    url_pattern = %r{https://github.com/([^/]+)/([^/]+)/releases/download/([^/]+)/(\S+)}
    unless @url =~ url_pattern
      raise CurlDownloadStrategyError, "Invalid url pattern for GitHub Release."
    end

    _, @owner, @repo, @tag, @filename = *@url.match(url_pattern)
  end

  def download_url
    "https://#{@github_token}@api.github.com/repos/#{@owner}/#{@repo}/releases/assets/#{asset_id}"
  end

  private

  def _fetch(url:, resolved_url:)
    # HTTP request header `Accept: application/octet-stream` is required.
    # Without this, the GitHub API will respond with metadata, not binary.
    curl_download download_url, "--header", "Accept: application/octet-stream", to: temporary_path
  end

  def asset_id
    @asset_id ||= resolve_asset_id
  end

  def resolve_asset_id
    release_metadata = fetch_release_metadata
    assets = release_metadata["assets"].select { |a| a["name"] == @filename }
    raise CurlDownloadStrategyError, "Asset file not found." if assets.empty?

    assets.first["id"]
  end

  def fetch_release_metadata
    release_url = "https://api.github.com/repos/#{@owner}/#{@repo}/releases/tags/#{@tag}"
    GitHub.open_api(release_url)
  end
end

+1, appreciate it @mike-carey . I spent a couple hours going down this rathole troubleshooting and finally ended up here!

Hi lads, hope this will be helpful for you if you are still using this strategy:

TL;DR you will need to update your strategy to override

both resolve_url_basename_time_file_size and resolved_basename** NEW
to address all use cases (see next section):

class GitHubPrivateRepositoryDownloadStrategy < CurlDownloadStrategy
  require "utils/formatter"
  require "utils/github"

  # fix issue: https://github.com/Homebrew/brew/issues/15169
  # bypass a HEAD request that does NOT contains token, which will fail
  def resolve_url_basename_time_file_size(url, timeout: nil)
    url = download_url
    super
  end

  # [2023-10-14] brew relies on this output to rename the downloaded file
  # See: https://github.com/Homebrew/brew/blob/fbe50bf280bff033b968d439d5441d338afec98f/Library/Homebrew/download_strategy.rb#L305
  # Not setting this will break formulas during install stage, symtoms: Errno::ENOENT: No such file or directory - path/to/file
  def resolved_basename
    @filename
  end
# ... skipped

I think at this point everyone should have overridden their resolve_url_basename_time_file_size method to deal
with the forced HEAD request as mentioned at Homebrew/brew#15169.

However, please read this the definition of cached_location:
https://github.com/Homebrew/brew/blob/fbe50bf280bff033b968d439d5441d338afec98f/Library/Homebrew/download_strategy.rb#L305

In short, AFAIK the practice in Homebrew is all download strategy should download stuff to temporary_path. The dependency is as follows:
temporary_path -> resolved_basename -> resolved_url_and_basename -> resolve_url_basename_time_file_size (for CurlDownloadStrategy only)

From this hierarchy we can see that if we overridden the former it will break download path too.
Symtoms are:

(during call to InstallRenamed.install_p)

Errno::ENOENT: No such file or directory - path/to/file

Of course an alternative is to just override resolve_url_basename_time_file_size with:

# from https://github.com/Homebrew/brew/issues/15169#issuecomment-1500653530
def resolve_url_basename_time_file_size(url, timeout: nil)
  [download_url, "", Time.now, 0, false]
end

However this will be less robust to changes to Homebrew's code.

---

Affected use case

From what I know this issue seems to only affect formulas that just installs a file via the bin.install directive.
For archives, my guess is extraction will be carried out transparently first, hence the file names inside the archive will be preserved.
I have NOT drill into how this the exact process is tho, so please share if you did.

However, for consistency sake I will suggest everyone to make this change.
Changing should also guard against behaviour changes on Homebrew's side, say if they decide to suffix all extracted files for whatever reasons.