Skip to content

Commit 88feed5

Browse files
coadometa-codesync[bot]
authored andcommitted
Decrease amount of requests send to Maven repos (#57765)
Summary: The Ruby scripts that resolve prebuilt artifacts (ReactNativeCore, ReactNativeDependencies, hermes-engine) re-resolve their tarball URLs on every call site and every podspec evaluation during a single pod install. Each resolution re-issues the same requests to the Maven repositories, so identical requests are made many times per install. This PR adds a shared cache in `ReactNativePodsUtils` for RNCore/RNDeps and a self-contained one in `hermes-utils.rb`. ## Changelog: [IOS] [CHANGED] - Cache Maven repository requests (artifact existence probes, nightly metadata) during pod install to avoid re-issuing identical requests on every podspec evaluation Pull Request resolved: #57765 Test Plan: I've added logs locally and inspected number of cache hits and misses from the single `pod install` in rn-tester: ``` RCT_USE_PREBUILT_RNCORE=1 RCT_TESTONLY_RNCORE_VERSION=0.81.0 RCT_USE_RN_DEP=1 RCT_DEPS_VERSION=0.81.0 bundle exec pod install 2>&1 | tee /tmp/pod.log ``` ``` [Cache] [bench] MISS probe (requesting): https://repo.reactnative.dev/maven2/com/facebook/react/react-native-artifacts/0.81.0/react-native-artifacts-0.81.0-reactnative-dependencies-debug.tar.gz [Cache] [bench] HIT probe (request saved): https://repo.reactnative.dev/maven2/com/facebook/react/react-native-artifacts/0.81.0/react-native-artifacts-0.81.0-reactnative-dependencies-debug.tar.gz [Cache] [bench] HIT probe (request saved): https://repo.reactnative.dev/maven2/com/facebook/react/react-native-artifacts/0.81.0/react-native-artifacts-0.81.0-reactnative-dependencies-debug.tar.gz [Cache] [bench] MISS probe (requesting): https://repo.reactnative.dev/maven2/com/facebook/react/react-native-artifacts/0.81.0/react-native-artifacts-0.81.0-reactnative-dependencies-release.tar.gz [Cache] [bench] MISS probe (requesting): https://repo.reactnative.dev/maven2/com/facebook/react/react-native-artifacts/0.81.0/react-native-artifacts-0.81.0-reactnative-core-debug.tar.gz [Cache] [bench] HIT probe (request saved): https://repo.reactnative.dev/maven2/com/facebook/react/react-native-artifacts/0.81.0/react-native-artifacts-0.81.0-reactnative-core-debug.tar.gz [Cache] [bench] MISS probe (requesting): https://repo.reactnative.dev/maven2/com/facebook/react/react-native-artifacts/0.81.0/react-native-artifacts-0.81.0-reactnative-core-release.tar.gz [Cache] [bench] HIT probe (request saved): https://repo.reactnative.dev/maven2/com/facebook/react/react-native-artifacts/0.81.0/react-native-artifacts-0.81.0-reactnative-core-debug.tar.gz [Cache] [bench] HIT probe (request saved): https://repo.reactnative.dev/maven2/com/facebook/react/react-native-artifacts/0.81.0/react-native-artifacts-0.81.0-reactnative-dependencies-debug.tar.gz [Cache] [bench] HIT probe (request saved): https://repo.reactnative.dev/maven2/com/facebook/react/react-native-artifacts/0.81.0/react-native-artifacts-0.81.0-reactnative-dependencies-debug.tar.gz [Cache] [bench] HIT probe (request saved): https://repo.reactnative.dev/maven2/com/facebook/react/react-native-artifacts/0.81.0/react-native-artifacts-0.81.0-reactnative-dependencies-release.tar.gz [Hermes] [bench] MISS probe (requesting): https://repo.reactnative.dev/maven2/com/facebook/hermes/hermes-ios/260318099.0.1/hermes-ios-260318099.0.1-hermes-ios-debug.tar.gz [Hermes] [bench] HIT probe (request saved): https://repo.reactnative.dev/maven2/com/facebook/hermes/hermes-ios/260318099.0.1/hermes-ios-260318099.0.1-hermes-ios-debug.tar.gz [Hermes] [bench] HIT probe (request saved): https://repo.reactnative.dev/maven2/com/facebook/hermes/hermes-ios/260318099.0.1/hermes-ios-260318099.0.1-hermes-ios-debug.tar.gz [Hermes] [bench] MISS probe (requesting): https://repo.reactnative.dev/maven2/com/facebook/hermes/hermes-ios/260318099.0.1/hermes-ios-260318099.0.1-hermes-ios-release.tar.gz ``` Reviewed By: fabriziocucci Differential Revision: D114341998 Pulled By: coado fbshipit-source-id: 7bf66967c5b767487a3448625f61d90fea49ab6b
1 parent 3a95e0e commit 88feed5

4 files changed

Lines changed: 74 additions & 15 deletions

File tree

packages/react-native/scripts/cocoapods/rncore.rb

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -371,7 +371,7 @@ def self.nightly_tarball_url(version, configuration, dsyms = false)
371371
artefact_name = "reactnative-core-#{dsyms ? "dSYM-" : ""}#{configuration ? configuration : "debug"}.tar.gz"
372372
xml_url = "https://central.sonatype.com/repository/maven-snapshots/com/facebook/react/#{artefact_coordinate}/#{version}-SNAPSHOT/maven-metadata.xml"
373373

374-
response = Net::HTTP.get_response(URI(xml_url))
374+
response = ReactNativePodsUtils.memoized_get_response(xml_url)
375375
if response.is_a?(Net::HTTPSuccess)
376376
xml = REXML::Document.new(response.body)
377377
timestamp = xml.elements['metadata/versioning/snapshot/timestamp'].text
@@ -476,11 +476,11 @@ def self.artifacts_dir()
476476
return File.join(Pod::Config.instance.project_pods_root, "ReactNativeCore-artifacts")
477477
end
478478

479-
# This function checks that ReactNativeCore artifact exists on the maven repo
479+
# This function checks that ReactNativeCore artifact exists on the maven repo.
480+
# The probe is memoized, so repeated podspec evaluations in one `pod install`
481+
# don't re-request the same URL.
480482
def self.artifact_exists(tarball_url)
481-
# -L is used to follow redirects, useful for the nightlies
482-
# I also needed to wrap the url in quotes to avoid escaping & and ?.
483-
return (`curl -o /dev/null --silent -Iw '%{http_code}' -L "#{tarball_url}"` == "200")
483+
return ReactNativePodsUtils.artifact_exists?(tarball_url)
484484
end
485485

486486
def self.rncore_log(message, level = :info)

packages/react-native/scripts/cocoapods/rndependencies.rb

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -248,7 +248,7 @@ def self.nightly_tarball_url(version, build_type)
248248
artifact_name = "reactnative-dependencies-#{build_type.to_s}.tar.gz"
249249
xml_url = "https://central.sonatype.com/repository/maven-snapshots/com/facebook/react/#{artifact_coordinate}/#{version}-SNAPSHOT/maven-metadata.xml"
250250

251-
response = Net::HTTP.get_response(URI(xml_url))
251+
response = ReactNativePodsUtils.memoized_get_response(xml_url)
252252
if response.is_a?(Net::HTTPSuccess)
253253
xml = REXML::Document.new(response.body)
254254
timestamp = xml.elements['metadata/versioning/snapshot/timestamp'].text
@@ -378,11 +378,11 @@ def self.artifacts_dir()
378378
return File.join(Pod::Config.instance.project_pods_root, "ReactNativeDependencies-artifacts")
379379
end
380380

381-
# This function checks that ReactNativeDependencies artifact exists on the maven repo
381+
# This function checks that ReactNativeDependencies artifact exists on the maven repo.
382+
# The probe is memoized, so repeated podspec evaluations in one `pod install`
383+
# don't re-request the same URL.
382384
def self.artifact_exists(tarball_url)
383-
# -L is used to follow redirects, useful for the nightlies
384-
# I also needed to wrap the url in quotes to avoid escaping & and ?.
385-
return (`curl -o /dev/null --silent -Iw '%{http_code}' -L "#{tarball_url}"` == "200")
385+
return ReactNativePodsUtils.artifact_exists?(tarball_url)
386386
end
387387

388388
def self.rndeps_log(message, level = :info)

packages/react-native/scripts/cocoapods/utils.rb

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
require 'shellwords'
77
require 'digest'
88
require 'uri'
9+
require 'net/http'
910

1011
require_relative "./helpers.rb"
1112
require_relative "./jsengine.rb"
@@ -757,6 +758,47 @@ def self.resolve_use_frameworks(spec, header_mappings_dir: nil, module_name: nil
757758
end
758759
end
759760

761+
# ============================ #
762+
# Network request memoization #
763+
# ============================ #
764+
# CocoaPods evaluates the prebuilt podspecs several times during a single
765+
# `pod install`, and every evaluation re-resolves the artifact URLs from
766+
# scratch: existence probes against the mirror/Maven Central and nightly
767+
# metadata lookups. The answers should not change within one install, so
768+
# each request is issued at most once per process and then served from
769+
# these in-memory caches.
770+
@@artifact_exists_cache = {}
771+
@@get_response_cache = {}
772+
773+
# Memoized existence probe (HTTP HEAD) for a prebuilt artifact URL.
774+
# Only conclusive answers are cached. If curl never got an HTTP status
775+
# (DNS failure, no route, ...) the probe is left uncached so that a
776+
# transient hiccup doesn't permanently mark the artifact as missing.
777+
def self.artifact_exists?(tarball_url)
778+
unless @@artifact_exists_cache.key?(tarball_url)
779+
# -L is used to follow redirects, useful for the nightlies
780+
# The url is wrapped in quotes to avoid escaping & and ?.
781+
http_code = `curl -o /dev/null --silent -Iw '%{http_code}' -L "#{tarball_url}"`
782+
return false if !$?.success? || http_code == "000"
783+
@@artifact_exists_cache[tarball_url] = (http_code == "200")
784+
end
785+
return @@artifact_exists_cache[tarball_url]
786+
end
787+
788+
# Memoized HTTP GET for small metadata lookups (Maven snapshot metadata).
789+
# Returns the Net::HTTPResponse. Only successful responses are cached:
790+
# raised network errors propagate uncached, and non-2xx responses (a
791+
# transient 5xx, a 404) are returned without being stored, so a later
792+
# call within the same process can retry.
793+
def self.memoized_get_response(url)
794+
unless @@get_response_cache.key?(url)
795+
response = Net::HTTP.get_response(URI(url))
796+
return response unless response.is_a?(Net::HTTPSuccess)
797+
@@get_response_cache[url] = response
798+
end
799+
return @@get_response_cache[url]
800+
end
801+
760802
# ==================== #
761803
# Shared download cache #
762804
# ==================== #

packages/react-native/sdks/hermes-engine/hermes-utils.rb

Lines changed: 22 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,14 @@
1010
MAVEN_CENTRAL_REPOSITORY = "https://repo1.maven.org/maven2"
1111
REACT_NATIVE_MAVEN_MIRROR_REPOSITORY = "https://repo.reactnative.dev/maven2"
1212

13+
# Memoized results of requests to the Maven repositories (mirror or central).
14+
# hermes-engine.podspec is evaluated several times during a single
15+
# `pod install`, and every evaluation re-resolves the artifact source from
16+
# scratch; without memoization that re-issues identical artifact existence
17+
# probes. The answers should not change within one install, so each request
18+
# is issued at most once per process.
19+
HERMES_ARTIFACT_EXISTS_CACHE = {}
20+
1321
module HermesEngineSourceType
1422
LOCAL_PREBUILT_TARBALL = :local_prebuilt_tarball
1523
DOWNLOAD_PREBUILD_RELEASE_TARBALL = :download_prebuild_release_tarball
@@ -339,14 +347,23 @@ def resolve_url_redirects(url)
339347

340348
# This function checks that Hermes artifact exists.
341349
# As of now it should check it on the Maven repo.
350+
# The probe is memoized, so repeated podspec evaluations in one `pod install`
351+
# don't re-request the same URL. Only conclusive answers are cached: if curl
352+
# never got an HTTP status (DNS failure, no route, ...) the probe is left
353+
# uncached so a transient hiccup doesn't permanently mark the artifact as
354+
# missing.
342355
#
343356
# Parameters
344-
# - version: the version of React Native
345-
# - build_type: debug or release
357+
# - tarball_url: the URL of the Hermes artifact to probe
346358
def hermes_artifact_exists(tarball_url)
347-
# -L is used to follow redirects, useful for the nightlies
348-
# I also needed to wrap the url in quotes to avoid escaping & and ?.
349-
return (`curl -o /dev/null --silent -Iw '%{http_code}' -L "#{tarball_url}"` == "200")
359+
unless HERMES_ARTIFACT_EXISTS_CACHE.key?(tarball_url)
360+
# -L is used to follow redirects, useful for the nightlies
361+
# I also needed to wrap the url in quotes to avoid escaping & and ?.
362+
http_code = `curl -o /dev/null --silent -Iw '%{http_code}' -L "#{tarball_url}"`
363+
return false if !$?.success? || http_code == "000"
364+
HERMES_ARTIFACT_EXISTS_CACHE[tarball_url] = (http_code == "200")
365+
end
366+
return HERMES_ARTIFACT_EXISTS_CACHE[tarball_url]
350367
end
351368

352369
def hermes_log(message, level = :warning)

0 commit comments

Comments
 (0)