Attempting to fix publishing (#9140)

* mark template and utils as non-publish

* switch to development version for testing

* activate unleash check

* maybe if I disable all rules...

* Fix isolated compilation of `max-encoded-len-derive` with `syn`

error[E0369]: binary operation `==` cannot be applied to type `syn::Path`
  --> src/lib.rs:88:29
   |
88 |             .filter(|attr| attr.path == parse_quote!(max_encoded_len_crate))
   |                            --------- ^^ ----------------------------------- _
   |                            |
   |                            syn::Path

error: aborting due to previous error

For more information about this error, try `rustc --explain E0369`.
Error: could not compile `max-encoded-len-derive`

* WIP: bump changes crates since v3 tag to next breaking

cargo unleash version bump-breaking --changed-since v3.0.0
cargo unleash version set-pre dev --changed-since v3.0.0

FIXME: Don't modify crates that are not yet released, e.g.
`max-encoded-len-derive`

* Update lockfile

* WIP: Bump sp-transaction-pool as well

* WIP: Bump sp-offchain as well

* WIP: Bump frame-system-rpc-runtime-api as well

* WIP: Bump sp-authority-discovery as well

* Manually deactivate dev-deps before `cargo unleash check`

Otherwise we run into `Cycle detected` error.

* Bump sp-consensus-slots

* Add missing Cargo.lock change

* Bump sp-consensus-vrf as well

* Bump sp-keyring as well

* Bump sp-consensus-pow as well

* Try to speed up the `unleash-check` job

Previously, the job took 106 minutes - let's see if explicitly
specifying a `CARGO_TARGET_DIR` will help

* fixup: Ensure the temp target dir exists for unleash check

* Bump pallet-transaction-payment-rpc-runtime-api as well

Needed for Polkadot

* Bump pallet-transaction-payment-rpc as well

Needed for Polkadot

* Try updating crates after patching in the Polkadot CI job

* Use another approach to update patched Substrate crates

* Try to update all sp-core versions in Polkadot CI job

* Simplify sp-core version checking

* Apply another shellcheck lint

* Just do the simplest thing I guess

* Welp don't do --offline then

* Clean up `unleash-check` job triggers

Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

* Fix a note in unleash-check cache step

* Add a note about temporary optimization in cargo-unleash

* Pin a newer version of cargo-unleash

Co-authored-by: Igor Matuszewski <xanewok@gmail.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>

Author: 3 people

.gitlab-ci.yml

@@ -40,7 +40,7 @@ variables:                         &default-vars
   ARCH:                            "x86_64"
   CI_IMAGE:                        "paritytech/ci-linux:production"
   # FIXME set to release
-  CARGO_UNLEASH_INSTALL_PARAMS:    "--version 1.0.0-alpha.11"
+  CARGO_UNLEASH_INSTALL_PARAMS:    "--version 1.0.0-alpha.12"
   CARGO_UNLEASH_PKG_DEF:           "--skip node node-* pallet-template pallet-example pallet-example-* subkey chain-spec-builder"
 
 default:
@@ -326,13 +326,14 @@ test-linux-stable:                 &test-linux
 unleash-check:
   stage:                           test
   <<:                              *docker-env
-  rules:
-    - if: $CI_PIPELINE_SOURCE == "pipeline"
-      when: never
-    - if: $CI_COMMIT_REF_NAME == "master"
-    - if: $CI_COMMIT_REF_NAME =~ /^v[0-9]+\.[0-9]+.*$/              # i.e. v1.0, v2.1rc1
+  <<:                              *test-refs-no-trigger
   script:
     - cargo install cargo-unleash ${CARGO_UNLEASH_INSTALL_PARAMS}
+    - cargo unleash de-dev-deps
+    # Reuse build artifacts when running checks (cuts down check time by 3x)
+    # TODO: Implement this optimization in cargo-unleash rather than here
+    - mkdir -p target/unleash
+    - export CARGO_TARGET_DIR=target/unleash
     - cargo unleash check ${CARGO_UNLEASH_PKG_DEF}
 
 test-frame-examples-compile-to-wasm:

.maintain/gitlab/check_polkadot_companion_build.sh

@@ -88,5 +88,12 @@ fi
 # Patch all Substrate crates in Polkadot
 diener patch --crates-to-patch ../ --substrate --path Cargo.toml
 
+# We need to update specifically our patched Substrate crates so that other
+# crates that depend on them (e.g. Polkadot, BEEFY) use this unified version
+# NOTE: There's no way to only update patched crates, so we use a heuristic
+# of updating a crucial Substrate crate (`sp-core`) to minimize the impact of
+# updating unrelated dependencies
+cargo update -p sp-core
+
 # Test Polkadot pr or master branch with this Substrate commit.
 time cargo test --all --release --verbose