Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI tool: recursive derives and attributes #1379

Merged
merged 70 commits into from
Jan 22, 2024
Merged
Changes from 68 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
38f3a16
restructure cli commands
tadeohepperle Nov 20, 2023
71e0ed7
config: Add `SkipCheckIfFeeless` signed extension (#1264)
lexnv Nov 16, 2023
6980c2f
subxt: Replace removed lint (#1270)
lexnv Nov 16, 2023
156dd84
lightclient: Add support for multi-chain usecase (#1238)
lexnv Nov 16, 2023
4ae6ffd
Bump futures from 0.3.28 to 0.3.29 (#1272)
dependabot[bot] Nov 20, 2023
62ea604
Bump zeroize from 1.6.0 to 1.7.0 (#1274)
dependabot[bot] Nov 20, 2023
e09a025
Bump tracing-subscriber from 0.3.17 to 0.3.18 (#1275)
dependabot[bot] Nov 20, 2023
0c28a7e
Bump tracing-subscriber from 0.3.17 to 0.3.18 (#1275)
dependabot[bot] Nov 20, 2023
14420b8
Bump getrandom from 0.2.10 to 0.2.11 (#1273)
dependabot[bot] Nov 20, 2023
871561c
impl RpcClientT for Arc<T> and Box<T> (#1277)
niklasad1 Nov 21, 2023
9f74bca
first iteration of using scale_typegen
tadeohepperle Nov 21, 2023
bbe8e4f
Merge branch 'master' into tadeohepperle/cli-support-runtime-apis
tadeohepperle Nov 21, 2023
5bf6c52
introduce indoc for formatting
tadeohepperle Nov 22, 2023
8052211
calls, constants and home are cleaner now
tadeohepperle Nov 22, 2023
5fd50ce
added event subcommand
tadeohepperle Nov 23, 2023
c5973a0
show runtime apis working
tadeohepperle Nov 23, 2023
ec8ac18
add better code formatting
tadeohepperle Nov 23, 2023
f6b8ddd
fix style
tadeohepperle Nov 23, 2023
af019a6
adjust tests, use owo_colorize to not add extra dependency
tadeohepperle Nov 24, 2023
b871375
fmt
tadeohepperle Nov 24, 2023
52b4013
adjust docs
tadeohepperle Nov 24, 2023
febd3a0
Merge branch 'master' into tadeohepperle/cli-support-runtime-apis
tadeohepperle Nov 24, 2023
cd3f660
move scale-typegen-description dependency to workspace
tadeohepperle Nov 29, 2023
a7835f6
improve `substrate-compat` (#1265)
yjhmelody Nov 24, 2023
6f17b5d
Bump proc-macro2 from 1.0.69 to 1.0.70 (#1292)
dependabot[bot] Nov 27, 2023
3b68cb3
Bump serde from 1.0.192 to 1.0.193 (#1291)
dependabot[bot] Nov 27, 2023
e36181c
lightclient: Fix wasm socket closure called after being dropped (#1289)
lexnv Nov 29, 2023
9040872
workflows: Install rustup component for building substrate (#1295)
lexnv Nov 29, 2023
6611a7e
cli: Command to fetch chainSpec and optimise its size (#1278)
lexnv Nov 29, 2023
16a67cd
conflict resolving
tadeohepperle Nov 29, 2023
afd1ce2
update to new scale-typegen interfaces
tadeohepperle Dec 8, 2023
36b7b8b
Merge branch 'master' into tadeohepperle/cli-support-runtime-apis
tadeohepperle Jan 2, 2024
6af9fa3
use released version of scale-typegen
tadeohepperle Jan 2, 2024
4f4d399
Merge branch 'master' into tadeohepperle/cli-support-runtime-apis
tadeohepperle Jan 4, 2024
e36e612
Merge branch 'master' into tadeohepperle/cli-support-runtime-apis
tadeohepperle Jan 4, 2024
b623e3c
remove unused debug file
tadeohepperle Jan 5, 2024
ecd252c
Merge branch 'master' into tadeohepperle/cli-support-runtime-apis
tadeohepperle Jan 8, 2024
f7c6699
Merge branch 'master' into tadeohepperle/cli-support-runtime-apis
tadeohepperle Jan 9, 2024
651a74b
resolve merge errors
tadeohepperle Jan 9, 2024
fc8b568
Merge branch 'master' into tadeohepperle/cli-support-runtime-apis
tadeohepperle Jan 12, 2024
221c9e1
adjustments
tadeohepperle Jan 12, 2024
6a8b345
constants file adjustment
tadeohepperle Jan 12, 2024
04c006e
Merge branch 'master' into tadeohepperle/cli-support-runtime-apis
tadeohepperle Jan 12, 2024
b6f2423
Merge branch 'master' into tadeohepperle/cli-support-runtime-apis
jsdw Jan 17, 2024
0b7704a
method renaming
tadeohepperle Jan 16, 2024
0a08a8c
fix issue with encoding runtime api params
tadeohepperle Jan 18, 2024
4223852
Add logging to submit_transaction and unstable driver, and ensure unp…
jsdw Jan 17, 2024
7f4a110
panic if None returned from subscription too, also with stats
jsdw Jan 17, 2024
e8f29bb
change panic to Err just to be on the safe side
jsdw Jan 17, 2024
1fcd3b0
clippy
jsdw Jan 17, 2024
c7b517d
make long tests run only after clippy + fmt pass
jsdw Jan 17, 2024
d3777fa
megre in light client test change pr
jsdw Jan 17, 2024
d95c8eb
chore(subxt/src): typo fix (#1370)
Pan-chao Jan 17, 2024
e39acda
Weekly Cronjob fetching artifacts and generating polkadot.rs file. (#…
tadeohepperle Jan 17, 2024
8c9e1e8
lightclient(fix): Ensure lightclient chainSpec is at least one block …
lexnv Jan 17, 2024
d6d258f
ci: Reduce the light client timeout to 15 minutes (#1373)
lexnv Jan 17, 2024
26607f1
actually only wait for machete+fmt, clippy can be much slower
jsdw Jan 17, 2024
fce3ef7
update CI file from Alex PR
jsdw Jan 18, 2024
240cf26
resolve clippy err
jsdw Jan 18, 2024
1c37385
Try a few RPC nodes in case one of them is not working
jsdw Jan 18, 2024
833beb2
fix submit_transaction debug logging of message
jsdw Jan 18, 2024
f29fbed
Improve Signed Extension and Block Decoding Examples/Book (#1357)
tadeohepperle Jan 18, 2024
9e9dcfa
fix formatting of returned sections
tadeohepperle Jan 18, 2024
d9570df
Merge branch 'master' into tadeohepperle/cli-support-runtime-apis
tadeohepperle Jan 18, 2024
e12dabc
Merge branch 'master' into tadeohepperle/cli-support-runtime-apis
tadeohepperle Jan 19, 2024
4e64857
add recursive derive and attribute options in the cli
tadeohepperle Jan 19, 2024
b76a0bc
format tuples uncaught
tadeohepperle Jan 19, 2024
f28d3fa
Merge branch 'master' into tadeo-hepperle-cli-recursive-derives
tadeohepperle Jan 19, 2024
f28cd05
add tests and rename type map parser
tadeohepperle Jan 22, 2024
5b45843
make parsing more strict
tadeohepperle Jan 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 60 additions & 24 deletions cli/src/commands/codegen.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,16 @@ pub struct Opts {
attributes: Vec<String>,
/// Additional derives for a given type.
///
/// Example `--derive-for-type my_module::my_type=serde::Serialize`.
/// Example 1: `--derive-for-type my_module::my_type=serde::Serialize`.
/// Example 2: `--derive-for-type my_module::my_type=serde::Serialize,recursive`.
#[clap(long = "derive-for-type", value_parser = derive_for_type_parser)]
derives_for_type: Vec<(String, String)>,
derives_for_type: Vec<DeriveForType>,
/// Additional attributes for a given type.
///
/// Example `--attributes-for-type my_module::my_type=#[allow(clippy::all)]`.
/// Example 1: `--attributes-for-type my_module::my_type=#[allow(clippy::all)]`.
/// Example 2: `--attributes-for-type my_module::my_type=#[allow(clippy::all)],recursive`.
#[clap(long = "attributes-for-type", value_parser = attributes_for_type_parser)]
attributes_for_type: Vec<(String, String)>,
attributes_for_type: Vec<AttributeForType>,
/// Substitute a type for another.
///
/// Example `--substitute-type sp_runtime::MultiAddress<A,B>=subxt::utils::Static<::sp_runtime::MultiAddress<A,B>>`
Expand Down Expand Up @@ -67,20 +69,52 @@ pub struct Opts {
allow_insecure: bool,
}

fn derive_for_type_parser(src: &str) -> Result<(String, String), String> {
let (ty, derive) = src
.split_once('=')
.ok_or_else(|| String::from("Invalid pattern for `derive-for-type`. It should be `type=derive`, like `my_type=serde::Serialize`"))?;
#[derive(Debug, Clone)]
struct DeriveForType {
type_path: String,
trait_path: String,
recursive: bool,
}

Ok((ty.to_string(), derive.to_string()))
#[derive(Debug, Clone)]
struct AttributeForType {
type_path: String,
attribute: String,
recursive: bool,
}

fn attributes_for_type_parser(src: &str) -> Result<(String, String), String> {
let (ty, attribute) = src
.split_once('=')
.ok_or_else(|| String::from("Invalid pattern for `attribute-type`. It should be `type=attribute`, like `my_type=serde::#[allow(clippy::all)]`"))?;
fn derive_for_type_parser(src: &str) -> Result<DeriveForType, String> {
let (type_path, trait_path, recursive) = str_equals_str_comma_maybe_recursive_parser(src)
.ok_or_else(|| String::from("Invalid pattern for `derive-for-type`. It should be `type=derive` or `type=derive,recursive`, like `my_type=serde::Serialize` or `my_type=serde::Serialize,recursive`"))?;
Ok(DeriveForType {
type_path,
trait_path,
recursive,
})
}

fn attributes_for_type_parser(src: &str) -> Result<AttributeForType, String> {
let (type_path, attribute, recursive) = str_equals_str_comma_maybe_recursive_parser(src)
.ok_or_else(|| String::from("Invalid pattern for `attributes-for-type`. It should be `type=attribute` like `my_type=serde::#[allow(clippy::all)]` or `type=attribute,recursive` like `my_type=serde::#[allow(clippy::all)], recursive`"))?;
Ok(AttributeForType {
type_path,
attribute,
recursive,
})
}

/// Parses a `&str` of the form `str1=str2` into `(str1, str2, false)` or `str1=str2,recursive` into `(str1, str2, true)`.
fn str_equals_str_comma_maybe_recursive_parser(src: &str) -> Option<(String, String, bool)> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could probably have a few unit-tests for this. Would str1=str2,a,b,c,recursive also be a valid syntax?

I'd also probably rename this to type_parser or something similar

let (str1, rest) = src.split_once('=')?;

let mut split_rest = rest.split(',');
let str2 = split_rest
.next()
.expect("split iter always returns at least one element; qed");

Ok((ty.to_string(), attribute.to_string()))
let rest_args = split_rest.collect::<Vec<_>>();
let recursive = rest_args.contains(&"recursive");
Some((str1.to_owned(), str2.to_owned(), recursive))
}

fn substitute_type_parser(src: &str) -> Result<(String, String), String> {
Expand Down Expand Up @@ -127,8 +161,8 @@ fn codegen(
metadata_bytes: &[u8],
raw_derives: Vec<String>,
raw_attributes: Vec<String>,
derives_for_type: Vec<(String, String)>,
attributes_for_type: Vec<(String, String)>,
derives_for_type: Vec<DeriveForType>,
attributes_for_type: Vec<AttributeForType>,
substitute_types: Vec<(String, String)>,
crate_path: Option<String>,
no_docs: bool,
Expand Down Expand Up @@ -168,13 +202,14 @@ fn codegen(
.map_err(|e| eyre!("Cannot parse global derives: {e}"))?;
codegen.set_additional_global_derives(global_derives);

for (ty_str, derive) in derives_for_type {
let ty: syn::TypePath = syn::parse_str(&ty_str)
for d in derives_for_type {
let ty_str = &d.type_path;
let ty: syn::TypePath = syn::parse_str(ty_str)
.map_err(|e| eyre!("Cannot parse derive for type {ty_str}: {e}"))?;
let derive = syn::parse_str(&derive)
let derive = syn::parse_str(&d.trait_path)
.map_err(|e| eyre!("Cannot parse derive for type {ty_str}: {e}"))?;
// Note: recursive derives and attributes not supported in the CLI => recursive: false
codegen.add_derives_for_type(ty, std::iter::once(derive), false);
codegen.add_derives_for_type(ty, std::iter::once(derive), d.recursive);
}

// Configure attribtues:
Expand All @@ -186,13 +221,14 @@ fn codegen(
.map_err(|e| eyre!("Cannot parse global attributes: {e}"))?;
codegen.set_additional_global_attributes(universal_attributes);

for (ty_str, attr) in attributes_for_type {
let ty = syn::parse_str(&ty_str)
for a in attributes_for_type {
let ty_str = &a.type_path;
let ty = syn::parse_str(ty_str)
.map_err(|e| eyre!("Cannot parse attribute for type {ty_str}: {e}"))?;
let attribute: OuterAttribute = syn::parse_str(&attr)
let attribute: OuterAttribute = syn::parse_str(&a.attribute)
.map_err(|e| eyre!("Cannot parse attribute for type {ty_str}: {e}"))?;
// Note: recursive derives and attributes not supported in the CLI => recursive: false
codegen.add_attributes_for_type(ty, std::iter::once(attribute.0), false);
codegen.add_attributes_for_type(ty, std::iter::once(attribute.0), a.recursive);
}

// Insert type substitutions:
Expand Down
Loading