Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(veritech): Run simple Joi validations in Rust instead of JS #5573

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jkeiser
Copy link
Contributor

@jkeiser jkeiser commented Feb 28, 2025

This PR intercepts the simplest Joi validations and runs them in Rust instead of dispatching to Veritech:

  • [boolean/string/number].required/valid(a,b,c)/invalid(a,b,c)
  • number.integer/min/max/less/greater/unsafe
  • string.min/max/length

Any validation not supported on that list, or with any unexpected options, is dispatched to Veritech instead and runs and returns exactly as before.

We produce the same results, as well as the same error message, for all of these, so user experience should be identical.

Why

Once Clover added automated validations to lots of fields on many assets, Veritech is under heavy load because every time you drag out a new asset, many Joi validations are run. All of the Clover validations are very straightforward, however, and encompass a very small subset of what is possible with Joi.

How

When a user does .setValidationFormat(Joi.string().min(2).max(3).required() in an asset function, we internally call describe() on that Joi schema, which produces JSON that looks like this:

{
  "type": "string",
  "flags": { "presence": "required" },
  "rules": [
       { "name": "min", "args": { "limit": 2 } },
       { "name": "max", "args": { "limit": 3 } }
  ]
}

Because there is no longer any JS involved, we can read this format in any language we like and interpret it.

Testing

  • New tests: tons and tons of tests of the new validation library, including both valid and invalid values for all supported rules and flags.
  • Dragged out an EC2::Subnet
    • watched validations run locally instead of remotely
    • Validated that simple fields still go red when they don't match, and green when you fix them
  • Dragged out S3::AccessPoint, with BucketAccountId (a pattern field requiring twelve digits)
    • Watched validation dispatch to Veritech
    • Ensured that BucketAccountId goes red when you enter bad data
    • Watched BucketAccountId go green when entering 12 digits

Risk Mitigation

  • This is a drop-in replacement for parts of the library and does not increase or shrink product surface area, limiting the number of new interactions that can happen
  • The library uses strict serde features like deny_unknown_fields to ensure we only deserialize exactly things we support, and tests those few things heavily

Later

Arguably the goal is to run all Clover validations in Rust instead of JS. However, pattern() isn't supported as part of this turn. There is a Rust library we may pick up, but regexes are a large surface area to test, and we want to evaluate whether this has a discernible impact before we reach out for that.

- [boolean/string/number].required/valid(a,b,c)/invalid(a,b,c)
- number.integer/min/max/less/greater/unsafe
- string.min/max/length

Any validation not supported on that list, or with any unexpected
options, is sent to Veritech instead.
Copy link

Dependency Review

✅ No vulnerabilities or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails

Scanned Files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant