Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let journey tests fail without find and enforce LF line endings #1676

Merged
merged 3 commits into from
Nov 14, 2024

Conversation

EliahKagan
Copy link
Member

@EliahKagan EliahKagan commented Nov 14, 2024

f1a171b (#1674) included the conservative step of correcting the last remaining with_program tree to with_program find and no longer installing tree on CI. However, we can go further: any environment without find is unsuitable for running journey tests (and, more broadly, broken for general purpose usage). find is required by POSIX, is in practice present on all but the most minimal Unix-like systems (and installable on those), and is present in Git Bash.

with_program skips tests when the named tool is not found, so using it does not prevent bugs from being concealed. More importantly, all the assertions covered under with_progam find were expect_run_sh $SUCCESSFULLY assertions, so if find were absent, they would fail, revealing the problem. I think that it is better for journey tests that use find to fail, when run in strange environments that don't have find.

This removes all such calls in bc6e4be. As detailed in that commit message, even with scoping considerations in mind, the subshells introduced with with_find calls are no longer needed either, so this removes those and dedents. (Thus, the changes here are significantly less than a +/- line count would suggest.) These were also the only remaining calls to with_program, so 05e176c removes that function definition.

While beginning to work on this, I noticed that shell scripts other than those related to fixtures don't have LF line endings enforced in .gitattributes and would end up being checked out with CRLF line endings on Windows. This is undesirable even if bash on Windows typically tolerates it, as detailed in 1a3f4e3, which therefore simplifies that part of .gitattributes to force LF for all shell scripts.

This adjustes `.gitattributes` to normalize line endings to LF
(Unix style line endings) for all shell scripts in the project, not
just shell scripts that are fixture scripts. Previously, shell
scripts related to journey tests were not normalized this way.

This is simpler to write, and `bash` and POSIX shell scripts are
required to have LF line endings to be correct in general. On
Windows, some `bash` builds (including the MSYS2 `bash` provided as
part of the Git Bash environment) are patched to automatically
accept Windows-style line endings, but I believe they are still
considered technically incorrect. In practice, the effect of not
using LF line endings when working with shell scripts on Windows
may usually be where other tools are involved: ShellCheck, and at
least some language servers for `bash` script editing, complain if
CRLF line endings are used.
This removes all uses of `with_program`, which is no longer used
with any command other than `find`. The rationale is:

- `find` is a standard command. Minimal systems may omit it, but
  they omit numerous other software that is needed for building and
  testing gitoxide. `find` is standardized and required by POSIX
  and is present on all common general-purpose installations of
  Unix-like operating systems. It is also present in the Git Bash
  environment (and most other MSYS2 environments) that may be used
  for running the test suite on Windows.

- One purpose of `with_program` is to check for missing commands
  where, if not checked for, then tests might run but wrongly pass
  due to assertions that commands fail (where they should fail for
  a different reason). However, that does not seem to apply to any
  `with_program find`: all assertions with `find` in code to which
  that had been applied are using `expect_run_sh $SUCCESSFULLY`.

`with_program` is typically applied in a `(` `)` subshell, which
itself can have an important effect in isolating changes made
within it. In this case, however, the code was either still
enclosed equivalently, or appeared at the end of a larger `(` `)`
scope. So this removes `(` `)` subshells that only existed to
facilitate the deleted `with_program` uses and dedents.
This removes the journey test helper function `with_program`, which
now has zero uses.

(This is done in its own commit to make it easy to revert or
examine later if a use case for it ever arises in the future.)
Copy link
Member

@Byron Byron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are on fire! Thanks again, it's much appreciated.

@Byron Byron merged commit 66c222c into GitoxideLabs:main Nov 14, 2024
19 checks passed
@EliahKagan EliahKagan deleted the run-ci/journey-find branch November 14, 2024 18:02
EliahKagan added a commit to EliahKagan/gitoxide that referenced this pull request Nov 18, 2024
In c957ab8 (GitoxideLabs#1340), I intended to mark the contents of
`gix-packetline-blocking/src` as auto-generated for
github-linguist, but I did not do so correctly, because giving the
directory as a path does not specify the `linguist-generated`
attribute as being unset for the contents.

This fixes that. The bug, and fix, is revealed by comparing the
output of `github-linguist --breakdown` before and after this:

	--- a   2024-11-18 15:45:57.376285107 -0500
	+++ b   2024-11-18 15:46:44.808328857 -0500
	@@ -1,6 +1,6 @@
	-94.14%  Rust
	-4.58%   Shell
	-1.19%   HTML
	+94.08%  Rust
	+4.63%   Shell
	+1.20%   HTML
	 0.09%   Makefile

	 Makefile:
	@@ -682,23 +682,6 @@
	 gix-pack/tests/pack/multi_index/mod.rs
	 gix-pack/tests/pack/multi_index/verify.rs
	 gix-pack/tests/pack/multi_index/write.rs
	-gix-packetline-blocking/src/decode.rs
	-gix-packetline-blocking/src/encode/async_io.rs
	-gix-packetline-blocking/src/encode/blocking_io.rs
	-gix-packetline-blocking/src/encode/mod.rs
	-gix-packetline-blocking/src/lib.rs
	-gix-packetline-blocking/src/line/async_io.rs
	-gix-packetline-blocking/src/line/blocking_io.rs
	-gix-packetline-blocking/src/line/mod.rs
	-gix-packetline-blocking/src/read/async_io.rs
	-gix-packetline-blocking/src/read/blocking_io.rs
	-gix-packetline-blocking/src/read/mod.rs
	-gix-packetline-blocking/src/read/sidebands/async_io.rs
	-gix-packetline-blocking/src/read/sidebands/blocking_io.rs
	-gix-packetline-blocking/src/read/sidebands/mod.rs
	-gix-packetline-blocking/src/write/async_io.rs
	-gix-packetline-blocking/src/write/blocking_io.rs
	-gix-packetline-blocking/src/write/mod.rs
	 gix-packetline/src/decode.rs
	 gix-packetline/src/encode/async_io.rs
	 gix-packetline/src/encode/blocking_io.rs

(The test was done on Ubuntu 18.04 LTS ESM with `github-linguist`
provided by the `ruby-github-linguist` package, version 5.3.3-1.)

In addition to fixing that, this also simplifies `.gitattributes`
files throughout the repository. This simplification includes
removing redundant `clrf=input` (discussed in GitoxideLabs#1676) and a
spurious unrecognized `-eof` (see GitoxideLabs#1689), changing lone `**` to `*`
where equivalent, and some other changes.

This also makes the style of `.gitattributes` files more consistent,
using `foo`/`-foo` rather than `foo=true`/`foo=false` everywhere
even where both work, and using whitespace more consistently.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants