Ability to choose workdir for temporary storage

[raybooysen]

The Archive Warrior running writes gigabytes of data before uploading. I would love a way to specify via env variable the location of the temporary storage to a file system or storage device that I'd prefer

[TheTechRobo]

Assuming you're using docker, you can use its -v parameter to bridge the current workdir to wherever you want on the host system.

[raybooysen]

I've been trying with some variants. /home/warrior, /home/warrior/data, /home/warrior/projects. All cause various amounts of errors.

Unsure which I should be using

[viniciushsantana]

I've been considering this issue as well. It seems that the working directories /home/warrior/projects and /home/warrior/data are being utilized in a manner that prevents exposing them as volumes, due to the absence of certain necessary files.

For example, within /home/warrior/data, the binaries wget-at and wget-at-gnutls should be present:

warrior-dockerfile/Dockerfile

Lines 16 to 17 in 86f9433

	&& ln -fs /usr/local/bin/wget-lua /home/warrior/data/wget-at
	COPY --from=atdr.meo.ws/archiveteam/grab-base:gnutls /usr/local/bin/wget-lua /home/warrior/data/wget-at-gnutls

I've discovered a potential workaround for this issue by altering Docker's data-root setting. However, this approach is not ideal and could lead to other complications.

Properly managing working directories and enabling the exposure of volumes would also facilitate the use of tmpfs mounts. This is particularly beneficial for users with ample RAM available who wish to conserve some SSD IOPS.

[raybooysen]

This was my primary usecase. My warriors run for long periods on machines with spare RAM, a tmpfs is a good use case here so avoid the SSD completely for temporal data.

[budde96]

I'm using Podman, you can mount /home/warrior/projects and /home/warrior/data/projects fine and that seems to cover the files you want for persistence and all the big temp files from what I can tell with my tests.

[ericblade]

Agree with others, attempting to use a volume for /home/warrior/data obscures the files that the image puts in there. attempting to use a volume for /home/warrior/projects breaks download/installation of a project with no obvious error, just "installation failed". Attempting to use a volume for a project itself results in a git error about not wanting to traverse across file system boundaries.

I keep my docker data on a "small" 2tb disk, and all my cold storage is elsewhere, so I'd need to restructure how this works to be able to stick a volume in here. Might have to do work on the software itself to get it to be compatible.

now i'm trying /home/warrior/data/projects .. that might work, at least it appears to be more successful than the others, in that it's trying to install the project. Still not being able to persist much of the work directory is not fun.

... that does seem to work after waiting many minutes for it to startup. all the other options i tried failed. at least it gets some of the data off my working disk