Do not consider failure to sync fatal

[kalikiana]

We still log the error but we assume it is okay to fail.

See: https://progress.opensuse.org/issues/112871

[perlpunk]

Hm, but this would ignore any error in read_files.sh?
I don't have a good idea how to find out the cause, but ignoring all errors might hide real problems in the future

[perlpunk]

So read_files.sh is doing

rsync -4 --list-only $rsync_pwd_option dist.suse.de::repos/SUSE:/SLE-15-SP6:/GA:/Staging:/A/images/repo/
 | grep -P '...' | awk '...' | grep -v IGNOREPATTERN
 | grep -E 'Module-Basesystem|...' | grep -E '...'  >> .../files_repo.lst

and if one of the grep fails, the script fails.
Maybe there is a way to make only the grep commands non-fatal.

[perlpunk]

This should work:

rsync ... | (grep ... | grep ... >>logfile || true)

I tested it with a mocked rsync.
So if rsync has a problem, the script would still fail.

[okurz]

How would this look in reality? I would like to see two cases, one with error and one without. And what about the explicit log message we planned?

[perlpunk]

This is a line from a genererated read_files.sh:

rsync -4 --list-only $rsync_pwd_option dist.suse.de::repos/SUSE:/SLE-15-SP6:/Update:/BCI/images//*repo/SLE-BCI-**
 | grep -P 'Media1(.license)?$'
 | awk '{ $1=$2=$3=$4=""; print substr($0,5); } '
 | grep -v IGNOREPATTERN
 | grep -E '' 
| grep -E 'aarch64|ppc64le|s390x|x86_64'  >> /opt/openqa-trigger-from-ibs/SUSE:SLE-15-SP6:Update:BCI/files_repo_SLE-BCI-*.lst

And I would guess that some of the grep commands is failing, but we probably want to ignore the return value of grep here.

[kalikiana]

Discussed in jitsi. This may be the wrong part of the generator, see #238 (comment) - I will look into that after coming up with a fake rsync that produces successful or error output, and see how grep and awk respond to that.

[kalikiana]

So it seems the read_files_repo value in cfg.py:23 is the basis for that rsync call and the p function in scriptgen.py actually replaces the target file that is being piped into i.e. __Envsub/files_repo.lst

[Martchus]

I'm guess I still don't get the full picture of how this repo works but the tests look like we've discussed (and you found an even better way to mock the commands).