differentiate security updates

[tpo]

The intent of this patch is to be able to differentiate between 'security' and other updates.

Package updates which are security updates get a status of 'U' instead of status 'u'.

The patch is the counterpart to this apt-dater commit ...

DE-IBH/apt-dater@0c4438e

... which make apt-dater display security updates with a flag 'U' instead of a 'u'.

This patch should be understood as a proof of concept that works.

What is missing from this patch is:

• a bump of the protocol version
• updates to the protocol documentation

I'd very much appreciate the inclusion of this /feature/. It does not need to be implemented the way I did it. In particular, maybe using a 'U' for signaling a security update could be debated.

I'd ask to please comment on this patch. (Merging it is of course also very fine ;-)

This implements the feature requested here: DE-IBH/apt-dater#24

[liske]

Thanks for your work to add this feature. I dislike the usage of the U flag since it breaks the compatibility to apt-dater w/o support for the new feature.

One way to keep compatibility may be to expand the STATUS line in ADP by another column:

STATUS: ${Package}|${InstVersion}|${Status}|${StatusData}

If ${Status} is update (u) than ${StatusData} might contain additional data like repository source or a security-patch flag.

[tpo]

So I tentatively suggest that with security updates we'd have:

STATUS: ${Package}|${InstVersion}|${Status}|${StatusData}

where ${StatusData} would be s=0.4.2 (i.e. the version of the package that would be installed when doing a security update).

I say "tentatively" because I'll try to implement this suggestion and then see how it works out in practice.

OK?

[tpo]

In commit 50fb8a7 I've implemented:

• getting security updates via a second run of apt-get with the fitting command line parameters to apt-get
• passing the status of the security update in an additional StatusData column as "s=0.1.2.3" where "s" signals a security update and the numbers indicate the available security update version
• note that the solver of apt-get (and also the one in aptitude) can deliver any combination of u=$update_version and s=$security_update_version. The solver could f.ex:
  • indicate there is no $update_version (because of conflicts when updating) but there's a $security_update_version
  • indicate there is a $security_update_version higher than a possible $update_version (again due to update conflicts)
  • et cetera - any combination is realistically possible, depending on your set of apt sources that get pulled and state of dependencies
• this is only the apt-dater-host side, the apt-dater side is not done yet

@liske: comments appreciated

[tpo]

So I tentatively suggest that with security updates we'd have:

STATUS: ${Package}|${InstVersion}|${Status}|${StatusData}

where ${StatusData} would be s=0.4.2 (i.e. the version of the package that would be installed when doing a security update).

I say "tentatively" because I'll try to implement this suggestion and then see how it works out in practice.

OK?

[tpo]

I wrote:

So I tentatively suggest that with security updates we'd have:

STATUS: ${Package}|${InstVersion}|${Status}|${StatusData}

and

note that the solver of apt-get (and also the one in aptitude) can deliver any combination of u=$update_version and s=$security_update_version.

The $update_version and the $security_update_version of a package are correlated. But the relation between $update_version and $security_update_version is not a trivial one (but instead $security_update_version = f($current_state, $available_versions_of_all_packages, $resolver_settings, $user_preferences) ) I propose to not use field names that are related, but to use unrelated field names, to emphasize the relative independence of those two field. Thus I propose:

STATUS: ${Package}|${InstVersion}|${Status}|${SecurityUpdateVersion}

where ${SecurityUpdateVersion} can either be non-existent, i.e. :

STATUS: ${Package}|${InstVersion}|${Status}

or ${SecurityUpdateVersion} contains a string like this:

s=5.4.45-0+deb7u13

giving:

STATUS: ${Package}|${InstVersion}|${Status}|s=5.4.45-0+deb7u13

The client, apt-dater side of this has been implemented at commit sourcepole/apt-dater@7d71138.

Thoughts?

I've added an apt-dater-host/doc/ADP-0.9 spec document that contains this change (see c479430). It is debatable if the protocol version needs to increment for this change. The new field should be compatible with older apt-dater clients, however it's a protocol change nevertheless.

[liske]

Hi @tpo,

sorry I was very busy in the last time. Thanks for updating your PR. I don't think it is useful to have the version number twice for security updates (in ${Status} and ${SecurityUpdateVersion}). I would prefer to change the additional field into the repository source label. So it could look like:

STATUS: foo|0.1-1|u=0.3-1-bpo1|s=Debian-Backports

STATUS: bar|0.3-1|u=0.3-2|s=Debian-Security

In apt-dater we could use some regex to check if the source repository is security related.

I would prefer to update the ADP version to 0.9 for this feature. This allows to check if apt-dater-host may support this feature.

[tpo]

Hello @liske,

sorry I was very busy in the last time

No problem! Thanks a lot for taking the time to look at this PR.

Thanks for updating your PR. I don't think it is useful to have the version number twice for security updates (in ${Status} and ${SecurityUpdateVersion}).

The problem is, as I explained here: #25 (comment), that Status und SecurityUpdateVersion can differ. I.e. it is possible that the repositories offer you a version that contains a security update and a version that contains a "regular" update. How do you handle that?

I would prefer to change the additional field into the repository source label.

That is not sufficient. Ubuntu does not have a labeled repository for security updates. Debian does. In Ubuntu instead security updates come from a given "suite". They do not come from a given "suite" in Debian. I have not looked at how other Debian based distrubutions (say Mint) are managing/tagging security updates. In short, if you consider both Ubuntu and Debian then you can not decide whether an update is a security update solely based on the repository source label. (see https://github.com/sourcepole/apt-dater-host/blob/c47943020329d6db5f703cb760f401d9cd77bbee/dpkg/apt-dater-host#L231 and following lines).

So I think either finding out whether an update is a security update happens "at the source" (i.e. in apt-dater-host) or all data about pending updates is given to apt-dater and then apt-dater has to decide what is what - maybe without having enough knowledge to do so. What happens when you add third party repositories that differentiate between security and non-security updates? Are you going to teach apt-dater about that?

Therefore I think the decision if something is a security update or not, is required to happen in apt-dater-host and not in apt-dater.

[liske]

The problem is, as I explained here: #25 (comment), that Status und SecurityUpdateVersion can differ. I.e. it is possible that the repositories offer you a version that contains a security update and a version that contains a "regular" update. How do you handle that?

The most recent version wins. If the package which sould be installed is from a normal release than it is not security specific. It may contain security fixes... but this is true for all update software. If the package comes from a security update repository I would expect to get it marked as security related.

That is not sufficient. Ubuntu does not have a labeled repository for security updates. Debian does. In Ubuntu instead security updates come from a given "suite".

Since apt-dater uses an abstract approach managing software updates the ADP needs to hide any linux distribution or package manager specific details. I would expect that any existing package manager has the concept of naming the package sources (repository, suite, ...).

Therefore I think the decision if something is a security update or not, is required to happen in apt-dater-host and not in apt-dater.

ACK. So we might need two fields: one for the source and one for the package release type (security, generic update, normal package).

For apt-based distris we might combine the repository label and suite to build the source string. This would result in the following strings:

Ubuntu trusty-security
Ubuntu trusty
Ubuntu trusty-proposed
Ubuntu trusty-backports

Debian-Security stable
Debian stable
Debian proposed-updates
Debian Backports stretch-backports

The source string could be used in the apt-dater UI, the type value to highlight updates depending if it is security related or not.

[tpo]

Good day @liske,

The problem is, as I explained here: #25 (comment), that Status und SecurityUpdateVersion can differ. I.e. it is possible that the repositories offer you a version that contains a security update and a version that contains a "regular" update. How do you handle that?

The most recent version wins.

I assume you mean the higher version?

I think I could live with that. However, I guess I'd patch my own version, just to keep that feature. So I'd like to ask you whether there's a possibility to transport both the "security version" and the "update version" from apt-dater-host to apt-dater?

Let me explain.

If "flag a security update but only transport the highest version" would land in apt-dater/apt-dater-host, then I could work with that, however, I'd probably just log in over ssh to the respective host and do a security update only. Resorting back to ssh, possibly needing to automate that aspect and splitting the work into multiple places, would of course remove an important part of the utility, that makes apt-dater so attractive.

Why would I not want to update to the highest version? Because of policy! At my clients often enought the policy is: "only security updates". Other updates have a different cycle, that often enough includes testing and go along with other stuff - f.ex. custom application updates.

That means in my work environment, I don't really have a choice, if I want to keep on working for my clients. I am guessing, that you need to deal with similar policies at your workplace.

So I ask you to please rethink this particular problem. Is there a way/possibility to have the cake and eat it?

Therefore I think the decision if something is a security update or not, is required to happen in apt-dater-host and not in apt-dater.

ACK. So we might need two fields: one for the source and one for the package release type (security, generic update, normal package).

For apt-based distris we might combine the repository label and suite to build the source string. This would result in the following strings:

Ubuntu trusty-security
Ubuntu trusty
Ubuntu trusty-proposed
Ubuntu trusty-backports

Debian-Security stable
Debian stable
Debian proposed-updates
Debian Backports stretch-backports

The source string could be used in the apt-dater UI, the type value to highlight updates depending if it is security related or not.

I am not really understanding this. Are you intending to have the decoding of repository suite label and so on on the apt-dater side? As I argued, IMO the decoding of the semantics should be as close to the source as possible. Which means that apt-dater-host should look at the data and tell apt-dater "this is a normal update" and "this is a security update", give the respective version(s) and that's it.

I can see some value to do the decoding of label and suite on the apt-dater side, because then apt-dater can support more aspects of updates than just "security" and "regular".

But then again, what's the interface, what is the abstraction, the model, that you want to implement here? What are the concerns that apt-dater shoud be dealing with and what are the concerns that apt-dater-host should be dealing with? If you start transporting more of the package properties, that the native package management provides to apt-dater, then (this is a bit of hyperbole) why not just do a "ssh target apt-get dist-upgrade" from apt-dater and parse everything on the apt-dater side?

Are you sure you want to do all of this for the other packages managers on the apt-dater side as well (maybe SuSE does security updates differently then RH does and yet another way than Fedora?).

On a related note, my current code doesn't detection non-security updates in Ubuntu reliably. I need to debug that.