My two cents: this resembles k8s's kubelet / apiserver versioning. a change that can potentially be incompatable is a minor version bump. so say, 1.10 to 1.11. upgrading kubelet 1.10.1 to 1.10.2 usually doesn't break existing things. at times they have said you must evacuate a node going from 1.x to 1.x+1 though. Which is similar to the use case below. no way to maintain state. They grantee kubelet 1.x will work with up to apiserver 1.x+2 So, kata-runtime is similar to the kube-apiserver in this case and the rest is kubelet. the vm/agenty things should be able to lag a few versions but until implemented, must match. It should probably bump the minor version, as it really is an incompatible version change and not just a bug fix release as 1.0.1 would imply. so I'd recommend calling the next version 1.1.0 instead of 1.0.1. And note in the upgrade notes that the node must be drained before upgrading. People might not look for a release note if it was called 1.0.1 as they won't really expect a breaking change in that case. Thanks, Kevin ________________________________ From: Ernst, Eric [eric.ernst@intel.com] Sent: Thursday, June 21, 2018 3:59 PM To: Jon Olson; Boeuf, Sebastien Cc: kata-dev@lists.katacontainers.io Subject: Re: [kata-dev] What to do about release versioning ? I agree on this stance, Jon. Perhaps until we have the protocol/API defined and versioned this shouldn’t be necessary. Stepping back, I think it makes more sense to fix the bugs existing (which I think are quite troubling) and documenting expectations of tags/packages rather than holding off releases. I do acknowledge I often like to break the rules, and think this is such a scenario. Bottom line, for many reasons we need to very well define the protocol and version it (this is not done yet), and I expect there will be changes that continue in this space until we have the major use cases of Frakti/Huawei figured out. Eric From: Jon Olson via kata-dev <kata-dev@lists.katacontainers.io> Reply-To: Jon Olson <jonolson@google.com> Date: Thursday, June 21, 2018 at 12:06 PM To: "Boeuf, Sebastien" <sebastien.boeuf@intel.com> Cc: "kata-dev@lists.katacontainers.io" <kata-dev@lists.katacontainers.io> Subject: Re: [kata-dev] What to do about release versioning ? This came up in Vancouver, but I know not everyone had the opportunity to be there. Here were/are my thoughts drawing from my experience working on similar problems for GCE VMs: currently iiuc (until in-place-upgrade and downgrade of releases is supported) the runtime and the guest image (including agent) are all part of the "release artifacts". I would argue that the protocol between the runtime and the agent is not (yet) part of the external contract for Kata. It probably should be, but until that protocol is itself formally documented and versioned I would say you have only changed internal contracts between components that are part of the same atomic set of release artifacts. I do think it's valuable to make the agent protocol part of the external contract for Kata, but I think that requires the protocol itself be explicitly versioned[0]. Jon [0]: I also still have an AI to put together a slide deck on gRPC API evolution best practices to minimize breaking changes generally. I should probably go do that. On Thu, Jun 21, 2018 at 11:54 AM Boeuf, Sebastien <sebastien.boeuf@intel.com<mailto:sebastien.boeuf@intel.com>> wrote: Hi folks, This morning we wanted to release 1.0.1 but held on because of some discussions about the semver semantic not being followed with this 1.0.1 version. The problem here, we have some "sort of" backward compatibility breakage between the runtime and the agent. And for any non compatible release, semver suggest that the new release should bump to the next X version (based on X.Y.Z notation). But before we can answer this, we have to figure out what is that we consider a "breakage". Let me try to explain the different possibility here, starting from an environment where you installed Kata 1.0.0 from packages: First, let's say we introduced a new PauseContainer() command to our agent protocol, this command will have one commit on the agent repo, updating the gRPC protocol and implementing the new feature. This will be followed by a new patch on the runtime to use this new feature, which means we'll have to revendor the new protocol and implement the runtime to rely on it. Now, let's say we bump both runtime and agent repos including those patches to the version 1.0.1, here is the issue we will run into: - The user with Kata 1.0.0 already installed on his system might update the kata-runtime package without updating the VM image containing the agent. And from his perspective, this is fine because he knows about semver semantics and thinks that a kata-runtime 1.0.1 might still work with the VM image container a kata-agent 1.0.0. Unfortunately, this won't work because the kata-runtime will try to call into the new command PauseContainer() that had been introduced, but the agent will have no idea about what this is, and the gRPC will return an error saying this is not a supported/known command. I have a second example, sorry this email is getting long but we have to talk about all cases here... Let's get back to 1.0.0 and say that some changes on both kata-runtime and kata-agent are modifying an existing command CreateContainer(). In this case, this is a more logical breakage since moving one or the other component to the newest version without bumping the other (either kata-runtime 1.0.0 and kata-agent 1.0.1, or kata-runtime 1.0.1 and kata-agent 1.0.0), will end up in functional issues. The second example is what I would call an obvious backward compatibility breakage, but the first one looks still valid to be taken into account. So now, to summarize, we have to agree on what we call a non compatible version that will be translated into a bump to the next X version. And I can see happening a lot of breakage in the future if we include both cases in what is considered as a "breakage", meaning we might end up on version 100.y.z pretty quickly. Numbers are cheap, but here is the status and I/we need some feedback before we can push the pending release (as we don't want to introduce a wrong version number in the history). Thanks, Sebastien _______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io<mailto:kata-dev@lists.katacontainers.io> http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev