rust kata-agent status update
Hi, folks: Let me update some progress of the rust agent. First of all, thanks for James' kindly help on the project. The functions of rust agent are almost finished. we have fixed several bugs to stablize it. Ci and osbuilder are still missing. We have to run end-to-end tests manually right now. - Refactor rust agent API. Remove overused matches, clean up code - Refactor communication between parent/child/grandchild, use raw bytes instead of serde_json to simplify communication - Refactor Process struct, remove duplicate/unused fields - Add Makefile to simplify building --- by jodh-intel - Fix up unit test for netlink - Fix up test run against musl and release - Implement debug console support - Implement command line parse partly - Implement Spec valaidator - Implement Hook infrastructure We are targeting to land the rust agent to Kata org by October, and you may check out the current WIP repo here: https://github.com/alipay/kata-rust-agent Cheers!
Hi, A couple of questions: 1) Do we know what the grps-rs memory footprint is? 2) What was the rationale behind picking grps-rs over e.g. grpcio? Cheers, Samuel. On Thu, Sep 12, 2019 at 04:53:29PM +0800, 杨波 via kata-dev wrote:
Hi, folks: Let me update some progress of the rust agent. First of all, thanks for James' kindly help on the project. The functions of rust agent are almost finished. we have fixed several bugs to stablize it. Ci and osbuilder are still missing. We have to run end-to-end tests manually right now.
- Refactor rust agent API. Remove overused matches, clean up code - Refactor communication between parent/child/grandchild, use raw bytes instead of serde_json to simplify communication - Refactor Process struct, remove duplicate/unused fields - Add Makefile to simplify building --- by jodh-intel - Fix up unit test for netlink - Fix up test run against musl and release - Implement debug console support - Implement command line parse partly - Implement Spec valaidator - Implement Hook infrastructure
We are targeting to land the rust agent to Kata org by October, and you may check out the current WIP repo here: https://github.com/alipay/kata-rust-agent
Cheers!
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
On Thu, Sep 12, 2019 at 02:42:57PM +0200, Samuel Ortiz wrote:
Hi,
A couple of questions:
1) Do we know what the grps-rs memory footprint is? 2) What was the rationale behind picking grps-rs over e.g. grpcio? Nevermind, it seems grpcio is the crate name for the grpc-rs project...
Cheers, Samuel.
footprint: RssAnon: 1700 kB RssFile: 1632 kB We choose grpc-rs because it is verified in product environment, grpcio is not. ------------------------------------------------------------------ 发件人：Samuel Ortiz <sameo@linux.intel.com> 发送时间：2019年9月12日(星期四) 20:44 收件人：杨波(兵卫) <yb203166@antfin.com> 抄 送：kata-dev <kata-dev@lists.katacontainers.io>; 王旭(循环) <gnawux@alibaba-inc.com> 主 题：Re: [kata-dev] rust kata-agent status update Hi, A couple of questions: 1) Do we know what the grps-rs memory footprint is? 2) What was the rationale behind picking grps-rs over e.g. grpcio? Cheers, Samuel. On Thu, Sep 12, 2019 at 04:53:29PM +0800, 杨波 via kata-dev wrote:
Hi, folks: Let me update some progress of the rust agent. First of all, thanks for James' kindly help on the project. The functions of rust agent are almost finished. we have fixed several bugs to stablize it. Ci and osbuilder are still missing. We have to run end-to-end tests manually right now.
- Refactor rust agent API. Remove overused matches, clean up code - Refactor communication between parent/child/grandchild, use raw bytes instead of serde_json to simplify communication - Refactor Process struct, remove duplicate/unused fields - Add Makefile to simplify building --- by jodh-intel - Fix up unit test for netlink - Fix up test run against musl and release - Implement debug console support - Implement command line parse partly - Implement Spec valaidator - Implement Hook infrastructure
We are targeting to land the rust agent to Kata org by October, and you may check out the current WIP repo here: https://github.com/alipay/kata-rust-agent
Cheers!
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
Hi, I love the footprint drop due to using Rust :) Instead of giving only the numbers from the Rust agent, could you show the difference between the Golang agent and Rust agent? Thanks, Sebastien ________________________________ From: 杨波 via kata-dev [kata-dev@lists.katacontainers.io] Sent: Sunday, September 15, 2019 6:52 PM To: Samuel Ortiz Cc: kata-dev Subject: [kata-dev] 回复： rust kata-agent status update footprint: RssAnon: 1700 kB RssFile: 1632 kB We choose grpc-rs because it is verified in product environment, grpcio is not. ------------------------------------------------------------------ 发件人：Samuel Ortiz <sameo@linux.intel.com> 发送时间：2019年9月12日(星期四) 20:44 收件人：杨波(兵卫) <yb203166@antfin.com> 抄 送：kata-dev <kata-dev@lists.katacontainers.io>; 王旭(循环) <gnawux@alibaba-inc.com> 主 题：Re: [kata-dev] rust kata-agent status update Hi, A couple of questions: 1) Do we know what the grps-rs memory footprint is? 2) What was the rationale behind picking grps-rs over e.g. grpcio? Cheers, Samuel. On Thu, Sep 12, 2019 at 04:53:29PM +0800, 杨波 via kata-dev wrote:
Hi, folks: Let me update some progress of the rust agent. First of all, thanks for James' kindly help on the project. The functions of rust agent are almost finished. we have fixed several bugs to stablize it. Ci and osbuilder are still missing. We have to run end-to-end tests manually right now.
- Refactor rust agent API. Remove overused matches, clean up code - Refactor communication between parent/child/grandchild, use raw bytes instead of serde_json to simplify communication - Refactor Process struct, remove duplicate/unused fields - Add Makefile to simplify building --- by jodh-intel - Fix up unit test for netlink - Fix up test run against musl and release - Implement debug console support - Implement command line parse partly - Implement Spec valaidator - Implement Hook infrastructure
We are targeting to land the rust agent to Kata org by October, and you may check out the current WIP repo here: https://github.com/alipay/kata-rust-agent
Cheers!
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
go-agent: about 11M rust-agent: about 1.1M It is also in README.md at here: https://github.com/alipay/kata-rust-agent/blob/master/README.md ------------------------------------------------------------------ 发件人：Boeuf, Sebastien <sebastien.boeuf@intel.com> 发送时间：2019年9月19日(星期四) 05:28 收件人：杨波(兵卫) <yb203166@antfin.com>; Samuel Ortiz <sameo@linux.intel.com> 抄 送：kata-dev <kata-dev@lists.katacontainers.io> 主 题：RE: [kata-dev] 回复： rust kata-agent status update Hi, I love the footprint drop due to using Rust :) Instead of giving only the numbers from the Rust agent, could you show the difference between the Golang agent and Rust agent? Thanks, Sebastien From: 杨波 via kata-dev [kata-dev@lists.katacontainers.io] Sent: Sunday, September 15, 2019 6:52 PM To: Samuel Ortiz Cc: kata-dev Subject: [kata-dev] 回复： rust kata-agent status update footprint: RssAnon: 1700 kB RssFile: 1632 kB We choose grpc-rs because it is verified in product environment, grpcio is not. ------------------------------------------------------------------ 发件人：Samuel Ortiz <sameo@linux.intel.com> 发送时间：2019年9月12日(星期四) 20:44 收件人：杨波(兵卫) <yb203166@antfin.com> 抄 送：kata-dev <kata-dev@lists.katacontainers.io>; 王旭(循环) <gnawux@alibaba-inc.com> 主 题：Re: [kata-dev] rust kata-agent status update Hi, A couple of questions: 1) Do we know what the grps-rs memory footprint is? 2) What was the rationale behind picking grps-rs over e.g. grpcio? Cheers, Samuel. On Thu, Sep 12, 2019 at 04:53:29PM +0800, 杨波 via kata-dev wrote:
Hi, folks: Let me update some progress of the rust agent. First of all, thanks for James' kindly help on the project. The functions of rust agent are almost finished. we have fixed several bugs to stablize it. Ci and osbuilder are still missing. We have to run end-to-end tests manually right now.
- Refactor rust agent API. Remove overused matches, clean up code - Refactor communication between parent/child/grandchild, use raw bytes instead of serde_json to simplify communication - Refactor Process struct, remove duplicate/unused fields - Add Makefile to simplify building --- by jodh-intel - Fix up unit test for netlink - Fix up test run against musl and release - Implement debug console support - Implement command line parse partly - Implement Spec valaidator - Implement Hook infrastructure
We are targeting to land the rust agent to Kata org by October, and you may check out the current WIP repo here: https://github.com/alipay/kata-rust-agent
Cheers!
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
* 杨波 via kata-dev (kata-dev@lists.katacontainers.io) wrote:
go-agent: about 11M rust-agent: about 1.1M
Do you understand why? That's a big difference - I wonder where the extra memory goes? Dave
It is also in README.md at here: https://github.com/alipay/kata-rust-agent/blob/master/README.md
------------------------------------------------------------------ 发件人：Boeuf, Sebastien <sebastien.boeuf@intel.com> 发送时间：2019年9月19日(星期四) 05:28 收件人：杨波(兵卫) <yb203166@antfin.com>; Samuel Ortiz <sameo@linux.intel.com> 抄 送：kata-dev <kata-dev@lists.katacontainers.io> 主 题：RE: [kata-dev] 回复： rust kata-agent status update
Hi,
I love the footprint drop due to using Rust :) Instead of giving only the numbers from the Rust agent, could you show the difference between the Golang agent and Rust agent?
Thanks, Sebastien
From: 杨波 via kata-dev [kata-dev@lists.katacontainers.io] Sent: Sunday, September 15, 2019 6:52 PM To: Samuel Ortiz Cc: kata-dev Subject: [kata-dev] 回复： rust kata-agent status update
footprint: RssAnon: 1700 kB RssFile: 1632 kB
We choose grpc-rs because it is verified in product environment, grpcio is not.
------------------------------------------------------------------ 发件人：Samuel Ortiz <sameo@linux.intel.com> 发送时间：2019年9月12日(星期四) 20:44 收件人：杨波(兵卫) <yb203166@antfin.com> 抄 送：kata-dev <kata-dev@lists.katacontainers.io>; 王旭(循环) <gnawux@alibaba-inc.com> 主 题：Re: [kata-dev] rust kata-agent status update
Hi,
A couple of questions:
1) Do we know what the grps-rs memory footprint is? 2) What was the rationale behind picking grps-rs over e.g. grpcio?
Cheers, Samuel.
On Thu, Sep 12, 2019 at 04:53:29PM +0800, 杨波 via kata-dev wrote:
Hi, folks: Let me update some progress of the rust agent. First of all, thanks for James' kindly help on the project. The functions of rust agent are almost finished. we have fixed several bugs to stablize it. Ci and osbuilder are still missing. We have to run end-to-end tests manually right now.
- Refactor rust agent API. Remove overused matches, clean up code - Refactor communication between parent/child/grandchild, use raw bytes instead of serde_json to simplify communication - Refactor Process struct, remove duplicate/unused fields - Add Makefile to simplify building --- by jodh-intel - Fix up unit test for netlink - Fix up test run against musl and release - Implement debug console support - Implement command line parse partly - Implement Spec valaidator - Implement Hook infrastructure
We are targeting to land the rust agent to Kata org by October, and you may check out the current WIP repo here: https://github.com/alipay/kata-rust-agent
Cheers!
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
-- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
I think the extra money is spent in go runtime, probably. ------------------------------------------------------------------ 发件人：Dr. David Alan Gilbert <dgilbert@redhat.com> 发送时间：2019年9月19日(星期四) 16:28 收件人：杨波(兵卫) <yb203166@antfin.com> 抄 送："Boeuf, Sebastien" <sebastien.boeuf@intel.com>; Samuel Ortiz <sameo@linux.intel.com>; kata-dev <kata-dev@lists.katacontainers.io> 主 题：Re: [kata-dev] 回复： 回复： rust kata-agent status update * 杨波 via kata-dev (kata-dev@lists.katacontainers.io) wrote:
go-agent: about 11M rust-agent: about 1.1M
Do you understand why? That's a big difference - I wonder where the extra memory goes? Dave
It is also in README.md at here: https://github.com/alipay/kata-rust-agent/blob/master/README.md
------------------------------------------------------------------ 发件人：Boeuf, Sebastien <sebastien.boeuf@intel.com> 发送时间：2019年9月19日(星期四) 05:28 收件人：杨波(兵卫) <yb203166@antfin.com>; Samuel Ortiz <sameo@linux.intel.com> 抄 送：kata-dev <kata-dev@lists.katacontainers.io> 主 题：RE: [kata-dev] 回复： rust kata-agent status update
Hi,
I love the footprint drop due to using Rust :) Instead of giving only the numbers from the Rust agent, could you show the difference between the Golang agent and Rust agent?
Thanks, Sebastien
From: 杨波 via kata-dev [kata-dev@lists.katacontainers.io] Sent: Sunday, September 15, 2019 6:52 PM To: Samuel Ortiz Cc: kata-dev Subject: [kata-dev] 回复： rust kata-agent status update
footprint: RssAnon: 1700 kB RssFile: 1632 kB
We choose grpc-rs because it is verified in product environment, grpcio is not.
------------------------------------------------------------------ 发件人：Samuel Ortiz <sameo@linux.intel.com> 发送时间：2019年9月12日(星期四) 20:44 收件人：杨波(兵卫) <yb203166@antfin.com> 抄 送：kata-dev <kata-dev@lists.katacontainers.io>; 王旭(循环) <gnawux@alibaba-inc.com> 主 题：Re: [kata-dev] rust kata-agent status update
Hi,
A couple of questions:
1) Do we know what the grps-rs memory footprint is? 2) What was the rationale behind picking grps-rs over e.g. grpcio?
Cheers, Samuel.
On Thu, Sep 12, 2019 at 04:53:29PM +0800, 杨波 via kata-dev wrote:
Hi, folks: Let me update some progress of the rust agent. First of all, thanks for James' kindly help on the project. The functions of rust agent are almost finished. we have fixed several bugs to stablize it. Ci and osbuilder are still missing. We have to run end-to-end tests manually right now.
- Refactor rust agent API. Remove overused matches, clean up code - Refactor communication between parent/child/grandchild, use raw bytes instead of serde_json to simplify communication - Refactor Process struct, remove duplicate/unused fields - Add Makefile to simplify building --- by jodh-intel - Fix up unit test for netlink - Fix up test run against musl and release - Implement debug console support - Implement command line parse partly - Implement Spec valaidator - Implement Hook infrastructure
We are targeting to land the rust agent to Kata org by October, and you may check out the current WIP repo here: https://github.com/alipay/kata-rust-agent
Cheers!
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
-- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
Typically the `go runtime` occupies about 20-30M memory under different test cases.
On Sep 20, 2019, at 9:54 AM, 杨波 via kata-dev <kata-dev@lists.katacontainers.io> wrote:
I think the extra money is spent in go runtime, probably.
------------------------------------------------------------------ 发件人：Dr. David Alan Gilbert <dgilbert@redhat.com> 发送时间：2019年9月19日(星期四) 16:28 收件人：杨波(兵卫) <yb203166@antfin.com> 抄 送："Boeuf, Sebastien" <sebastien.boeuf@intel.com>; Samuel Ortiz <sameo@linux.intel.com>; kata-dev <kata-dev@lists.katacontainers.io> 主 题：Re: [kata-dev] 回复： 回复： rust kata-agent status update
* 杨波 via kata-dev (kata-dev@lists.katacontainers.io) wrote:
go-agent: about 11M rust-agent: about 1.1M
Do you understand why? That's a big difference - I wonder where the extra memory goes?
Dave
It is also in README.md at here: https://github.com/alipay/kata-rust-agent/blob/master/README.md
------------------------------------------------------------------ 发件人：Boeuf, Sebastien <sebastien.boeuf@intel.com> 发送时间：2019年9月19日(星期四) 05:28 收件人：杨波(兵卫) <yb203166@antfin.com>; Samuel Ortiz <sameo@linux.intel.com> 抄 送：kata-dev <kata-dev@lists.katacontainers.io> 主 题：RE: [kata-dev] 回复： rust kata-agent status update
Hi,
I love the footprint drop due to using Rust :) Instead of giving only the numbers from the Rust agent, could you show the difference between the Golang agent and Rust agent?
Thanks, Sebastien
From: 杨波 via kata-dev [kata-dev@lists.katacontainers.io] Sent: Sunday, September 15, 2019 6:52 PM To: Samuel Ortiz Cc: kata-dev Subject: [kata-dev] 回复： rust kata-agent status update
footprint: RssAnon: 1700 kB RssFile: 1632 kB
We choose grpc-rs because it is verified in product environment, grpcio is not.
------------------------------------------------------------------ 发件人：Samuel Ortiz <sameo@linux.intel.com> 发送时间：2019年9月12日(星期四) 20:44 收件人：杨波(兵卫) <yb203166@antfin.com> 抄 送：kata-dev <kata-dev@lists.katacontainers.io>; 王旭(循环) <gnawux@alibaba-inc.com> 主 题：Re: [kata-dev] rust kata-agent status update
Hi,
A couple of questions:
1) Do we know what the grps-rs memory footprint is? 2) What was the rationale behind picking grps-rs over e.g. grpcio?
Cheers, Samuel.
On Thu, Sep 12, 2019 at 04:53:29PM +0800, 杨波 via kata-dev wrote:
Hi, folks: Let me update some progress of the rust agent. First of all, thanks for James' kindly help on the project. The functions of rust agent are almost finished. we have fixed several bugs to stablize it. Ci and osbuilder are still missing. We have to run end-to-end tests manually right now.
- Refactor rust agent API. Remove overused matches, clean up code - Refactor communication between parent/child/grandchild, use raw bytes instead of serde_json to simplify communication - Refactor Process struct, remove duplicate/unused fields - Add Makefile to simplify building --- by jodh-intel - Fix up unit test for netlink - Fix up test run against musl and release - Implement debug console support - Implement command line parse partly - Implement Spec valaidator - Implement Hook infrastructure
We are targeting to land the rust agent to Kata org by October, and you may check out the current WIP repo here: https://github.com/alipay/kata-rust-agent
Cheers!
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
-- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK _______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
Thanks! ________________________________ From: 杨波 [yb203166@antfin.com] Sent: Wednesday, September 18, 2019 7:47 PM To: Boeuf, Sebastien; Samuel Ortiz Cc: kata-dev Subject: 回复：[kata-dev] 回复： rust kata-agent status update go-agent: about 11M rust-agent: about 1.1M It is also in README.md at here: https://github.com/alipay/kata-rust-agent/blob/master/README.md ------------------------------------------------------------------ 发件人：Boeuf, Sebastien <sebastien.boeuf@intel.com> 发送时间：2019年9月19日(星期四) 05:28 收件人：杨波(兵卫) <yb203166@antfin.com>; Samuel Ortiz <sameo@linux.intel.com> 抄 送：kata-dev <kata-dev@lists.katacontainers.io> 主 题：RE: [kata-dev] 回复： rust kata-agent status update Hi, I love the footprint drop due to using Rust :) Instead of giving only the numbers from the Rust agent, could you show the difference between the Golang agent and Rust agent? Thanks, Sebastien ________________________________ From: 杨波 via kata-dev [kata-dev@lists.katacontainers.io] Sent: Sunday, September 15, 2019 6:52 PM To: Samuel Ortiz Cc: kata-dev Subject: [kata-dev] 回复： rust kata-agent status update footprint: RssAnon: 1700 kB RssFile: 1632 kB We choose grpc-rs because it is verified in product environment, grpcio is not. ------------------------------------------------------------------ 发件人：Samuel Ortiz <sameo@linux.intel.com> 发送时间：2019年9月12日(星期四) 20:44 收件人：杨波(兵卫) <yb203166@antfin.com> 抄 送：kata-dev <kata-dev@lists.katacontainers.io>; 王旭(循环) <gnawux@alibaba-inc.com> 主 题：Re: [kata-dev] rust kata-agent status update Hi, A couple of questions: 1) Do we know what the grps-rs memory footprint is? 2) What was the rationale behind picking grps-rs over e.g. grpcio? Cheers, Samuel. On Thu, Sep 12, 2019 at 04:53:29PM +0800, 杨波 via kata-dev wrote:
Hi, folks: Let me update some progress of the rust agent. First of all, thanks for James' kindly help on the project. The functions of rust agent are almost finished. we have fixed several bugs to stablize it. Ci and osbuilder are still missing. We have to run end-to-end tests manually right now.
- Refactor rust agent API. Remove overused matches, clean up code - Refactor communication between parent/child/grandchild, use raw bytes instead of serde_json to simplify communication - Refactor Process struct, remove duplicate/unused fields - Add Makefile to simplify building --- by jodh-intel - Fix up unit test for netlink - Fix up test run against musl and release - Implement debug console support - Implement command line parse partly - Implement Spec valaidator - Implement Hook infrastructure
We are targeting to land the rust agent to Kata org by October, and you may check out the current WIP repo here: https://github.com/alipay/kata-rust-agent
Cheers!
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
participants (5)
-
Boeuf, Sebastien
-
Dr. David Alan Gilbert
-
Jiang Liu
-
Samuel Ortiz
-
杨波