settingsLogin | Registersettings

[OpenStack-DefCore] [interop-challenge] VIO 2.5 Test Result Report

0 votes

Hi Tong et al,

Attached are results for VIO 2.5. Note that Tong and I discussed the issues that we found on IRC and he has a patch up here that could use your reviews to make the workload code more portable:

https://review.openstack.org/#/c/366784

I’ve documented the problems in these results anyway since I think it’ll be useful learning coming out of the challenge. To summarize: if you are concerned about workload portability, best practice for your workload code should be to either parameterize names for things like NICs and block devices or ideally to discover them. FWIW, both of these issues are quite trivial to work around in the workloads we’re testing with.

At Your Service,

Mark T. Voelker


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee

asked Sep 7, 2016 in defcore-committee by Mark_Voelker (4,000 points)   2 3

5 Responses

0 votes

Hi Mark,

The RefStack result [1] in this result set shows 1187 pass tests. Just
want to confirm whether this RefStack test result was run with admin
credential. Thanks for sharing the data!

[1]
https://refstack.openstack.org/#/results/fc80592b-4503-481c-8aa6-49d414961f2d

Catherine Diep
----- Forwarded by Catherine Cuong Diep/San Jose/IBM on 09/07/2016 12:02 PM


From: Mark Voelker mvoelker@vmware.com
To: "defcore-committee@lists.openstack.org"

Date: 09/07/2016 11:32 AM
Subject: [OpenStack-DefCore] [interop-challenge] VIO 2.5 Test Result
Report

Hi Tong et al,

Attached are results for VIO 2.5. Note that Tong and I discussed the
issues that we found on IRC and he has a patch up here that could use your
reviews to make the workload code more portable:

https://review.openstack.org/#/c/366784

I’ve documented the problems in these results anyway since I think it’ll be
useful learning coming out of the challenge. To summarize: if you are
concerned about workload portability, best practice for your workload code
should be to either parameterize names for things like NICs and block
devices or ideally to discover them. FWIW, both of these issues are quite
trivial to work around in the workloads we’re testing with.

At Your Service,

Mark T. Voelker

[attachment "vio2.5results.txt" deleted by Catherine Cuong Diep/San
Jose/IBM] [attachment "ATT00001.txt" deleted by Catherine Cuong Diep/San
Jose/IBM] _______________________________________________
Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Sep 7, 2016 by Catherine_Cuong_Diep (1,320 points)   2
0 votes

Hmm, that particular tempest run might have had admin credentials in the config (that particular result is from earlier this summer). I was planning to do a new run this week or next anyway to take care of a couple of extraneous failures for tests that just became required in 2016.08, so I’ll double check then.

At Your Service,

Mark T. Voelker
OpenStack Architect

On Sep 7, 2016, at 3:12 PM, Catherine Cuong Diep cdiep@us.ibm.com wrote:

Hi Mark,

The RefStack result [1] in this result set shows 1187 pass tests. Just want to confirm whether this RefStack test result was run with admin credential. Thanks for sharing the data!

[1] https://refstack.openstack.org/#/results/fc80592b-4503-481c-8aa6-49d414961f2d

Catherine Diep
----- Forwarded by Catherine Cuong Diep/San Jose/IBM on 09/07/2016 12:02 PM -----

From: Mark Voelker mvoelker@vmware.com
To: "defcore-committee@lists.openstack.org" defcore-committee@lists.openstack.org
Date: 09/07/2016 11:32 AM
Subject: [OpenStack-DefCore] [interop-challenge] VIO 2.5 Test Result Report

Hi Tong et al,

Attached are results for VIO 2.5. Note that Tong and I discussed the issues that we found on IRC and he has a patch up here that could use your reviews to make the workload code more portable:

https://review.openstack.org/#/c/366784

I’ve documented the problems in these results anyway since I think it’ll be useful learning coming out of the challenge. To summarize: if you are concerned about workload portability, best practice for your workload code should be to either parameterize names for things like NICs and block devices or ideally to discover them. FWIW, both of these issues are quite trivial to work around in the workloads we’re testing with.

At Your Service,

Mark T. Voelker

[attachment "vio2.5results.txt" deleted by Catherine Cuong Diep/San Jose/IBM] [attachment "ATT00001.txt" deleted by Catherine Cuong Diep/San Jose/IBM] _______________________________________________
Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Sep 7, 2016 by Mark_Voelker (4,000 points)   2 3
0 votes

FYI for those not watching IRC, Catherine and I just had a quick chat about this (thanks Catherine):

http://eavesdrop.openstack.org/irclogs/%23openstack-defcore/%23openstack-defcore.2016-09-07.log.html#t2016-09-07T19:33:42

I’ll defer to the interop challenge organizers on whether or not they’d prefer to have admin credentials removed before running refstack for purposes of the interop challenge--I don’t have much of a preference and will probably be running RefStack again in the near future anyway. However I’d suggest that if the decision is that not putting admin credentials in the tempest config would make things easier to compare, it would be good to document that before a lot of other folks start running tests and submitting results.

At Your Service,

Mark T. Voelker

On Sep 7, 2016, at 3:30 PM, Mark Voelker mvoelker@vmware.com wrote:

Hmm, that particular tempest run might have had admin credentials in the config (that particular result is from earlier this summer). I was planning to do a new run this week or next anyway to take care of a couple of extraneous failures for tests that just became required in 2016.08, so I’ll double check then.

At Your Service,

Mark T. Voelker
OpenStack Architect

On Sep 7, 2016, at 3:12 PM, Catherine Cuong Diep cdiep@us.ibm.com wrote:

Hi Mark,

The RefStack result [1] in this result set shows 1187 pass tests. Just want to confirm whether this RefStack test result was run with admin credential. Thanks for sharing the data!

[1] https://refstack.openstack.org/#/results/fc80592b-4503-481c-8aa6-49d414961f2d

Catherine Diep
----- Forwarded by Catherine Cuong Diep/San Jose/IBM on 09/07/2016 12:02 PM -----

From: Mark Voelker mvoelker@vmware.com
To: "defcore-committee@lists.openstack.org" defcore-committee@lists.openstack.org
Date: 09/07/2016 11:32 AM
Subject: [OpenStack-DefCore] [interop-challenge] VIO 2.5 Test Result Report

Hi Tong et al,

Attached are results for VIO 2.5. Note that Tong and I discussed the issues that we found on IRC and he has a patch up here that could use your reviews to make the workload code more portable:

https://review.openstack.org/#/c/366784

I’ve documented the problems in these results anyway since I think it’ll be useful learning coming out of the challenge. To summarize: if you are concerned about workload portability, best practice for your workload code should be to either parameterize names for things like NICs and block devices or ideally to discover them. FWIW, both of these issues are quite trivial to work around in the workloads we’re testing with.

At Your Service,

Mark T. Voelker

[attachment "vio2.5results.txt" deleted by Catherine Cuong Diep/San Jose/IBM] [attachment "ATT00001.txt" deleted by Catherine Cuong Diep/San Jose/IBM] _______________________________________________
Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Sep 7, 2016 by Mark_Voelker (4,000 points)   2 3
0 votes

Thanks Mark! I have updated the RefStack test instruction
https://wiki.openstack.org/wiki/Interop_Challenge#RefStack_Testing_and_Results_Upload
.

Catherine Diep
----- Forwarded by Catherine Cuong Diep/San Jose/IBM on 09/07/2016 04:05 PM


From: Mark Voelker mvoelker@vmware.com
To: Catherine Cuong Diep/San Jose/IBM@IBMUS
Cc: "defcore-committee@lists.openstack.org"

Date: 09/07/2016 01:19 PM
Subject: Re: [OpenStack-DefCore] [interop-challenge] VIO 2.5 Test Result
Report

FYI for those not watching IRC, Catherine and I just had a quick chat about
this (thanks Catherine):

http://eavesdrop.openstack.org/irclogs/%23openstack-defcore/%23openstack-defcore.2016-09-07.log.html#t2016-09-07T19:33:42

I’ll defer to the interop challenge organizers on whether or not they’d
prefer to have admin credentials removed before running refstack for
purposes of the interop challenge--I don’t have much of a preference and
will probably be running RefStack again in the near future anyway. However
I’d suggest that if the decision is that not putting admin credentials in
the tempest config would make things easier to compare, it would be good to
document that before a lot of other folks start running tests and
submitting results.

At Your Service,

Mark T. Voelker

On Sep 7, 2016, at 3:30 PM, Mark Voelker mvoelker@vmware.com wrote:

Hmm, that particular tempest run might have had admin credentials in the
config (that particular result is from earlier this summer). I was
planning to do a new run this week or next anyway to take care of a couple
of extraneous failures for tests that just became required in 2016.08, so
I’ll double check then.

At Your Service,

Mark T. Voelker
OpenStack Architect

On Sep 7, 2016, at 3:12 PM, Catherine Cuong Diep cdiep@us.ibm.com
wrote:

Hi Mark,

The RefStack result [1] in this result set shows 1187 pass tests. Just
want to confirm whether this RefStack test result was run with admin
credential. Thanks for sharing the data!

[1]
https://refstack.openstack.org/#/results/fc80592b-4503-481c-8aa6-49d414961f2d

Catherine Diep
----- Forwarded by Catherine Cuong Diep/San Jose/IBM on 09/07/2016 12:02
PM -----

From: Mark Voelker mvoelker@vmware.com
To: "defcore-committee@lists.openstack.org"

Date: 09/07/2016 11:32 AM
Subject: [OpenStack-DefCore] [interop-challenge] VIO 2.5 Test Result
Report

Hi Tong et al,

Attached are results for VIO 2.5. Note that Tong and I discussed the
issues that we found on IRC and he has a patch up here that could use your
reviews to make the workload code more portable:

https://review.openstack.org/#/c/366784

I’ve documented the problems in these results anyway since I think it’ll
be useful learning coming out of the challenge. To summarize: if you are
concerned about workload portability, best practice for your workload code
should be to either parameterize names for things like NICs and block
devices or ideally to discover them. FWIW, both of these issues are quite
trivial to work around in the workloads we’re testing with.

At Your Service,

Mark T. Voelker

[attachment "vio2.5results.txt" deleted by Catherine Cuong Diep/San
Jose/IBM] [attachment "ATT00001.txt" deleted by Catherine Cuong Diep/San
Jose/IBM] _______________________________________________
Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Sep 7, 2016 by Catherine_Cuong_Diep (1,320 points)   2
0 votes

H Mark,

It seems like the goal is to collect as much data as possible for all API
tests. As such, users should feel free to run with admin or multi-users.
The wiki has been updated accordingly.
https://wiki.openstack.org/wiki/Interop_Challenge#RefStack_Testing_and_Results_Upload

Catherine Diep
----- Forwarded by Catherine Cuong Diep/San Jose/IBM on 09/07/2016 05:58 PM


From: Mark Voelker mvoelker@vmware.com
To: Catherine Cuong Diep/San Jose/IBM@IBMUS
Cc: "defcore-committee@lists.openstack.org"

Date: 09/07/2016 01:19 PM
Subject: Re: [OpenStack-DefCore] [interop-challenge] VIO 2.5 Test Result
Report

FYI for those not watching IRC, Catherine and I just had a quick chat about
this (thanks Catherine):

http://eavesdrop.openstack.org/irclogs/%23openstack-defcore/%23openstack-defcore.2016-09-07.log.html#t2016-09-07T19:33:42

I’ll defer to the interop challenge organizers on whether or not they’d
prefer to have admin credentials removed before running refstack for
purposes of the interop challenge--I don’t have much of a preference and
will probably be running RefStack again in the near future anyway. However
I’d suggest that if the decision is that not putting admin credentials in
the tempest config would make things easier to compare, it would be good to
document that before a lot of other folks start running tests and
submitting results.

At Your Service,

Mark T. Voelker

On Sep 7, 2016, at 3:30 PM, Mark Voelker mvoelker@vmware.com wrote:

Hmm, that particular tempest run might have had admin credentials in the
config (that particular result is from earlier this summer). I was
planning to do a new run this week or next anyway to take care of a couple
of extraneous failures for tests that just became required in 2016.08, so
I’ll double check then.

At Your Service,

Mark T. Voelker
OpenStack Architect

On Sep 7, 2016, at 3:12 PM, Catherine Cuong Diep cdiep@us.ibm.com
wrote:

Hi Mark,

The RefStack result [1] in this result set shows 1187 pass tests. Just
want to confirm whether this RefStack test result was run with admin
credential. Thanks for sharing the data!

[1]
https://refstack.openstack.org/#/results/fc80592b-4503-481c-8aa6-49d414961f2d

Catherine Diep
----- Forwarded by Catherine Cuong Diep/San Jose/IBM on 09/07/2016 12:02
PM -----

From: Mark Voelker mvoelker@vmware.com
To: "defcore-committee@lists.openstack.org"

Date: 09/07/2016 11:32 AM
Subject: [OpenStack-DefCore] [interop-challenge] VIO 2.5 Test Result
Report

Hi Tong et al,

Attached are results for VIO 2.5. Note that Tong and I discussed the
issues that we found on IRC and he has a patch up here that could use your
reviews to make the workload code more portable:

https://review.openstack.org/#/c/366784

I’ve documented the problems in these results anyway since I think it’ll
be useful learning coming out of the challenge. To summarize: if you are
concerned about workload portability, best practice for your workload code
should be to either parameterize names for things like NICs and block
devices or ideally to discover them. FWIW, both of these issues are quite
trivial to work around in the workloads we’re testing with.

At Your Service,

Mark T. Voelker

[attachment "vio2.5results.txt" deleted by Catherine Cuong Diep/San
Jose/IBM] [attachment "ATT00001.txt" deleted by Catherine Cuong Diep/San
Jose/IBM] _______________________________________________
Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee


Defcore-committee mailing list
Defcore-committee@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee
responded Sep 8, 2016 by Catherine_Cuong_Diep (1,320 points)   2
...