settingsLogin | Registersettings

[openstack-dev] [qa][cinder] RFC: Cinder test on Tempest

0 votes

Hi,

Now we need to update Tempest for following Cinder API status.
I have an idea for restructuring and happy to see feedback about that.

Now Cinder API status is
V1: Deprecated
V2: Deprecated
V3: Current
V1 API tests have been removed from Tempest side already, so we just
need to concentrate on V2 and V3 now.

Gate jobs
Most Cinder tests are implemented for V2 API on Tempest side and the
base microversion of V3 is the same as V2.
Then we can re-use V2 API tests for the base microversion of V3 API.
One idea is that we can have Cinder V3 API tests as the default on the
gate jobs and the V2 API tests as another job like the following
because the V2 API is deprecated.

gate-tempest-dsvm-neutron-full-ubuntu-xenial - (existing job):
testing Cinder V3 API
gate-tempest-dsvm-py35-ubuntu-xenial - (existing job): testing Cinder V3 API
...
gate-tempest-dsvm-neutron-full-ubuntu-xenial-cinder-v2: (new job):
testing Cinder V2 API

We had the same testing way for Nova V2 API and V2.1 API before, and
we could avoid copy&paste V2 test code for V2.1 API on Tempest.

Test Structure
Current test structure is like:
tempest/api/volume/ - V2 API tests
tempest/api/volume/v2 - V2 API tests
tempest/api/volume/v3 - V3 API tests
Yes, this is mess.
For re-using V2 API tests for V3 API, it would be better to remove
"v2" from V2 API tests for avoiding confusions.

A new structure could be
tempest/api/volume/ - All tests for V2 API and the base
microversion of V3 API
tempest/api/volume/v3 - V3 API specific tests for newer microversions
or
tempest/api/volume/ - All tests for V2 API and V3 API which
includes newer microversions

As the reference, Nova API structure is like the later.

Any thoughts?

Thanks
Ken Ohmichi


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
asked Mar 23, 2017 in openstack-dev by Ken'ichi_Ohmichi (8,640 points)   2 3 3

7 Responses

0 votes

On Wed, Mar 22, 2017 at 01:08:23PM -0700, Ken'ichi Ohmichi wrote:
Hi,

Now we need to update Tempest for following Cinder API status.
I have an idea for restructuring and happy to see feedback about that.

Now Cinder API status is
V1: Deprecated
V2: Deprecated
V3: Current
V1 API tests have been removed from Tempest side already, so we just
need to concentrate on V2 and V3 now.

Gate jobs
Most Cinder tests are implemented for V2 API on Tempest side and the
base microversion of V3 is the same as V2.
Then we can re-use V2 API tests for the base microversion of V3 API.
One idea is that we can have Cinder V3 API tests as the default on the
gate jobs and the V2 API tests as another job like the following
because the V2 API is deprecated.

gate-tempest-dsvm-neutron-full-ubuntu-xenial - (existing job):
testing Cinder V3 API
gate-tempest-dsvm-py35-ubuntu-xenial - (existing job): testing Cinder V3 API
...
gate-tempest-dsvm-neutron-full-ubuntu-xenial-cinder-v2: (new job):
testing Cinder V2 API

+1 I like this idea.

We had the same testing way for Nova V2 API and V2.1 API before, and
we could avoid copy&paste V2 test code for V2.1 API on Tempest.

Test Structure
Current test structure is like:
tempest/api/volume/ - V2 API tests
tempest/api/volume/v2 - V2 API tests
tempest/api/volume/v3 - V3 API tests
Yes, this is mess.
For re-using V2 API tests for V3 API, it would be better to remove
"v2" from V2 API tests for avoiding confusions.

A new structure could be
tempest/api/volume/ - All tests for V2 API and the base
microversion of V3 API
tempest/api/volume/v3 - V3 API specific tests for newer microversions
or
tempest/api/volume/ - All tests for V2 API and V3 API which
includes newer microversions

As the reference, Nova API structure is like the later.

I like the last one better as well.

Any thoughts?

Thanks
Ken Ohmichi


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 22, 2017 by Sean_McGinnis (11,820 points)   2 3 6
0 votes

On Wed, Mar 22, 2017 at 8:31 PM Sean McGinnis sean.mcginnis@gmx.com wrote:

On Wed, Mar 22, 2017 at 01:08:23PM -0700, Ken'ichi Ohmichi wrote:

Hi,

Now we need to update Tempest for following Cinder API status.
I have an idea for restructuring and happy to see feedback about that.

Now Cinder API status is
V1: Deprecated
V2: Deprecated
V3: Current
V1 API tests have been removed from Tempest side already, so we just
need to concentrate on V2 and V3 now.

Gate jobs
Most Cinder tests are implemented for V2 API on Tempest side and the
base microversion of V3 is the same as V2.
Then we can re-use V2 API tests for the base microversion of V3 API.
One idea is that we can have Cinder V3 API tests as the default on the
gate jobs and the V2 API tests as another job like the following
because the V2 API is deprecated.

gate-tempest-dsvm-neutron-full-ubuntu-xenial - (existing job):
testing Cinder V3 API
gate-tempest-dsvm-py35-ubuntu-xenial - (existing job): testing Cinder
V3 API
...
gate-tempest-dsvm-neutron-full-ubuntu-xenial-cinder-v2: (new job):
testing Cinder V2 API

I guess this job would run against tempest and cinder only?

+1 I like this idea.

We had the same testing way for Nova V2 API and V2.1 API before, and
we could avoid copy&paste V2 test code for V2.1 API on Tempest.

Test Structure
Current test structure is like:
tempest/api/volume/ - V2 API tests
tempest/api/volume/v2 - V2 API tests
tempest/api/volume/v3 - V3 API tests
Yes, this is mess.
For re-using V2 API tests for V3 API, it would be better to remove
"v2" from V2 API tests for avoiding confusions.

A new structure could be
tempest/api/volume/ - All tests for V2 API and the base
microversion of V3 API
tempest/api/volume/v3 - V3 API specific tests for newer microversions
or
tempest/api/volume/ - All tests for V2 API and V3 API which
includes newer microversions

As the reference, Nova API structure is like the later.

I like the last one better as well.

My favourite option would be that that generates less churn in the code :)
One folder for everything means moving 4 or 5 modules only, so I think that
would
be a good option.

I would prefer to avoid though having a lot of v3 test classes that inherit
from
v2 test classes, and just set apiversion = 3.

As long as we can assume we will never run v2 and v3 in the same job, we
could
have cinderapiversion as a configuration setting, to determine which
cinder
endpoint to hit when running the volume tests.

Any thoughts?

Thanks
Ken Ohmichi


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Apart from the volume tests, if we split the gate jobs into standard one
running v3
plus and extra v2 one, we should make sure that all tests that use the
volume API
use a consistent version of the volume API. Nova as well should be
configured if
possible to use the same volume API version.

andrea


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 22, 2017 by Andrea_Frittoli (5,920 points)   1 2 3
0 votes

2017-03-22 14:32 GMT-07:00 Andrea Frittoli andrea.frittoli@gmail.com:

On Wed, Mar 22, 2017 at 8:31 PM Sean McGinnis sean.mcginnis@gmx.com wrote:

On Wed, Mar 22, 2017 at 01:08:23PM -0700, Ken'ichi Ohmichi wrote:

Hi,

Now we need to update Tempest for following Cinder API status.
I have an idea for restructuring and happy to see feedback about that.

Now Cinder API status is
V1: Deprecated
V2: Deprecated
V3: Current
V1 API tests have been removed from Tempest side already, so we just
need to concentrate on V2 and V3 now.

Gate jobs
Most Cinder tests are implemented for V2 API on Tempest side and the
base microversion of V3 is the same as V2.
Then we can re-use V2 API tests for the base microversion of V3 API.
One idea is that we can have Cinder V3 API tests as the default on the
gate jobs and the V2 API tests as another job like the following
because the V2 API is deprecated.

gate-tempest-dsvm-neutron-full-ubuntu-xenial - (existing job):
testing Cinder V3 API
gate-tempest-dsvm-py35-ubuntu-xenial - (existing job): testing Cinder
V3 API
...
gate-tempest-dsvm-neutron-full-ubuntu-xenial-cinder-v2: (new job):
testing Cinder V2 API

I guess this job would run against tempest and cinder only?

A nice point, I think so.

+1 I like this idea.

We had the same testing way for Nova V2 API and V2.1 API before, and
we could avoid copy&paste V2 test code for V2.1 API on Tempest.

Test Structure
Current test structure is like:
tempest/api/volume/ - V2 API tests
tempest/api/volume/v2 - V2 API tests
tempest/api/volume/v3 - V3 API tests
Yes, this is mess.
For re-using V2 API tests for V3 API, it would be better to remove
"v2" from V2 API tests for avoiding confusions.

A new structure could be
tempest/api/volume/ - All tests for V2 API and the base
microversion of V3 API
tempest/api/volume/v3 - V3 API specific tests for newer microversions
or
tempest/api/volume/ - All tests for V2 API and V3 API which
includes newer microversions

As the reference, Nova API structure is like the later.

I like the last one better as well.

My favourite option would be that that generates less churn in the code :)
One folder for everything means moving 4 or 5 modules only, so I think that
would
be a good option.

I would prefer to avoid though having a lot of v3 test classes that inherit
from
v2 test classes, and just set apiversion = 3.

Yeah, I agree :-)

As long as we can assume we will never run v2 and v3 in the same job, we
could
have cinderapiversion as a configuration setting, to determine which
cinder
endpoint to hit when running the volume tests.

Or it would be enough to have the existing "catalogtype",
"min
microversion" and "maxmicroversion" only without apiv1/v2/v3 to
control the target API version, because of the above separated gate
jobs.

Apart from the volume tests, if we split the gate jobs into standard one
running v3
plus and extra v2 one, we should make sure that all tests that use the
volume API
use a consistent version of the volume API. Nova as well should be
configured if
possible to use the same volume API version.

This also is a nice point.
Nova team also has a plan to use cinder v3 as the default in Pike.
We have consistent direction now.

Thanks


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 22, 2017 by Ken'ichi_Ohmichi (8,640 points)   2 3 3
0 votes

On Thu, Mar 23, 2017 at 7:02 AM, Ken'ichi Ohmichi ken1ohmichi@gmail.com
wrote:

2017-03-22 14:32 GMT-07:00 Andrea Frittoli andrea.frittoli@gmail.com:

On Wed, Mar 22, 2017 at 8:31 PM Sean McGinnis sean.mcginnis@gmx.com
wrote:

On Wed, Mar 22, 2017 at 01:08:23PM -0700, Ken'ichi Ohmichi wrote:

Hi,

Now we need to update Tempest for following Cinder API status.
I have an idea for restructuring and happy to see feedback about that.

Now Cinder API status is
V1: Deprecated
V2: Deprecated
V3: Current
V1 API tests have been removed from Tempest side already, so we just
need to concentrate on V2 and V3 now.

Gate jobs
Most Cinder tests are implemented for V2 API on Tempest side and the
base microversion of V3 is the same as V2.
Then we can re-use V2 API tests for the base microversion of V3 API.
One idea is that we can have Cinder V3 API tests as the default on the
gate jobs and the V2 API tests as another job like the following
because the V2 API is deprecated.

gate-tempest-dsvm-neutron-full-ubuntu-xenial - (existing job):
testing Cinder V3 API
gate-tempest-dsvm-py35-ubuntu-xenial - (existing job): testing
Cinder
V3 API
...
gate-tempest-dsvm-neutron-full-ubuntu-xenial-cinder-v2: (new job):
testing Cinder V2 API

I guess this job would run against tempest and cinder only?

A nice point, I think so.

+1 I like this idea.

We had the same testing way for Nova V2 API and V2.1 API before, and
we could avoid copy&paste V2 test code for V2.1 API on Tempest.

Test Structure
Current test structure is like:
tempest/api/volume/ - V2 API tests
tempest/api/volume/v2 - V2 API tests
tempest/api/volume/v3 - V3 API tests
Yes, this is mess.
For re-using V2 API tests for V3 API, it would be better to remove
"v2" from V2 API tests for avoiding confusions.

A new structure could be
tempest/api/volume/ - All tests for V2 API and the base
microversion of V3 API
tempest/api/volume/v3 - V3 API specific tests for newer
microversions
or
tempest/api/volume/ - All tests for V2 API and V3 API which
includes newer microversions

​+1, this looks better as no more version specific tests and all v2 tests
should run as it is on v3 base version.​

As the reference, Nova API structure is like the later.

I like the last one better as well.

My favourite option would be that that generates less churn in the code
:)
One folder for everything means moving 4 or 5 modules only, so I think
that
would
be a good option.

I would prefer to avoid though having a lot of v3 test classes that
inherit
from
v2 test classes, and just set apiversion = 3.

Yeah, I agree :-)

​yea we should not have that.

As long as we can assume we will never run v2 and v3 in the same job, we
could
have cinderapiversion as a configuration setting, to determine which
cinder
endpoint to hit when running the volume tests.

Or it would be enough to have the existing "catalogtype",
"min
microversion" and "maxmicroversion" only without apiv1/v2/v3 to
control the target API version, because of the above separated gate
jobs.

​Yes, so devstack does set different catalog for v2 and v3 [0]​. Based on
those catalog_type configured on tempest config(we already have that for
volume API config) auth can select the right endpoints to make the API call.

All existing tests can be run for both API without any extra class or
change. This way we can get rid of 'apiversion' in all volumes clients and
have them as single copy for v2 and v3 base.
Further v3 microversion tests can be implemented as accordingly by sending
the microversion header on API request. and devstack can tell Temepst not
to set microversion if catalog
type volume_v2 is being asked to run the
tests.

As you mentioned it can be same way we handle compute v2, v2.1 and +
microversion tests.

Apart from the volume tests, if we split the gate jobs into standard one
running v3
plus and extra v2 one, we should make sure that all tests that use the
volume API
use a consistent version of the volume API. Nova as well should be
configured if
possible to use the same volume API version.

This also is a nice point.
Nova team also has a plan to use cinder v3 as the default in Pike.
We have consistent direction now.

Thanks


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

​.. 0
http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/cinder#n403

Thanks
gmann​


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 23, 2017 by GHANSHYAM_MANN (5,700 points)   1 3 4
0 votes

To have only one folder (tempest/api/volume/ ) looks really good, and, do we plan to deem "api_version" and "microversion" as one thing?

i.e., we will use the mechanism of microversion to skip v3 new functional tests when the environment only supports v2?

Original Mail

Sender: <ghanshyammann@gmail.com>
To: <openstack-dev@lists.openstack.org>
Date: 2017/03/23 08:30
Subject: Re: [openstack-dev] [qa][cinder] RFC: Cinder test on Tempest

On Thu, Mar 23, 2017 at 7:02 AM, Ken'ichi Ohmichi <ken1ohmichi@gmail.com> wrote:
2017-03-22 14:32 GMT-07:00 Andrea Frittoli <andrea.frittoli@gmail.com>:


> On Wed, Mar 22, 2017 at 8:31 PM Sean McGinnis <sean.mcginnis@gmx.com> wrote:
>>
>> On Wed, Mar 22, 2017 at 01:08:23PM -0700, Ken'ichi Ohmichi wrote:
>> > Hi,
>> >
>> > Now we need to update Tempest for following Cinder API status..
>> > I have an idea for restructuring and happy to see feedback about that.
>> >
>> > Now Cinder API status is
>> > V1: Deprecated
>> > V2: Deprecated
>> > V3: Current
>> > V1 API tests have been removed from Tempest side already, so we just
>> > need to concentrate on V2 and V3 now.
>>
>> >
>> > Gate jobs
>> > Most Cinder tests are implemented for V2 API on Tempest side and the
>> > base microversion of V3 is the same as V2.
>> > Then we can re-use V2 API tests for the base microversion of V3 API.
>> > One idea is that we can have Cinder V3 API tests as the default on the
>> > gate jobs and the V2 API tests as another job like the following
>> > because the V2 API is deprecated.
>> >
>> > gate-tempest-dsvm-neutron-full-ubuntu-xenial - (existing job):
>> > testing Cinder V3 API
>> > gate-tempest-dsvm-py35-ubuntu-xenial - (existing job): testing Cinder
>> > V3 API
>> > ...
>> > gate-tempest-dsvm-neutron-full-ubuntu-xenial-cinder-v2: (new job):
>> > testing Cinder V2 API
>> >
>>
> I guess this job would run against tempest and cinder only?

A nice point, I think so.

>> +1 I like this idea.
>>
>> > We had the same testing way for Nova V2 API and V2.1 API before, and
>> > we could avoid copy&paste V2 test code for V2.1 API on Tempest.
>> >
>> > Test Structure
>> > Current test structure is like:
>> > tempest/api/volume/ - V2 API tests
>> > tempest/api/volume/v2 - V2 API tests
>> > tempest/api/volume/v3 - V3 API tests
>> > Yes, this is mess.
>> > For re-using V2 API tests for V3 API, it would be better to remove
>> > "v2" from V2 API tests for avoiding confusions.
>> >
>> > A new structure could be
>> > tempest/api/volume/ - All tests for V2 API and the base
>> > microversion of V3 API
>> > tempest/api/volume/v3 - V3 API specific tests for newer microversions
>> > or
>> > tempest/api/volume/ - All tests for V2 API and V3 API which
>> > includes newer microversions

+1, this looks better as no more version specific tests and all v2 tests should run as it is on v3 base version.

>> >
>> > As the reference, Nova API structure is like the later.
>>
>> I like the last one better as well.
>>
> My favourite option would be that that generates less churn in the code :)
> One folder for everything means moving 4 or 5 modules only, so I think that
> would
> be a good option.

> I would prefer to avoid though having a lot of v3 test classes that inherit
> from
> v2 test classes, and just set apiversion = 3.

Yeah, I agree :-)

yea we should not have that.

> As long as we can assume we will never run v2 and v3 in the same job, we
> could
> have cinderapiversion as a configuration setting, to determine which
> cinder
> endpoint to hit when running the volume tests.

Or it would be enough to have the existing "catalogtype",
"min
microversion" and "maxmicroversion" only without apiv1/v2/v3 to
control the target API version, because of the above separated gate
jobs.

Yes, so devstack does set different catalog for v2 and v3 [0]. Based on those catalog_type configured on tempest config(we already have that for volume API config) auth can select the right endpoints to make the API call.

All existing tests can be run for both API without any extra class or change. This way we can get rid of 'api_version' in all volumes clients and have them as single copy for v2 and v3 base.

Further v3 microversion tests can be implemented as accordingly by sending the microversion header on API request. and devstack can tell Temepst not to set microversion if catalogtype volumev2 is being asked to run the tests.

As you mentioned it can be same way we handle compute v2, v2.1 and + microversion tests.

> Apart from the volume tests, if we split the gate jobs into standard one
> running v3
> plus and extra v2 one, we should make sure that all tests that use the
> volume API
> use a consistent version of the volume API. Nova as well should be
> configured if
> possible to use the same volume API version.

This also is a nice point.
Nova team also has a plan to use cinder v3 as the default in Pike.
We have consistent direction now.

Thanks


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

.. 0 http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/cinder#n403

Thanks

gmann__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

responded Mar 23, 2017 by zhu.fanglei_at_zte.c (500 points)  
0 votes

2017-03-22 19:31 GMT-07:00 zhu.fanglei@zte.com.cn:

To have only one folder (tempest/api/volume/ ) looks really good, and, do we
plan to deem "api_version" and "microversion" as one thing?

i.e., we will use the mechanism of microversion to skip v3 new functional
tests when the environment only supports v2?

Yeah, that is right.
Tempest has the range of microversions with the config options
min/maxmicroversions and we can select the target microversions.
If both min
microversion and max_microversion are not specified(means
None), microversion tests run as the help message1.

So the configuration would be like

gate-tempest-dsvm-neutron-full-ubuntu-xenial - (existing job): testing
Cinder V3 API
catalogtype: Specify Cinder V3 API's one
min
microversion: Don't specify
max_microversion: Specify max microversion of the branch (master, stable)

gate-tempest-dsvm-neutron-full-ubuntu-xenial-cinder-v2: (new job):
testing Cinder V2 API
catalogtype: Specify Cinder V2 API's one
min
microversion: Don't specify
max_microversion: Don't specify

Thanks


On Thu, Mar 23, 2017 at 7:02 AM, Ken'ichi Ohmichi <ken1ohmichi@gmail.com>
wrote:

2017-03-22 14:32 GMT-07:00 Andrea Frittoli <andrea.frittoli@gmail.com>:


> On Wed, Mar 22, 2017 at 8:31 PM Sean McGinnis <sean.mcginnis@gmx.com>
wrote:
>>
>> On Wed, Mar 22, 2017 at 01:08:23PM -0700, Ken'ichi Ohmichi wrote:
>> > Hi,
>> >
>> > Now we need to update Tempest for following Cinder API status..
>> > I have an idea for restructuring and happy to see feedback about
that.

>> >
>> > Now Cinder API status is
>> > V1: Deprecated
>> > V2: Deprecated
>> > V3: Current
>> > V1 API tests have been removed from Tempest side already, so we just
>> > need to concentrate on V2 and V3 now.
>>
>> >
>> > Gate jobs
>> > Most Cinder tests are implemented for V2 API on Tempest side and the
>> > base microversion of V3 is the same as V2.
>> > Then we can re-use V2 API tests for the base microversion of V3 API.
>> > One idea is that we can have Cinder V3 API tests as the default on
the
>> > gate jobs and the V2 API tests as another job like the following
>> > because the V2 API is deprecated.
>> >
>> > gate-tempest-dsvm-neutron-full-ubuntu-xenial - (existing job):
>> > testing Cinder V3 API
>> > gate-tempest-dsvm-py35-ubuntu-xenial - (existing job): testing
Cinder
>> > V3 API
>> > ...
>> > gate-tempest-dsvm-neutron-full-ubuntu-xenial-cinder-v2: (new job):
>> > testing Cinder V2 API
>> >
>>
> I guess this job would run against tempest and cinder only?

A nice point, I think so.

>> +1 I like this idea.
>>
>> > We had the same testing way for Nova V2 API and V2.1 API before, and
>> > we could avoid copy&paste V2 test code for V2.1 API on Tempest.
>> >
>> > Test Structure
>> > Current test structure is like:
>> > tempest/api/volume/ - V2 API tests
>> > tempest/api/volume/v2 - V2 API tests
>> > tempest/api/volume/v3 - V3 API tests
>> > Yes, this is mess.
>> > For re-using V2 API tests for V3 API, it would be better to remove
>> > "v2" from V2 API tests for avoiding confusions.
>> >
>> > A new structure could be
>> > tempest/api/volume/ - All tests for V2 API and the base
>> > microversion of V3 API
>> > tempest/api/volume/v3 - V3 API specific tests for newer
microversions
>> > or
>> > tempest/api/volume/ - All tests for V2 API and V3 API which
>> > includes newer microversions

+1, this looks better as no more version specific tests and all v2 tests
should run as it is on v3 base version.

>> >
>> > As the reference, Nova API structure is like the later.
>>
>> I like the last one better as well.
>>
> My favourite option would be that that generates less churn in the code
:)
> One folder for everything means moving 4 or 5 modules only, so I think
that
> would
> be a good option.

> I would prefer to avoid though having a lot of v3 test classes that
inherit
> from
> v2 test classes, and just set apiversion = 3.

Yeah, I agree :-)

yea we should not have that.

> As long as we can assume we will never run v2 and v3 in the same job, we
> could
> have cinderapiversion as a configuration setting, to determine which
> cinder
> endpoint to hit when running the volume tests.

Or it would be enough to have the existing "catalogtype",
"min
microversion" and "maxmicroversion" only without apiv1/v2/v3 to
control the target API version, because of the above separated gate
jobs.

Yes, so devstack does set different catalog for v2 and v3 [0]. Based on
those catalog_type configured on tempest config(we already have that for
volume API config) auth can select the right endpoints to make the API call.

All existing tests can be run for both API without any extra class or
change. This way we can get rid of 'apiversion' in all volumes clients and
have them as single copy for v2 and v3 base.
Further v3 microversion tests can be implemented as accordingly by sending
the microversion header on API request. and devstack can tell Temepst not to
set microversion if catalog
type volume_v2 is being asked to run the tests.

As you mentioned it can be same way we handle compute v2, v2.1 and +
microversion tests.

> Apart from the volume tests, if we split the gate jobs into standard one
> running v3
> plus and extra v2 one, we should make sure that all tests that use the
> volume API
> use a consistent version of the volume API. Nova as well should be
> configured if
> possible to use the same volume API version.

This also is a nice point.
Nova team also has a plan to use cinder v3 as the default in Pike.
We have consistent direction now.

Thanks


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

.. 0
http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/cinder#n403

Thanks
gmann


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 23, 2017 by Ken'ichi_Ohmichi (8,640 points)   2 3 3
0 votes

The current mechanism of microversion looks a bit strange to me.

https://github.com/openstack/tempest/blob/c0223906280619b6eb1ffb3fa200136fd3050528/tempest/api/volume/v3/base.py#L49-L52

that means we set microversion at setUp and clear it at tearDown, but that is strange,

1) we never set microvertion at testcase level, i.e., we only set microversion at class level, somethink like:

    class KeyPairsV210TestJSON(base.BaseKeypairTest):

        min_microversion = '2.10'

2) if microversion is set to None in tearDown, then in resource_cleanup the client will not have the correct microversion to work.

to sum up, why we will set and clear microversion at testcase level while not at testclass level?

Original Mail

Sender: <ken1ohmichi@gmail.com>
To: <openstack-dev@lists.openstack.org>
Date: 2017/03/23 11:24
Subject: Re: [openstack-dev] [qa][cinder] RFC: Cinder test on Tempest

2017-03-22 19:31 GMT-07:00 <zhu.fanglei@zte.com.cn>:
> To have only one folder (tempest/api/volume/ ) looks really good, and, do we
> plan to deem "api_version" and "microversion" as one thing?

> i.e., we will use the mechanism of microversion to skip v3 new functional
> tests when the environment only supports v2?

Yeah, that is right.
Tempest has the range of microversions with the config options
min/maxmicroversions and we can select the target microversions.
If both min
microversion and max_microversion are not specified(means
None), microversion tests run as the help message1.

So the configuration would be like

gate-tempest-dsvm-neutron-full-ubuntu-xenial - (existing job): testing
Cinder V3 API
catalogtype: Specify Cinder V3 API's one
min
microversion: Don't specify
max_microversion: Specify max microversion of the branch (master, stable)

gate-tempest-dsvm-neutron-full-ubuntu-xenial-cinder-v2: (new job):
testing Cinder V2 API
catalogtype: Specify Cinder V2 API's one
min
microversion: Don't specify
max_microversion: Don't specify

Thanks


> On Thu, Mar 23, 2017 at 7:02 AM, Ken'ichi Ohmichi <ken1ohmichi@gmail.com>
> wrote:
>>
>> 2017-03-22 14:32 GMT-07:00 Andrea Frittoli <andrea.frittoli@gmail.com>:
>> >
>> >
>> > On Wed, Mar 22, 2017 at 8:31 PM Sean McGinnis <sean.mcginnis@gmx.com>
>> wrote:
>> >>
>> >> On Wed, Mar 22, 2017 at 01:08:23PM -0700, Ken'ichi Ohmichi wrote:
>> >> > Hi,
>> >> >
>> >> > Now we need to update Tempest for following Cinder API status..
>> >> > I have an idea for restructuring and happy to see feedback about
>> that.
>>
>> >> >
>> >> > Now Cinder API status is
>> >> > V1: Deprecated
>> >> > V2: Deprecated
>> >> > V3: Current
>> >> > V1 API tests have been removed from Tempest side already, so we just
>> >> > need to concentrate on V2 and V3 now.
>> >>
>> >> >
>> >> > Gate jobs
>> >> > Most Cinder tests are implemented for V2 API on Tempest side and the
>> >> > base microversion of V3 is the same as V2.
>> >> > Then we can re-use V2 API tests for the base microversion of V3 API.
>> >> > One idea is that we can have Cinder V3 API tests as the default on
>> the
>> >> > gate jobs and the V2 API tests as another job like the following
>> >> > because the V2 API is deprecated.
>> >> >
>> >> > gate-tempest-dsvm-neutron-full-ubuntu-xenial - (existing job):
>> >> > testing Cinder V3 API
>> >> > gate-tempest-dsvm-py35-ubuntu-xenial - (existing job): testing
>> Cinder
>> >> > V3 API
>> >> > ...
>> >> > gate-tempest-dsvm-neutron-full-ubuntu-xenial-cinder-v2: (new job):
>> >> > testing Cinder V2 API
>> >> >
>> >>
>> > I guess this job would run against tempest and cinder only?
>>
>> A nice point, I think so.
>>
>> >> +1 I like this idea.
>> >>
>> >> > We had the same testing way for Nova V2 API and V2.1 API before, and
>> >> > we could avoid copy&paste V2 test code for V2.1 API on Tempest.
>> >> >
>> >> > Test Structure
>> >> > Current test structure is like:
>> >> > tempest/api/volume/ - V2 API tests
>> >> > tempest/api/volume/v2 - V2 API tests
>> >> > tempest/api/volume/v3 - V3 API tests
>> >> > Yes, this is mess.
>> >> > For re-using V2 API tests for V3 API, it would be better to remove
>> >> > "v2" from V2 API tests for avoiding confusions.
>> >> >
>> >> > A new structure could be
>> >> > tempest/api/volume/ - All tests for V2 API and the base
>> >> > microversion of V3 API
>> >> > tempest/api/volume/v3 - V3 API specific tests for newer
>> microversions
>> >> > or
>> >> > tempest/api/volume/ - All tests for V2 API and V3 API which
>> >> > includes newer microversions


> +1, this looks better as no more version specific tests and all v2 tests
> should run as it is on v3 base version.


>>
>> >> >
>> >> > As the reference, Nova API structure is like the later.
>> >>
>> >> I like the last one better as well.
>> >>
>> > My favourite option would be that that generates less churn in the code
>> :)
>> > One folder for everything means moving 4 or 5 modules only, so I think
>> that
>> > would
>> > be a good option.
>> >
>> > I would prefer to avoid though having a lot of v3 test classes that
>> inherit
>> > from
>> > v2 test classes, and just set apiversion = 3.
>>
>> Yeah, I agree :-)



> yea we should not have that.


>>
>>
>> > As long as we can assume we will never run v2 and v3 in the same job, we
>> > could
>> > have cinderapiversion as a configuration setting, to determine which
>> > cinder
>> > endpoint to hit when running the volume tests.
>>
>> Or it would be enough to have the existing "catalogtype",
>> "min
microversion" and "maxmicroversion" only without apiv1/v2/v3 to
>> control the target API version, because of the above separated gate
>> jobs.
>>

> Yes, so devstack does set different catalog for v2 and v3 [0]. Based on
> those catalogtype configured on tempest config(we already have that for
> volume API config) auth can select the right endpoints to make the API call.

> All existing tests can be run for both API without any extra class or
> change. This way we can get rid of 'api
version' in all volumes clients and
> have them as single copy for v2 and v3 base.
> Further v3 microversion tests can be implemented as accordingly by sending
> the microversion header on API request. and devstack can tell Temepst not to
> set microversion if catalogtype volumev2 is being asked to run the tests.

> As you mentioned it can be same way we handle compute v2, v2.1 and +
> microversion tests.


>>
>> > Apart from the volume tests, if we split the gate jobs into standard one
>> > running v3
>> > plus and extra v2 one, we should make sure that all tests that use the
>> > volume API
>> > use a consistent version of the volume API. Nova as well should be
>> > configured if
>> > possible to use the same volume API version.
>>
>> This also is a nice point.
>> Nova team also has a plan to use cinder v3 as the default in Pike.
>> We have consistent direction now.
>>
>> Thanks
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



> .. 0
http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/cinder#n403

> Thanks
> gmann




> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
responded Mar 23, 2017 by zhu.fanglei_at_zte.c (500 points)  
...